How to Create Dynamic Sitemaps in Next.js Pages Directory

Sitemaps play a crucial role in helping search engines discover and index your website's content. In Next.js, creating a dynamic sitemap can be a bit tricky, especially when using the pages router. In this blog post, we'll explore how to generate dynamic sitemaps in Next.js using server-side rendering, even if you don't have access to the app router or are using page extensions.

The problem

When building a website with Next.js, you may encounter a common problem related to sitemaps: you want a dynamic sitemap that reflects your website's content, rather than a static sitemap.xml file. This becomes particularly challenging when you have hundreds of dynamic pages that need to be accounted for in the sitemap.

Next.js app router provides a sitemap.ts file that can solve this problem, but if you are only using the pages router or using file extensions that don't work with the app router, you need to find a way to dynamically create the sitemap in the pages directory.

Exploring Solutions

Ideally, if you have access to the Next.js app router, you can leverage the sitemap.(js|ts) to programmatically generate a sitemap. This is a built-in functionality in Next.js and will work out of the box.

However, this might not be an option sometimes. In such cases, the alternative solution is to generate a dynamic sitemap on the server-side using getServerSideProps function in the pages directory.

Solution

Here's how you can create a dynamic sitemap in Next.js using server-side rendering:

1. Create a sitemap.xml.tsx (or sitemap.xml.page.tsx if using page extensions) file in the pages directory and add the following server-side rendering boilerplate code:

// pages/sitemap.xml.tsx
export default function SiteMap() {
  return; 
}

export async function getServerSideProps({ res }: GetServerSidePropsContext) {
  // We will write the sitemap to the browser here
}

Great! Now that we have this boilerplate code, let's go ahead and generate the actual sitemap.

2. Create an asynchronous function called generateSiteMap that will handle queries to the database and generate the XML sitemap string:

async function generateSiteMap(): Promise<string> {
  // We will fetch data from our DB and generate the sitemap string here
}

3. Inside this function, query data for the pages that you want to display in the sitmap. In this example, we will display the "TEACHER" users and the "Public" classes.

async function generateSiteMap(): Promise<string> {
  // Database queries
  const [classes, users] = await Promise.all([
    prisma.class.findMany({
      where: {
        status: "Public",
      },
      select: {
        classUrlSlug: true,
        lastModified: true,
      },
    }),
    prisma.user.findMany({
      where: {
        userProfileSlug: {
          not: null,
        },
        role: "TEACHER",
      },
      select: {
        userProfileSlug: true,
        lastModified: true,
      },
    }),
  ]);

  // TODO: Build sitemap
}

Note: The sitemap is a tool to facilitate crawling, but just because a page url isn't displayed in the sitemap doesn't mean that the page will not be indexed. If you want to avoid indexing certain pages you can add the no-index option in the meta robots tag

3. Once we have our data, all we need to do is return the xml string with the following format:

const BASE_URL = [YOUR_BASE_URL]; // i.e. https://www.google.com

async function generateSiteMap(): Promise<string> {
  // Database queries
  const [classes, users] = await Promise.all([
    prisma.class.findMany({
      where: {
        status: "Public",
      },
      select: {
        classUrlSlug: true,
        lastModified: true,
      },
    }),
    prisma.user.findMany({
      where: {
        userProfileSlug: {
          not: null,
        },
        role: "TEACHER",
      },
      select: {
        userProfileSlug: true,
        lastModified: true,
      },
    }),
  ]);

  // Build sitemap
  return `<?xml versionn="1.0" encodingg="UTF-8"?>
  <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
    <url>
      <loc>${baseUrl}</loc>
      <lastmod>${format(new Date(), "yyyy-MM-dd")}</lastmod>
      <changefreq>daily</changefreq>
      <priority>1.0</priority>
    </url>
    ${classes
      .map((x) => {
        return `<url>
        <loc>${BASE_URL + "/class/" + x.classUrlSlug}</loc>
        <lastmod>${format(new Date(x.lastModified), "yyyy-MM-dd")}</lastmod>
        <changefreq>monthly</changefreq>
        <priority>0.5</priority>
      </url>`;
      })
      .join("")}
      ${users
        .map((x) => {
          return `<url>
          <loc>${BASE_URL + "/profile/" + x.userProfileSlug}</loc>
          <lastmod>${format(new Date(x.lastModified), "yyyy-MM-dd")}</lastmod>
          <changefreq>monthly</changefreq>
          <priority>0.5</priority>
        </url>`;
        })
        .join("")}
  </urlset>`;
}

Feel free to experiment with the lastmod, changefreq, and priority fields.

4. Once we have the generateSiteMap function, update the getServerSideProps function to retrieve the sitemap and write it to the browser:

export async function getServerSideProps({ res }: GetServerSidePropsContext) {
  // Generate the XML sitemap
  const sitemap = await generateSiteMap();

  // We send the XML to the browser
  res.setHeader("Content-Type", "text/xml");
  res.write(sitemap);
  res.end();

  return { props: {} };
}

Conclusion

Creating dynamic sitemaps in Next.js using the pages router and server-side rendering is a practical solution when the app router's sitemap.ts file cannot be useDebugValue.

By leveraging the power of getServerSideProps and generating the XML sitemap on the server, you can ensure that your website's content, including hundreds of dynamic pages, is easily discoverable by search engines. With this dynamic sitemap setup, you can focus on creating great content while letting search engines efficiently index your website.