Creating a sitemap generator for Next.js

From sitemap creation to search engine index requests

In this story I want to share the SEO work I did while migrating the Volla bridge page from Vue.js to Next.js—and how I built a dynamic sitemap generator for Next.js.

1. Create a sitemap generator script that follows the Next.js folder structure

Next.js automatically creates URLs based on the folder and file names inside the pages directory. So I'll write the XML generator to walk those names while excluding Next.js's special files (_document.js, _app.js, etc.) using globby.

Create a scripts folder in the root directory and add sitemap-common.js:

Running this script produces an XML file like this (example):

2. A sitemap generator script for an external API

It's relatively easy to generate sitemaps for static pages, as in the example above. For dynamic pages (e.g., pages that use a userId), we need a slightly different script. In the example I used JSONPlaceholder's API.

I wrote sitemap-posts.js:

Running this script produces an XML file like this (example):

3. A script that compresses all sitemap files to gzip

Sitemaps compressed to gzip (.gz) work the same way as XML sitemaps but are much smaller. Using zlib, I'll compress the XML files into gzip:

This script compresses every XML file generated in steps 1 and 2 into .gz format.

4. A sitemap-index generator script for the files created above

Submitting multiple sitemaps to search engines (Google Search Console, Naver Search Advisor, etc.) requires a sitemap index file.

I created separate sitemaps for various dynamic URL patterns such as /seller/[_id], /product/[_id], and /video/[_id]. To submit them to Google Search Console, a single sitemap must be submitted, so I wrote this script to build a sitemap index:

Running it produces an XML file like this (example):

5. A bash script that builds new sitemaps on every master deployment, plus a script that pings Google Search Console from GitHub Actions

# yarn sitemap
$ cd public
$ rm -rf sitemap
$ mkdir sitemap
$ cd ..
$ cd scripts
$ node ./sitemap-common.js
$ node ./sitemap-posts.js
$ node ./sitemap.js

If you want Google Search Console to reindex your pages, just add this line at the end of the script:

$ curl http://google.com/ping?sitemap=http://website.com/sitemap.xml

I wrote a bash script to generate the XML sitemaps, compress them to gzip, and remove unused XML files automatically.

I then updated the workflow file to run these scripts via GitHub Actions on each master deployment.

# Create a sitemap and submit it to Google.
- run: yarn sitemap
  name: ping sitemap

Takeaway

I hope you enjoyed this short tutorial on generating sitemaps for Next.js. Feel free to reach out via LinkedIn or by email at ghsspower@gmail.com. Thanks!