Lessons learned writing our own static site generator
The journey from Gatsby to custom React SSR SSG. Article originally published on TableCheck's Tech Blog
But when websites have mostly static content (except for dynamic news/blog pages), that’s when SSGs (Static Site Generators) become a great option. One of the most popular frameworks out there is Gatsby. That is the one we used to build our previous website and for a time, it served us well.
One of the issues we discovered a couple of years after, was that the build times were taking longer and longer. This was due to the huge amount of pages that the site was accumulating–specifically because of the multiple languages–and the difficulty of building just one page. Instead, we had to build the whole website whenever a single page was updated.
Arriving at this point, we learned that Gatsby was releasing incremental builds, but even with that potentially helpful feature, our minds were already set: we were going to build our static site generator.
The selected tech stack
At TableCheck we are highly invested in AWS, Typescript, React, Emotion, and our own UI Toolkit called TableKit. It became quite clear from the beginning that using these technologies could accelerate development and simplify maintenance. As a comparison exercise, Wahid Farid (project lead) also did a POC with Hugo and Joan Mira (front-end manager) with the EJS templating system, but eventually, we preferred to continue with the React SSR approach.
At the same time, our previous Gatsby website was building its pages by gathering the content from the Storyblok API. This was something that we were quite happy about, specifically because of the flexibility that provides this headless CMS for building pages using blocks with custom schemas. This allows us and content editors to build pages for different regions in the world by reusing common layouts and blocks defined in the CMS.
We also considered the possibility of using WordPress and Elementor (mostly because of the UX for content editors). But the fact that it’s based on PHP requires other technologies and it’s often the target of hackers was something that we were not very keen on. Hence, we decided to continue with Storyblok as our CMS.
Regarding the infrastructure, Alexander Nicholson (SRE lead) suggested that we use a serverless approach (Seed.run) with a lambda function to build the HTML static pages.
This is the approved tech stack for our custom statically generated website:
- AWS, serverless, and Seed.run
- React SSR
- Emotion, TableKit & CSS variables
- Algolia and Calendly
Once we decided on the technologies to use, TableCheck’s principal front-end engineer, Simeon Cheeseman, suggested using React SSR to build the pages. We brainstormed about how to make it work and ended up with the following:
- Content gets added/updated in Storyblok and the user clicks the save button
- A webhook gets triggered in Storyblok
- A NodeJS Express server running on a lambda function receives the webhook call
- A server-side script reads the story ID from the URL passed to the server
- The script calls the Storyblok API with that ID and gets all the data needed to build that specific page
- Then it generates the HTML using React components
- And finally, it uploads the file to an S3 bucket from where it’d be served to the Internet
We all agreed with this approach and Alexandr Shostyr (project technical lead) built a POC.
From the POC to the finish line
Once the POC was approved and the architecture validated, we continued the development with the help of Irina Soupel (front-end engineer). We soon realized that we'd have to do certain things slightly different from other TableCheck projects, especially because we were not using React in the front-end:
- Use emotion for SSR to support
- Figure out how to use CSS variables with Tablekit's theme provider
- Create a simple Button and Input styled component to bypass an issue with hexToRgba
Another considerable hiccup we encountered was regarding building individual pages vs all pages. Some Storyblok blocks are common to all pages, like the top navigation or the footer. Therefore, whenever a content editor changes one of these blocks, we have to regenerate all the pages on the website. This represented a challenge since we would need to have a way to tell the build script if all pages need to be built or not. We are currently still investigating this point and will update the article once we find the appropriate solution.
At the same time, the blog listing pages were also tricky. First, we implemented them dynamically using Algolia. That worked fine but it required many API requests, which would make the navigation slow and costly. Finally, we found a way to build all these pages statically, even for the category and paginated listing pages.
We also had other challenges regarding the sitemap, legacy redirects, build complications due to having to share the same S3 bucket with another website, etc. Nonetheless, these issues are not representative of the migration process from Gatsby to our custom SSG.
Going forward, we still want to continue doing improvements. We will update the article in the future with more details. Here are some of the things we are considering doing:
- The build speed: especially with supporting individual page builds
- Preload critical assets and pages (like Gatsby does) to make the website load even faster than it already does, probably because we have very little JS code on it
Overall, it was a great experience building something new from scratch that we have full control over. More cool things will come. Stay tuned and follow us!
March 2022 Update
Good news! after a bit of work, we were able to implement individual page builds with StoryBlok.
The way we ended up doing it is by copying/pasting the page ID into an input within a modal dialog that appears when clicking on a custom task. This allows us to pass the ID of the page we want to build to the Lambda function.