Lessons learned writing our own static site generator

19 January 2022

Nowadays, websites and web apps can be built and served in many different ways. Using pure HTML, Javascript, and CSS, with frameworks such as React, Vue, or Angular, following a CSR or SSR approach, MPA, using back-end frameworks like Phoenix, with a CMS like WordPress or Drupal, implementing static site generators like Gatsby, Next js, Hugo, Jekyll, etc. The options are overwhelming.

If that wasn't enough, you can still dive into 0kb of JS frameworks, AlpineJS, Stimulus, Petite Vue, Astro, Iles, MarjoJS, Qwik, Turbo, or Remix, the new kid on the block.

But when websites have mostly static content (except for dynamic news/blog pages), that’s when SSGs (Static Site Generators) become a great option. One of the most popular frameworks out there is Gatsby. That is the one we used to build our previous website and for a time, it served us well.

One of the issues we discovered a couple of years after, was that the build times were taking longer and longer. This was due to the huge amount of pages that the site was accumulating–specifically because of the multiple languages–and the difficulty of building just one page. Instead, we had to build the whole website whenever a single page was updated.

We also realized that static sites don’t need to have a lot of JavaScript and NPM dependencies. They should feel light and easy to maintain. The main requirements are just HTML, CSS, and some JavaScript for a side or top navigation, a contact form, and some other bits and bobs.

Arriving at this point, we learned that Gatsby was releasing incremental builds, but even with that potentially helpful feature, our minds were already set: we were going to build our static site generator.

The selected tech stack

At TableCheck we are highly invested in AWS, Typescript, React, Emotion, and our own UI Toolkit called TableKit. It became quite clear from the beginning that using these technologies could accelerate development and simplify maintenance. As a comparison exercise, Wahid Farid (project lead) also did a POC with Hugo and Joan Mira (front-end manager) with the EJS templating system, but eventually, we preferred to continue with the React SSR approach.

At the same time, our previous Gatsby website was building its pages by gathering the content from the Storyblok API. This was something that we were quite happy about, specifically because of the flexibility that provides this headless CMS for building pages using blocks with custom schemas. This allows us and content editors to build pages for different regions in the world by reusing common layouts and blocks defined in the CMS.

We also considered the possibility of using WordPress and Elementor (mostly because of the UX for content editors). But the fact that it’s based on PHP requires other technologies and it’s often the target of hackers was something that we were not very keen on. Hence, we decided to continue with Storyblok as our CMS.

Regarding the infrastructure, Alexander Nicholson (SRE lead) suggested that we use a serverless approach (Seed.run) with a lambda function to build the HTML static pages.

And finally, for the search functionality, Joan Mira suggested using Algolia and for the product demo scheduling, Eri Koyano (product manager) suggested using Calendly.

This is the approved tech stack for our custom statically generated website:

AWS, serverless, and Seed.run
Typescript
React SSR
Emotion, TableKit & CSS variables
Storyblok
Algolia and Calendly

Website architecture

Once we decided on the technologies to use, TableCheck’s principal front-end engineer, Simeon Cheeseman, suggested using React SSR to build the pages. We brainstormed about how to make it work and ended up with the following:

Content gets added/updated in Storyblok and the user clicks the save button
A webhook gets triggered in Storyblok
A NodeJS Express server running on a lambda function receives the webhook call
A server-side script reads the story ID from the URL passed to the server
The script calls the Storyblok API with that ID and gets all the data needed to build that specific page
Then it generates the HTML using React components
And finally, it uploads the file to an S3 bucket from where it’d be served to the Internet

We all agreed with this approach and Alexandr Shostyr (project technical lead) built a POC.

From the POC to the finish line

Once the POC was approved and the architecture validated, we continued the development with the help of Irina Soupel (front-end engineer). We soon realized that we'd have to do certain things slightly different from other TableCheck projects, especially because we were not using React in the front-end:

Use emotion for SSR to support nth-child selectors
Figure out how to use CSS variables with Tablekit's theme provider
Create a simple Button and Input styled component to bypass an issue with hexToRgba

Another considerable hiccup we encountered was regarding building individual pages vs all pages. Some Storyblok blocks are common to all pages, like the top navigation or the footer. Therefore, whenever a content editor changes one of these blocks, we have to regenerate all the pages on the website. This represented a challenge since we would need to have a way to tell the build script if all pages need to be built or not. We are currently still investigating this point and will update the article once we find the appropriate solution.

At the same time, the blog listing pages were also tricky. First, we implemented them dynamically using Algolia. That worked fine but it required many API requests, which would make the navigation slow and costly. Finally, we found a way to build all these pages statically, even for the category and paginated listing pages.

We also had other challenges regarding the sitemap, legacy redirects, build complications due to having to share the same S3 bucket with another website, etc. Nonetheless, these issues are not representative of the migration process from Gatsby to our custom SSG.

Next steps

Going forward, we still want to continue doing improvements. We will update the article in the future with more details. Here are some of the things we are considering doing:

The build speed: especially with supporting individual page builds
Preload critical assets and pages (like Gatsby does) to make the website load even faster than it already does, probably because we have very little JS code on it

Overall, it was a great experience building something new from scratch that we have full control over. More cool things will come. Stay tuned and follow us!

March 2022 Update

Good news! after a bit of work, we were able to implement individual page builds with StoryBlok.

The way we ended up doing it is by copying/pasting the page ID into an input within a modal dialog that appears when clicking on a custom task. This allows us to pass the ID of the page we want to build to the Lambda function.