How we reduced our website build time by 59%
How we reduced our website build time by 59%
My name is Aikansh, and I am an intern with the Ops Engineering team at Fyle. During my tenure here, I've absolutely loved working with such talented people over the 6+ months I've spent here. I've learned more about Expense Management than I ever thought I would, and I'm grateful for all the knowledge, support, and guidance the team has provided me. :)
What do I do at Fyle?
During my internship, I've loved working on the usability issues module, which gave me exposure to varied microservices and how they work in sync.
Our web strategists, content specialists, and web designers help create the desired results together. However, as developers, we need to tie all the pieces together with the right code for the website. We also run multiple tests to ensure that the site is bug-free and stay alert to be the troubleshooters when necessary.
This internship has helped me hone my skills in front end technologies, workflow in a start-up, management among teams, the importance of design docs, and understanding products we build from the users' perspective. Here’s my experience in reducing our website build time by 59%.
Context to the situation
For our website, we use the versatile webpack as our module bundler and the excellent jekyll as our site generator. In summary, what we do is compile our ES6 code into an ES5 bundle and place it in Jekyll's main /assets directory. During development, we have Jekyll's server watch for changes and reload the page every time it detects a change in our JavaScript code.
The Problem
As is always the case given the speed of development, our web pages, components, and webpack config grew organically, and the speed of the pipeline was an after-thought.
Often we had to wait 10 whole seconds for our build to apply simple changes like color:black; to color:blue; :/
But there came a tipping point. Eventually, we snapped and decided to get that build time way down.
tl;dr version
We reduced the build time by 59% with some minor tweaks. (As a bonus, my teammates also like me a lot more now!) ;)
The Approach
After being assigned this challenging yet interesting task, I started by measuring our performance. This really helps to find current bottlenecks, and compare progress as we make changes.
On running npm run dev, our script for getting the development server up running, we had to wait for ~20-25s for webpack and jekyll to do their jobs.
Then, I noted down the basic points against which we could test our output later. We call these functional requirements:
Production build should not be affected; don't sacrifice the quality of application for small performance gains.
npm script commands should remain unaltered.
There should be noticeable differences in the resolving speed of the build & watch process.
Next up was research; following lots of articles, stackoverflow discussions, github issues threads, and official documentations.
The Solution
Using some of the suggestions and experimenting with multiple configurations to accelerate the webpack and jekyll build performance, I came up with five 3 second changes to reduce the build time by over 59%.
Sounds good? Let’s get started!
Enable persistent caching with the webpack cache-loader
The cache-loader allows caching the result of following loaders on disk (default) or in the database. Cache-loaders can be placed in any chain of loaders and cache the results of previous loaders. By default, it stores the cache in the .cache-loader folder in the project root.
Replace Uglifyjs with Terser webpack plugin to minify your JavaScript
Attached below is a snippet from npm trends. As we can see, Terser has gained significant popularity in the past one year. This can be directly attributed to its better performance compared to other minification tools.
Use Shopify’s Liquid-C gem to speed up Liquid parsing
Liquid is a secure templating language developed by Shopify. Liquid is designed for end-users to execute logic within template files without imposing any security risk on the hosting server. Jekyll uses Liquid to generate the post content within the final page layout structure and as the primary interface for working with your site and post/page data.
We took advantage of Shopify’s Liquid-C gem which is written in C to speed up this parsing. For this to work, we simply needed to update our Gemfile.
Use Jekyll Include Cache plugin
To see how Jekyll _includes affects site build times, I observed the build profile by running jekyll build --profile. Right away, it was visible that some include files are being called hundreds of times, creating a performance bottleneck.
Now was the right time to make use of jekyll-include-cache, a Jekyll plugin created by Ben Balter that will cache the rendering of our Liquid includes.
“If you have a computationally expensive include (such as a sidebar or navigation), Jekyll Include Cache renders the include once, and then reuses the output any time that includes is called with the same arguments, potentially speeding up your site’s build significantly.”
I started by caching the navigation in the header, footer, cookie consent banner, hello bar, and the schedule demo form on our site. The reason; these components were being called on every page and didn’t change that frequently.
Enable Jekyll Incremental flag in development
Incremental regeneration helps shorten build times by only generating documents and pages that were updated since the previous build. It does this by keeping track of both file modification times and inter-document dependencies in the .jekyll-metadata file.
This is how our script to run local setup looks like:
"dev": "webpack --mode development --devtool inline-source-map --watch & jekyll serve --livereload --incremental"
In summary
These are some of the hacks that helped us reduce our build time considerably. But webpack is a complex creature, and optimization is a dark art. I’m sure of having missed many tricks, but do reach out to me and let me know your thoughts!
By the way, let me show you an actual picture of our website build now ;)
