Ultra-Fast Thumbnail Generation with Jekyll and libvips

How we started: thumbnails with smartcropper

In the very early days of OpsLevel, our marketing website was powered by WordPress.  Even though our site then was small, WordPress was a pretty big moving part that required more maintenance than it was worth.  We found ourselves spending time on upgrading both WordPress and its plugins, debugging when things broke, and managing performance.  We also found that drafts were not a great workflow for previewing or staging changes as the live production site wouldn’t always look the same as a draft edit.

In mid-2019, we hopped aboard the Jamstack train and rebuilt our site (this site you’re reading now) using Jekyll.

One feature we wanted for our blog posts was automatic thumbnail generation.  Every blog post has a hero image (like the speedometer above) and we wanted smaller versions of this image for the blog index page.

We found Kari Silvennoinen’s blog post which describes a mechanism for using the smartcropper gem in a Jekyll generator.

We refactored things a bit for our use case (we generate multiple thumbnail sizes and also allow selectively overriding thumbnails).  However, the main call to smartcropper was nearly identical to Kari’s:

At the time, this change increased the build time of our site slightly.  Building the entire site would take about 10 seconds in devlocal and a few minutes to build on Netlify for production.  Not terrible and hey, we’re generating thumbnails, so obviously that’s going to take some time.

Bigger Site, Slower Build

Fast forward two years and we now have a lot more blog posts, which meant more thumbnails to generate.  During this period, the build time of our site creeped up to 30s - 40s for devlocal and 12+ minutes on Netlify.  Developing or changing our site was painful.  95% of the time was spent generating thumbnails, so we investigated to see if there was a better way.

Under the covers, smartcropper uses RMagick, which is Ruby interface for ImageMagick.

After some research, we found jekyll_picture_tag, which is based on libvips.  libvips bills itself as:

 libvips is a demand-driven, horizontally threaded image processing library. Compared to similar libraries, libvips runs quickly and uses little memory.

That sounds promising.  Let’s put it to the test.

libvips supports the same entropy-based cropping as smartcropper, so it was a straightforward replacement in our generator.  Here’s our new implementation of crop_and_scale:

libvips is fast

We profiled site build time with both smartcropper and libvips.

This is the output of jekyll build --profile:

🤯

Generating thumbnails with libvips is nearly 10x faster than smartcropper.

Ten.  Times.  Faster.

Full site builds now take < 4 seconds in devlocal and < 60 seconds on Netlify for production.  In addition, using Jekyll’s --incremental option in devlocal makes editing nearly instant.

The full implementation

Here’s our full implementation for generating thumbnails in Jekyll.

It’s pretty simple, but supports generating multiple thumbnails from a single image.  It also supports overriding thumbnails with an explicit file.

Put the following in _plugins/post_thumbnail_generator.rb:

‍

To specify the various thumbnail dimensions, add or customize the following in your _config.yml:

‍

In your posts, you can set thumbnail in the front matter.  For example, here’s this post’s front matter:

‍

Check us out

If you’re interested in performance, microservices, or helping teams adopt DevOps and service ownership, check out our open roles.