apps/www/_blog/2022-12-13-storage-image-resizing-smart-cdn.mdx
Today we're introducing three new features for Supabase Storage: Image resizing, webhooks, and a Smart CDN.
These features are designed to work together to deliver a next-gen image resizing system. Let's explore how it works.
<div className="bg-gray-300 rounded-lg px-6 py-2 italic">🆕 What is new in Storage: Supabase Storage v3: Resumable Uploads with support for 50GB files
</div>Image resizing has been a popular request since we released Supabase Storage. For good reason - if you're providing an image upload function to your users, you can't expect them to optimize the images before uploading.
Supabase developers have been hacking together all sorts of workarounds using client-side compression, or external providers like Cloudinary and Imgix.
Today's release changes that. Resizing is now as easy as adding a few lines of code:
supabase.storage.from('bucket').getPublicUrl('image.jpg', {
transform: {
width: 500,
height: 600,
},
})
Supabase Storage uses imgproxy. As usual, we assessed various open source tools available, including Imaginary (MIT) and Sharp (Apache2).
All of these have very friendly open source licenses for self-hosting, so ultimately it came down to features and performance.
While Imaginary doesn't support some modern image formats yet (like AVIF), Sharp and imgproxy are both very feature-complete and well-maintained
In a head-to-head benchmark, we found the performance to be almost identical, with the deciding factor instead coming down to resources.
We spent a long time considering what a good developer experience would be for image resizing.
We landed on a simple, on-demand interface. Downloading a resized image in JavaScript looks like this:
supabase.storage.from('bucket').download('image.jpg', {
transform: {
width: 800,
height: 300,
resize: 'contain', // 'contain' | 'cover' | 'fill'
},
})
Or you can make a direct request to the Storage API (see width and height params):
GET https://project_id.supabase.co/storage/v1/render/image/public/bucket/image.jpg?width=500&height=600
If you've never used a resizing service before, you might be interested to learn that there are various approaches. Some services require you to resize your images “server side” before you can download them in the browser. Other services allow you to resize directly from the client. We opted for this client-based approach. It feels more developer friendly.
We might additionally launch server-side presets as we add transformations because they provide a good DX for chaining multiple transformations (resize, rotate, and watermark for example). With server-side presets, transformations can be bundled together into a single command - ?preset=profile instead of adding ?width=200&rotate=90&watermark=image to all your image URLs.
Behind the scenes, we've provided a few small niceties:
coverImage Resizing is in Beta. While in Beta, it is only enabled for Pro Plan and Enterprise customers.
</div>Resizing images can get expensive, so before this release we knew we needed a better caching system. We already provide a “basic” CDN for all projects (including Free Plan), so the important update here is some smarter cache invalidation whenever there are changes.
Our Basic CDN works by adding a 1-hour cache header to every request. After this header expires, we re-check the origin (S3), to see if an image is still available. This parameter is configurable (for example, you can make it one day, or one year).
This approach introduces the possibility of a “stale” image. If you update an image in S3, users will continue to see the old, stale image for one hour. Similarly for deleted images - the cache will continue serving the deleted image until the cache expires.
With this release we're introducing a Smart CDN. Whenever you update or delete an image, the cache will be busted within 60 seconds - for every region in the world.
As a developer, you no longer need to worry about adjusting your cache parameters - the cache “just works”.
If you're interested in how the cache works, see the deep-dive later in this blog post.
As a “pre-release”, we've added webhooks and rate-limiting to the Storage API. We haven't exposed these parameters on our platform yet, but they are coming soon.
If you're self-hosting then you can use them today. It's as simple as enabling a few environment variables.
The webhook events we added are:
ObjectCreated:Post # A new object is created
ObjectCreated:Put # A new object is updated
ObjectCreated:Copy # An object is duplicated
ObjectCreated:Move # An object is created as part of a move operation
ObjectRemoved:Delete # An object is deleted
ObjectRemoved:Move # An object is deleted as part of a move operation
ObjectUpdated:Metadata # An object's metadata is updated
Let's put everything together to see how the new Smart CDN works with Imgproxy.
Some developers have a high “resize” workload, and some have a high “read” workload. This means that there are different scaling requirements for every developer. As a result, we kept the image resizer and the Storage API as two separate containers. This allows them to scale independently depending on your workload.
Let's step through a few different flows to see how it all works:
Let's imagine one of your users makes a request for a resized cat (kitten?) image in your application. Here's the flow:
The cat image goes viral, and so now multiple users want to see the resized image.
To complete this architecture, we need to keep the global KV store (which stores the image metadata) synchronized whenever an image is changed. To avoid coupling any CDN-specific logic into the Storage server, we added Webhooks that publish events for every object created, updated and deleted.
Using our cat picture from above:
ObjectCreated:Put event from the Supabase Storage server.Most CDNs offer this functionality “natively”. However there is usually a hard-limit to the number of daily cache purges. This implementation gives us greater control over when the cache is cleared.
Besides solving issues with stale data, one of the biggest benefits this architecture offers is higher cache hits. Why is that important? Because image resizing is slow. Every cache miss is increased latency for your users.
The green line here shows the number of cache hits to our CDN. As you can see, when we enabled the Smart CDN, the number of requests cache hits increased significantly (which means faster cat images).
Image Resizing is in Beta. While in Beta, it is only enabled for Pro Plan and Enterprise customers.
When choosing a pricing model there were a few options that competitors offered:
We tried to find a middle ground, where you pay for the number of images you want to transform rather than the number of transformations:
Of course, all of this functionality is free in the self-hosting setup too. You can self-host the Storage API either as a standalone service or as part of the Supabase stack.
For local development, this is available in our CLI from v1.23.0.
Upgrade to supabase-js v2.2.0.
A few features to look out for in the future:
Upgrade to Pro to get started with Image Resizing today.