Optimised data retrieval can provide an improvement in performance and cost by several orders of magnitude.
Following discusses the various strategies that can be used to bring your data closer to your compute, compare and decide the solution that works the best, and high level implementations on how to integrate it into you current system.
As your request volumes start to go up, optimization starts to become vital for speed and costs. This is when you start to realise that simple REST i/o might not cut it. When it comes to information retrieval, you ideally want to process requests with minimal use of compute resources and minimal use of databse reads. Depending on the frequency of the request or the amount of compute required to realise the response, there are many interesting strategies that can be utilised to optimise reads
1. Client Side Caching
The most precise of all, client side storage ensures data is readily available on the client itself. As this doesnt rely on any server-calls, this is the fastest method that can be implemented if the trade-offs are acceptable. The trade-offs being data exposure to users and data rigidity.
There are multiple ways of caching on the client side, whether through session storage, cookies or as proper dbs like sqlite, shared preferences, hive etc.
As a ground rule, it should be understood that client cached data will be visible and manipulatable by the client. There are various encryptions and obfuscations that can be employed but sensitive data is yet not recomended to be stored on the client.
Special logic also usually needs to be written if client stored data needs to be invalidated without invoking the primary APIs that fetch the data. Or wait for the TTL of the data to expire. This makes the stored data rigid.
UI configurations, user preferences and non-sensitive data that only changes on static frequencies are good candidates for this method.
✅ Fastest retrieval
✅ No network calls
❌ Rigid data
❌ Not recommended for sensitive data
2. CDN Caching
CDNs provide readily available endpoints that are extremely fast in serving data. CDNs can prove to be a great solution for data that is user-agnostic and with a low-frequency update rate.
CDNs can be attached in front of an API Gateway to cache API responses. CDNs can also be attached in front of static files similar to how static webpages get served. This method has proven to be a really effective way of serving extremely fast data. Simply create a public S3 bucket with a few txt/json files and link these files to a Cloudfront Distribution.
This method allows you to skip requests reaching your server altogether. It also gives you better flexibility to change your cached data on your own timelines. If used effectively, CDNs tend to contribute to significant cost savings due to it usually being extremely generous in its free tier.
There are a few caveats to mention. For one, (1) CDNs arent favorable for data suited for querying. (2) Also, as there is no compute involved, building authentication for these requests are dificult. This would require storing encrypted data and decoding it on the client using separately passed down keys. (3) Updating the data would require a small workflow which updates your files and invalidates the caches which may take some time to reach the end user.
Global configs, frequently used assets, application layer data are good candidates for this method
✅ Fast, edge optimised
✅ Cost efficient
❌ Not suitable for relational data or sensitive data
3. API Gateway Caching
This is another great way of improving the performance of your system. API Gateways sit between the client and the collection of backend services of your system. Each Gateway usually has the capacity to accept millions of requests. And there is also a built-in functionality provided by most API Gateway providers to build persistence in the response delivered via the API Gateway.
If you’re already using an API Gateway provided by Apigee/AWS/Azure (GCP doesn’t currently provide caching), you can enable caching on the entire gateway or even individual endpoints, and specify custom ttls for each, which will flush and refresh the cache based on it.
This approach allows the data to be served to all incoming requests by only invoking your business logic once. An added versatality to this method of caching is how it constructs its cache keys. API Gateway caching may use the entire request url including the query params for caching, making the cache a lot more specific and a lot more versatile.
A few things to keep in mind here is that (1)an API Gateway will need to be configured with an authoriser to authenticate your requests. This could be a static key or a serverless function for other forms of auth. (2) It is difficult to flush the cache of an API Gateway before its TTL unless you’re doing it manually. (3)Based on experience, API Gateway caching has proved to be useful but expensive so should be used sparingly.
User profile requests, lazy leaderboards, offers and promotions and such can be good candidates for this method.
✅ Versatile data storage
❌ Costly
4. In-memory cache
Redis and memcache are probably the most familiar caching instruments for many and rightly so. Redis being much more mainstream. It essentially acts like a key-value HashMap directly working in the RAM, making data reads and writes really fast.
Implementing redis requires setting it up as a client in your server. If you’re unsure about the data scaling and performance, you can go with a managed Redis solution like Elasticache or Redis Labs, which will handle the monitoring, backups and scaling on its own. Once integrated, i/o with redis will be akin to a key-value store.
Any data that is actively read and updated at high frequencies is a good candidate for redis-level caching. This could be user session data, metadata, leaderboard lists, distributed locks, feature flags and so on.
While a silver bullet for many data types, in-memory db still isn't suited well for (1)complex-relational data with foreign key constraints or (2) very big dumps of data or (3)data with very low access frequency.
There is much to be explored in Redis that are outside the scope of this post. Several teams go a lot further and treat Redis as primary DBs as well employing RDBs to build a persistence. Redis itself has Redis sorted sets, transactions, pub/subs and so on that adds more capabillities.
✅ Fast, simple, versatile. Suitable for many applications
❌ Not suitable for relational data / large data / full text search
5. Database caching
Certain databases will go a bit further and provide their own inherent caching mechanisms. For example, DynamoDB provides DAX (DynamoDB Accelarator), an in-memory cache, as an added functionality that will improve read times by many folds by caching.
Lookup speeds can also be improved by indexing. Indexing complex and frequent queries can improve performance by several folds both in relational and noSQL databases.
If you’re using a managed service such as AWS RDS and are dealing with significantly high reads to the database directly, read replicas can be setup that reduce the load on the primary database and provide higher availability and better performance. This however wouldn't be categorised as caching and is outside the scope of this post.
✅ Increases database read speed by several folds
❌ Usually not a packaged feature in most open source databases
As you keep scaling, adding more jumps before reaching the database can increase the performance of serving data by a several orders of magnitude. And as pointed above, good caching strategies dont necessairly come at the cost of higher spending, but in fact can help reduce your costs in most cases. Employing these strategies should be seen as recipe for better performance, lower costs, lower use of compute and improved security.
