Let’s assume we have an async function that is somewhat expensive and takes a considerable amount of time to execute. We probably want to cache its result. If we used Redis for example, it might look something like this:
We try to get our content from DB, check if the result is truthy and then either return it or generate new content which we subsequently store in the database.
This is all good, but what happens if the function gets called 10 times within
the timeframe of its execution? E.g. the
expensiveFn takes 5 seconds to
execute, and before it completes the first time we call it a few more times.
All of them would check for
"mykey" and get a response. Since this is all
before the first call was done executing, all the responses will be empty and
each of our ten calls will generate new content and store it in the database.
Redis is pretty fast, but we still don’t want to do 20 calls where 2 would
suffice, or even worse, 10 potentially expensive function calls.
Redis is pretty fast, and it can surely handle setting the same key several times, but I feel like that’s what you call a race condition.
You might say that this is an edge case and if the function is inexpensive then who cares if it runs a few more times. On the other hand if it is expensive enough to warrant caching then this scenario might only apply for the first few calls before it gets cached.
And I agree with that, but what even is coding without premature optimization and overengineered solutions?
#Caching a Promise
Asynchronous functions in JS return a value of type Promise. Think of it as
a pointer to something that might contain a value in the future. We can use
await to halt execution until the value is there, but we can also just store
the promise inside a variable and do something with it.
The function checks if
promise is empty and assigns the result of
getContent to it. Now if we call the function 10 times it will only run once
but return all 10 times. After the promise gets fulfilled we set the variable
This is not extremely useful as functions usually take parameters, in which case the promises can be stored in a Map with an argument as a key.
Instead of writing this manually every time and having a variable for storing a promise above every function I like to define a class, that does it for me and looks like this:
* using TypeScript for type annotations, but it works the same in JS
It stores state in a variable fittingly named
store and has a
method, which takes a key and a function as its arguments. If the key exists
the function doesn’t get called and the existing promise is returned instead.
Given a function that takes a single argument we can:
Most of the time (but not always) the key I wanted to use would be the same as the function’s parameter, so I added some more abstraction for this.
This function initializes a
PromiseStore and then generates a function that
uses said store for caching.
cachedAsyncFn could become just one line. Obviously at that
point we could just change the implementation of
asyncFn and wrap it with
promiseStoreFn, but that’s beside the point.
#Putting it Together
Let’s go back to the very first code block and redo it:
It has barely changed, but now if we were to call the function 1000 times at
the same time with the same
key, it would only run once instead of a thousand
A simple snippet to verify that this approach works:
Running it gives us this console output: