Caching promises in JavaScript August 25, 2023 source/commit

#Reasoning

Let’s assume we have an async function that is somewhat expensive and takes a considerable amount of time to execute. We probably want to cache its result. If we used Redis for example, it might look something like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

async function getContent(key) {
    return await redis.get(key).then(async (v) => {
        if (v) {
            return v;
        } else {
            const content = expensiveFn(key);
            await redis.set(key, content);
            return content;
        }
    }
}

We try to get our content from DB, check if the result is truthy and then either return it or generate new content which we subsequently store in the database.

This is all good, but what happens if the function gets called 10 times within the timeframe of its execution? E.g. the expensiveFn takes 5 seconds to execute, and before it completes the first time we call it a few more times.

All of them would check for "mykey" and get a response. Since this is all before the first call was done executing, all the responses will be empty and each of our ten calls will generate new content and store it in the database. Redis is pretty fast, but we still don’t want to do 20 calls where 2 would suffice, or even worse, 10 potentially expensive function calls.

Redis is pretty fast, and it can surely handle setting the same key several times, but I feel like that’s what you call a race condition.

You might say that this is an edge case and if the function is inexpensive then who cares if it runs a few more times. On the other hand if it is expensive enough to warrant caching then this scenario might only apply for the first few calls before it gets cached.

And I agree with that, but what even is coding without premature optimization and overengineered solutions?

#Caching a promise

Asynchronous functions in JS return a value of type Promise. Think of it as a pointer to something that might contain a value in the future. We can use await to halt execution until the value is there, but we can also just store the promise inside a variable and do something with it.

1
2
3
4
5
6
7
8
9

let promise;

async function cachedGetContent() {
    if (!promise) {
        promise = getContent().finally(() => promise = null);
    }

    return await promise;
}

The function checks if promise is empty and assigns the result of getContent to it. Now if we call the function 10 times it will only run once but return all 10 times. After the promise gets fulfilled we set the variable to null.

This is not extremely useful as functions usually take parameters, in which case the promises can be stored in a Map with an argument as a key.

#Generic implementation

Instead of writing this manually every time and having a variable for storing a promise above every function I like to define a class, that does it for me and looks like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

class PromiseStore<T, U> {
    private store = new Map<T, Promise<U>>();

    async resolve(key: T, fn: () => Promise<U>) {
        if (!this.store.has(key)) {
            this.store.set(
                key,
                fn().finally(() => this.store.delete(key)),
            );
        }

        return await this.store.get(key)!;
    }
}

* using TypeScript for type annotations, but it works the same in JS

It stores state in a variable fittingly named store and has a resolve method, which takes a key and a function as its arguments. If the key exists the function doesn’t get called and the existing promise is returned instead.

Given a function that takes a single argument we can:

1
2
3
4
5

const store = new PromiseStore();

async function cachedAsyncFn(param: string) => {
    return await store.resolve(param, async () => await asyncFn(param));
}

Most of the time (but not always) the key I wanted to use would be the same as the function’s parameter, so I added some more abstraction for this.

1
2
3
4
5
6
7

export function promiseStoreFn(fn) {
    const store = new PromiseStore();

    return async (param) => {
        return await store.resolve(param, async () => await fn(param));
    };
}

This function initializes a PromiseStore and then generates a function that uses said store for caching.

Afterward the cachedAsyncFn could become just one line. Obviously at that point we could just change the implementation of asyncFn and wrap it with promiseStoreFn, but that’s beside the point.

1

const cachedAsyncFn = promiseStoreFn(asyncFn);

#Putting it together

Let’s go back to the very first code block and redo it:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

const getContent = promiseStoreFn(async (key) => {
    return await redis.get(key).then(async (v) => {
        if (v) {
            return v;
        } else {
            const content = expensiveFn(key);
            await redis.set(key, content);
            return content;
        }
    }
});

It has barely changed, but now if we were to call the function 1000 times at the same time with the same key, it would only run once instead of a thousand times.

A simple snippet to verify that this approach works:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

const testFn = promiseStoreFn(async (param) => {
    // just sleep for 3 seconds
    await new Promise((r) => setTimeout(r, 3000));

    console.log("I will run once");
    return param;
});

const promises = [];

for (let i = 0; i <= 1000; i++) {
    promises.push(testFn("I will return thousand times"));
}

console.log(await Promise.all(promises));

Running it gives us this console output:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

I will run once
[
  "I will return thousand times",
  "I will return thousand times",
  "I will return thousand times",
  "I will return thousand times",
  "I will return thousand times",
  "I will return thousand times",
  "I will return thousand times",
  "I will return thousand times",
  "I will return thousand times",
  "I will return thousand times",
  "I will return thousand times",
  "I will return thousand times",
  "I will return thousand times",
  "I will return thousand times",
  "I will return thousand times",
  "I will return thousand times",
  "I will return thousand times",
  "I will return thousand times",
  "I will return thousand times",
  "I will return thousand times",

  ... the same thing for many more lines
]