You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
nice package! I have gotten a question if this package supports my use case.
My setup:
3 separate webservers running nodeJS.
Currently,
each webserver holds 80mb worth of data, that is refreshed every 10 minutes using setInterval() with a call to a database. One database call can be quite expensive and take long time to compute (roughly a minute), so essentially around 10 minutes I'll have 3 (because of 3 webservers) very expensive / large db calls go off. (For the sake of this question I have simplified it, in reality I've got multiple webservers with 20+ clusters, so there will be a lot of db calls).
I am now investigating how to basically make 1 database call in every 10 minutes and have everyone fetch the cached db data from redis instead.
The part I don't seem to understand is how do I ensure if the redis cache expires, that only 1 webserver is actually going to do an expensive db call & update the value while all the other webservers still use the old data?
Ideally I would like to use something like this (pseudocode):
const getCachedItem = (keyId, fetchFn) => {
if(localCache[keyId] && !localCache[keyId].stale) return localCache[keyId];
while(true) {
// lets see if anything exists in distributed cache
if(redisCache[keyId]){
localCache[keyId]=redisCache[keyId]
return localCache[keyId];
}
// looks like nothing in distributed cache,
// lets try to acquire the lock and compute the value!
await tryToAcquireDistributedLockIfItsAvailable(`lock${keyId}`, async () => {
// We have acquired distributed lock
// verify if the cache is still empty
// as there could have been other servers that actually already did the expensive work
if (redisCache[keyId]){
localCache[keyId]=redisCache[keyId]
return localCache[keyId];
}
const expensive = await fetchFn();
redisCache[keyId]=expensive; // items in redis cache expire by themselves after XXX by having TTL.
localCache[keyId]=expensive;
return localCache[keyId];
});
// Failed to acquire the distributed lock,
// which can indicate that someone else has the lock and is doing the expensive computation
// If we have local stale data available, we'll return that
if (localCache[keyId]) return localCache[keyId];
// If we have no data at all available
// We are going to wait a little, and try again to acquire the lock
await sleep(200);
}
}
The text was updated successfully, but these errors were encountered:
nice package! I have gotten a question if this package supports my use case.
My setup:
3 separate webservers running nodeJS.
Currently,
each webserver holds 80mb worth of data, that is refreshed every 10 minutes using setInterval() with a call to a database. One database call can be quite expensive and take long time to compute (roughly a minute), so essentially around 10 minutes I'll have 3 (because of 3 webservers) very expensive / large db calls go off. (For the sake of this question I have simplified it, in reality I've got multiple webservers with 20+ clusters, so there will be a lot of db calls).
I am now investigating how to basically make 1 database call in every 10 minutes and have everyone fetch the cached db data from redis instead.
The part I don't seem to understand is how do I ensure if the redis cache expires, that only 1 webserver is actually going to do an expensive db call & update the value while all the other webservers still use the old data?
Ideally I would like to use something like this (pseudocode):
The text was updated successfully, but these errors were encountered: