HerdCache with set expire_time of Redis key (non-infinity TTL)

[Pajinek]

Checklist

• Python version 3.8.7

Steps to reproduce

I created cache object of HerdCache with positive value of expire_time.

My scenario:

• I want to keep behaviour of TTL Redis key (remove after expiration), but still resolve thundering herd problem.

But this part of code doesn't clear for me.

expected_expired_ts = int(time.time())
if expire_time:
   expected_expired_ts += expire_time
expected_expired_ts += herd_timeout or self.default_herd_timeout
value = self._pack([value, expected_expired_ts])
return await self.client.set(identity, value, ex=expire_time)

Expected behavior

The value of expected_expired_ts should be less then expire_time, because data will be expired sooner - Redis returns value None and check of expected_expired_ts can't be used.

Actual behavior

• Currently it works with infinity TTL for Redis key or negative herd_timeout or self.default_herd_timeout and positive expire_time.

For example herd_timeout is set to -10 (s), default_herd_timeout is set to 10 (s) and expire_time is set to 110 (s)

1. during 0-89s. nothing happens
2. in 90s the expected_expired_ts expires, the method get returns None (but data is still store in Redis). Data is extended about 10s. And for this one process is opened window 10s for operation to store new data to the Redis.