Skip to content

Commit

Permalink
Elevated rate limits incremental quota (#63)
Browse files Browse the repository at this point in the history 
* feat:join all erl configuration and allow quota modification

* feat: added takeElevated to client

* fix: return correct bucket size [PSERV-2452] (#64)

* fix: init buckets for non-elevated take

---------

Co-authored-by: Ece Tavasli <[email protected]>

* adding erl_configured_for_bucket as a param to the lua script

* fix:erl_active_for_bucket only if it's also configured

* bumping version

---------

Co-authored-by: Lewis Metcalf <[email protected]>
Co-authored-by: Ece Tavasli <[email protected]>
Co-authored-by: Ece Tavasli <[email protected]>
  • Loading branch information
@pubalokta @LeweyM @ecita
4 people authored May 7, 2024
1 parent 946dba5 commit 99a7685
Show file tree
Hide file tree
Showing 9 changed files with 728 additions and 295 deletions.
47 changes: 32 additions & 15 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,6 @@ To be able to allow its use within limitd-redis, you need to:
2. pass the `elevated_limits` parameter with the following properties:
- `erl_is_active_key`: the identifier of the ERL activation for the bucket. This works similarly to the `key` you pass to `limitd.take`, which is the identifier of the bucket; however it's used to track the ERL activation for the bucket instead
- `erl_quota_key`: the identifier of the ERL quota bucket name.
- `per_calendar_month`: the amount of tokens that the quota bucket will receive on every calendar month.
3. make sure that the bucket definition has ERL configured.

### Configuration
Expand All @@ -137,10 +136,11 @@ buckets = {
ip: {
size: 10,
per_second: 5,
elevated_limits: {
size: 100, // new bucket size. already used tokens will be deducted from current bucket content upon ERL activation.
per_second: 50, // new bucket refill rate. You can use all the other refill rate configurations defined above, such as per_minute, per_hour, per_interval etc.
erl_activation_period_seconds: 300, // for how long the ERL configuration should remain active once activated.
elevated_limits: { // Optional. ERL configuration if needed for the bucket. If not defined, the bucket will not use ERL.
size: 100, // Optional. New bucket size. already used tokens will be deducted from current bucket content upon ERL activation. Default: same as the original bucket.
per_second: 50, // Optional. New bucket refill rate. You can use all the other refill rate configurations defined above, such as per_minute, per_hour, per_interval etc. Default: same as the original bucket.
erl_activation_period_seconds: 300, // Mandatory. For how long the ERL configuration should remain active once activated. No default value.
quota_per_calendar_month: 100, // Mandatory. The amount of ERL activations that can be done in a calendar month. Each activation will remain active during erl_activation_period_seconds.
}
}
}
Expand All @@ -151,8 +151,7 @@ ERL quota represents the number of ERL activations that can be performed in a ca

When ERL is triggered, it will keep activated for the `erl_activation_period_seconds` defined in the bucket configuration.

The amount of minutes per month allowed in ERL mode is defined by: `per_calendar_month * erl_activation_period_seconds / 60`.

The amount of minutes per month allowed in ERL mode is defined by: `quota_per_calendar_month * erl_activation_period_seconds / 60`.

The overrides in ERL work the same way as for the regular bucket. Both size and per_interval are mandatory when specifying an override.

Expand Down Expand Up @@ -200,16 +199,15 @@ limitd.takeElevated(type, key, { count, configOverride, elevated_limits }, (err,
- `elevated_limits`: (object)
- `erl_is_active_key`: (string) the identifier of the ERL activation for the bucket.
- `erl_quota_key`: (string) the identifier of the ERL quota bucket name.
- `per_calendar_month`: (number) the amount of tokens that the quota bucket will receive on every calendar month.

`erlQuota.per_calendar_month` is the only refill rate available for ERL quota buckets at the moment.
`quota_per_calendar_month` is the only refill rate available for ERL quota buckets at the moment.
The quota bucket will be used to track the amount of ERL activations that can be done in a calendar month.
If the quota bucket is empty, the ERL activation will not be possible.
The quota bucket will be refilled at the beginning of every calendar month.

For instance, if you want to allow a user to activate ERL for a bucket only 5 times in a month, you can define a quota bucket with `per_calendar_month: 5`.
For instance, if you want to allow a user to activate ERL for a bucket only 5 times in a month, you can define a quota bucket with `quota_per_calendar_month: 5`.
That means that the user can activate ERL for the bucket 5 times in a month, and after that, the ERL activation will not be possible until the start of the next month.
The total minutes allowed for ERL activation in a calendar month is calculated as follows: `per_calendar_month * erl_activation_period_seconds / 60`.
The total minutes allowed for ERL activation in a calendar month is calculated as follows: `quota_per_calendar_month * erl_activation_period_seconds / 60`.

The result object has:
- `conformant` (boolean): true if the requested amount is conformant to the limit.
Expand All @@ -219,13 +217,14 @@ The result object has:
- `elevated_limits` (object)
- `triggered` (boolean): true if ERL was triggered in the current request.
- `activated` (boolean): true if ERL is activated. Not necessarily triggered in this call.
- `quota_count` (int): **[Only valid if triggered=true]** If `triggered=true`, this value contains the current quota count left for the given `erl_quota_key`. Otherwise, it will return -1, which is not valid to be interpreted as a quota count.
- `quota_remaining` (int): **[Only valid if triggered=true]** If `triggered=true`, this value contains the remaining quota count for the given `erl_quota_key`. Otherwise, it will return -1, which is not valid to be interpreted as a quota count.
- `quota_allocated`: (int): amount of quota allocated in the bucket configuration. This value is defined in the bucket configuration and is the same as `quota_per_calendar_month`.
- `erl_activation_period_seconds`: (int): the ERL activation period as defined in the bucket configuration used in the current request.

Example of interpretation:
``` javascript
if erl_activated && erl_quota_count >= 0; // quota left in the quotaKey bucket
if erl_activated && erl_quota_count = -1; // ERL is activated, but it wasn't triggered in this call, so we haven't identified the quota for this call.
if !erl_activated; // ERL is not activated, hence the quota hasn't been identified for this call.
if erl_triggered // quota left in the quotaKey bucket
if !erl_triggered // ERL wasn't triggered in this call, so we haven't identified the remaining quota.
```

## PUT
Expand Down Expand Up @@ -267,6 +266,24 @@ limitd.take(type, key, { count: 3, configOverride }, (err, result) => {

Config overrides follow the same rules as Bucket configuration elements with respect to default size when not provided and ttl.

### Overriding Configuration at Runtime with ERL
We can also override the configuration for ERL buckets at runtime. The shape of this `configOverride` parameter is the same as `Buckets` above.

An example configuration override call for ERL might look like this:

```js
const configOverride = {
size: 45,
per_hour: 15,
elevated_limits: {
size: 100,
per_hour: 50,
erl_activation_period_seconds: 300,
quota_per_calendar_month: 100
}
}
```

## Author

[Auth0](auth0.com)
Expand Down
59 changes: 33 additions & 26 deletions lib/db.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,9 @@ const async = require('async');
const LRU = require('lru-cache');
const utils = require('./utils');
const Redis = require('ioredis');
const { validateParams, validateConfigIsForElevatedBucket, validateERLParams } = require('./validation');
const { validateParams, validateERLParams } = require('./validation');
const DBPing = require("./db_ping");
const { calculateQuotaExpiration } = require('./utils');
const EventEmitter = require('events').EventEmitter;

const TAKE_LUA = fs.readFileSync(`${__dirname}/take.lua`, "utf8");
Expand Down Expand Up @@ -289,36 +290,37 @@ class LimitDBRedis extends EventEmitter {
}

takeElevated(params, callback) {
const valERLParamsError = validateERLParams(params.elevated_limits);
if (valERLParamsError) {
return callback(valERLParamsError)
const valError = validateERLParams(params.elevated_limits);
if (valError) {
return callback(valError)
}
const erlParams = utils.getERLKeysQuotaAmountAndExpiration(params.elevated_limits);
const erlParams = utils.getERLParams(params.elevated_limits);

this._doTake(params, callback, (key, bucketKeyConfig, count) => {
const valERLConfigError = validateConfigIsForElevatedBucket(key, bucketKeyConfig)
const isERLEnabledForBucket = !valERLConfigError;
if (valERLConfigError) {

bucketKeyConfig.elevated_limits = {
ms_per_interval: bucketKeyConfig.ms_per_interval,
size: bucketKeyConfig.size,
erl_activation_period_seconds: 0
}
}

// provide default values for elevated_limits unless the bucketKeyConfig has them
const elevated_limits = {
ms_per_interval: bucketKeyConfig.ms_per_interval,
size: bucketKeyConfig.size,
erl_activation_period_seconds: 900,
erl_quota_amount: 0,
erl_quota_interval: 'quota_per_calendar_month',
erl_configured_for_bucket: false,
...bucketKeyConfig.elevated_limits
};

const erl_quota_expiration = calculateQuotaExpiration(elevated_limits);
this.redis.takeElevated(key, erlParams.erl_is_active_key, erlParams.erl_quota_key,
bucketKeyConfig.ms_per_interval || 0,
bucketKeyConfig.size,
count,
Math.ceil(bucketKeyConfig.ttl || this.globalTTL),
bucketKeyConfig.drip_interval || 0,
bucketKeyConfig.elevated_limits.ms_per_interval || 0,
bucketKeyConfig.elevated_limits.size,
bucketKeyConfig.elevated_limits.erl_activation_period_seconds,
erlParams.erl_quota_amount,
erlParams.erl_quota_expiration,
isERLEnabledForBucket,
elevated_limits.ms_per_interval,
elevated_limits.size,
elevated_limits.erl_activation_period_seconds,
elevated_limits.erl_quota_amount,
erl_quota_expiration,
elevated_limits.erl_configured_for_bucket ? 1 : 0,
(err, results) => {
if (err) {
return callback(err);
Expand All @@ -328,18 +330,23 @@ class LimitDBRedis extends EventEmitter {
const currentMS = parseInt(results[2], 10);
const reset = parseInt(results[3], 10);
const erl_triggered = parseInt(results[4], 10) ? true : false;
const erl_activated = parseInt(results[5], 10) ? true : false;
let erl_activate_for_bucket = parseInt(results[5], 10) ? true : false;
// if the bucket is not configured for elevated limits, then it shouldn't be activated
erl_activate_for_bucket = erl_activate_for_bucket && elevated_limits.erl_configured_for_bucket;
const erl_quota_count = parseInt(results[6], 10);
const res = {
conformant,
remaining,
reset: Math.ceil(reset / 1000),
limit: bucketKeyConfig.size,
limit: erl_activate_for_bucket ? elevated_limits.size : bucketKeyConfig.size,
delayed: false,
elevated_limits : {
erl_configured_for_bucket: elevated_limits.erl_configured_for_bucket,
triggered: erl_triggered,
activated: erl_activated,
quota_count: erl_quota_count
activated: erl_activate_for_bucket,
quota_remaining: erl_quota_count,
quota_allocated: elevated_limits.erl_quota_amount,
erl_activation_period_seconds: elevated_limits.erl_activation_period_seconds,
},
};
if (bucketKeyConfig.skip_n_calls > 0) {
Expand Down
40 changes: 25 additions & 15 deletions lib/take_elevated.lua
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,14 @@ local erl_bucket_size = tonumber(ARGV[7])
local erl_activation_period_seconds = tonumber(ARGV[8])
local erl_quota_amount = tonumber(ARGV[9])
local erl_quota_expiration_epoch = tonumber(ARGV[10])
local is_erl_enabled = ARGV[11] == "true" and true or false
local erl_configured_for_bucket = tonumber(ARGV[11]) == 1

-- the key to use for pulling last bucket state from redis
local lastBucketStateKey = KEYS[1]

-- the key for checking in redis if elevated rate limits (erl) were activated earlier
local erlKey = KEYS[2]
local is_erl_activated = redis.call('EXISTS', erlKey)
local is_erl_activated = erl_configured_for_bucket and redis.call('EXISTS', erlKey) or 0

-- the key for erl quota counting
local erl_quota_key = KEYS[3]
Expand Down Expand Up @@ -48,16 +48,28 @@ local function calculateNewBucketContent(current, tokens_per_ms, bucket_size, cu
end

local function takeERLQuota(erl_quota_key, erl_quota_amount, erl_quota_expiration_epoch)
local erl_quota = erl_quota_amount
if erl_quota_amount <= 0 then
-- no quota available to take
return 0
end

local get_quota_result = redis.call('GET', erl_quota_key)
if type(get_quota_result) == 'string' then
erl_quota = tonumber(get_quota_result)
if type(get_quota_result) ~= 'string' then
-- first activation. Set quota to 1 and return.
redis.call('SET', erl_quota_key, 1, 'PXAT', string.format('%.0f', erl_quota_expiration_epoch))
return 1
end

if erl_quota > 0 then
redis.call('SET', erl_quota_key, erl_quota -1, 'PXAT', string.format('%.0f', erl_quota_expiration_epoch))
local erl_quota_used = tonumber(get_quota_result)
if erl_quota_used >= erl_quota_amount then
-- quota is exceeded. Return the current quota.
return erl_quota_used
end
return erl_quota

-- quota is not exceeded. Increment and return.
local new_quota = erl_quota_used + 1
redis.call('SET', erl_quota_key, new_quota, 'PXAT', string.format('%.0f', erl_quota_expiration_epoch))
return new_quota
end

-- Enable verbatim replication to ensure redis sends script's source code to all masters
Expand All @@ -67,7 +79,7 @@ redis.replicate_commands()

-- calculate new bucket content
local bucket_content_after_refill
if is_erl_activated == 1 then
if erl_configured_for_bucket and is_erl_activated == 1 then
bucket_content_after_refill = calculateNewBucketContent(current, erl_tokens_per_ms, erl_bucket_size, current_timestamp_ms)
else
bucket_content_after_refill = calculateNewBucketContent(current, tokens_per_ms, bucket_size, current_timestamp_ms)
Expand All @@ -86,23 +98,21 @@ if enough_tokens then
end
else
-- if tokens are not enough, see if activating erl will help.
if is_erl_activated == 0 and is_erl_enabled then
if erl_configured_for_bucket and is_erl_activated == 0 and erl_quota_amount > 0 then
local used_tokens = bucket_size - bucket_content_after_refill
local bucket_content_after_erl_activation = erl_bucket_size - used_tokens
local enough_tokens_after_erl_activation = bucket_content_after_erl_activation >= tokens_to_take
if enough_tokens_after_erl_activation then
local erl_quota = takeERLQuota(erl_quota_key, erl_quota_amount, erl_quota_expiration_epoch)
-- erl_quota contains the quota before taking one to avoid confusing erl_quota=0 with the last element of the quota
if erl_quota > 0 then
local erl_quota_used = takeERLQuota(erl_quota_key, erl_quota_amount, erl_quota_expiration_epoch)
if erl_quota_used < erl_quota_amount then
enough_tokens = enough_tokens_after_erl_activation -- we are returning this value, thus setting it
bucket_content_after_take = math.min(bucket_content_after_erl_activation - tokens_to_take, erl_bucket_size)
-- save erl state
redis.call('SET', erlKey, '1')
redis.call('EXPIRE', erlKey, erl_activation_period_seconds)
is_erl_activated = 1
-- now we remove one from the quota to return what's left in the bucket
erl_quota_left = erl_quota - 1
erl_triggered = true
erl_quota_left = erl_quota_amount - erl_quota_used
end
end
end
Expand Down
69 changes: 41 additions & 28 deletions lib/utils.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,10 @@ const INTERVAL_TO_MS = {

const INTERVAL_SHORTCUTS = Object.keys(INTERVAL_TO_MS);

const ERL_QUOTA_INTERVAL_PER_CALENDAR_MONTH = 'per_calendar_month';
const ERL_QUOTA_INTERVALS = {};
ERL_QUOTA_INTERVALS[ERL_QUOTA_INTERVAL_PER_CALENDAR_MONTH] = () => endOfMonthTimestamp();

const ERL_DEFAULT_ACTIVATION_PERIOD_SECONDS = 15 * 60;
const ERL_QUOTA_INTERVAL_PER_CALENDAR_MONTH = 'quota_per_calendar_month';
const ERL_QUOTA_INTERVALS = {
[ERL_QUOTA_INTERVAL_PER_CALENDAR_MONTH]: () => endOfMonthTimestamp()
};
const ERL_QUOTA_INTERVALS_SHORTCUTS = Object.keys(ERL_QUOTA_INTERVALS);

function normalizeTemporals(params) {
Expand Down Expand Up @@ -53,18 +52,28 @@ function normalizeTemporals(params) {
}

function normalizeElevatedTemporals(params) {
if (!params) {
return;
if (!params.erl_activation_period_seconds) {
throw new LimitdRedisConfigurationError('erl_activation_period_seconds is required for elevated limits', {code: 201});
}

let type = normalizeTemporals(params);
type.erl_activation_period_seconds = params.erl_activation_period_seconds;

if (!params.erl_activation_period_seconds) {
type.erl_activation_period_seconds = ERL_DEFAULT_ACTIVATION_PERIOD_SECONDS;
} else {
type.erl_activation_period_seconds = params.erl_activation_period_seconds;
// extract erl quota information
ERL_QUOTA_INTERVALS_SHORTCUTS.forEach(intervalShortcut => {
if (!(intervalShortcut in params)) {
return;
}
type.erl_quota_amount = params[intervalShortcut];
type.erl_quota_interval = intervalShortcut;
});

if (!type.erl_quota_interval || type.erl_quota_amount === undefined) {
throw new LimitdRedisConfigurationError('a valid quota amount per interval is required for elevated limits', {code: 202});
}

type.erl_configured_for_bucket = true;

return type;
}

Expand Down Expand Up @@ -139,30 +148,34 @@ function randomBetween(min, max) {
return Math.random() * (max - min) + min;
}

function getERLKeysQuotaAmountAndExpiration(params) {
const type = _.pick(params, [
function getERLParams(params) {
return _.pick(params, [
'erl_is_active_key',
'erl_quota_key',
'erl_quota_amount',
'erl_quota_expiration'
'erl_quota_key'
]);
}

ERL_QUOTA_INTERVALS_SHORTCUTS.forEach(intervalShortcut => {
if (!(intervalShortcut in params)) {
return;
}
type.erl_quota_amount = params[intervalShortcut];
type.erl_quota_expiration = ERL_QUOTA_INTERVALS[intervalShortcut]();
});

return type;
function calculateQuotaExpiration(params) {
return ERL_QUOTA_INTERVALS[params.erl_quota_interval]();
}

function endOfMonthTimestamp() {
const curDate = new Date();
return Date.UTC(curDate.getUTCFullYear(), curDate.getUTCMonth() + 1, 1, 0, 0, 0, 0);
}

class LimitdRedisConfigurationError extends Error {
constructor(msg, extra) {
super();
this.name = this.constructor.name;
this.message = msg;
Error.captureStackTrace(this, this.constructor);
if (extra) {
this.extra = extra;
}
}
}

module.exports = {
buildBuckets,
buildBucket,
Expand All @@ -172,7 +185,7 @@ module.exports = {
normalizeType,
functionOrFalse,
randomBetween,
ERL_DEFAULT_ACTIVATION_PERIOD_SECONDS,
getERLKeysQuotaAmountAndExpiration,
endOfMonthTimestamp
getERLParams,
endOfMonthTimestamp,
calculateQuotaExpiration
};
Loading

0 comments on commit 99a7685

Please sign in to comment.