Add rate-limiting and throttling support for log messages

[tarun-t]

Is your feature request related to a problem? Please describe.

When debugging or monitoring applications, there are scenarios where certain events or log messages occur very frequently (e.g., in tight loops or high-frequency operations), leading to log file bloat and reduced visibility of other important messages. Currently, Quill lacks a built-in mechanism to rate-limit or throttle specific log messages, either based on occurrence count or time intervals. This forces developers to implement custom solutions outside the logging framework.

Describe the solution you'd like

Add a rate-limiting feature to Quill that allows developers to specify logging frequency constraints in two ways:

1. Time-based throttling:

LOG_THROTTLE(logger, 5min, "High-frequency event occurred {} times", count);

2. Count-based throttling:

LOG_EVERY_N(logger, 1000, "Processing item {}", item_id);

Additional context

Similar features exist in other logging frameworks:

• Google's glog has LOG_EVERY_N and VLOG_EVERY_N
• Boost.Log supports filtering based on custom attributes
• spdlog has a basic rate-limiting feature

[odygrd]

Hey, we have the LOG_LIMIT macros have you tried those? For example

quill/examples/console_logging.cpp

Line 58 in 62450f9

LOG_INFO_LIMIT(std::chrono::seconds{1}, logger, "A {} message with number {}", l, a);

list of all available macros

https://quillcpp.readthedocs.io/en/latest/logging_macros.html

let me know if that helps

[tarun-t]

Thank you for recommending these macros - they've been helpful for implementing time-based rate limiting. Would it be possible to also add a counter-based macro? Additionally, for the time-based limiting feature, it would be useful to have functionality to retrieve the number of occurrences within a specified time window.

[odygrd]

The count-based approach should be straightforward to add if it’s as simple as:

LOG_EVERY_N(logger, 1000, "Processing item {}", item_id);

However, I’m not certain how easy it would be to retrieve the count within the timing macros while keeping the API simple. If you have any ideas, please feel free to share

[odygrd]

• Added LOG_LEVEL_LIMIT_EVERY_N count based macros

• The count of log messages for the time based LOG_LEVEL_LIMIT macros is appended to the end of the log message for example

  A log message with number 123 (21x)