-
Notifications
You must be signed in to change notification settings - Fork 375
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
DEBUG-3175 DI snapshot/rate limit benchmark #4207
base: master
Are you sure you want to change the base?
Conversation
👋 Hey @p-datadog, please fill "Change log entry" section in the pull request description. If changes need to be present in CHANGELOG.md you can state it this way **Change log entry**
Yes. A brief summary to be placed into the CHANGELOG.md (possible answers Yes/Yep/Yeah) Or you can opt out like that **Change log entry**
None. (possible answers No/Nope/None) Visited at: 2024-12-09 21:35:47 UTC |
2406cc3
to
2671908
Compare
2671908
to
b5045c8
Compare
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #4207 +/- ##
==========================================
- Coverage 97.77% 97.76% -0.01%
==========================================
Files 1357 1357
Lines 81973 81981 +8
Branches 4168 4170 +2
==========================================
+ Hits 80145 80150 +5
- Misses 1828 1831 +3 ☔ View full report in Codecov by Sentry. |
ca27dae
to
4f443b9
Compare
Datadog ReportBranch report: ✅ 0 Failed, 22056 Passed, 1457 Skipped, 5m 28.91s Total Time |
What does this PR do?
Adds dynamic instrumentation benchmarks verifying that the rate at which probes can capture, serialize and submit snapshots to the agent are below the rate limits.
It also makes the probe notifier worker queue capacity configurable via an internal setting. With default DI settings, DI will permit no more than 30 snapshots/second on average across all probes which is well short of the 5000/second/probe that this benchmark is aiming to achieve. The benchmark raises the limit (and makes flushes more frequent) to demonstrate theoretical performance of DI implementation, however more work is needed to permit higher limits by default - flushing logic must be carefully written to not starve the application of CPU. The default limits are very conservative in this regard.
Motivation:
We want to ensure that the rate limits we set are achievable, meaning they are not set so high that DI code can starve customer applications of CPU and still not hit rate limits.
Change log entry
None.
Additional Notes:
The benchmarks perform as many invocations of target methods as each probe type would permit (5000/second for basic probes, 1/second for enriched probes). A rate of greater than 1 instruction/second in the benchmark means the rate limit was not reached by the test code and thus DI performance is within expectations.
How to test the change?
N/A