-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(otel): adding in basic tracing via otel #794
Conversation
Plan Result (user-api-cdk-production)
|
Plan Result (user-list-search-production)
|
Plan Result (list-api-cdk-production)
|
Plan Result (v3-proxy-api-cdk-production)
|
Plan Result (parser-graphql-wrapper-cdk-production)
|
Plan Result (client-api-cdk-production)
|
}, | ||
'@opentelemetry/instrumentation-undici': { | ||
headersToSpanAttributes: { | ||
requestHeaders: [ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These are headers that we add to our trace data, we could expand this in the future. This should be enough to correlate with sentry
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
does applicationname map to consumer key?
🎉 This PR is included in version @pocket-tools/feature-flags-client-v1.0.0 🎉 The release is available on:
Your semantic-release bot 📦🚀 |
🎉 This PR is included in version @pocket-tools/sentry-v1.0.0 🎉 The release is available on:
Your semantic-release bot 📦🚀 |
Goal
Setup APM via Otel to gain insights into system performance.
Note: In dev testing this increased memory by 2%. The OTEL collector also has memory protection in place to drop spans if it gets too big.
We also have Logs in GCP.
Moving forward
Based on our system and how tracing works, I believe our future iterations should focus mainly on Sampling from within Client API and allow the downstream services to sample based on their parent trace OR if they have an error only.
Right now having the ability to sample each service based on a % is useful for debugging as we roll this out, but for day to day observability it is more helpful to only capture full system traces, which we will only get if Client API decides to sample a trace and all downstream calls. Follow up ticket here.
Local Screenshot
GCP Screenshot
Example