Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GRPC integration logging error even when tracing is disabled #3724

Open
sco11morgan opened this issue Jun 18, 2024 · 3 comments
Open

GRPC integration logging error even when tracing is disabled #3724

sco11morgan opened this issue Jun 18, 2024 · 3 comments
Assignees
Labels
bug Involves a bug community Was opened by a community member integrations Involves tracing integrations

Comments

@sco11morgan
Copy link
Contributor

Current behaviour

GRPC integration logging error even when tracing is disabled. These errors are logged locally when running tests with tracing disabled.

E, [2024-06-18T14:45:33.966658 #817] ERROR -- ddtrace: [ddtrace] (/usr/local/bundle/ruby/3.1.0/gems/ddtrace-1.23.0/lib/datadog/tracing/distributed/propagation.rb:65:in `rescue in block in inject!') Error injecting distributed trace data. Cause: can't convert nil into Integer Location: /usr/local/bundle/ruby/3.1.0/gems/ddtrace-1.23.0/lib/datadog/tracing/distributed/trace_context.rb:124:in `format'

Expected behaviour

With tracing is disabled we'd expect no error logged.

As a workaround we are explicitly skipping the integration with

  c.tracing.instrument :grpc if Rails.env.production? || Rails.env.qa?

and that stops the errors from showing.

Environment

  • datadog version: 1.23.0
  • Configuration block (Datadog.configure ...):
Datadog.configure do |c|
  
  c.tracing.enabled = Rails.env.production? || Rails.env.qa?
  c.tracing.analytics.enabled = Rails.env.production? || Rails.env.qa?
  c.tracing.instrument :grpc
end
  • Ruby version: 3.1.5
  • Operating system: Debian Bullseye
  • Relevant library versions:
@sco11morgan sco11morgan added bug Involves a bug community Was opened by a community member labels Jun 18, 2024
@delner
Copy link
Contributor

delner commented Jul 8, 2024

@sco11morgan Thanks for the report! Sorry for getting back to you late.

From a quick look, to me, it looks like distributed tracing is attempting to propagate trace IDs, but since tracing is disabled, the trace ID is nil, which causes the computation to fail.

It also looks like we merged (what could be) a fix to this in our master branch for our 2.x series via #3730, possibly in response to this report. Maybe we just need to backport this to 1.x? @TonyCTHsu @marcotc WDYT?

@delner delner self-assigned this Jul 8, 2024
@delner delner added the integrations Involves tracing integrations label Jul 8, 2024
@bernardobarreto
Copy link

I'm having this issue while using version 1.23.
We do slow upgrades, so it would be nice if guys could backport it

@swrobel
Copy link

swrobel commented Nov 26, 2024

I, too, would love to see a backport to the 1.x series @TonyCTHsu @marcotc

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Involves a bug community Was opened by a community member integrations Involves tracing integrations
Projects
None yet
Development

No branches or pull requests

4 participants