Integration with Jaeger 1.49 is broken.

[danielbdias]

Users that update Jaeger to use the latest v1.49.0 are receiving the following gRPC error: Reason: cannot fetch trace: jaeger stream recv err: rpc error: code = Unknown desc = cannot retrieve trace: trace not found.

One example where this error happened: https://github.com/kubeshop/tracetest/actions/runs/6149707536/job/16686384831

[danielbdias]

One internal error that we are seeing is here: {"level":"warn","ts":1694453658.4716527,"caller":"[email protected]/clientconn.go:1515","msg":"[core][Channel #1 SubChannel #2] grpc: addrConn.createTransport failed to connect to {Addr: \"localhost:4317\", ServerName: \"localhost:4317\", }. Err: connection error: desc = \"transport: Error while dialing: dial tcp 127.0.0.1:4317: connect: connection refused\"","system":"grpc","grpc_log":true}

Updating Jaeger connection with OTel Collector to use OTLP seems to stabilize this problem: https://github.com/kubeshop/tracetest/blob/main/examples/tracetest-jaeger/collector.config.yaml but we need to investigate more about it.

[danielbdias]

Root cause pinpointed by @mathnogueira : due to this change https://github.com/jaegertracing/jaeger/pull/4742/files#diff-0ff00d4c60f3eba3addc36a2bd3222cd75fec1587416d74c397ff9704e22272bR47, jaeger started to send a error message that Tracetest was not able to decode.

#3149 fix that