Comments (6)
delner
commented on August 22, 2024
2
@anandhasudhan Thanks for reporting this! Just want to let you know the team has seen this, and is starting to look at it. I'll tag in someone here to walk through this issue with you.
from dd-trace-rb.
marcotc
commented on August 22, 2024
1
I'm taking a look into this issue next week. I'll report back.
from dd-trace-rb.
Ataraxic
commented on August 22, 2024
1
Ran into this as well. Subscribed. Thanks.
from dd-trace-rb.
anandhasudhan
commented on August 22, 2024
@marcotc Thanks for looking into this.
For additional context, I initialized 1_datadog.rb first and then 2_opentelemetry.rb. I tried changing the order to initialize 1_opentelemetry.rb first and then 2_datadog.rb. The opposite seems to happen. ie. open telemetry code ends up calling data dog code and throws the ArgumentError.
Argument Error Stack trace
ArgumentError: wrong number of arguments (given 2, expected 3)
/usr/src/app/vendor/bundle/ruby/2.6.0/gems/ddtrace-1.10.1/lib/datadog/tracing/contrib/http/instrumentation.rb:91:in `annotate_span_with_response!': wrong number of arguments (given 2, expected 3) (ArgumentError)
from /usr/src/app/vendor/bundle/ruby/2.6.0/gems/opentelemetry-instrumentation-net_http-0.21.1/lib/opentelemetry/instrumentation/net/http/patches/instrumentation.rb:39:in `block (2 levels) in request'
from /usr/src/app/vendor/bundle/ruby/2.6.0/gems/opentelemetry-instrumentation-net_http-0.21.1/lib/opentelemetry/instrumentation/net/http/patches/instrumentation.rb:38:in `tap'
from /usr/src/app/vendor/bundle/ruby/2.6.0/gems/opentelemetry-instrumentation-net_http-0.21.1/lib/opentelemetry/instrumentation/net/http/patches/instrumentation.rb:38:in `block in request'
from /usr/src/app/vendor/bundle/ruby/2.6.0/gems/opentelemetry-api-1.1.0/lib/opentelemetry/trace.rb:72:in `block in with_span'
from /usr/src/app/vendor/bundle/ruby/2.6.0/gems/opentelemetry-api-1.1.0/lib/opentelemetry/context.rb:87:in `with_value'
from /usr/src/app/vendor/bundle/ruby/2.6.0/gems/opentelemetry-api-1.1.0/lib/opentelemetry/trace.rb:72:in `with_span'
from /usr/src/app/vendor/bundle/ruby/2.6.0/gems/opentelemetry-api-1.1.0/lib/opentelemetry/trace/tracer.rb:29:in `in_span'
from /usr/src/app/vendor/bundle/ruby/2.6.0/gems/opentelemetry-instrumentation-net_http-0.21.1/lib/opentelemetry/instrumentation/net/http/patches/instrumentation.rb:31:in `request'
from /usr/src/app/vendor/bundle/ruby/2.6.0/gems/ddtrace-1.10.1/lib/datadog/tracing/contrib/http/instrumentation.rb:55:in `block in request'
from /usr/src/app/vendor/bundle/ruby/2.6.0/gems/ddtrace-1.10.1/lib/datadog/tracing/trace_operation.rb:192:in `block in measure'
from /usr/src/app/vendor/bundle/ruby/2.6.0/gems/ddtrace-1.10.1/lib/datadog/tracing/span_operation.rb:150:in `measure'
from /usr/src/app/vendor/bundle/ruby/2.6.0/gems/ddtrace-1.10.1/lib/datadog/tracing/trace_operation.rb:192:in `measure'
from /usr/src/app/vendor/bundle/ruby/2.6.0/gems/ddtrace-1.10.1/lib/datadog/tracing/tracer.rb:380:in `start_span'
from /usr/src/app/vendor/bundle/ruby/2.6.0/gems/ddtrace-1.10.1/lib/datadog/tracing/tracer.rb:160:in `block in trace'
from /usr/src/app/vendor/bundle/ruby/2.6.0/gems/ddtrace-1.10.1/lib/datadog/tracing/context.rb:43:in `activate!'
from /usr/src/app/vendor/bundle/ruby/2.6.0/gems/ddtrace-1.10.1/lib/datadog/tracing/tracer.rb:159:in `trace'
from /usr/src/app/vendor/bundle/ruby/2.6.0/gems/ddtrace-1.10.1/lib/datadog/tracing.rb:16:in `trace'
from /usr/src/app/vendor/bundle/ruby/2.6.0/gems/ddtrace-1.10.1/lib/datadog/tracing/contrib/http/instrumentation.rb:40:in `request'
from /usr/src/app/vendor/bundle/ruby/2.6.0/gems/opentelemetry-exporter-otlp-0.23.0/lib/opentelemetry/exporter/otlp/exporter.rb:163:in `block (2 levels) in send_bytes'
from /usr/src/app/vendor/bundle/ruby/2.6.0/gems/opentelemetry-exporter-otlp-0.23.0/lib/opentelemetry/exporter/otlp/exporter.rb:237:in `measure_request_duration'
from /usr/src/app/vendor/bundle/ruby/2.6.0/gems/opentelemetry-exporter-otlp-0.23.0/lib/opentelemetry/exporter/otlp/exporter.rb:163:in `block in send_bytes'
from /usr/src/app/vendor/bundle/ruby/2.6.0/gems/opentelemetry-exporter-otlp-0.23.0/lib/opentelemetry/exporter/otlp/exporter.rb:131:in `block in around_request'
from /usr/src/app/vendor/bundle/ruby/2.6.0/gems/opentelemetry-common-0.19.7/lib/opentelemetry/common/utilities.rb:92:in `block in untraced'
from /usr/src/app/vendor/bundle/ruby/2.6.0/gems/opentelemetry-api-1.1.0/lib/opentelemetry/context.rb:87:in `with_value'
from /usr/src/app/vendor/bundle/ruby/2.6.0/gems/opentelemetry-common-0.19.7/lib/opentelemetry/common/utilities.rb:91:in `untraced'
from /usr/src/app/vendor/bundle/ruby/2.6.0/gems/opentelemetry-exporter-otlp-0.23.0/lib/opentelemetry/exporter/otlp/exporter.rb:131:in `around_request'
from /usr/src/app/vendor/bundle/ruby/2.6.0/gems/opentelemetry-exporter-otlp-0.23.0/lib/opentelemetry/exporter/otlp/exporter.rb:155:in `send_bytes'
from /usr/src/app/vendor/bundle/ruby/2.6.0/gems/opentelemetry-exporter-otlp-0.23.0/lib/opentelemetry/exporter/otlp/exporter.rb:89:in `export'
from /usr/src/app/vendor/bundle/ruby/2.6.0/gems/opentelemetry-sdk-1.1.0/lib/opentelemetry/sdk/trace/export/batch_span_processor.rb:187:in `block in export_batch'
from /usr/src/app/vendor/bundle/ruby/2.6.0/gems/opentelemetry-sdk-1.1.0/lib/opentelemetry/sdk/trace/export/batch_span_processor.rb:187:in `synchronize'
from /usr/src/app/vendor/bundle/ruby/2.6.0/gems/opentelemetry-sdk-1.1.0/lib/opentelemetry/sdk/trace/export/batch_span_processor.rb:187:in `export_batch'
from /usr/src/app/vendor/bundle/ruby/2.6.0/gems/opentelemetry-sdk-1.1.0/lib/opentelemetry/sdk/trace/export/batch_span_processor.rb:170:in `block in work'
from /usr/src/app/vendor/bundle/ruby/2.6.0/gems/opentelemetry-sdk-1.1.0/lib/opentelemetry/sdk/trace/export/batch_span_processor.rb:159:in `loop'
from /usr/src/app/vendor/bundle/ruby/2.6.0/gems/opentelemetry-sdk-1.1.0/lib/opentelemetry/sdk/trace/export/batch_span_processor.rb:159:in `work'
from /usr/src/app/vendor/bundle/ruby/2.6.0/gems/opentelemetry-sdk-1.1.0/lib/opentelemetry/sdk/trace/export/batch_span_processor.rb:180:in `block in reset_on_fork'
from /usr/src/app/vendor/bundle/ruby/2.6.0/gems/hamelin-1.1.3/lib/hamelin/core_ext/threads.rb:14:in `block in initialize'
I came across this PR where it indicates that opentelemetry tracers have been built using the datadog auto-instrumentation library as a starting point and wonder if that could have any impact here.
I've provided my Opentelemetry configuration below.
OpenTelemetry configuration
require 'opentelemetry/sdk'
require 'opentelemetry/instrumentation/all'
require 'opentelemetry-exporter-otlp'
OpenTelemetry::SDK.configure do |c|
c.service_name = 'basic-web'
c.use 'OpenTelemetry::Instrumentation::ActiveSupport'
c.use 'OpenTelemetry::Instrumentation::Rack'
c.use 'OpenTelemetry::Instrumentation::AwsSdk'
c.use 'OpenTelemetry::Instrumentation::HTTP'
c.use 'OpenTelemetry::Instrumentation::ConcurrentRuby'
c.use 'OpenTelemetry::Instrumentation::Dalli'
c.use 'OpenTelemetry::Instrumentation::Ethon'
c.use 'OpenTelemetry::Instrumentation::Faraday'
c.use 'OpenTelemetry::Instrumentation::HttpClient'
c.use 'OpenTelemetry::Instrumentation::Net::HTTP'
c.use 'OpenTelemetry::Instrumentation::Rake'
c.use 'OpenTelemetry::Instrumentation::Redis'
c.use 'OpenTelemetry::Instrumentation::RestClient'
c.use 'OpenTelemetry::Instrumentation::Sidekiq'
c.add_span_processor(
OpenTelemetry::SDK::Trace::Export::BatchSpanProcessor.new(
OpenTelemetry::Exporter::OTLP::Exporter.new(endpoint: "http://#{ENV['DD_AGENT_HOST']}:4318/v1/traces")
)
)
endfrom dd-trace-rb.
agunn-so
commented on August 22, 2024
Seeing this as well, following
from dd-trace-rb.
anandhasudhan
commented on August 22, 2024
Based on my investigation, I realized that it is not possible to have both data dog and opentelemetry auto-instrumentation code running in parallel. Please correct me if I'm wrong.
I took a deep dive at the implementation of both datadog and opentelemetry auto-instrumentation code. Both overload the request method in the Net::HTTP module. So when datadog is trying to instrument the HTTP requests, it ends up calling opentelemetry code since that overloads the same set of methods (annotate_span_with_response!) in the Net::HTTP module leading to argument errors. To run both datadog and opentelemetry code in parallel, we would need to create our own wrapper class that encapsulates both the datadog and opentelemetry tracing logic at which point it's better to avoid and just use one library.
from dd-trace-rb.
Related Issues (20)
- ddtrace - throws error when deploying a rails 6.1 instance HOT 1
- Sinatra - missing trace_id when trying to correlate logs with traces HOT 2
- Qless support is missing from 2.0 HOT 1
- Capture tracing when a specific header is passed in an HTTP request HOT 3
- Instrumenting both Redis and Sidekiq leads to "stack level too deep (SystemStackError)" HOT 1
- GraphQL queries will error out after datadog upgrade ("Found two visible definitions") HOT 2
- With `datadog` 2.x, how can I mimic the 1.x test adapter that prints traces to `$stdout`? HOT 2
- Failed to report profiling data, connection closed before message completed HOT 3
- Anyway to make the faraday instrumentation split by domain but allow an override to not split by domain in some cases? HOT 5
- "Incompatible version of the passenger gem" even when Passenger has the right version HOT 7
- GRPC integration logging error even when tracing is disabled HOT 1
- We had a use case for Datadog::Tracing::Contrib::HTTP::Instrumentation.after_request HOT 3
- Wrong MIME/Content-Type for code provenance multipart data HOT 2
- Support tracing of Mysql2 gem prepared statements HOT 2
- Datadog::Tracing.correlation without trace information HOT 1
- AppSec crashes when parsing integer HTTP headers (undefined method `tr' for nil:NilClass (NoMethodError)) HOT 3
- Rails tracing produces duplicate mysql2 spans
- GraphQL unified tracer not autoloaded HOT 2
- suppress logs by default when running minitest tests HOT 2
- ActiveSupport::Cache instrumentation missing namespace
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dd-trace-rb.