Comments (6)
Yes, nothing has not changed to improve the agent configuration. I can continue to replicate ongoing issues with the TCP connector. Documentation has not been improved. I continue to contemplate life as a lumberjack.
from datadog-plugin.
Thank you for your note. We will look deeper into our current process for metrics and logs ingestion, but not something we can change in the short term. I would also like to take the opportunity to clarify a few points here
TCP is a backup method
In the context of Jenkins plugin, there is no easy alternative to collect logs. TCP is the next best alternative to when the Agent cannot tail logs from a file. So while TCP is considered a backup, here it makes sense for a TCP connection.
TCP does not use encryption
Yes, while the TCP connection between the Jenkins plugin and the Agent is unencrypted, the Agent itself further processes these logs and sends them over an encrypted HTTPS connection to Datadog.
Plugin generates spammy Jenkins logs upon connection error
Thank you for your feedback. We will take a look to improve the logging and have a more graceful failure logging
from datadog-plugin.
In the context of Jenkins plugin, there is no easy alternative to collect logs. TCP is the next best alternative to when the Agent cannot tail logs from a file.
I find this confusing, which is probably my own lack of familiarity. However, forwarding logs over HTTPS to Datadog, directly, which is the default setting, suits my purposes and works consistently. I don't note much complexity in either the writer or the client it invokes. Why is it not possible to offer a configuration option to use the agent for trace data and the like, while sending logs over HTTP? There's multiple other tickets/issues indicating issues with the lack of per-service configuration that this would solve, and outwardly the approach seems simple.
Yes, while the TCP connection between the Jenkins plugin and the Agent is unencrypted, the Agent itself further processes these logs and sends them over an encrypted HTTPS connection to Datadog.
Ah, my misunderstanding, then. "TCP log shipment" is a bit of an overloaded term, and the TCP connector feature isn't terribly well-documented. I certainly don't have much concern about encrypting Jenkins -> agent traffic, but assumed the TCP connector feature was opening a direct port-forward to Datadog's receivers.
TCP is a backup method
This is my primary issue with the approach as-implemented. While I agree it's the agreed-upon solution for the agent to ship in this context, it's not a well-implemented, or well-documented feature. It took me nearly a month to convince DD support that a TCP listener was even a feature that could be implemented by an agent container. Even as such, implementing it using the Docker agent requires a rather un-Dockery bind mount of a configuration YAML file, or otherwise a custom container. I'd consider both considerable drawbacks over simply sending the logs over HTTP. I'm not sure what utility the agent is providing in this context past ensuring consistent tagging.
from datadog-plugin.
Thanks for your contribution!
This issue has been automatically marked as stale because it has not had activity in the last 30 days. Note that the issue will not be automatically closed, but this notification will remind us to investigate why there's been inactivity. Thank you for participating in the Datadog open source community.
If you would like this issue to remain open:
-
Verify that you can still reproduce the issue in the latest version of this project.
-
Comment that the issue is still reproducible and include updated details requested in the issue template.
from datadog-plugin.
Hi, I would like to apologize for my tardiness. I have spoken to my team, and we will add a feature request to decouple metrics and logs ingest. We are also currently working on the documentation and should have it ready within a couple of weeks at max.
Regarding the log level - can you share a sample log message. We would like to understand if the message here pertains to failure to send a log line or a connection failure. TIA!
from datadog-plugin.
Hi @bradq, just bumping this in case you missed @shris-github's message!
from datadog-plugin.
Related Issues (20)
- Cannot use CI Visibility with DogStatsD over Unix Domain Socket HOT 4
- Failed to serialize org.datadog.jenkins.plugins.datadog.model.PipelineQueueInfoAction occasionally thrown during pipeline execution HOT 3
- Datadog plugin incompatible with Jenkins 2.338 and up HOT 5
- Java errors after enabling log collection in Jenkins HOT 13
- Support different hosts for DogStatsD and Traces Collection Port HOT 1
- CasC: default values are not respected HOT 2
- Metrics and/or service checks for available upgrades or plugins with warnings / security issues HOT 3
- Plugin configured with CI metrics prevents logs from being displayed in the pipeline stage view HOT 7
- `branch` tag missing on metrics and events HOT 4
- Datadog Log Collection Port is not set properly in jenkins logs HOT 2
- Failed to send log payload for Jenkins workers
- Invalid configuration elements `emitConfigChangeEvents` HOT 3
- StackOverflowError after enabling log collection HOT 3
- All classic jobs in jenkins turn into failed on datadog plugin v6.0.0 HOT 8
- jenkins.job.waiting is in seconds, not ms HOT 1
- CI Traces not being sent to agent because BuildSpanAction is null HOT 1
- Missing git client dependency HOT 2
- Getting exception NoClassDefFoundError HOT 8
- Build history destroyed with `7.0.0` upgrade: builds marked "1 Jan. 1970" and stuck HOT 9
- Unbounded memory usage in generating traces from stage data HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from datadog-plugin.