Comments (2)
@acstevens this was built with Spark 2.2.1. Had no chance to build against Spark 2.3. Will try to make some time next week to compile it for Spark 2.3 as well.
Since the Sink
interface that we have to implement is not public in Spark codebase thus we need to compile Spark 2.3 first with the Sink
interface made public. Once we have the 'patched' jar we can rebuild spark-metrics
and make the necessary changes to bring it in line with the interface changes made to 2.3. I don't expect any changes in spark-metrics apart the ones required by interface changes made in Spark 2.3
Once the Sink
interface is made public in Spark building and releasing a new version of spark-metrics is going to much simpler.
On a side note if you're running long running Java apps on k8s and interested in how you can enable Prometheus monitoring check out: JMX Exporter Operator. If you're keen to learn what JMX Exporter Operator does behind the scenes read the Kubernetes Operator SDK blog.
from spark-metrics.
@acstevens spark-metrics
has been rebuilt for Spark 2.3 now. Please give it a try and let us know should you find any issues?
from spark-metrics.
Related Issues (20)
- Not seeing executor metrics, only driver HOT 13
- upgrade prometheus simpleclient to version 0.8.1
- Q: Expose AccumulatorSource HOT 2
- Metrics namespace HOT 11
- Pushgateway Read timed out HOT 2
- Want to understand that this spark matrics repo will work with prometheus in Hadoop cluster ? HOT 4
- Release Spark provided fix to maven HOT 2
- No Metrics From Spark Executors (Classes are being instantiated) HOT 10
- Metrics name pre-processing by custom Prometheus sink is working for only one component(driver/executor/applicationMaster) HOT 10
- Filter metrics HOT 10
- Metric Name RegEx Replacement doesn't work with JMX HOT 2
- Spark Metrics Stop Pushing After Pushgateway Restarts
- Configure sink to stop sending job as label/group-key HOT 3
- Metrics filter doesn't work
- Add remote_write to PrometheusSink
- Security Policy violation Binary Artifacts HOT 315
- Security Policy violation Branch Protection HOT 314
- Prometheus Sink is not working with SparkPi
- VictoriaMetrics HOT 2
- Repetitions of last metric value HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from spark-metrics.