Git Product home page Git Product logo

spark-root-applications's Introduction

Spark ROOT Applications

Collection of various examples/utilities for Spark ROOT.

Metrics Listener

Simple Metrics Listener that extends SparkListener. All the jobs are collected in a buffer and dumped into json at the application's end. Directory/Filename of the metrics output are configurable (see below the conf options)

source for MetricsListener is located here: src/main/scala/org/dianahep/sparkrootapplications/metrics
sources for BenchMarksApp are here: src/main/scala/org/dianahep/sparkrootapplications/benchmarks

./spark-submit  --packages org.diana-hep:spark-root_2.11:0.1.7,org.diana-hep:histogrammar-sparksql_2.11:1.0.3,org.json4s:json4s-native_2.11:3.2.11  --class "org.dianahep.sparkrootapplications.benchmarks.AODPublicBenchmarkApp"  --conf spark.extraListeners="org.dianahep.sparkrootapplications.metrics.MetricsListener" --conf spark.executorEnv.pathToMetrics=/Users/vk/software/diana-hep/intel/metrics --conf spark.executorEnv.metricsFileName=test.json /Users/vk/software/diana-hep/spark-root-applications/target/scala-2.11/spark-root-applications_2.11-0.0.1.jar  file:/Users/vk/software/diana-hep/test_data/0029E804-C77C-E011-BA94-00215E22239A.root  /Users/vk/software/diana-hep/test_data/benchmarks/test 1 5

Example Applications

  • ReductionExample App
  • AODPublicExample App
  • Bacon/Higgs Example Apps

spark-root-applications's People

Contributors

vkhristenko avatar mrtnzrm2 avatar

Watchers

James Cloos avatar avatar

Forkers

siewyan

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.