Q: Expose AccumulatorSource about spark-metrics HOT 2 OPEN

Hi @f1yegor, as I know org.apache.spark.metrics.source.LongAccumulatorSource is available only from Spark 3.0 which is still in preview and we don't yet support it with spark-metrics.

from spark-metrics.

Ok, I will answer my question (I tested with Spark 3.0, but I believe it will work with 2.4+):

you need to register you accumulator like

val successCounter = spark.sparkContext.longAccumulator("records_success")

LongAccumulatorSource.register(
      spark.sparkContext,
      Map(
        successCounter.name.get -> successCounter)
    )

and include

*.source.jvm.class=org.apache.spark.metrics.source.LongAccumulatorSource

in $SPARK_HOME/metrics.properties
metric will be reported with timeseries
driver_accumulatorsource_records_success

from spark-metrics.