Git Product home page Git Product logo

stackdriver_exporter's Introduction

Google Stackdriver Prometheus Exporter

Build Status

A Prometheus exporter for Google Stackdriver Monitoring metrics. It acts as a proxy that requests Stackdriver API for the metric's time-series everytime prometheus scrapes it.

Installation

Binaries

Download the already existing binaries for your platform:

$ ./stackdriver_exporter <flags>

From source

Using the standard go install (you must have Go already installed in your local machine):

$ go install github.com/prometheus-community/stackdriver_exporter
$ stackdriver_exporter <flags>

Docker

To run the stackdriver exporter as a Docker container, run:

$ docker run -p 9255:9255 prometheuscommunity/stackdriver-exporter <flags>

Cloud Foundry

The exporter can be deployed to an already existing Cloud Foundry environment:

$ git clone https://github.com/prometheus-community/stackdriver_exporter.git
$ cd stackdriver_exporter

Modify the included application manifest file to include the desired properties. Then you can push the exporter to your Cloud Foundry environment:

$ cf push

BOSH

This exporter can be deployed using the Prometheus BOSH Release.

Usage

Credentials and Permissions

The Google Stackdriver Exporter uses the Google Golang Client Library, which offers a variety of ways to provide credentials. Please refer to the Google Application Default Credentials documentation to see how the credentials can be provided.

If you are using IAM roles, the roles/monitoring.viewer IAM role contains the required permissions. See the Access Control Guide for more information.

If you are still using the legacy Access scopes, the https://www.googleapis.com/auth/monitoring.read scope is required.

Flags

Flag Required Default Description
google.project-id No GCloud SDK auto-discovery Comma seperated list of Google Project IDs
monitoring.metrics-ingest-delay No Offsets metric collection by a delay appropriate for each metric type, e.g. because bigquery metrics are slow to appear
monitoring.metrics-type-prefixes Yes Comma separated Google Stackdriver Monitoring Metric Type prefixes (see example and available metrics)
monitoring.metrics-interval No 5m Metric's timestamp interval to request from the Google Stackdriver Monitoring Metrics API. Only the most recent data point is used
monitoring.metrics-offset No 0s Offset (into the past) for the metric's timestamp interval to request from the Google Stackdriver Monitoring Metrics API, to handle latency in published metrics
monitoring.filters No Formatted string to allow filtering on certain metrics type
web.listen-address No :9255 Address to listen on for web interface and telemetry
web.telemetry-path No /metrics Path under which to expose Prometheus metrics

Metrics

The exporter returns the following metrics:

Metric Description Labels
stackdriver_monitoring_api_calls_total Total number of Google Stackdriver Monitoring API calls made project_id
stackdriver_monitoring_scrapes_total Total number of Google Stackdriver Monitoring metrics scrapes project_id
stackdriver_monitoring_scrape_errors_total Total number of Google Stackdriver Monitoring metrics scrape errors project_id
stackdriver_monitoring_last_scrape_error Whether the last metrics scrape from Google Stackdriver Monitoring resulted in an error (1 for error, 0 for success) project_id
stackdriver_monitoring_last_scrape_timestamp Number of seconds since 1970 since last metrics scrape from Google Stackdriver Monitoring project_id
stackdriver_monitoring_last_scrape_duration_seconds Duration of the last metrics scrape from Google Stackdriver Monitoring project_id

Metrics gathered from Google Stackdriver Monitoring are converted to Prometheus metrics:

  • Metric's names are normalized according to the Prometheus specification using the following pattern:
    1. namespace is a constant prefix (stackdriver)
    2. subsystem is the normalized monitored resource type (ie gce_instance)
    3. name is the normalized metric type (ie compute_googleapis_com_instance_cpu_usage_time)
  • Labels attached to each metric are an aggregation of:
    1. the unit in which the metric value is reported
    2. the metric type labels (see Metrics List)
    3. the monitored resource labels (see Monitored Resource Types)
  • For each timeseries, only the most recent data point is exported.
  • Stackdriver GAUGE and DELTA metric kinds are reported as Prometheus Gauge metrics; Stackdriver CUMULATIVE metric kinds are reported as Prometheus Counter metrics.
  • Only BOOL, INT64, DOUBLE and DISTRIBUTION metric types are supported, other types (STRING and MONEY) are discarded.
  • DISTRIBUTION metric type is reported as a Prometheus Histogram, except the _sum time series is not supported.

Example

If we want to get all CPU (compute.googleapis.com/instance/cpu) and Disk (compute.googleapis.com/instance/disk) metrics for all Google Compute Engine instances, we can run the exporter with the following options:

stackdriver_exporter \
  --google.project-id my-test-project \
  --monitoring.metrics-type-prefixes "compute.googleapis.com/instance/cpu,compute.googleapis.com/instance/disk"

Using extra filters:

stackdriver_exporter \
 --google.project-id my-test-project \
 --monitoring.metrics-type-prefixes='pubsub.googleapis.com/subscription' \
 --monitoring.filters='pubsub.googleapis.com/subscription:resource.labels.subscription_id=monitoring.regex.full_match("us-west4.*my-team-subs.*")'

Filtering enabled collectors

The stackdriver_exporter collects all metrics type prefixes by default.

For advanced uses, the collection can be filtered by using a repeatable URL param called collect. In the Prometheus configuration you can use you can use this syntax under the scrape config.

params:
  collect:
  - compute.googleapis.com/instance/cpu
  - compute.googleapis.com/instance/disk

Contributing

Refer to the contributing guidelines.

License

Apache License 2.0, see LICENSE.

stackdriver_exporter's People

Contributors

frodenas avatar superq avatar prombot avatar thepalbi avatar everesio avatar jameshartig avatar alexey-medvedchikov avatar gidesh avatar rossmmurray avatar pracucci avatar kinghrothgar avatar jihonrado avatar glightfoot avatar fredr avatar eitah avatar dcaba avatar dfredell avatar vrunoa avatar xbglowx avatar anas-aso avatar daquinoaldo avatar

Stargazers

Roman avatar

Watchers

James Cloos avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.