Provide support for scraping multi-dimensional metrics.

Configuration should be expanded to include optional dimension information.

This should cover:

• Name of the dimension to use (mandatory)
• Target of partition in dimension you are interested in (optional)

The reported metric in the scraping endpoint should use labels to report the metric with a general name but labeled for every dimension regardless if a target was specified or not.

Names are just examples, open for feedback!

Example

name: demo_containerinstances_cpu
description: "Average cpu usage of our 'promitor-container-instance' container instance"
resourceType: ContainerInstance
containerGroup: promitor-container-instance
azureMetricConfiguration:
  metricName: CpuUsage
  aggregation:
    type: Average
  dimension:
    name: containerName # name of the dimension to split on
    target: redis-single-node # name of the targetted partition in the dimension you're interested in

Portal

You can see this in action in the Azure Portal:

Provide capability to go back to the landing page. This should be possible by clicking on the title.

Suggestion by @samneirinck

Provide integration with Azure Monitor where it will automatically query the Azure Monitor REST API for a set of metrics and expose them via the scrape endpoint.

Depends on #3.

Specifications

• Integrate with the Azure Monitor REST API
• Provide a way to configure one or more metrics to query
• Provide a way to configure how often you should pull latest representation of a metric
• Provide documentation on how to configure the metrics
• Provide documentation on how often you should pull latest representation of a metric
• Provide documentation on how to configure the authentication with Azure Monitor

Configuration

Metrics to scrape:

metrics:
- metric:
  name: "my-name"
  resource-type: "Microsoft.ServiceBus"
- metric:
  name: "my-name"
  resource-type: "Microsoft.WebApps"

Scrape frequency:

• PROMITOR_SCRAPE_SCHEDULE

Azure authentication:

• PROMITOR_AUTH_APPID
• PROMITOR_AUTH_APPKEY

Introduce unit testing

Specification

• Provide unit testing in codebase, without mocking
• Automatically runs on CI build

Validate the Azure authentication configuration on startup.

• Make sure that we don't start when either PROMITOR_AUTH_APPID or PROMITOR_AUTH_APPKEY are not configured
• Optionally authenticate to see if it works

Publish Docker image on hub.docker.com

Specifications

• Automate building the DOCKERFILE on master commit
• Automate tagging of the docker image
• Automate pushing a new version of container image
• Automate applying git tag on release
• Automate creation of GitHub release

Add generic scrape source for Azure Monitor

Specification

• Add new scrape source
• Add new metric configuration deserialization
• Add validation for new scrape source
• Add unit tests for new scrape source
• Document new scrape source usage

Add Event Hubs scrape source for Azure Monitor

Here is how you can add one.

Specification

• Add new scrape source
• Document new scrape source

Document what services we are using and are so generous to provide this offering for free:

These should all be on README & promitor.io, except for GitHub Apps which should only end up in README.

Used 3rd parties

Services:

• Travis CI for building PRs #6
• Snyk.io for vulnerability scanning (currently only docs because of #143) #91
• Renovate for maintaining dependencies #70

GitHub Apps:

• WIP for blocking WIP PRs
• Reminders for setting reminders
• Request Info for requesting information for PRs without info
• Stale for managing issues

Provide issue template

Add Storage Queue scrape source for Azure Monitor, if they do not support it we look for alternative data source

Here is how you can add one.

Specification

• Add new scrape source
• Document new scrape source

Provide documentation on how to configure the image

Specification

• How to configure the scraping endpoint
• How to configure how often to scrape
• How to configure the metrics to scrape in Azure Monitor
  • What resource types are supported
• How to authenticate to Azure Monitor
  • Requirement for Monitoring Reader role (More information)

Run integration tests in CI build that are introduced in #33

Specification

• CI should start Docker image
• CI should run integration tests on the running instance

Add telemetry sink for Application Insights

Part of #11.

Provide integration with Azure Key Vault for storing secrets

Specifications

• Provide capability to get secrets from the vault
• Provide in-memory caching for secrets (limited duration)
• Provide way to refresh a secret
• Document how to configure it

Provide documentation on how to run the image.

Specification

• Where to pull the image
• How to configure the container

Add support for custom metrics from Azure Monitor. These can originate from services such as Azure Application Insights.

Here is how you can add one.

Specification

• Add new scrape source
• Document new scrape source

Migrate to Microsoft.Azure.Management.Monitor.Fluent instead of Microsoft.Azure.Management.Monitor.

It would allow us to:

• Use a full featured package which is better supported
• Use fluent pattern

Provide support for Windows containers.

• Provide support for Windows containers
• Build Windows & Linux container in CI
• Build Windows & Linux container in release
• Document supported containers
• Update tagging strategy

Should be fairly easy via multi-arch containers

Provide capability to update a metric to scrape via the management API.

Specification

• Provide OpenAPI specs
• Provide endpoint
• Provide validation
• Provide integration test

Ignore Docker Compose project on CI given Microsoft.Docker.Sdk is not supported by Travis CI.
travis-ci/travis-ci#8494

Integrate snyk.io for security vulnerability scanning

• Configure for docs
• Configure for runtime (Docker)
• Configure for runtime (.NET)

Upgrade to .NET Core 2.1

More information about .NET Core 2.0 End of Life on September 1, 2018

Provide resiliency for Azure Monitor by implementing a few improvements:

• Retry
• Circuit Breaker

This should be handled via Polly.

Provide initial telemetry to Azure Application Insights. This is to have a operable MVP before starting with #29.

Specification

• Trace validation steps
• Track exceptions in Application Insights
• Track Azure Monitor as dependency
• Provide documentation on what to expect, how to configure, etc.

Build an advanced release pipeline to automatically release a new version and extends #172.

Specification

• Use YAML configuration over UI in Azure DevOps
• Every build should use the following flow
• Automatically push image tags to Docker Hub when building from master

Versioning

Every build is queued by specifying the version to release. This supports the following formats:

• {major}.{minor}.{patch} for full releases
  • Creates a GitHub release
• {major}.{minor}.{patch}-{description} for preview releases
  • Creates a GitHub pre-release

Docker tags

Every release should tag the docker image twice:

• {major}.{minor}.{patch} which is specific for this release
• {major}.{minor} which gives you all the features for this combination, but always pulls in the latest patch version. This should not be done when it is a preview release.

This is important as this enables consumers to always run on the latest bug fixes and most importantly security fixes.

Build flow

1. Build the codebase
2. Run unit tests (relates to #32)
3. Run docker build
4. Deploy new docker image (can be on Azure Container Instances)
5. Run integration testing (relates to #33)
6. Create git tag, if running on master
7. Create GitHub release, if running on master
8. Push Docker image to Docker Hub, if running on master

Provide kubernetes YAML declaration that deploys promitor with example configuration.

Provide health endpoint for Kubernetes

Specification

• Expose health endpoint
• Update documentation
• Add endpoint to Kubernetes YAML declaration (depends on #1)

API Specification

• Verb: GET
• Path: api/v1/health
• Reponse Codes:
  • 200: Healthy
  • 503: Unhealthy

Introduce integration testing to run on a deployed container or Promitor.

Specification

• Provide integration testing in codebase

Setup Travis CI to automatically build pull requests

Specifications

• Automatically build pull requests with Travis CI
• Add badge to README
• Protect master-branch

Promitor deserves to have a mascot and a basic logo.

I would like to refer to an animal that is good at monitoring, overwatching or scraping things.

Ideas:

• Laughing kookaburra (kudos to @jacqinthebox)
• Eagle (kudos to @SamVanhoutte)
• Falcon (kudos to @SamVanhoutte)
• Vulture (kudos to @SamVanhoutte)
• Meerkat (kudos to @sqldbawithbeard)
• Owl (kudos to @rikhepworth)
• Giraffe (kudos to @maartenba & @chton)
• Shepherd Dog (kudos to @maartenba)
• Stingray (kudos to @kvaes)

Logo should end up on our Twitter account as well.

Determine if we need to introduce bulkheads so that the Azure Monitoring scraper does not crash the whole container instance.

One option is to use Bulkheads with Polly.

Introduce default location for PROMITOR_CONFIGURATION_PATH in case users do not care where it is stored.

Setup documentation on Promitor.io.

Relates to #37.

Specification

• Deployed via GitHub Pages
• Deprecate wiki

Document how to add support for new scrape sources.

Back buttons are not always working.

Reported by @samneirinck

Currently every time a metric is scraped it will get a list of all measure points with a fixed interval, currently 1 minute.

This means that the scraping schedule, which is a cron, is not taken into account and can give false positives when using metrics such as average.

Initial documentation on promitor.io

Specification

• Setup documentation via GitHub Pages (#38)
• Why this project? (#8)
• Provide documentation on how to run the image (#15)
• Provide documentation on configuration #35)
• Remove wiki for business features
• Write blog post

Build Docker images in CI build to ensure they are still valid.

This can be done in Travis CI. (docs)

• Build Docker image in CI build
• Move running of tests to Dockerfile

Validate configured metrics declaration on startup.

Specification

• Avoid duplicate metric names
• Verify that the declaration file is present
• Verify that the format of the configuration is valid
• Verify that every declared metric is valid
• Every metric has a name
• Every metric has a resourceType
• Every metric has an azureMetricConfiguration

travis-ci.org is going away and we need to shift to travis-ci.com.

See official blog post and how to migrate.

Provide support for quering metrics for all Azure Clouds.

This should be configured in the metric declaration.

Add Service Bus Topic scrape source for Azure Monitor

Here is how you can add one.

Specification

• Add new scrape source
• Document new scrape source

Configuration should look like the following:

name: demo_topic_size
description: "Amount of active messages of the 'mytopic' topic"
resourceType: ServiceBusTopic
namespace: promitor-messaging
topicName: orders
subscriptionName: sello-vendor
azureMetricConfiguration:
  metricName: ActiveMessages
  aggregation:
    type: Total

Where subscriptionName is optional - If that is the case, metric will be reported for all subscriptions for a given topic.

Provide capability to send telemetry. It should allow us to have a variety of telemetry sinks.

For this we will use Serilog and will be implemented in #29, but we will start without serilog and provide Application Insights first #82.

Supported Sinks

• Azure Application Insights (#28)

Provide scraping endpoint for Prometheus

Specifications

• Configurable endpoint
• Document how to configure

Configuration

scrape-endpoint:
   uri-path: "prometheus/scrape"

Validate all configuration on startup and fail when something is not valid.

Specification

• Should signal container runtime that the instance is in an invalid state
• Enough information should be provided about what is not configured correctly

Provide documentation why this project has initiated and why it can be useful.

Should include the following:

• Using Kubernetes Horizontal Pod Autoscaler and how it works with custom metrics
• Why using Azure Monitor metrics in Kubernetes is not obvious

All telemetry should be written via Serilog so we can use a variety of sinks.

Part of #11.

Specifications

• Send exception information
• Send general tracing information
• Provide individual feature flags to control what is being sent
• Document how to configure this

Supported Sinks

• Console (on by default)
• Application Insights (opt-in)

Sign every Docker image that is being pushed

Provide Kubernetes Helm Chart to streamline deployments