Exported data contains long floating point numbers such as 0.0000032120027740023963. Due to the nature of the floating points numbers, aggregating those can lead to unstable results as the addition is non-commutative. This is problematic as it prevents us from defining meaningful aggregation rules over exported cost data.

Please see prometheus/prometheus#2951 for details.

We don't need full precision here, so we should round the results to 2 or 3 digits before emitting.

The metric name as it appears in prometheus is hardcoded at the moment. This should be made configurable via application.cfg.

Blog post should be updated in the README.md to https://tech.jda.com/public-cloud-cost-control/ or the server side redirect needs to be fixed as it currently redirects to a bad URL (missing the root /): https://tech.jda.compublic-cloud-cost-control/

Hi,

I am unable to follow properly on how to configure this exporter and then what to configure in prometheus

Use X-Prometheus-Scrape-Timeout-Seconds header sent by Prometheus instead of a hardcoded network timeout in the request to the Azure API.

To foster re-use and make the code easier to approach for the Prometheus community, we should switch to the official Prometheus client (https://github.com/prometheus/client_python)

Example how the integration could look like:

from prometheus_client.core import CounterMetricFamily, REGISTRY

class AzureCollector(object):
    def collect(self):
        c = CounterMetricFamily('my_counter_total', 'Help text', labels=['foo'])
        c.add_metric(['bar'], 1.7)
        c.add_metric(['baz'], 3.8)
        yield c

REGISTRY.register(AzureCollector())

We could either use the built-in HTTP server (via start_http_server), built-in WSGI app (via make_wsgi_app) or continue to use Flask by returning output of generate_latest() with content type CONTENT_TYPE_LATEST from a custom /metrics function.

As Microsoft has retired the old and switched to new billing APIs (https://docs.microsoft.com/en-us/azure/billing/billing-enterprise-api), are there any plans to introduce compatibility with them?

cryptography 2.2.2 is listed as a vulnerability.

Currently the labels of different components of this exporter are not aligned.

• azure_costs_eur labels are written in CamelCase, while azure_vms_allocated et al are written in snake_case.
• ResourceGroup values are lowercase, resource_group values are UPPERCASE
• azure_costs_eur has the label SubscriptionName="<literal name>", while azure_vms_allocated has the label subscription="<UUID>"

Currently, the test_token test runs through but actually should fail. The test obviously is not sensitive to the token setting. In practice it works, but the test should represent that behaviour.

I run applications like these in a docker container. For this I don't want a dependancy to a file with hardcoded variables. It would be nice if there is an option to read the configuration from the environment.

e.g. I would like to see a docker-compose.yml like this

version: '3'

services:
azure-billing-exporter:
    image: blue-yonder/azure-cost-mon
    ports:
    - 8000:8000
    environment:
      PROMETHEUS_METRIC_NAME: "azure_costs_eur"
      ENROLLMENT_NUMBER: "31415"
      BILLING_API_ACCESS_KEY: "3408795poitwiqeotu934t5pqweiut"

My workaround is to use the following Dockerfile

FROM python       

ADD https://github.com/blue-yonder/azure-cost-mon/archive/v0.4.1.tar.gz /tmp

WORKDIR /tmp
RUN tar -zxvf /tmp/v0.4.1.tar.gz \
  &&  mv /tmp/azure-cost-mon-0.4.1 /azure_cost_mon

WORKDIR /azure_cost_mon
RUN pip install -r requirements.txt \
  && pip install azure-costs-exporter gunicorn

ADD startup.py /azure_cost_mon/startup.py
EXPOSE 8000
ENTRYPOINT ["python","/azure_cost_mon/startup.py"]

And the following startup.py

import os

metricName=os.environ['PROMETHEUS_METRIC_NAME']
enrollmentNumber = os.environ['ENROLLMENT_NUMBER']
apiAccessKey= os.environ['BILLING_API_ACCESS_KEY']

file = open('/azure_cost_mon/application.cfg','w') 
file.write('ENROLLMENT_NUMBER=\''+enrollmentNumber+ '\'\n') 
file.write('BILLING_API_ACCESS_KEY=\''+apiAccessKey+ '\'\n') 
file.write('PROMETHEUS_METRIC_NAME=\''+metricName+ '\'\n') 
file.close()

os.system("gunicorn --log-level=info azure_costs_exporter:app -b :8000")

There seems to be an Azure test enrollment with ID 100 that returns realistic data. It would be useful to perform a real integration test against live data. The API key can be found here.

The exporter crashes if for a requested month, no usage details are available. The API doesn't return a JSON document in that case.