albertodonato / query-exporter Goto Github PK

View Code? Open in Web Editor NEW

429.0 14.0 101.0 460 KB

Export Prometheus metrics from SQL queries

License: GNU General Public License v3.0

Python 98.23% Shell 0.14% Dockerfile 1.63%

prometheus-exporter prometheus database query sql metrics metrics-exporter

query-exporter's People

Contributors

Stargazers

Watchers

Forkers

bambash wellingh homeguru geographicags ckseven ifanrx alan-an mdennis07 hbasria gianlucaficarelli jiportilla marlongehringer justmiles maverick-crank-grey ildarminaev tmkdev enchobelezirev jpouba richardwilko docjones zhuziyuan richardmoe lenalebt polohex hiteshsondhi88 escline psemeniuk mrsool kearme rayminr bouthib kajla ganeshdudwadkar cybernetics alexhenker chunoo anatoliis invokermain jancis-liang bigbadhobbit infa-ssimon italanchan sharathkumar13 bigcoster otani88 yeborisov applovin roman-ismailov kamalmemon arnovbr more-forks mdovhopo b4ldr ralfsilvadba grzegorznowak tyagian bourneye skillfactorycoding mrbilit leodemoniac khaoulab1 williamsdj tedigom mbailey mike-sixd juliocleal14 ulimit65535 jamesgouldsonos wally110 usermale jasonwanga rachlenko habitatenergy dacou mcraynest nuagic vincentchwong victorlopess shuaiwuyue ilanni2460 kayleejacques xuhua912 mohitmutha habitatenergy ericyu lfyorke habitatenergy darioef nqdev-fork gkuo06 paman6415 jmichael7 abhishek-mallick ilantnt andrewjw newrushbolt edisonx-sudo longxiangzhu nobl9 ausuwardi

query-exporter's Issues

Oracle not supported in docker image

We use oracledb as datasource, But I have test the images: adonato/query-exporter:2.4.0. It does not support the oracledb.

The error message: module "cx_Oracle" not found

The connection is as :

databases: testdb: dsn: oracle+cx_oracle://_account_:_password_@(DESCRIPTION_LIST = (LOAD_BALANCE=OFF) (FAILOVER=ON) (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = _test01_)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = _servicename_))) (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = _test02_)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = _servicename_))))

More extensive config validation

The config should be validated

against a JSON-schema, to prevent extra options (and detect typos)
to validate queries (and params) as much as possible statically

Multi-rows results

It seems an output with multiple rows is not supported.

I have queries with a clause GROUP BY which provides results in multi-rows like this:

> SELECT column2,COUNT(*) AS value FROM table GROUP BY column2;
+---------+-------+
| column2 | value |
+---------+-------+
| Type1   |     4 |
| Type2   |    73 |
| Type3   |    38 |
+---------+-------+

It would be practical to support multi-rows results that provides this kind of metrics:

databases:
  dbname:
    dsn: driver://foo:bar@baz/dbname
metrics:
  dbname_metric_by_column2:
    description: Total values by column2
    type: gauge
queries:
  dbname_metric_by_column2:
    databases: [dbname]
    metrics: [dbname_metric_by_column2]
    sql: SELECT column2,COUNT(*) FROM table GROUP BY column2

dbname_metric_by_column2{label="Type1"} 4
dbname_metric_by_column2{label="Type2"} 73
dbname_metric_by_column2{label="Type3"} 38

What do you think?

Multiple Queries

I'm trying run a SQL with multiple queries, like a script:

databases:
    xxx:
      dsn: env:DB2_DATABASE_DSN

  metrics:
    xxx_last_max_data:
      type: counter
      description: A counter

  queries:
    last_max_data:
      interval: 5
      databases: [xxx]
      metrics: [xxx_last_max_data]
      sql: |
        SET CURRENT QUERY ACCELERATION = NONE;
        SELECT
            MAX(ACTION) AS xxx_last_max_data
        FROM
            XXX.XXX
        WHERE
            ACTION BETWEEN xxx
            and yyy;

What you think about another optional (true/false) parameter that allow script execution?

How to run query exporter service to listen to 0.0.0.0/0

At present the exporter is bind to localhost only and not exposed for outside Prometheus to scrape.Any idea how can i add/change bind address to 0.0.0.0/0

Please add Docker support?

ibm_db not usable in Docker

The dns for db2 should be db2+ibm_db://....
Can you try if that works for you?

That doesn't work either, i am sorry... i tried the following:

db2+ibm_db_sa://<user>:<pass>@<ip>:<port>/<database>;
ibm_db://<user>:<pass>@<ip>:<port>/<database>;

these two give an "Invalid database DSN"

ibm_db_sa://<user>:<pass>@<ip>:<port>/<database>;
db2+ibm_db://<user>:<pass>@<ip>:<port>/<database>;

these two give an "module "ibm_db" not found"

The rest of the testing configuration is pretty much forward:

databases:
  c03:
    dsn: <...>

metrics:
  metric1:
    type: gauge
    description: test

queries:
  query1:
    interval: 5
    databases: [c03]
    metrics: [metric1]
    sql: SELECT RAND() as metric1 FROM SYSIBM.SYSDUMMY1

I'll push the fix for the Dockerfile in the next release soon.

Thanks :)

Originally posted by @Docjones in #39 (comment)

Do i need to set timeout for query execution?

Hello, i have a question - when DB query take too long time for execution does it impact all metrics return time.
Do i need to set timeout for query execution to solve problem with long executing queries or does async/await already solves it?

IBM DB2 table integration

Hi - Can this feature be used for IBM DB2 tables? It is not listed as one of the supported databases. I would like to confirm if that is the case. Where can I get the complete list of supported database engines ?

Thanks

Docker + MSSQL how to specify the connection string

I'm trying to connect to a local MSSQL-docker using the Docker image of query-exporter ( in a docker-compose env) with various connection strings, but no one works.

Installation details

docker image: adonato/query-exporter:latest
SQL Server image official MSSQL image

Installation through running pipenv shell -> pip3 install . -> query-exporter config.yaml works fine, so I can get metrics working when installed on local system ✅

To Reproduce

Prerequisite: MS SQL server running, exposing port 1433

Config file content

databases:
  db1:
    dsn: mssql+pyodbc://<username>:<password>@localhost:1433/master
    labels:
      region: us1
      app: app1

sections [metrics] and [queries] unaltered from your example

Ran query-exporter with the following
docker run -p 9560:9560/tcp -v "$PWD/config.yaml:/config.yaml" --rm -it \ adonato/query-exporter:latest -- /config.yaml
Error log:

query-exporter - error from database "db1": (pyodbc.InterfaceError) ('IM002', '[IM002] [unixODBC][Driver Manager]Data source name not found, and no default driver specified (0) (SQLDriverConnect)')

Encrypt and decrypt connection string

Hi Alberto!
Improve field "dsn":
add possibility to encrypt and decrypt connection string.

Startup in k8s results in error unless extra config applied

Describe the bug
I am playing around with the software in a kubernetes environment. Doing so, I stumbled upon a problem of the used asyncio-library: aio-libs/aiohttp#4554

This is the stack trace I am seeing:

/usr/local/lib/python3.8/asyncio/base_events.py:641: RuntimeWarning: coroutine 'Queue.get' was never awaited
  self._ready.clear()
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
Traceback (most recent call last):
  File "/virtualenv/bin/query-exporter", line 10, in <module>
    sys.exit(script())
  File "/virtualenv/lib/python3.8/site-packages/toolrack/script.py", line 105, in __call__
    self.main(parsed_args)
  File "/virtualenv/lib/python3.8/site-packages/prometheus_aioexporter/script.py", line 138, in main
    exporter.run()
  File "/virtualenv/lib/python3.8/site-packages/prometheus_aioexporter/web.py", line 70, in run
    run_app(
  File "/virtualenv/lib/python3.8/site-packages/aiohttp/web.py", line 419, in run_app
    loop.run_until_complete(_run_app(app,
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/virtualenv/lib/python3.8/site-packages/aiohttp/web.py", line 359, in _run_app
    await site.start()
  File "/virtualenv/lib/python3.8/site-packages/aiohttp/web_runner.py", line 100, in start
    self._server = await loop.create_server(  # type: ignore
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 1463, in create_server
    raise OSError(err.errno, 'error while attempting '
OSError: [Errno 99] error while attempting to bind on address ('::1', 9560, 0, 0): cannot assign requested address

It seems to try to bind to an IPv6 address, although this may not be available in the environment.

I do have a workaround: start the application using the parameters -H 0.0.0.0 to force it to using IPv4. I found the possibility by having a look at the documentation of the underlying https://github.com/albertodonato/prometheus-aioexporter.

While I do think that this issue is not directly related to this project, I wanted to raise this bug (even if it is rejected) to document this workaround for others that may run into this issue.

Installation details

Using Kubernetes 1.15 in GCloud
Using docker image of version 2.2.0 of the query-exporter

Add support for static label for DB

Is your feature request related to a problem? Please describe.
ex.
have oracle server with 4 instance: uz1/uz2/uz3/uz4
simple metric up:

queries:
  query1:
    interval: 10
    databases: [uz1, uz2,, uz3,, uz4]
    metrics: [up]
    sql: select 1 as up from dual

and generated metric:

up{database="uz1"} 1.0
up{database="uz2"} 1.0
up{database="uz3"} 1.0
up{database="uz4"} 1.0

But i need combine databases with one label server/stand/zone, etc
like:

up{database="uz1",server="1.2.3.4"} 1.0
up{database="uz2",server="1.2.3.4"} 1.0
up{database="uz3",server="1.2.3.4"} 1.0
up{database="uz4",server="1.2.3.4"} 1.0

yes it canbe resolved by

sql: select '1.2.3.4' as server, 1 as up from dual

but when add new database in list with other server ip need add new metric

Describe the solution you'd like
Add static_label + static_value for databases config.

databases:
  uz1:
    dsn: oracle+cx_oracle://
    static_label: server
    static_value: 1.2.3.4
  uz2:
    dsn: oracle://
    static_label: server
    static_value: 4.3.2.1

Describe alternatives you've considered
Or crazy but flexible variant:

queries:
  query1:
    interval: 10
    databases: [uz1, uz2,, uz3,, uz4, uz_other]
    metrics: [up]
    sql: select :param as server, 1 as up from dual
    parameters:
       - param1: 1.2.3.4
             databases_to_send: [uz1, uz2,, uz3,, uz4]
       - param1: 4.3.2.1
              databases_to_send: [uz_other]

with new key to separate send param1 with list db.

query-exporter stop working after for a while

I setup two query-exporter instances to query from different postgres separately. After running for a while, one of query-exporter always return the same result and queries_total stop increasing. Another one looks normal.

I compared the two query-exporter result, the stopped work one has large result(4.37 M) than the work one(1.65 M).

There's no any different setup for the two query-exporter instance. Is it possible the large metric result cause it stop working? Any limit of the size of postgres result for query-exporter?

Function Sequence Error

My exporter was working last week but for some reason now I am getting errors like:

2019-08-07 14:09:38,634 - ERROR - query-exporter - query "query2" on database "my database" failed: (pyodbc.Error) ('HY000', '[HY000] [Microsoft][ODBC SQL Server Driver]Connection is busy with results for another hstmt (0) (SQLExecDirectW)')

2019-08-07 14:09:38,650 - ERROR - query-exporter - query "query1" on database "*MY Database" failed: (pyodbc.Error) ('HY010', '[HY010] [Microsoft][ODBC SQL Server Driver]Function sequence error (0) (SQLFetch)')
(Background on this error at: http://sqlalche.me/e/dbapi)

Wrong result count from query

I am running into an issue with query count and unsure if this is a bug, required feature, or I am simply not grasping the query-exporter configuration correctly.

metrics:
  wait_time:
      type: gauge
      description: Generic metric from v$waitclassmetric view in Oracle.
      lables: [wait_class]

queries:
  wait_time:
      interval: 15     
      databases: [oracle]
      metrics: [wait_time]
      sql: |

SELECT
n.wait_class as "WAIT_CLASS",
round(m.time_waited/m.INTSIZE_CSEC,3) as "WAIT_TIME"
FROM
v$waitclassmetric  m, v$system_wait_class n
WHERE
m.wait_class_id=n.wait_class_id AND n.wait_class != 'Idle'

This results in the following error:
ERROR - query-exporter - query "wait_time" on database "oracle" failed: Wrong result count from query: expected 1, got 2

If I add another arbitrary metric I received an error indicating that a string from the wait_class view, which I have defined as a metric label is being treated as a second query value.
exception=ValueError("could not convert string to float: 'Application'",)>

In another prometheus database exporter tool there is a "field to append" option (e.g. fieldtoappend= "wait_class") used with these sorts of queries.

Exporter is stuck in transaction

Hello,

query-exporter: master
sqlalchemy: 1.3.5

The exporter executes a SET autocommit=0 command with mysql at the beginning and never gets out of the transaction, because all values are wrong after a few minutes of scraping e.g. all incoming updates/insertions/deletions transactions are not seen.

A hack described in the documentation of sqlalchemy does the trick.
https://docs.sqlalchemy.org/en/13/core/connections.html?highlight=engine#understanding-autocommit

diff --git a/query_exporter/db.py b/query_exporter/db.py
index 5f3f826..75b0713 100644
--- a/query_exporter/db.py
+++ b/query_exporter/db.py
@@ -203,7 +203,7 @@ class DataBase(_DataBase):
         self._pending_queries += 1
         self._conn: Engine
         try:
-            result = await self._conn.execute(query.sql)
+            result = await self._conn.execute(sqlalchemy.text(query.sql).execution_options(autocommit=True))
             return query.results(await QueryResults.from_results(result))
         except Exception as error:
             raise self._query_db_error(

Best,
Mike

Support disconnecting from db after each query

It can be useful to support not keeping open connection to a database if queries are run with a very long interval.

A flag could be specified in the databases.<db> config to force close after each query.

Improve documentation

Hi,
could you improve the documentation specially about the configuration file , is this a yml ?
And also how to start the exporter passing the config file?

Thanks
Nelson

feature request: use env variables in config

I would like to dockerise query-exporter for our test/prelive/live environments. Only difference between envs is connection url that I would like to expose/pass in docker image as environment variable

ala config.yml would be

databases:
  db1:
    dsn: ${CONNECTION_URL}

and then start

export CONNECTION_URL=oracle+cx_oracle://user:password@localhost:1521/XE 
query-exporter config.yml

I'm no python wizard but it seems doable https://stackoverflow.com/questions/52412297/how-to-replace-environment-variable-value-in-yaml-file-to-be-parsed-using-python

TypeError: unsupported operand type(s) for +=: 'float' and 'decimal.Decimal'

Thanks for taking the time to write this! I've found it quite useful.

I'm using this (v1.9.2) against Postgres (v9.2). It looks like the psycopg2 library casts floats as a decimal.Decimal here. I was able to get this to work for my use-case by forcing the value to a float (justmiles@d209076) but that might not be the most elegant solution - it could impact other database drivers.

Help wanted: Connect to Postgres using SSL certificates

Hi,
currently trying to connect to a postgres database that also uses client / server certificates. As far as I understand, I cannot add the ssl connection parameters to the DSN string with sqlalchemy. If I am wrong, I'd love to get some info about how to do it :).

Please explain how I can use it with client/server certificates (best would be an entry in the README) or add the possibility to configure ssl options explicitly.

Metrics for query time

Is your feature request related to a problem? Please describe.

Is it possible to get the execution time of a query as a metric?

Describe the solution you'd like
Have a config section for timed_queries where you could specify sql queries. This would return the time it took to execute the query.

Describe alternatives you've considered

Additional context

Help getting started

I am attempting to use query-exporter.

here is my config file:

databases:
  db1:
    dsn: postgresql://<user>:<pw>@<host>:5432/<database>

metrics:
  metric1:
    type: gauge
    description: A sample gauge

queries:
  query1:
    interval: 5
    databases: [db1]
    metrics: [metric1]
    sql: SELECT random() AS metric1

(where I have filled in <user>, <password>, etc)

And here is the command I run

docker run -p 9560:9560/tcp -v "$PWD/config.yaml:/config.yaml" --rm -it adonato/query-exporter:latest -- /config.yaml

When I run the command to start the query exporter, I get no response, it just hangs.

Log query execution timings

Is your feature request related to a problem? Please describe.
Maybe I have overlooked a configuration, but I could not find this in the Readme.

We use the query-exporter with different kinds of queries: some are short-running, and run quite often - others are rarely running, and can run for quite long (minutes, 10s of minutes). While I too think that those long-running queries are problematic, we cannot easily get rid of them.

Especially while setting them up initially, it's hard to see though when the queries run, and whether they should already have returned with data or not. Making it possible to activate logging (for start & end of a query) would really help when debugging queries.

Describe the solution you'd like
I'd love to either have a general configuration to switch on query logging for all queries, or to do it on a per-query-basis. Having the possibility for all queries would already help a lot - we could turn it on when needed, and turn it off normally.

Describe alternatives you've considered
n/a

Additional context
n/a

Disable queries if they fail because of invalid row length

If a query returns a number of entries for each row that doesn't match the expected count, it should be disabled as it will never succeed.

Query result count error if fields don't have unique names

The following error

2019-12-23 11:26:17,097 - ERROR - query-exporter - query "query3" on database "db3" failed: Wrong result count from query: expected 2, got 1

happens if columns don't have unique names (e.g. when automatically named after functions):

  query3:
    interval: 10
    databases: [db3]
    metrics: [metric1, metric2]
    sql: |
      SELECT floor(random() * 100 + 1)::int,
             floor(random() * 100 + 1)::int

(both columns are named floor in Postgres)

Issue connecting to ProxySQL admin SQL interface

Describe the bug

Connecting to the ProxySQL admin interface gives me the following error:

ERROR - query-exporter - error from database "proxysql_0": 'NoneType' object is not subscriptable

I am able to connect to normal MySQL instances no problem but ProxySQL fails.

Installation details

operating system: Debian Stretch (Docker)

To Reproduce

Connect to a ProxySQL admin interface.

I created a ticket in the upstream lib, I was able to validate that after applying my fix to the lib the problem goes away (though I do get some other warnings about SQL_MODE and CAST, but that's not a big deal to me).

sqlalchemy/sqlalchemy#5239

I'm looking for possibly some fix that doesn't require waiting for upstream, right now I'm considering building my own docker image with the appropriate fix built in.

Correctly escape sql queries

I have a query which contains DATE_FORMAT(CURDATE(), '%Y-%m-01 00:00:00'). In my config yaml, I have to escape the %s (ie. DATE_FORMAT(CURDATE(), '%%Y-%%m-01 00:00:00')

Otherwise I get the error

ERROR - query-exporter - query "query1" on database "db" failed: unsupported format character 'Y' (0x59) at index 144

Obviously escaping the % character myself wasn't hard to do (after a bit of googling), but ideally this would be handled in the application.

Getting errors when trying to connect to DB2 Sailfish appliance

2020-04-02 15:15:27,610 - ERROR - query-exporter - query "process_stats" on database "db2" failed: (ibm_db_dbi.Error) ibm_db_dbi::Error: [IBM][CLI Driver] CLI0125E Function sequence error. SQLSTATE=HY010 SQLCODE=-99999

sailfish.yaml:
databases:
db2:
dsn: ibm_db_sa://xxxxxxxxxxxxxx:50000/bludb

metrics:
db2_transactionlog_used:
type: counter
description: Percentage of transaction log used

queries:
process_stats:
databases: [db2]
interval: 30
metrics: [db2_transactionlog_used]
sql: 'select max(int((float(total_log_used) / float(total_log_used+total_log_available)) * 100)) as db2_transactionlog_used from table(mon_get_transaction_log(-2))'

Query as seen on the DB2 console:
select max(int((float(total_log_used) / float(total_log_used+total_log_available))*CAST (? AS INT))) "db2_transactionlog_used" from table(mon_get_transaction_log(CAST (? AS INT)))

It runs OK and returns 1 row as per the console.

The following code executes fine:
import sqlalchemy
from sqlalchemy import *
import ibm_db_sa

db2 = sqlalchemy.create_engine('ibm_db_sa://xxxxxxxxxxxxxxxxxx:50000/bludb')

count = db2.engine.execute('select max(int((float(total_log_used) / float(total_log_used+total_log_available))*100)) from table(mon_get_transaction_log(-2))').scalar()

print(count)

query-exporter binaries

Hi,

is it possible to get query-exporter binaries without snap..
problem is that snap cannot be installed on system which is highly secured and without internet access

Many thanks
tomislav

Query Exporter stops working after short while

Describe the bug
I am in the process of evaluating query exporter. Using the docker image adonato/query-exporter:2.1.0, after approximately six minutes requests to the service start timing out.

Running with -L DEBUG, I receive the following error when the service starts to time out:

query-exporter_1 | 2020-03-02 18:40:58,936 - ERROR - query-exporter - error from database "slave": QueuePool limit of size 5 overflow 10 reached, connection timed out, timeout 30 (Background on this error at: http://sqlalche.me/e/3o7r)

On the database server, I can see 30 sleeping connections opened from the service:

MySQL Process List

+--------+--------------+---------------------+----------+-----------+------+-----------------------------------------------------------------------------+------------------+----------+
| Id     | User         | Host                | db       | Command   | Time | State                                                                       | Info             | Progress |
+--------+--------------+---------------------+----------+-----------+------+-----------------------------------------------------------------------------+------------------+----------+
| 113949 | databaseuser | 192.168.10.54:46486 | database | Sleep     | 1491 |                                                                             | NULL             |    0.000 |
| 113953 | databaseuser | 192.168.10.54:46532 | database | Sleep     | 1466 |                                                                             | NULL             |    0.000 |
| 113955 | databaseuser | 192.168.10.54:46536 | database | Sleep     | 1451 |                                                                             | NULL             |    0.000 |
| 113960 | databaseuser | 192.168.10.54:46554 | database | Sleep     | 1446 |                                                                             | NULL             |    0.000 |
| 113963 | databaseuser | 192.168.10.54:46556 | database | Sleep     | 1421 |                                                                             | NULL             |    0.000 |
| 113964 | databaseuser | 192.168.10.54:46564 | database | Sleep     | 1391 |                                                                             | NULL             |    0.000 |
| 113969 | databaseuser | 192.168.10.54:46602 | database | Sleep     | 1376 |                                                                             | NULL             |    0.000 |
| 113974 | databaseuser | 192.168.10.54:46618 | database | Sleep     | 1361 |                                                                             | NULL             |    0.000 |
| 113977 | databaseuser | 192.168.10.54:46636 | database | Sleep     | 1346 |                                                                             | NULL             |    0.000 |
| 113980 | databaseuser | 192.168.10.54:46640 | database | Sleep     | 1331 |                                                                             | NULL             |    0.000 |
| 113983 | databaseuser | 192.168.10.54:46644 | database | Sleep     | 1316 |                                                                             | NULL             |    0.000 |
| 113986 | databaseuser | 192.168.10.54:46650 | database | Sleep     | 1286 |                                                                             | NULL             |    0.000 |
| 113989 | databaseuser | 192.168.10.54:46662 | database | Sleep     | 1271 |                                                                             | NULL             |    0.000 |
| 113994 | databaseuser | 192.168.10.54:46764 | database | Sleep     | 1256 |                                                                             | NULL             |    0.000 |
| 113997 | databaseuser | 192.168.10.54:46776 | database | Sleep     | 1240 |                                                                             | NULL             |    0.000 |
| 114056 | databaseuser | 192.168.10.54:47192 | database | Sleep     |  811 |                                                                             | NULL             |    0.000 |
| 114059 | databaseuser | 192.168.10.54:47210 | database | Sleep     |  791 |                                                                             | NULL             |    0.000 |
| 114060 | databaseuser | 192.168.10.54:47226 | database | Sleep     |  761 |                                                                             | NULL             |    0.000 |
| 114065 | databaseuser | 192.168.10.54:47256 | database | Sleep     |  741 |                                                                             | NULL             |    0.000 |
| 114070 | databaseuser | 192.168.10.54:47282 | database | Sleep     |  733 |                                                                             | NULL             |    0.000 |
| 114073 | databaseuser | 192.168.10.54:47288 | database | Sleep     |  732 |                                                                             | NULL             |    0.000 |
| 114074 | databaseuser | 192.168.10.54:47292 | database | Sleep     |  731 |                                                                             | NULL             |    0.000 |
| 114075 | databaseuser | 192.168.10.54:47296 | database | Sleep     |  726 |                                                                             | NULL             |    0.000 |
| 114076 | databaseuser | 192.168.10.54:47298 | database | Sleep     |  725 |                                                                             | NULL             |    0.000 |
| 114079 | databaseuser | 192.168.10.54:47300 | database | Sleep     |  721 |                                                                             | NULL             |    0.000 |
| 114080 | databaseuser | 192.168.10.54:47302 | database | Sleep     |  716 |                                                                             | NULL             |    0.000 |
| 114081 | databaseuser | 192.168.10.54:47304 | database | Sleep     |  686 |                                                                             | NULL             |    0.000 |
| 114084 | databaseuser | 192.168.10.54:47308 | database | Sleep     |  671 |                                                                             | NULL             |    0.000 |
| 114089 | databaseuser | 192.168.10.54:47320 | database | Sleep     |  641 |                                                                             | NULL             |    0.000 |
| 114092 | databaseuser | 192.168.10.54:47330 | database | Sleep     |  625 |                                                                             | NULL             |    0.000 |

Installation details

operating system: ubuntu:19.10
query-exporter installation type:
- docker image: 2.1.0

To Reproduce

docker-compose.yml

  query-exporter:
    image: adonato/query-exporter:2.1.0
    mem_limit: 64M
    volumes:
    - ./config/query-exporter/config.yml:/config.yml
    command:
    - "-L"
    - "DEBUG"
    - "--process-stats"
    - "/config.yml"
    ports:
    - "$GATEWAY_IP:9560:9560"
    environment:
      DB_DSN: $DB_DSN

config.yml

databases:
  slave:
    dsn: env:DB_DSN
    keep-connected: false

metrics:
  transaction_declines:
    type: gauge
  transaction_sales:
    type: gauge

queries:
  transaction_declines:
    interval: null
    databases: [slave]
    metrics: [transaction_declines]
    sql: |
      -- Quick query that takes <0.1ms to run
      [REDACTED]

  transaction_sales:
    interval: null
    databases: [slave]
    metrics: [transaction_sales]
    sql: |
      -- Quick query that takes <0.1ms to run
      [REDACTED]

Is it possible to change where the metrics are exposed to?

My job wants the metrics to be exposed to our network's website instead of being a localhost. Is it possible I can change the website from being localhost to our server the query-exporter is being run on?

Failing queries should be disabled on a per-database basis

A query might permanently fail on a database but not on another. It shouldn't be unconditionally removed.

Query-Exporter issue

Hi,

Have developed db2 query exporter for Prometheus,
build this select query,
select name as "db2_mycounter_name", total as "db2_mycounter_total" from myfakecounter

and inserted 2 values into table
insert into myfakecounter values ( 'counter1', 100 )
insert into myfakecounter values ( 'counter2, 200 )

I'm able to see both values in Prometheus console.

However, after a while, i have deleted counter2 value i.e 200 in this case from database.

But in Prometheus from query exporter select query i'm seeing 200 value still .
Its not showing me the existing results of select query i.e only counter 1 value

Question: dynamic reload of configuration file

Not a bug or feature request (yet) .. but is the configuration file automatically reload when its changed?
That way we could manage the configuration file outside of the docker container for the exporter and use a Volume to link the configuration file into the docker.
And by using env variables we can also remove any user/password from the configuration file so new Databases/Metrics/Queries can be added without changing the running instance (and possibly impacting metrics)

Oracle exporter queries

My data base is oracle. My requirement is alarms for : DB down, DB Slow, Some other Oracle erros like lack of table space and make a dashboard with the metrics.

I have done the below steps:

Copied the source code from the URL: https://github.com/albertodonato/query-exporter
Entered into query_exporter folder
Created config.yml file and script drafted as per format given in the github.
Questions:
How to startup the oracle exporter?
Do I need to touch any of the file with .py? which means do I need to configure something in main.py or setup.py?

I have attached word document for your reference.

Please help to clarify my queries.

Add support for setting a timeout for queries

Cool project, I have been using another SQL exporter but they are not actively maintained, so I am keen to switch to this project instead (also Python is nice).

Quick question as this is crucial for us to use this exporter in production. Is it possible to set query timeouts on a global or per query basis? I can't find anything while reading (skimming) over the docs.

query-exporter gets stuck and doesn't refresh metrics

Describe the bug

Exporter stops getting data from database after some time (sometimes a day, sometime a few minutes after start). After restarting (via snap/docker) it works correctly until it gets stuck again.

Installation details

operating system: Debian 10
query-exporter installation type (both were tried):
- docker image: latest (2.4.0)
- snap: 2.4.0

To Reproduce

config.yaml:

databases:
  db1:
    dsn: mysql://user:passw#[email protected]:3306/user

metrics:
  aAmount:
    type: gauge
    labels: [username]
  bAmount:
    type: gauge
    labels: [username]
  distribution:
    type: gauge
    labels: [aCount]

queries:
  getStats:
    interval: 15
    databases: [db1]
    metrics: [aAmount, bAmount]
    sql: {query that takes about 5 seconds to execute}
  getDistribution:
    interval: 15
    databases: [db1]
    metrics: [distributionDiscoveryBotAmount]
    sql: {query that takes about 0.5 seconds to execute}

Log (with `-L DEBUG`):

,2020-06-29 13:54:34,323 - INFO - aiohttp.access - 172.17.0.1 "GET /metrics HTTP/1.1" 200 20818 "-" "Prometheus/2.14.0"
,2020-06-29 13:54:44,592 - DEBUG - query-exporter - running query "getStats" on database "db1"
,2020-06-29 13:54:44,593 - DEBUG - query-exporter - running query "getDistribution" on database "db1"
,2020-06-29 13:54:49,185 - DEBUG - query-exporter - updating metric "aAmount" set 23735 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,185 - DEBUG - query-exporter - updating metric "bAmount" set 94403.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,185 - DEBUG - query-exporter - updating metric "aAmount" set 707 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,185 - DEBUG - query-exporter - updating metric "bAmount" set 2810.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,185 - DEBUG - query-exporter - updating metric "aAmount" set 214 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,185 - DEBUG - query-exporter - updating metric "bAmount" set 855.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,185 - DEBUG - query-exporter - updating metric "aAmount" set 4503 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,185 - DEBUG - query-exporter - updating metric "bAmount" set 18008.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,186 - DEBUG - query-exporter - updating metric "aAmount" set 525 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,186 - DEBUG - query-exporter - updating metric "bAmount" set 2100.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,186 - DEBUG - query-exporter - updating metric "aAmount" set 1046 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,186 - DEBUG - query-exporter - updating metric "bAmount" set 2439.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,186 - DEBUG - query-exporter - updating metric "aAmount" set 501 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,186 - DEBUG - query-exporter - updating metric "bAmount" set 2004.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,186 - DEBUG - query-exporter - updating metric "aAmount" set 30017 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,186 - DEBUG - query-exporter - updating metric "bAmount" set 119980.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,186 - DEBUG - query-exporter - updating metric "aAmount" set 2263 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,186 - DEBUG - query-exporter - updating metric "bAmount" set 9049.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,187 - DEBUG - query-exporter - updating metric "aAmount" set 500 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,187 - DEBUG - query-exporter - updating metric "bAmount" set 1995.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,187 - DEBUG - query-exporter - updating metric "aAmount" set 147 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,187 - DEBUG - query-exporter - updating metric "bAmount" set 588.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,187 - DEBUG - query-exporter - updating metric "aAmount" set 8000 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,187 - DEBUG - query-exporter - updating metric "bAmount" set 30599.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,187 - DEBUG - query-exporter - updating metric "aAmount" set 1758 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,187 - DEBUG - query-exporter - updating metric "bAmount" set 7032.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,187 - DEBUG - query-exporter - updating metric "aAmount" set 7346 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,188 - DEBUG - query-exporter - updating metric "bAmount" set 29371.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,188 - DEBUG - query-exporter - updating metric "aAmount" set 23742 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,188 - DEBUG - query-exporter - updating metric "bAmount" set 94913.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,188 - DEBUG - query-exporter - updating metric "aAmount" set 176 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,188 - DEBUG - query-exporter - updating metric "bAmount" set 703.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,188 - DEBUG - query-exporter - updating metric "aAmount" set 11915 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,188 - DEBUG - query-exporter - updating metric "bAmount" set 47660.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,188 - DEBUG - query-exporter - updating metric "aAmount" set 1709 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,189 - DEBUG - query-exporter - updating metric "bAmount" set 6558.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,189 - DEBUG - query-exporter - updating metric "aAmount" set 501 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,189 - DEBUG - query-exporter - updating metric "bAmount" set 2000.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,189 - DEBUG - query-exporter - updating metric "aAmount" set 152 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,189 - DEBUG - query-exporter - updating metric "bAmount" set 608.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,189 - DEBUG - query-exporter - updating metric "aAmount" set 26619 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,189 - DEBUG - query-exporter - updating metric "bAmount" set 103949.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,189 - DEBUG - query-exporter - updating metric "aAmount" set 930 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,190 - DEBUG - query-exporter - updating metric "bAmount" set 930.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,190 - DEBUG - query-exporter - updating metric "aAmount" set 409 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,190 - DEBUG - query-exporter - updating metric "bAmount" set 1636.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,190 - DEBUG - query-exporter - updating metric "aAmount" set 87 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,190 - DEBUG - query-exporter - updating metric "bAmount" set 347.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,190 - DEBUG - query-exporter - updating metric "aAmount" set 564 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,190 - DEBUG - query-exporter - updating metric "bAmount" set 2247.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,190 - DEBUG - query-exporter - updating metric "aAmount" set 969 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,190 - DEBUG - query-exporter - updating metric "bAmount" set 3870.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,190 - DEBUG - query-exporter - updating metric "aAmount" set 2045 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,191 - DEBUG - query-exporter - updating metric "bAmount" set 8115.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,191 - DEBUG - query-exporter - updating metric "aAmount" set 937 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,191 - DEBUG - query-exporter - updating metric "bAmount" set 3748.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,191 - DEBUG - query-exporter - updating metric "aAmount" set 661 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,191 - DEBUG - query-exporter - updating metric "bAmount" set 2573.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,191 - DEBUG - query-exporter - updating metric "aAmount" set 1820 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,191 - DEBUG - query-exporter - updating metric "bAmount" set 6696.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,191 - DEBUG - query-exporter - updating metric "aAmount" set 2084 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,191 - DEBUG - query-exporter - updating metric "bAmount" set 8033.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,192 - DEBUG - query-exporter - updating metric "aAmount" set 658 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,192 - DEBUG - query-exporter - updating metric "bAmount" set 2632.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,192 - DEBUG - query-exporter - updating metric "aAmount" set 2 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,192 - DEBUG - query-exporter - updating metric "bAmount" set 3.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,192 - DEBUG - query-exporter - updating metric "aAmount" set 8 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,192 - DEBUG - query-exporter - updating metric "bAmount" set 29.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,192 - DEBUG - query-exporter - updating metric "aAmount" set 220 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,192 - DEBUG - query-exporter - updating metric "bAmount" set 879.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,192 - DEBUG - query-exporter - updating metric "aAmount" set 5006 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,193 - DEBUG - query-exporter - updating metric "bAmount" set 19835.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,193 - DEBUG - query-exporter - updating metric "aAmount" set 5554 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,193 - DEBUG - query-exporter - updating metric "bAmount" set 22216.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,193 - DEBUG - query-exporter - updating metric "aAmount" set 16195 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,193 - DEBUG - query-exporter - updating metric "bAmount" set 64720.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,193 - DEBUG - query-exporter - updating metric "aAmount" set 3435 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,193 - DEBUG - query-exporter - updating metric "bAmount" set 13343.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,193 - DEBUG - query-exporter - updating metric "aAmount" set 1357 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,193 - DEBUG - query-exporter - updating metric "bAmount" set 5426.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,193 - DEBUG - query-exporter - updating metric "aAmount" set 14651 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,194 - DEBUG - query-exporter - updating metric "bAmount" set 58589.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,194 - DEBUG - query-exporter - updating metric "aAmount" set 1698 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,194 - DEBUG - query-exporter - updating metric "bAmount" set 6792.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,194 - DEBUG - query-exporter - updating metric "aAmount" set 571 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,194 - DEBUG - query-exporter - updating metric "bAmount" set 2284.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,194 - DEBUG - query-exporter - updating metric "aAmount" set 16037 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,194 - DEBUG - query-exporter - updating metric "bAmount" set 64148.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,194 - DEBUG - query-exporter - updating metric "aAmount" set 3 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,195 - DEBUG - query-exporter - updating metric "bAmount" set 6.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,195 - DEBUG - query-exporter - updating metric "aAmount" set 760 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,195 - DEBUG - query-exporter - updating metric "bAmount" set 3036.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,195 - DEBUG - query-exporter - updating metric "aAmount" set 2400 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,195 - DEBUG - query-exporter - updating metric "bAmount" set 9600.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,195 - DEBUG - query-exporter - updating metric "aAmount" set 589 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,195 - DEBUG - query-exporter - updating metric "bAmount" set 2354.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,195 - DEBUG - query-exporter - updating metric "aAmount" set 517 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,195 - DEBUG - query-exporter - updating metric "bAmount" set 2068.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,196 - DEBUG - query-exporter - updating metric "aAmount" set 2640 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,196 - DEBUG - query-exporter - updating metric "bAmount" set 8479.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,196 - DEBUG - query-exporter - updating metric "aAmount" set 575 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,196 - DEBUG - query-exporter - updating metric "bAmount" set 2300.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,196 - DEBUG - query-exporter - updating metric "aAmount" set 1280 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,196 - DEBUG - query-exporter - updating metric "bAmount" set 5056.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,196 - DEBUG - query-exporter - updating metric "aAmount" set 193 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,196 - DEBUG - query-exporter - updating metric "bAmount" set 772.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,196 - DEBUG - query-exporter - updating metric "aAmount" set 2844 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,197 - DEBUG - query-exporter - updating metric "bAmount" set 11373.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,197 - DEBUG - query-exporter - updating metric "aAmount" set 150 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,197 - DEBUG - query-exporter - updating metric "bAmount" set 450.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,197 - DEBUG - query-exporter - updating metric "aAmount" set 1488 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,197 - DEBUG - query-exporter - updating metric "bAmount" set 5932.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,197 - DEBUG - query-exporter - updating metric "aAmount" set 1371 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,197 - DEBUG - query-exporter - updating metric "bAmount" set 5480.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,197 - DEBUG - query-exporter - updating metric "aAmount" set 9234 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,197 - DEBUG - query-exporter - updating metric "bAmount" set 36927.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,198 - DEBUG - query-exporter - updating metric "aAmount" set 1450 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,198 - DEBUG - query-exporter - updating metric "bAmount" set 5793.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,198 - DEBUG - query-exporter - updating metric "aAmount" set 1860 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,198 - DEBUG - query-exporter - updating metric "bAmount" set 7439.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,198 - DEBUG - query-exporter - updating metric "aAmount" set 5054 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,198 - DEBUG - query-exporter - updating metric "bAmount" set 20195.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,198 - DEBUG - query-exporter - updating metric "aAmount" set 2266 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,198 - DEBUG - query-exporter - updating metric "bAmount" set 8883.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,198 - DEBUG - query-exporter - updating metric "aAmount" set 1478 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,199 - DEBUG - query-exporter - updating metric "bAmount" set 3977.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,199 - DEBUG - query-exporter - updating metric "aAmount" set 1696 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,199 - DEBUG - query-exporter - updating metric "bAmount" set 5874.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,199 - DEBUG - query-exporter - updating metric "aAmount" set 870 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,199 - DEBUG - query-exporter - updating metric "bAmount" set 3467.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,199 - DEBUG - query-exporter - updating metric "aAmount" set 1375 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,199 - DEBUG - query-exporter - updating metric "bAmount" set 4180.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,199 - DEBUG - query-exporter - updating metric "aAmount" set 45074 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,199 - DEBUG - query-exporter - updating metric "bAmount" set 179977.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,200 - DEBUG - query-exporter - updating metric "aAmount" set 526 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,200 - DEBUG - query-exporter - updating metric "bAmount" set 2104.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,200 - DEBUG - query-exporter - updating metric "aAmount" set 20 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,200 - DEBUG - query-exporter - updating metric "bAmount" set 80.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,200 - DEBUG - query-exporter - updating metric "aAmount" set 1672 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,200 - DEBUG - query-exporter - updating metric "bAmount" set 6519.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,200 - DEBUG - query-exporter - updating metric "aAmount" set 192 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,200 - DEBUG - query-exporter - updating metric "bAmount" set 569.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,200 - DEBUG - query-exporter - updating metric "aAmount" set 431 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,201 - DEBUG - query-exporter - updating metric "bAmount" set 1715.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,201 - DEBUG - query-exporter - updating metric "aAmount" set 847 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,201 - DEBUG - query-exporter - updating metric "bAmount" set 3348.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,201 - DEBUG - query-exporter - updating metric "aAmount" set 1614 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,201 - DEBUG - query-exporter - updating metric "bAmount" set 6442.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,201 - DEBUG - query-exporter - updating metric "aAmount" set 17460 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,201 - DEBUG - query-exporter - updating metric "bAmount" set 68370.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,201 - DEBUG - query-exporter - updating metric "aAmount" set 1956 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,201 - DEBUG - query-exporter - updating metric "bAmount" set 5168.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,201 - DEBUG - query-exporter - updating metric "aAmount" set 2724 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,202 - DEBUG - query-exporter - updating metric "bAmount" set 10892.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,202 - DEBUG - query-exporter - updating metric "aAmount" set 1204 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,202 - DEBUG - query-exporter - updating metric "bAmount" set 4814.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,202 - DEBUG - query-exporter - updating metric "aAmount" set 300 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,202 - DEBUG - query-exporter - updating metric "bAmount" set 1100.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,202 - DEBUG - query-exporter - updating metric "aAmount" set 800 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,202 - DEBUG - query-exporter - updating metric "bAmount" set 3200.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,202 - DEBUG - query-exporter - updating metric "aAmount" set 10829 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,202 - DEBUG - query-exporter - updating metric "bAmount" set 43304.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,203 - DEBUG - query-exporter - updating metric "aAmount" set 10819 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,203 - DEBUG - query-exporter - updating metric "bAmount" set 43246.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,203 - DEBUG - query-exporter - updating metric "aAmount" set 13363 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,203 - DEBUG - query-exporter - updating metric "bAmount" set 53426.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,203 - DEBUG - query-exporter - updating metric "aAmount" set 2053 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,203 - DEBUG - query-exporter - updating metric "bAmount" set 8201.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,203 - DEBUG - query-exporter - updating metric "aAmount" set 500 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,203 - DEBUG - query-exporter - updating metric "bAmount" set 2000.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,203 - DEBUG - query-exporter - updating metric "aAmount" set 7493 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,203 - DEBUG - query-exporter - updating metric "bAmount" set 27972.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,204 - DEBUG - query-exporter - updating metric "aAmount" set 2274 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,204 - DEBUG - query-exporter - updating metric "bAmount" set 9094.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,204 - DEBUG - query-exporter - updating metric "aAmount" set 351 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,204 - DEBUG - query-exporter - updating metric "bAmount" set 1404.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,204 - DEBUG - query-exporter - updating metric "aAmount" set 4192 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,204 - DEBUG - query-exporter - updating metric "bAmount" set 16666.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,204 - DEBUG - query-exporter - updating metric "aAmount" set 1464 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,204 - DEBUG - query-exporter - updating metric "bAmount" set 5421.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,204 - DEBUG - query-exporter - updating metric "aAmount" set 2001 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,204 - DEBUG - query-exporter - updating metric "bAmount" set 7897.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,205 - DEBUG - query-exporter - updating metric "aAmount" set 3783 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,205 - DEBUG - query-exporter - updating metric "bAmount" set 15128.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,205 - DEBUG - query-exporter - updating metric "aAmount" set 6508 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,205 - DEBUG - query-exporter - updating metric "bAmount" set 26014.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,205 - DEBUG - query-exporter - updating metric "aAmount" set 822 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,205 - DEBUG - query-exporter - updating metric "bAmount" set 3285.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,205 - DEBUG - query-exporter - updating metric "aAmount" set 303 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,205 - DEBUG - query-exporter - updating metric "bAmount" set 1209.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,205 - DEBUG - query-exporter - updating metric "aAmount" set 504 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,206 - DEBUG - query-exporter - updating metric "bAmount" set 2013.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,206 - DEBUG - query-exporter - updating metric "aAmount" set 3969 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,206 - DEBUG - query-exporter - updating metric "bAmount" set 15871.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,206 - DEBUG - query-exporter - updating metric "aAmount" set 510 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,206 - DEBUG - query-exporter - updating metric "bAmount" set 2031.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,206 - DEBUG - query-exporter - updating metric "aAmount" set 811 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,206 - DEBUG - query-exporter - updating metric "bAmount" set 3212.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,206 - DEBUG - query-exporter - updating metric "aAmount" set 17380 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,206 - DEBUG - query-exporter - updating metric "bAmount" set 69511.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,206 - DEBUG - query-exporter - updating metric "aAmount" set 173 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,206 - DEBUG - query-exporter - updating metric "bAmount" set 690.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,207 - DEBUG - query-exporter - updating metric "aAmount" set 39112 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,207 - DEBUG - query-exporter - updating metric "bAmount" set 154937.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,207 - DEBUG - query-exporter - updating metric "aAmount" set 3700 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,207 - DEBUG - query-exporter - updating metric "bAmount" set 14482.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,207 - DEBUG - query-exporter - updating metric "aAmount" set 931 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,207 - DEBUG - query-exporter - updating metric "bAmount" set 3724.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,207 - DEBUG - query-exporter - updating metric "aAmount" set 15030 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,207 - DEBUG - query-exporter - updating metric "bAmount" set 60093.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,207 - DEBUG - query-exporter - updating metric "aAmount" set 800 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,207 - DEBUG - query-exporter - updating metric "bAmount" set 3194.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,207 - DEBUG - query-exporter - updating metric "aAmount" set 2332 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,208 - DEBUG - query-exporter - updating metric "bAmount" set 9321.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,208 - DEBUG - query-exporter - updating metric "aAmount" set 589 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,208 - DEBUG - query-exporter - updating metric "bAmount" set 2356.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,208 - DEBUG - query-exporter - updating metric "aAmount" set 183 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,208 - DEBUG - query-exporter - updating metric "bAmount" set 550.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,208 - DEBUG - query-exporter - updating metric "aAmount" set 31107 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,208 - DEBUG - query-exporter - updating metric "bAmount" set 124422.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,208 - DEBUG - query-exporter - updating metric "aAmount" set 571 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,208 - DEBUG - query-exporter - updating metric "bAmount" set 2014.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,208 - DEBUG - query-exporter - updating metric "aAmount" set 8576 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,209 - DEBUG - query-exporter - updating metric "bAmount" set 34180.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,209 - DEBUG - query-exporter - updating metric "aAmount" set 2557 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,209 - DEBUG - query-exporter - updating metric "bAmount" set 10228.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,209 - DEBUG - query-exporter - updating metric "aAmount" set 3946 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,209 - DEBUG - query-exporter - updating metric "bAmount" set 15784.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,209 - DEBUG - query-exporter - updating metric "aAmount" set 4987 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,209 - DEBUG - query-exporter - updating metric "bAmount" set 19860.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,209 - DEBUG - query-exporter - updating metric "aAmount" set 94 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,209 - DEBUG - query-exporter - updating metric "bAmount" set 376.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,210 - DEBUG - query-exporter - updating metric "aAmount" set 13 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,210 - DEBUG - query-exporter - updating metric "bAmount" set 52.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,210 - DEBUG - query-exporter - updating metric "aAmount" set 569 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,210 - DEBUG - query-exporter - updating metric "bAmount" set 2096.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,210 - DEBUG - query-exporter - updating metric "aAmount" set 585 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,210 - DEBUG - query-exporter - updating metric "bAmount" set 2336.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,210 - DEBUG - query-exporter - updating metric "aAmount" set 6579 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,210 - DEBUG - query-exporter - updating metric "bAmount" set 26308.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,210 - DEBUG - query-exporter - updating metric "aAmount" set 547 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,210 - DEBUG - query-exporter - updating metric "bAmount" set 2187.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,210 - DEBUG - query-exporter - updating metric "aAmount" set 5918 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,211 - DEBUG - query-exporter - updating metric "bAmount" set 19364.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,211 - DEBUG - query-exporter - updating metric "aAmount" set 226 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,211 - DEBUG - query-exporter - updating metric "bAmount" set 901.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,211 - DEBUG - query-exporter - updating metric "aAmount" set 3 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,211 - DEBUG - query-exporter - updating metric "bAmount" set 12.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,211 - DEBUG - query-exporter - updating metric "aAmount" set 82702 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,211 - DEBUG - query-exporter - updating metric "bAmount" set 330605.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,211 - DEBUG - query-exporter - updating metric "aAmount" set 7369 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,211 - DEBUG - query-exporter - updating metric "bAmount" set 29406.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,211 - DEBUG - query-exporter - updating metric "aAmount" set 426 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,212 - DEBUG - query-exporter - updating metric "bAmount" set 1704.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,212 - DEBUG - query-exporter - updating metric "aAmount" set 13407 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,212 - DEBUG - query-exporter - updating metric "bAmount" set 51165.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,212 - DEBUG - query-exporter - updating metric "aAmount" set 378 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,212 - DEBUG - query-exporter - updating metric "bAmount" set 1512.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,212 - DEBUG - query-exporter - updating metric "aAmount" set 65 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,212 - DEBUG - query-exporter - updating metric "bAmount" set 260.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,212 - DEBUG - query-exporter - updating metric "aAmount" set 94 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,212 - DEBUG - query-exporter - updating metric "bAmount" set 376.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,213 - DEBUG - query-exporter - updating metric "aAmount" set 185 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,213 - DEBUG - query-exporter - updating metric "bAmount" set 553.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,213 - DEBUG - query-exporter - updating metric "aAmount" set 202 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,213 - DEBUG - query-exporter - updating metric "bAmount" set 807.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,213 - DEBUG - query-exporter - updating metric "aAmount" set 400 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,213 - DEBUG - query-exporter - updating metric "bAmount" set 1600.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,214 - DEBUG - query-exporter - updating metric "aAmount" set 1900 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,214 - DEBUG - query-exporter - updating metric "bAmount" set 7590.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,214 - DEBUG - query-exporter - updating metric "aAmount" set 1331 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,214 - DEBUG - query-exporter - updating metric "bAmount" set 5312.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,214 - DEBUG - query-exporter - updating metric "aAmount" set 606 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,214 - DEBUG - query-exporter - updating metric "bAmount" set 2424.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,214 - DEBUG - query-exporter - updating metric "aAmount" set 107 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,214 - DEBUG - query-exporter - updating metric "bAmount" set 428.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,214 - DEBUG - query-exporter - updating metric "aAmount" set 2687 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,214 - DEBUG - query-exporter - updating metric "bAmount" set 10742.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,215 - DEBUG - query-exporter - updating metric "aAmount" set 3584 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,215 - DEBUG - query-exporter - updating metric "bAmount" set 14336.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,215 - DEBUG - query-exporter - updating metric "aAmount" set 251 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,215 - DEBUG - query-exporter - updating metric "bAmount" set 1003.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,215 - DEBUG - query-exporter - updating metric "aAmount" set 375 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,215 - DEBUG - query-exporter - updating metric "bAmount" set 1500.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,215 - DEBUG - query-exporter - updating metric "aAmount" set 604 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,215 - DEBUG - query-exporter - updating metric "bAmount" set 2413.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,215 - DEBUG - query-exporter - updating metric "aAmount" set 667 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,216 - DEBUG - query-exporter - updating metric "bAmount" set 2458.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,216 - DEBUG - query-exporter - updating metric "aAmount" set 987 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,216 - DEBUG - query-exporter - updating metric "bAmount" set 3924.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,216 - DEBUG - query-exporter - updating metric "aAmount" set 3083 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,216 - DEBUG - query-exporter - updating metric "bAmount" set 12329.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,216 - DEBUG - query-exporter - updating metric "aAmount" set 174 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,216 - DEBUG - query-exporter - updating metric "bAmount" set 694.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,216 - DEBUG - query-exporter - updating metric "aAmount" set 95 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,217 - DEBUG - query-exporter - updating metric "bAmount" set 379.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,217 - DEBUG - query-exporter - updating metric "aAmount" set 106 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,217 - DEBUG - query-exporter - updating metric "bAmount" set 318.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,217 - DEBUG - query-exporter - updating metric "aAmount" set 400 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,217 - DEBUG - query-exporter - updating metric "bAmount" set 1300.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,217 - DEBUG - query-exporter - updating metric "aAmount" set 16479 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,217 - DEBUG - query-exporter - updating metric "bAmount" set 65787.0 {database="db1",username="REDACTED"}
,2020-06-29 13:54:49,217 - DEBUG - query-exporter - updating metric "query_latency" observe 4.1880284189246595 {database="db1",query="getStats"}
,2020-06-29 13:54:49,217 - DEBUG - query-exporter - updating metric "queries" inc 1 {database="db1",status="success"}
,2020-06-29 13:54:49,218 - DEBUG - query-exporter - updating metric "distribution" set 2811 {database="db1",aCount="1"}
,2020-06-29 13:54:49,218 - DEBUG - query-exporter - updating metric "distribution" set 3815 {database="db1",aCount="2"}
,2020-06-29 13:54:49,218 - DEBUG - query-exporter - updating metric "distribution" set 20418 {database="db1",aCount="3"}
,2020-06-29 13:54:49,218 - DEBUG - query-exporter - updating metric "distribution" set 667564 {database="db1",aCount="4"}
,2020-06-29 13:54:49,219 - DEBUG - query-exporter - updating metric "query_latency" observe 0.38984500290825963 {database="db1",query="getDistribution"}
,2020-06-29 13:54:49,219 - DEBUG - query-exporter - updating metric "queries" inc 1 {database="db1",status="success"}
,2020-06-29 13:54:49,325 - INFO - aiohttp.access - 172.17.0.1 "GET /metrics HTTP/1.1" 200 20819 "-" "Prometheus/2.14.0"
,2020-06-29 13:54:59,594 - DEBUG - query-exporter - running query "getStats" on database "db1"
,2020-06-29 13:54:59,594 - DEBUG - query-exporter - running query "getDistribution" on database "db1"
,2020-06-29 13:55:04,327 - INFO - aiohttp.access - 172.17.0.1 "GET /metrics HTTP/1.1" 200 20819 "-" "Prometheus/2.14.0"
,2020-06-29 13:55:04,345 - DEBUG - query-exporter - updating metric "aAmount" set 23735 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,345 - DEBUG - query-exporter - updating metric "bAmount" set 94403.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,345 - DEBUG - query-exporter - updating metric "aAmount" set 707 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,345 - DEBUG - query-exporter - updating metric "bAmount" set 2810.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,346 - DEBUG - query-exporter - updating metric "aAmount" set 214 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,346 - DEBUG - query-exporter - updating metric "bAmount" set 855.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,346 - DEBUG - query-exporter - updating metric "aAmount" set 4503 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,346 - DEBUG - query-exporter - updating metric "bAmount" set 18008.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,346 - DEBUG - query-exporter - updating metric "aAmount" set 525 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,346 - DEBUG - query-exporter - updating metric "bAmount" set 2100.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,346 - DEBUG - query-exporter - updating metric "aAmount" set 1046 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,346 - DEBUG - query-exporter - updating metric "bAmount" set 2439.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,346 - DEBUG - query-exporter - updating metric "aAmount" set 501 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,346 - DEBUG - query-exporter - updating metric "bAmount" set 2004.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,347 - DEBUG - query-exporter - updating metric "aAmount" set 30017 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,347 - DEBUG - query-exporter - updating metric "bAmount" set 119980.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,347 - DEBUG - query-exporter - updating metric "aAmount" set 2263 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,347 - DEBUG - query-exporter - updating metric "bAmount" set 9049.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,347 - DEBUG - query-exporter - updating metric "aAmount" set 500 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,347 - DEBUG - query-exporter - updating metric "bAmount" set 1995.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,347 - DEBUG - query-exporter - updating metric "aAmount" set 147 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,348 - DEBUG - query-exporter - updating metric "bAmount" set 588.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,348 - DEBUG - query-exporter - updating metric "aAmount" set 8000 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,348 - DEBUG - query-exporter - updating metric "bAmount" set 30599.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,348 - DEBUG - query-exporter - updating metric "aAmount" set 1758 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,348 - DEBUG - query-exporter - updating metric "bAmount" set 7032.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,348 - DEBUG - query-exporter - updating metric "aAmount" set 7346 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,348 - DEBUG - query-exporter - updating metric "bAmount" set 29371.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,348 - DEBUG - query-exporter - updating metric "aAmount" set 23742 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,348 - DEBUG - query-exporter - updating metric "bAmount" set 94913.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,349 - DEBUG - query-exporter - updating metric "aAmount" set 176 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,349 - DEBUG - query-exporter - updating metric "bAmount" set 703.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,349 - DEBUG - query-exporter - updating metric "aAmount" set 11915 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,349 - DEBUG - query-exporter - updating metric "bAmount" set 47660.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,349 - DEBUG - query-exporter - updating metric "aAmount" set 1709 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,349 - DEBUG - query-exporter - updating metric "bAmount" set 6558.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,349 - DEBUG - query-exporter - updating metric "aAmount" set 501 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,349 - DEBUG - query-exporter - updating metric "bAmount" set 2000.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,349 - DEBUG - query-exporter - updating metric "aAmount" set 152 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,349 - DEBUG - query-exporter - updating metric "bAmount" set 608.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,350 - DEBUG - query-exporter - updating metric "aAmount" set 26619 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,350 - DEBUG - query-exporter - updating metric "bAmount" set 103949.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,350 - DEBUG - query-exporter - updating metric "aAmount" set 930 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,350 - DEBUG - query-exporter - updating metric "bAmount" set 930.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,350 - DEBUG - query-exporter - updating metric "aAmount" set 409 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,350 - DEBUG - query-exporter - updating metric "bAmount" set 1636.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,350 - DEBUG - query-exporter - updating metric "aAmount" set 87 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,350 - DEBUG - query-exporter - updating metric "bAmount" set 347.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,350 - DEBUG - query-exporter - updating metric "aAmount" set 564 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,350 - DEBUG - query-exporter - updating metric "bAmount" set 2247.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,351 - DEBUG - query-exporter - updating metric "aAmount" set 969 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,351 - DEBUG - query-exporter - updating metric "bAmount" set 3870.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,351 - DEBUG - query-exporter - updating metric "aAmount" set 2045 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,351 - DEBUG - query-exporter - updating metric "bAmount" set 8115.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,351 - DEBUG - query-exporter - updating metric "aAmount" set 937 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,351 - DEBUG - query-exporter - updating metric "bAmount" set 3748.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,351 - DEBUG - query-exporter - updating metric "aAmount" set 661 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,351 - DEBUG - query-exporter - updating metric "bAmount" set 2573.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,352 - DEBUG - query-exporter - updating metric "aAmount" set 1820 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,352 - DEBUG - query-exporter - updating metric "bAmount" set 6696.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,352 - DEBUG - query-exporter - updating metric "aAmount" set 2084 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,352 - DEBUG - query-exporter - updating metric "bAmount" set 8033.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,352 - DEBUG - query-exporter - updating metric "aAmount" set 658 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,352 - DEBUG - query-exporter - updating metric "bAmount" set 2632.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,352 - DEBUG - query-exporter - updating metric "aAmount" set 2 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,352 - DEBUG - query-exporter - updating metric "bAmount" set 3.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,353 - DEBUG - query-exporter - updating metric "aAmount" set 8 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,353 - DEBUG - query-exporter - updating metric "bAmount" set 29.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,353 - DEBUG - query-exporter - updating metric "aAmount" set 220 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,353 - DEBUG - query-exporter - updating metric "bAmount" set 879.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,353 - DEBUG - query-exporter - updating metric "aAmount" set 5006 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,353 - DEBUG - query-exporter - updating metric "bAmount" set 19835.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,353 - DEBUG - query-exporter - updating metric "aAmount" set 5554 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,353 - DEBUG - query-exporter - updating metric "bAmount" set 22216.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,353 - DEBUG - query-exporter - updating metric "aAmount" set 16195 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,354 - DEBUG - query-exporter - updating metric "bAmount" set 64720.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,354 - DEBUG - query-exporter - updating metric "aAmount" set 3435 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,354 - DEBUG - query-exporter - updating metric "bAmount" set 13343.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,354 - DEBUG - query-exporter - updating metric "aAmount" set 1357 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,354 - DEBUG - query-exporter - updating metric "bAmount" set 5426.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,354 - DEBUG - query-exporter - updating metric "aAmount" set 14651 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,354 - DEBUG - query-exporter - updating metric "bAmount" set 58589.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,354 - DEBUG - query-exporter - updating metric "aAmount" set 1698 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,354 - DEBUG - query-exporter - updating metric "bAmount" set 6792.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,354 - DEBUG - query-exporter - updating metric "aAmount" set 571 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,355 - DEBUG - query-exporter - updating metric "bAmount" set 2284.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,355 - DEBUG - query-exporter - updating metric "aAmount" set 16037 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,355 - DEBUG - query-exporter - updating metric "bAmount" set 64148.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,355 - DEBUG - query-exporter - updating metric "aAmount" set 3 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,355 - DEBUG - query-exporter - updating metric "bAmount" set 6.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,355 - DEBUG - query-exporter - updating metric "aAmount" set 760 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,355 - DEBUG - query-exporter - updating metric "bAmount" set 3036.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,356 - DEBUG - query-exporter - updating metric "aAmount" set 2400 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,356 - DEBUG - query-exporter - updating metric "bAmount" set 9600.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,356 - DEBUG - query-exporter - updating metric "aAmount" set 589 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,356 - DEBUG - query-exporter - updating metric "bAmount" set 2354.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,356 - DEBUG - query-exporter - updating metric "aAmount" set 517 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,356 - DEBUG - query-exporter - updating metric "bAmount" set 2068.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,356 - DEBUG - query-exporter - updating metric "aAmount" set 2640 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,356 - DEBUG - query-exporter - updating metric "bAmount" set 8479.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,357 - DEBUG - query-exporter - updating metric "aAmount" set 575 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,357 - DEBUG - query-exporter - updating metric "bAmount" set 2300.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,357 - DEBUG - query-exporter - updating metric "aAmount" set 1280 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,357 - DEBUG - query-exporter - updating metric "bAmount" set 5056.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,357 - DEBUG - query-exporter - updating metric "aAmount" set 193 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,357 - DEBUG - query-exporter - updating metric "bAmount" set 772.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,357 - DEBUG - query-exporter - updating metric "aAmount" set 2844 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,357 - DEBUG - query-exporter - updating metric "bAmount" set 11373.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,357 - DEBUG - query-exporter - updating metric "aAmount" set 150 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,357 - DEBUG - query-exporter - updating metric "bAmount" set 450.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,358 - DEBUG - query-exporter - updating metric "aAmount" set 1488 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,358 - DEBUG - query-exporter - updating metric "bAmount" set 5932.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,358 - DEBUG - query-exporter - updating metric "aAmount" set 1371 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,358 - DEBUG - query-exporter - updating metric "bAmount" set 5480.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,358 - DEBUG - query-exporter - updating metric "aAmount" set 9234 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,358 - DEBUG - query-exporter - updating metric "bAmount" set 36927.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,358 - DEBUG - query-exporter - updating metric "aAmount" set 1450 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,358 - DEBUG - query-exporter - updating metric "bAmount" set 5793.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,358 - DEBUG - query-exporter - updating metric "aAmount" set 1860 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,359 - DEBUG - query-exporter - updating metric "bAmount" set 7439.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,359 - DEBUG - query-exporter - updating metric "aAmount" set 5054 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,359 - DEBUG - query-exporter - updating metric "bAmount" set 20195.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,359 - DEBUG - query-exporter - updating metric "aAmount" set 2266 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,359 - DEBUG - query-exporter - updating metric "bAmount" set 8883.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,359 - DEBUG - query-exporter - updating metric "aAmount" set 1478 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,359 - DEBUG - query-exporter - updating metric "bAmount" set 3977.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,359 - DEBUG - query-exporter - updating metric "aAmount" set 1696 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,360 - DEBUG - query-exporter - updating metric "bAmount" set 5874.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,360 - DEBUG - query-exporter - updating metric "aAmount" set 870 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,360 - DEBUG - query-exporter - updating metric "bAmount" set 3467.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,360 - DEBUG - query-exporter - updating metric "aAmount" set 1375 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,360 - DEBUG - query-exporter - updating metric "bAmount" set 4180.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,360 - DEBUG - query-exporter - updating metric "aAmount" set 45074 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,360 - DEBUG - query-exporter - updating metric "bAmount" set 179977.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,360 - DEBUG - query-exporter - updating metric "aAmount" set 526 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,361 - DEBUG - query-exporter - updating metric "bAmount" set 2104.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,361 - DEBUG - query-exporter - updating metric "aAmount" set 20 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,361 - DEBUG - query-exporter - updating metric "bAmount" set 80.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,361 - DEBUG - query-exporter - updating metric "aAmount" set 1672 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,361 - DEBUG - query-exporter - updating metric "bAmount" set 6519.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,361 - DEBUG - query-exporter - updating metric "aAmount" set 192 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,361 - DEBUG - query-exporter - updating metric "bAmount" set 569.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,361 - DEBUG - query-exporter - updating metric "aAmount" set 431 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,361 - DEBUG - query-exporter - updating metric "bAmount" set 1715.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,361 - DEBUG - query-exporter - updating metric "aAmount" set 847 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,361 - DEBUG - query-exporter - updating metric "bAmount" set 3348.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,361 - DEBUG - query-exporter - updating metric "aAmount" set 1614 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,361 - DEBUG - query-exporter - updating metric "bAmount" set 6442.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,362 - DEBUG - query-exporter - updating metric "aAmount" set 17460 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,362 - DEBUG - query-exporter - updating metric "bAmount" set 68370.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,362 - DEBUG - query-exporter - updating metric "aAmount" set 1956 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,362 - DEBUG - query-exporter - updating metric "bAmount" set 5168.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,362 - DEBUG - query-exporter - updating metric "aAmount" set 2724 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,362 - DEBUG - query-exporter - updating metric "bAmount" set 10892.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,362 - DEBUG - query-exporter - updating metric "aAmount" set 1204 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,362 - DEBUG - query-exporter - updating metric "bAmount" set 4814.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,362 - DEBUG - query-exporter - updating metric "aAmount" set 300 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,362 - DEBUG - query-exporter - updating metric "bAmount" set 1100.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,362 - DEBUG - query-exporter - updating metric "aAmount" set 800 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,362 - DEBUG - query-exporter - updating metric "bAmount" set 3200.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,362 - DEBUG - query-exporter - updating metric "aAmount" set 10829 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,362 - DEBUG - query-exporter - updating metric "bAmount" set 43304.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,362 - DEBUG - query-exporter - updating metric "aAmount" set 10819 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,362 - DEBUG - query-exporter - updating metric "bAmount" set 43246.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,362 - DEBUG - query-exporter - updating metric "aAmount" set 13363 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,363 - DEBUG - query-exporter - updating metric "bAmount" set 53426.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,363 - DEBUG - query-exporter - updating metric "aAmount" set 2053 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,363 - DEBUG - query-exporter - updating metric "bAmount" set 8201.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,363 - DEBUG - query-exporter - updating metric "aAmount" set 500 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,363 - DEBUG - query-exporter - updating metric "bAmount" set 2000.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,363 - DEBUG - query-exporter - updating metric "aAmount" set 7493 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,363 - DEBUG - query-exporter - updating metric "bAmount" set 27972.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,363 - DEBUG - query-exporter - updating metric "aAmount" set 2274 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,363 - DEBUG - query-exporter - updating metric "bAmount" set 9094.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,363 - DEBUG - query-exporter - updating metric "aAmount" set 351 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,363 - DEBUG - query-exporter - updating metric "bAmount" set 1404.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,363 - DEBUG - query-exporter - updating metric "aAmount" set 4192 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,363 - DEBUG - query-exporter - updating metric "bAmount" set 16666.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,363 - DEBUG - query-exporter - updating metric "aAmount" set 1464 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,363 - DEBUG - query-exporter - updating metric "bAmount" set 5421.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,364 - DEBUG - query-exporter - updating metric "aAmount" set 2001 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,364 - DEBUG - query-exporter - updating metric "bAmount" set 7897.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,364 - DEBUG - query-exporter - updating metric "aAmount" set 3783 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,364 - DEBUG - query-exporter - updating metric "bAmount" set 15128.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,364 - DEBUG - query-exporter - updating metric "aAmount" set 6508 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,364 - DEBUG - query-exporter - updating metric "bAmount" set 26014.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,364 - DEBUG - query-exporter - updating metric "aAmount" set 822 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,364 - DEBUG - query-exporter - updating metric "bAmount" set 3285.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,364 - DEBUG - query-exporter - updating metric "aAmount" set 303 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,364 - DEBUG - query-exporter - updating metric "bAmount" set 1209.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,364 - DEBUG - query-exporter - updating metric "aAmount" set 504 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,364 - DEBUG - query-exporter - updating metric "bAmount" set 2013.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,364 - DEBUG - query-exporter - updating metric "aAmount" set 3969 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,364 - DEBUG - query-exporter - updating metric "bAmount" set 15871.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,364 - DEBUG - query-exporter - updating metric "aAmount" set 510 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,364 - DEBUG - query-exporter - updating metric "bAmount" set 2031.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,365 - DEBUG - query-exporter - updating metric "aAmount" set 811 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,365 - DEBUG - query-exporter - updating metric "bAmount" set 3212.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,365 - DEBUG - query-exporter - updating metric "aAmount" set 17380 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,365 - DEBUG - query-exporter - updating metric "bAmount" set 69511.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,365 - DEBUG - query-exporter - updating metric "aAmount" set 173 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,365 - DEBUG - query-exporter - updating metric "bAmount" set 690.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,365 - DEBUG - query-exporter - updating metric "aAmount" set 39112 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,365 - DEBUG - query-exporter - updating metric "bAmount" set 154937.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,365 - DEBUG - query-exporter - updating metric "aAmount" set 3700 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,365 - DEBUG - query-exporter - updating metric "bAmount" set 14482.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,365 - DEBUG - query-exporter - updating metric "aAmount" set 931 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,365 - DEBUG - query-exporter - updating metric "bAmount" set 3724.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,365 - DEBUG - query-exporter - updating metric "aAmount" set 15030 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,365 - DEBUG - query-exporter - updating metric "bAmount" set 60093.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,365 - DEBUG - query-exporter - updating metric "aAmount" set 800 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,365 - DEBUG - query-exporter - updating metric "bAmount" set 3194.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,366 - DEBUG - query-exporter - updating metric "aAmount" set 2332 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,366 - DEBUG - query-exporter - updating metric "bAmount" set 9321.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,366 - DEBUG - query-exporter - updating metric "aAmount" set 589 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,366 - DEBUG - query-exporter - updating metric "bAmount" set 2356.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,366 - DEBUG - query-exporter - updating metric "aAmount" set 183 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,366 - DEBUG - query-exporter - updating metric "bAmount" set 550.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,366 - DEBUG - query-exporter - updating metric "aAmount" set 31107 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,366 - DEBUG - query-exporter - updating metric "bAmount" set 124422.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,366 - DEBUG - query-exporter - updating metric "aAmount" set 571 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,366 - DEBUG - query-exporter - updating metric "bAmount" set 2014.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,366 - DEBUG - query-exporter - updating metric "aAmount" set 8576 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,366 - DEBUG - query-exporter - updating metric "bAmount" set 34180.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,366 - DEBUG - query-exporter - updating metric "aAmount" set 2557 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,366 - DEBUG - query-exporter - updating metric "bAmount" set 10228.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,366 - DEBUG - query-exporter - updating metric "aAmount" set 3946 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,366 - DEBUG - query-exporter - updating metric "bAmount" set 15784.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,367 - DEBUG - query-exporter - updating metric "aAmount" set 4987 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,367 - DEBUG - query-exporter - updating metric "bAmount" set 19860.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,367 - DEBUG - query-exporter - updating metric "aAmount" set 94 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,367 - DEBUG - query-exporter - updating metric "bAmount" set 376.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,367 - DEBUG - query-exporter - updating metric "aAmount" set 13 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,367 - DEBUG - query-exporter - updating metric "bAmount" set 52.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,367 - DEBUG - query-exporter - updating metric "aAmount" set 569 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,367 - DEBUG - query-exporter - updating metric "bAmount" set 2096.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,367 - DEBUG - query-exporter - updating metric "aAmount" set 585 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,367 - DEBUG - query-exporter - updating metric "bAmount" set 2336.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,367 - DEBUG - query-exporter - updating metric "aAmount" set 6579 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,367 - DEBUG - query-exporter - updating metric "bAmount" set 26308.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,367 - DEBUG - query-exporter - updating metric "aAmount" set 547 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,367 - DEBUG - query-exporter - updating metric "bAmount" set 2187.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,367 - DEBUG - query-exporter - updating metric "aAmount" set 5918 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,367 - DEBUG - query-exporter - updating metric "bAmount" set 19364.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,367 - DEBUG - query-exporter - updating metric "aAmount" set 226 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,368 - DEBUG - query-exporter - updating metric "bAmount" set 901.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,368 - DEBUG - query-exporter - updating metric "aAmount" set 3 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,368 - DEBUG - query-exporter - updating metric "bAmount" set 12.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,368 - DEBUG - query-exporter - updating metric "aAmount" set 82702 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,368 - DEBUG - query-exporter - updating metric "bAmount" set 330605.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,368 - DEBUG - query-exporter - updating metric "aAmount" set 7369 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,368 - DEBUG - query-exporter - updating metric "bAmount" set 29406.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,368 - DEBUG - query-exporter - updating metric "aAmount" set 426 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,368 - DEBUG - query-exporter - updating metric "bAmount" set 1704.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,368 - DEBUG - query-exporter - updating metric "aAmount" set 13407 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,368 - DEBUG - query-exporter - updating metric "bAmount" set 51165.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,368 - DEBUG - query-exporter - updating metric "aAmount" set 378 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,368 - DEBUG - query-exporter - updating metric "bAmount" set 1512.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,368 - DEBUG - query-exporter - updating metric "aAmount" set 65 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,368 - DEBUG - query-exporter - updating metric "bAmount" set 260.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,368 - DEBUG - query-exporter - updating metric "aAmount" set 94 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,369 - DEBUG - query-exporter - updating metric "bAmount" set 376.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,369 - DEBUG - query-exporter - updating metric "aAmount" set 185 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,369 - DEBUG - query-exporter - updating metric "bAmount" set 553.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,369 - DEBUG - query-exporter - updating metric "aAmount" set 202 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,369 - DEBUG - query-exporter - updating metric "bAmount" set 807.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,369 - DEBUG - query-exporter - updating metric "aAmount" set 400 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,369 - DEBUG - query-exporter - updating metric "bAmount" set 1600.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,369 - DEBUG - query-exporter - updating metric "aAmount" set 1900 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,369 - DEBUG - query-exporter - updating metric "bAmount" set 7590.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,369 - DEBUG - query-exporter - updating metric "aAmount" set 1331 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,369 - DEBUG - query-exporter - updating metric "bAmount" set 5312.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,369 - DEBUG - query-exporter - updating metric "aAmount" set 606 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,369 - DEBUG - query-exporter - updating metric "bAmount" set 2424.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,369 - DEBUG - query-exporter - updating metric "aAmount" set 107 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,369 - DEBUG - query-exporter - updating metric "bAmount" set 428.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,369 - DEBUG - query-exporter - updating metric "aAmount" set 2687 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,369 - DEBUG - query-exporter - updating metric "bAmount" set 10742.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,370 - DEBUG - query-exporter - updating metric "aAmount" set 3584 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,370 - DEBUG - query-exporter - updating metric "bAmount" set 14336.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,370 - DEBUG - query-exporter - updating metric "aAmount" set 251 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,370 - DEBUG - query-exporter - updating metric "bAmount" set 1003.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,370 - DEBUG - query-exporter - updating metric "aAmount" set 375 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,370 - DEBUG - query-exporter - updating metric "bAmount" set 1500.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,370 - DEBUG - query-exporter - updating metric "aAmount" set 604 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,370 - DEBUG - query-exporter - updating metric "bAmount" set 2413.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,370 - DEBUG - query-exporter - updating metric "aAmount" set 667 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,370 - DEBUG - query-exporter - updating metric "bAmount" set 2458.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,370 - DEBUG - query-exporter - updating metric "aAmount" set 987 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,370 - DEBUG - query-exporter - updating metric "bAmount" set 3924.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,370 - DEBUG - query-exporter - updating metric "aAmount" set 3083 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,370 - DEBUG - query-exporter - updating metric "bAmount" set 12329.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,370 - DEBUG - query-exporter - updating metric "aAmount" set 174 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,370 - DEBUG - query-exporter - updating metric "bAmount" set 694.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,370 - DEBUG - query-exporter - updating metric "aAmount" set 95 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,370 - DEBUG - query-exporter - updating metric "bAmount" set 379.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,370 - DEBUG - query-exporter - updating metric "aAmount" set 106 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,370 - DEBUG - query-exporter - updating metric "bAmount" set 318.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,371 - DEBUG - query-exporter - updating metric "aAmount" set 400 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,371 - DEBUG - query-exporter - updating metric "bAmount" set 1300.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,371 - DEBUG - query-exporter - updating metric "aAmount" set 16479 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,371 - DEBUG - query-exporter - updating metric "bAmount" set 65787.0 {database="db1",username="REDACTED"}
,2020-06-29 13:55:04,371 - DEBUG - query-exporter - updating metric "query_latency" observe 4.27717144228518 {database="db1",query="getStats"}
,2020-06-29 13:55:04,371 - DEBUG - query-exporter - updating metric "queries" inc 1 {database="db1",status="success"}
,2020-06-29 13:55:04,371 - DEBUG - query-exporter - updating metric "distribution" set 2811 {database="db1",aCount="1"}
,2020-06-29 13:55:04,372 - DEBUG - query-exporter - updating metric "distribution" set 3815 {database="db1",aCount="2"}
,2020-06-29 13:55:04,372 - DEBUG - query-exporter - updating metric "distribution" set 20418 {database="db1",aCount="3"}
,2020-06-29 13:55:04,372 - DEBUG - query-exporter - updating metric "distribution" set 667564 {database="db1",aCount="4"}
,2020-06-29 13:55:04,372 - DEBUG - query-exporter - updating metric "query_latency" observe 0.46246094815433025 {database="db1",query="getDistribution"}
,2020-06-29 13:55:04,372 - DEBUG - query-exporter - updating metric "queries" inc 1 {database="db1",status="success"}
,2020-06-29 13:55:14,596 - DEBUG - query-exporter - running query "getStats" on database "db1"
,2020-06-29 13:55:14,596 - DEBUG - query-exporter - running query "getDistribution" on database "db1"
,2020-06-29 13:55:19,325 - INFO - aiohttp.access - 172.17.0.1 "GET /metrics HTTP/1.1" 200 20819 "-" "Prometheus/2.14.0"
,2020-06-29 13:55:29,595 - DEBUG - query-exporter - running query "getStats" on database "db1"
,2020-06-29 13:55:29,595 - DEBUG - query-exporter - running query "getDistribution" on database "db1"
,2020-06-29 13:55:34,325 - INFO - aiohttp.access - 172.17.0.1 "GET /metrics HTTP/1.1" 200 20819 "-" "Prometheus/2.14.0"
,2020-06-29 13:55:44,585 - DEBUG - query-exporter - running query "getStats" on database "db1"
,2020-06-29 13:55:44,586 - DEBUG - query-exporter - running query "getDistribution" on database "db1"
,2020-06-29 13:55:49,326 - INFO - aiohttp.access - 172.17.0.1 "GET /metrics HTTP/1.1" 200 20819 "-" "Prometheus/2.14.0"
,2020-06-29 13:55:59,586 - DEBUG - query-exporter - running query "getStats" on database "db1"
,2020-06-29 13:55:59,586 - DEBUG - query-exporter - running query "getDistribution" on database "db1"
,2020-06-29 13:56:04,325 - INFO - aiohttp.access - 172.17.0.1 "GET /metrics HTTP/1.1" 200 20819 "-" "Prometheus/2.14.0"
,2020-06-29 13:56:14,585 - DEBUG - query-exporter - running query "getStats" on database "db1"
,2020-06-29 13:56:14,585 - DEBUG - query-exporter - running query "getDistribution" on database "db1"
,2020-06-29 13:56:19,325 - INFO - aiohttp.access - 172.17.0.1 "GET /metrics HTTP/1.1" 200 20819 "-" "Prometheus/2.14.0"
,2020-06-29 13:56:29,585 - DEBUG - query-exporter - running query "getStats" on database "db1"
,2020-06-29 13:56:29,585 - DEBUG - query-exporter - running query "getDistribution" on database "db1"
,2020-06-29 13:56:34,325 - INFO - aiohttp.access - 172.17.0.1 "GET /metrics HTTP/1.1" 200 20819 "-" "Prometheus/2.14.0"
,2020-06-29 13:56:44,585 - DEBUG - query-exporter - running query "getStats" on database "db1"
,2020-06-29 13:56:44,585 - DEBUG - query-exporter - running query "getDistribution" on database "db1"
,2020-06-29 13:56:49,325 - INFO - aiohttp.access - 172.17.0.1 "GET /metrics HTTP/1.1" 200 20819 "-" "Prometheus/2.14.0"
,2020-06-29 13:56:59,585 - DEBUG - query-exporter - running query "getStats" on database "db1"
,2020-06-29 13:56:59,586 - DEBUG - query-exporter - running query "getDistribution" on database "db1"
,2020-06-29 13:57:04,326 - INFO - aiohttp.access - 172.17.0.1 "GET /metrics HTTP/1.1" 200 20819 "-" "Prometheus/2.14.0"
,2020-06-29 13:57:14,586 - DEBUG - query-exporter - running query "getStats" on database "db1"
,2020-06-29 13:57:14,586 - DEBUG - query-exporter - running query "getDistribution" on database "db1"
,2020-06-29 13:57:19,322 - INFO - aiohttp.access - 172.17.0.1 "GET /metrics HTTP/1.1" 200 20819 "-" "Prometheus/2.14.0"
,2020-06-29 13:57:29,585 - DEBUG - query-exporter - running query "getStats" on database "db1"
,2020-06-29 13:57:29,585 - DEBUG - query-exporter - running query "getDistribution" on database "db1"
,2020-06-29 13:57:34,323 - INFO - aiohttp.access - 172.17.0.1 "GET /metrics HTTP/1.1" 200 20819 "-" "Prometheus/2.14.0"
,2020-06-29 13:57:44,585 - DEBUG - query-exporter - running query "getStats" on database "db1"
,2020-06-29 13:57:44,585 - DEBUG - query-exporter - running query "getDistribution" on database "db1"
,2020-06-29 13:57:49,326 - INFO - aiohttp.access - 172.17.0.1 "GET /metrics HTTP/1.1" 200 20819 "-" "Prometheus/2.14.0"
,2020-06-29 13:57:59,586 - DEBUG - query-exporter - running query "getStats" on database "db1"
,2020-06-29 13:57:59,586 - DEBUG - query-exporter - running query "getDistribution" on database "db1"
,2020-06-29 13:58:04,326 - INFO - aiohttp.access - 172.17.0.1 "GET /metrics HTTP/1.1" 200 20819 "-" "Prometheus/2.14.0"
,

Support dynamic labels for metrics from query results

Arbitrary labels could be defined for metrics so that queries return their value together with metric values

Register exporter with prometheus and define unique port

Please register your exporter with prometheus on this page: Default-port-allocations. As part of this, please pick a unique port.

query-exporter cache expired metrics

Describe the bug
query-exporter will always cache expired metrics from db unless the query-exporter restarted

Installation details

operating system: kubernetes
query-exporter installation type:
- pip: [output from pip list]
- docker image: adonato/query-exporter:2.0.2
- snap: [output from snap list | grep query-exporter]

To Reproduce

create database test in postgres. It has table pipeline and query metrics with following sql:
SELECT pipeline_starttime_seconds, environment, pipelineid, pipelinename FROM pipeline WHERE pipeline._start_time >= (NOW() - INTERVAL '2 DAYS')
From query-exporter endpoints, I can see the metrics which matches to the sql output. Like:
pipeline_starttime_seconds{database="test",environment="staging",pipelineid="5e4c22125282e765b7cbdf3f",pipelinename="job_name"} 1.582047762272e+09
after one day, the pipelineid 5e4c22125282e765b7cbdf3f is not in the output of same sql "SELECT * FROM pipeline WHERE pipeline._start_time >= (NOW() - INTERVAL '2 DAYS')" since pipeline._start_time was expired. That's expected.
However, the same metric in 1 still existed in query-exporter endpoints. Looks like query-exporter always keep the old metrics even the sql output does not have the records.
After restarting the query-exporter container, then the metric in 1 was gone.

I think the query-exporter should match to the sql result. Otherwise, it'll accumulate the metrics by the up of query-exporter. Most of metrics are never used since they're not from the output of sql. And some old metrics may mess up the alerts.

@albertodonato Could you please take a look when you get a chance? Thanks a lot!

(psycopg2.errors.SyntaxError) syntax error at or near ":"

2020-01-31 21:00:29,509 - ERROR - query-exporter - query "pipeline_test_stats[params0]" on database "pg" failed: (psycopg2.errors.SyntaxError) syntax error at or near ":"
LINE 9: FROM pipeline INNER JOIN test ON :x INNER JOIN test_aggregat...

Password rotation

Hey @albertodonato! I've just found out your project and it fits the need that we have in our project 100% Thank you for developing it, looks great!

The only problem that I see is that the Postgres database in our environment rotates password every few minutes. Therefore if we deploy the query-exporter it would have to update the connection password as well.

I've found this issue #41 which would suggest that dynamic config reloads are not possible. However, maybe connection settings are somewhat different and there is a way to do it?

I'd be happy to try and create a PR for such a feature, yet it would be nice to hear your thoughts and suggestions on this as a project's creator :)

Add new date format for field "Interval"

Hi Alberto!
Improve field "Interval":
add possibility use date format like a crontab:
[Minute] [hour] [Day_of_the_Month] [Month_of_the_Year] [Day_of_the_Week]

Multiple set of parameters

I'm lost about use of parameters. I may not have understood, but shouldn't each set of parameters generate a different metric?

databases:
  xxx:
    dsn: env:DB2_DATABASE_DSN
    connect-sql:
      - SET CURRENT QUERY ACCELERATION = NONE

  xxx_acc:
    dsn: env:DB2_DATABASE_DSN
    connect-sql:
      - SET CURRENT QUERY ACCELERATION = ALL

metrics:
  xxx_xxx_max:
    type: gauge
    description: A gauge

queries:
  xxx_xxx_max:
    interval: 5
    databases: [xxx, xxx_acc]
    metrics: [xxx_xxx_max]
    parameters:
      - min: 1001
        max: 2000
      - min: 2001
        max: 3000
      - min: 3001
        max: 4000
    sql: |
      SELECT
          MAX(XXX) AS xxx_xxx_max
      FROM
          XXX.XXX
      WHERE
          XXX BETWEEN :min
          and :max;

Thanks!

Add Success Metric

Just a thought, if something were to go wrong etc perhaps add a result_code metric to denote successful completion of the query or other issues such as database not reachable, host not found, table not found etc.

Wrong result count from query - too many "AS" used in query?

I am attempting to pull data out of a server to get the value of some fields that were recorded the previous day. In taking a deeper look, I am wondering if the script is counting the number of "AS" statements in the query in order to determine the expected number of results from the query.

Error:
2020-01-17T18:59:14Z query-exporter.daemon[9204]: 2020-01-17 18:59:14,244 - ERROR - query-exporter - query "ob_stats" on database "pg" failed: Wrong result count from query: expected 8, got 3

config.yaml is set up as follows:

metrics:
  ob_avg_elapsed_time:
    type: gauge
    labels: [rd, avg]
  ob_max_elapsed_time:
    type: gauge
    labels: [rd, max]
  ob_min_elapsed_time:
    type: gauge
    labels: [rd, min]

queries:
  ob_stats:
    interval: 1d
    databases: [pg]
    metrics: [ob_avg_elapsed_time, ob_max_elapsed_time, ob_min_elapsed_time]
    sql: >
      SELECT
        coalesce(AVG(((EXTRACT(EPOCH FROM (CAST(_http_requests.completed_at AS TIMESTAMP) - _http_requests.created_at)) / (60.0 * 60 * 24)) * 86400)), 0) AS ob_avg_elapsed_time,
        coalesce(MAX(((EXTRACT(EPOCH FROM (CAST(_http_requests.completed_at AS TIMESTAMP) - _http_requests.created_at)) / (60.0 * 60 * 24)) * 86400)), 0) AS ob_max_elapsed_time,
        coalesce(MIN(((EXTRACT(EPOCH FROM (CAST(_http_requests.completed_at AS TIMESTAMP) - _http_requests.created_at)) / (60.0 * 60 * 24)) * 86400)), 0) AS ob_min_elapsed_time
      FROM public._http_requests _http_requests
      LEFT JOIN public._users _users ON (_http_requests.user_id = _users.id)
      LEFT JOIN public._sessions _sessions ON (_http_requests.session_id = _sessions.session_id)
      LEFT JOIN public._sites _sites ON (_http_requests.site_id = _sites.id)
      WHERE ((_http_requests.currentsheet IN ('ob/OrgBreakdown')) AND (_sites.name = 'team1')) and (CAST(TRUNC(EXTRACT(DAY FROM _http_requests.created_at)) AS INTEGER)) = (cast(TRUNC(extract(day from CURRENT_DATE - interval '1 DAY')) as INTEGER))

The result of the query run on its own is similar to the following:

ob_avg_elapsed_time | ob_max_elapsed_time | ob_min_elapsed_time
---------------------+---------------------+---------------------
                   0 |                   0 |                   0

Function Sequence Error

As I previously said, My job wants our query-exporter run on a windows virtual server. When I run the query exporter on my regular computer it works completely fine. When I run it on the windows server, it get function sequence errors like I had in an old issue. I only have one query so I'm confused on why it's throwing this error.

ERROR - query-exporter - query "query1" on database "DPA_Kapow_PRD2" failed: (pyodbc.Error) ('HY010', '[HY010] [Microsoft][ODBC Driver Manager] Function sequence error (0) (SQLFetch)')
(Background on this error at: http://sqlalche.me/e/dbapi)

mysql database / histogram

I have an event_length table in my mysql database with a column event_length (int) containing the length of an event. There is an event_date (datetime) column with the timestamp when the event happens.

I would like to create a histogram metric from this data. My config.yaml is the following:

=== config.yaml ===

databases:
db1:
dsn: mysql://xx_user_xx:xx_password_xx@xx_ip_xx:xx_port_xx/xx_database_xx

metrics:

metric2:
type: histogram
description: A sample histogram
buckets: [10, 20, 50, 100]

queries:

query2:
interval: 20
databases: [db1]
metrics: [metric2]
sql: |
SELECT event_length AS metric2 from event_length

=== config.yaml ===

The exporter works excellent and I can see the histogram buckets. But every time the query fires, it aggregates the select result set to the existing buckets, causing a continuously increasing number in the bucket. For example: if there is 100 rows in the table. First run will fill the buckets with the 100 rows (_COUNTER = 100). Next run (20 sec later) the 100 record and the new records is aggregated again to the buckets and (_COUNTER = 200+) and so on.

Is there any way to aggregate only the new records, or reset the buckets every time when the query fires?

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.