cherts / pgscv Goto Github PK

View Code? Open in Web Editor NEW

This project forked from lesovsky/pgscv

13.0 0.0 2.0 1.3 MB

pgSCV is a PostgreSQL ecosystem metrics collector

License: BSD 3-Clause "New" or "Revised" License

Shell 0.48% Go 98.47% Makefile 0.50% Dockerfile 0.26% Smarty 0.30%

pgscv's Introduction

pgSCV - PostgreSQL ecosystem metrics collector

По-русски / In Russian

pgSCV

collects a lot of stats about PostgreSQL environment.
exposes metrics through the HTTP /metrics endpoint in Prometheus metrics exposition format.

IMPORTANT NOTES This project is a continuation of the development of the original pgSCV by Alexey Lesovsky

Features

Supported services: support collecting metrics of PostgreSQL, Pgbouncer and Patroni.
OS metrics: support collecting metrics of operating system.
TLS and authentication. /metrics endpoint could be protected with basic authentication and TLS.
Collecting metrics from multiple services. pgSCV can collect metrics from many databases instances.
User-defined metrics. pgSCV could be configured in a way to collect metrics defined by user.
Collectors management. Collectors could be disabled if necessary.
Collectors filters. Collectors could be adjusted to skip collecting metrics based on labels values, like block devices, network interfaces, filesystems, users, databases, etc.

Requirements

Can run on Linux only; can connect to remote services running on other OS/PaaS.
Requisites for connecting to the services, such as login and password.
Database user should have privileges for executing stats functions and reading views. For more details see security considerations.

Quick start

Download the archive from releases. Unpack the archive. Create minimum config file. Start pgSCV systemd service under postgres user.

curl -s -L https://github.com/cherts/pgscv/releases/download/v0.9.1/pgscv_0.9.1_linux_amd64.tar.gz -o - | tar xzf - -C /tmp && \
mv /tmp/pgscv.yaml /etc && \
mv /tmp/pgscv.service /etc/systemd/system &&  \
mv /tmp/pgscv.default /etc/default/pgscv && \
mv /tmp/pgscv /usr/sbin && \
chown postgres:postgres /etc/pgscv.yaml && \
systemctl daemon-reload && \
systemctl enable pgscv --now

or using Docker, use DATABASE_DSN for setting up a connection to PostgreSQL:

docker pull cherts/pgscv:latest
docker run -ti -d --name pgscv \
   -e PGSCV_LISTEN_ADDRESS=0.0.0.0:9890 \
   -e PGSCV_DISABLE_COLLECTORS="system" \
   -e DATABASE_DSN="postgresql://postgres:password@dbhost:5432/postgres" \
   -p 9890:9890 \
   --restart=always \
   cherts/pgscv:latest

or using Docker, save deploy/pgscv.yaml config file to local directory /etc/pgscv:

docker pull cherts/pgscv:latest
docker run -ti -d --name pgscv \
   -v /etc/pgscv:/etc/app \
   -p 9890:9890 \
   --restart=always \
   cherts/pgscv:latest \
   --config-file=/etc/app/pgscv.yaml

or using Docker-compose, edit file docker-compose.yaml for setting up a connection to PostgreSQL:

mkdir ~/pgscv
curl -s -L https://raw.githubusercontent.com/CHERTS/pgscv/master/deploy/docker-compose.yaml -o ~/pgscv/docker-compose.yaml && cd ~/pgscv
docker-compose up -d

When pgSCV has been started it is ready to accept HTTP requests at http://127.0.0.1:9890/metrics.

or using k8s deployment

curl -s -L https://raw.githubusercontent.com/CHERTS/pgscv/master/deploy/deployment.yaml -o ~/deployment.yaml
kubectl apply -f ~/deployment.yaml

or using k8s helm chart

git clone https://github.com/CHERTS/pgscv.git && cd pgscv
kubectl create ns pgscv-ns
helm install -n pgscv-ns pgscv deploy/helm-chart/

Complete setup

Checkout complete setup guide.

Documentation

For further documentation see wiki.

Grafana dashboards

See directory deploy/grafana or use Grafana Lab repo:

Support and feedback

If you need help using pgSCV feel free to open discussion via email or Telegram @cherts or create an issue

Development and contribution

To help development you are encouraged to:

provide feedback via email or Telegram @cherts or create an issue
pull requests for new features
star the project

Current developer and maintaner

Mikhail Grigorev

Authors of original version

Alexey Lesovsky

License

BSD-3. See LICENSE for more details.

pgscv's People

Contributors

Stargazers

Forkers

sstolbov dbulashev

pgscv's Issues

Segmentation Fault

Describe the bug
PGSCV is running in a docker container (packaged inside our Kubernetes cluster), it already access several POSTGRESQL servers.
upon adding a new server, the PGSCV process started to crash when trying to execute command with the monitoring user on the server.

Others scrape endpoints are running normally if the buggy one is removed

Steps to reproduce

apiVersion: v1
kind: ConfigMap
metadata:
  name: pgscv-config
data:
  PGSCV_CONFIG_FILE: /opt/pgscv/pgscv.yaml
  pgscv.yaml: |
    services:
      "SERVER-1":
        service_type: "postgres"
        conninfo: "postgres://postgres_exporter:<PLACEHOLDER>@10.10.10.1:5432/postgres"
      "SERVER-2":
        service_type: "postgres"
        conninfo: "postgres://postgres_exporter:<PLACEHOLDER>@10.10.10.2:5432/postgres"
      "SERVER-3":
        service_type: "postgres"
        conninfo: "postgres://postgres_exporter:<PLACEHOLDER>@10.10.10.3:5432/postgres"
    listen_address: "0.0.0.0:9890"

"registered new service [system:0]"
"registered new service [SERVER-1]"
"registered new service [SERVER-2]"
"registered new service [SERVER-3]"
"listen on http://0.0.0.0:9890"}

the process starts normally but then it rapidelly crashes

Expected behavior
No crash

pgSCV startup options
No special configuration, /metrics is exposed on 0.0.0.0

Errors and Logs

{"level":"warn","service":"pgscv","time":"2024-07-29T13:37:21Z","message":"read 'kernel.sched_migration_cost_ns' failed: open /proc/sys/kernel/sched_migration_cost_ns: no such file or directory; skip"} |  
-- | --
  |   | {"level":"warn","service":"pgscv","time":"2024-07-29T13:37:21Z","message":"pg_stat_statements not found, skip collecting statements metrics"} |  
  |   | {"level":"warn","service":"pgscv","time":"2024-07-29T13:37:21Z","message":"pg_stat_statements not found, skip collecting statements metrics"} |  
  |   | {"level":"error","service":"pgscv","time":"2024-07-29T13:37:21Z","message":"postgres/archiver collector failed; ERROR: permission denied for function pg_ls_archive_statusdir (SQLSTATE 42501)"} |  
  |   | {"level":"warn","service":"pgscv","time":"2024-07-29T13:37:22Z","message":"get in-flight temp files failed: ERROR: permission denied for function pg_ls_tmpdir (SQLSTATE 42501); skip"} |  
  |   | panic: runtime error: invalid memory address or nil pointer dereference |  
  |   | [signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x9e0ee8] |  
  |   |   |  
  |   | goroutine 157 [running]: |  
  |   | github.com/cherts/pgscv/internal/collector.parsePostgresTempFileInflght(0x0) |  
  |   | /app/internal/collector/postgres_storage.go:189 +0x48 |  
  |   | github.com/cherts/pgscv/internal/collector.(*postgresStorageCollector).Update(0xc000356e08, {{0xc000012110, 0x8}, {0xc0000380a0, 0x48}, {0x0, 0x0}, 0x0, {0x0, 0x2000, ...}, ...}, ...) |  
  |   | /app/internal/collector/postgres_storage.go:126 +0x132 |  
  |   | github.com/cherts/pgscv/internal/collector.collect({0xb9de05, 0x10}, {{0xc000012110, 0x8}, {0xc0000380a0, 0x48}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...) |  
  |   | /app/internal/collector/collector.go:243 +0x7b |  
  |   | github.com/cherts/pgscv/internal/collector.PgscvCollector.Collect.func1({0xb9de05?, 0x0?}, {0xc9e600?, 0xc000356e08?}) |  
  |   | /app/internal/collector/collector.go:207 +0x66 |  
  |   | created by github.com/cherts/pgscv/internal/collector.PgscvCollector.Collect in goroutine 38 |  
  |   | /app/internal/collector/collector.go:206 +0x2ec

Environment :

Docker - Kubernetes v1.28.9+rke2r1
pgscv v0.8.7
PostgreSQL 12.18 (other working postgres are on the same version)
All services are distant

**Additional Information **
The bug appears only on this specific server, other server running same ROLE configuration / OS version / PostgreSQL version, aren't affected.
It certainly is a configuration problem on my side but i cannot see any link, maybe you already encountered this error ?

Fix tests

$ LC_ALL=C make test
go mod download
go env -w GOFLAGS="-buildvcs=false"
golangci-lint run --timeout 5m -E golint -e '(struct field|type|method|func) [a-zA-Z`]+ should be [a-zA-Z`]+'
WARN The linter 'golint' is deprecated (since v1.41.0) due to: The repository of the linter has been archived by the owner. Replaced by revive. 
ERRO [linters_context] golint: This linter is fully inactivated: it will not produce any reports. 
make: *** [Makefile:55: lint] Error 7

Configuring additional features

Is your feature request related to a problem? Please describe.
I see some errors but they don't block the scraping

permission denied for function pg_ls_tmpdir
permission denied for function pg_ls_archive_statusdir
pg_stat_statements not found

It would be also nice to be able to disable scrapping localhost (in my configuration, pgscv is just a docker scraping distant postgreSQL, i have nothing to scrap locally)

"message":"registered new service [system:0]"

Describe the solution you'd like
some configuration / documentation about these features

Describe alternatives you've considered
Ignoring error messages, i don't know if they prevent me from having some metrics but i didn't go that far with the tool yet.
It's just spamming the errors in the error / war loglevel

Additional context
in my configuration, pgscv is just a docker scraping distant postgreSQL.
pgscv is packaged inside a kubernetes cluster (i would like to propose a helm chart in the future as well)