Git Product home page Git Product logo

cosmos-validators-exporter's Introduction

cosmos-validators-exporter

Latest release Actions Status codecov

cosmos-validators-exporter is a Prometheus scraper that fetches validators stats from an LCD server exposed by a fullnode.

What can I use it for?

You can use it for building a validator(s) dashboard(s) to visualize your validators metrics (like total staked amount, delegators count, etc.) as well as building alerts for your validators' metrics (like validator status/ranking/total staked amount/etc.)

Here's an example of the dashboard we're using in production:

Validators dashboard

How can I set it up?

First, you need to download the latest release from the releases page. After that, you should unzip it, and you are ready to go:

wget <the link from the releases page>
tar xvfz <filename you just downloaded>
./cosmos-validators-exporter

Alternatively, you can build from source:

git clone https://github.com/QuokkaStake/cosmos-validators-exporter
cd cosmos-validators-exporter
# This will produce a binary at ./cosmos-validators-exporter.
make build
# This will produce a binary at $GOPATH/bin/cosmos-validators-exporter.
make install

To run it detached, you need to run it as a systemd service. First, we have to copy the file to the system apps folder:

sudo cp ./cosmos-validators-exporter /usr/bin

Then we need to create a systemd service for our app:

sudo nano /etc/systemd/system/cosmos-validators-exporter.service

You can use this template (change the user to whatever user you want this to be executed from. It's advised to create a separate user for that instead of running it from root):

[Unit]
Description=Cosmos Validators Exporter
After=network-online.target

[Service]
User=<username>
TimeoutStartSec=0
CPUWeight=95
IOWeight=95
ExecStart=cosmos-validators-exporter --config <path to config>
Restart=always
RestartSec=2
LimitNOFILE=800000
KillSignal=SIGTERM

[Install]
WantedBy=multi-user.target

Then we'll add this service to autostart and run it:

sudo systemctl daemon-reload # reflect the systemd file change
sudo systemctl enable cosmos-validators-exporter # enable the scraper to run on system startup
sudo systemctl start cosmos-validators-exporter # start it
sudo systemctl status cosmos-validators-exporter # validate it's running

If you need to, you can also see the logs of the process:

sudo journalctl -u cosmos-validators-exporter -f --output cat

How can I scrape data from it?

Here's the example of the Prometheus config you can use for scraping data:

scrape-configs:
  - job_name:       'cosmos-validators-exporter'
    scrape_interval: 60s
    scrape_timeout: 60s
    static_configs:
      - targets:
        - localhost:9560 # replace localhost with scraper IP if it's on the other host

Then restart Prometheus and you're good to go!

Important: consider setting quite big intervals/timeouts, both in app config and in Prometheus config. This is due to some requests taking a lot of time, and with a shorter timeout there's a chance the whole scrape request will time out. If you face scrape errors, consider increasing the timeout.

All the metrics provided by cosmos-validators-exporter have the cosmos_validators_exporter_ as a prefix. For the full list of metrics, try running curl localhost:9560/metrics (or your host/port, if it's non-standard) and see the list of metrics there.

Queries examples

When developing, we aimed to only return metrics that are required, and avoid creating metrics that can be computed on Grafana/Prometheus side. This decreases the amount of time series that this exporter will return, but will make writing queries more complex. Here are some examples of queries that we consider useful:

  • count(cosmos_validators_exporter_info) - number of validators monitored
  • sum((cosmos_validators_exporter_total_delegations) * on (chain) group_left(denom) cosmos_validators_exporter_base_denom / on (chain, denom) cosmos_validators_exporter_denom_coefficient * on (chain, denom) cosmos_validators_exporter_price) - total delegated tokens in $
  • sum(cosmos_validators_exporter_delegations_count) - total delegators count
  • cosmos_validators_exporter_total_delegations / on (chain) cosmos_validators_exporter_tokens_bonded_total - voting power percent of your validator
  • 1 - (cosmos_validators_exporter_missed_blocks / on (chain) cosmos_validators_exporter_missed_blocks_window) - validator's uptime in %

How can I configure it?

All configuration is done via the .toml config file, which is passed to the application via the --config app parameter. Check config.example.toml for a config reference.

How can I contribute?

Bug reports and feature requests are always welcome! If you want to contribute, feel free to open issues or PRs.

cosmos-validators-exporter's People

Contributors

freak12techno avatar fossabot avatar

Stargazers

 avatar Dima K. avatar  avatar bylit avatar Nymph avatar  avatar Javed Khan avatar Greg North avatar Chris avatar

Watchers

Kostas Georgiou avatar  avatar b3rt one avatar

cosmos-validators-exporter's Issues

Dexscreener only price source for asset

We tried to use this against our Provenance config, but Provenance was removed from coingecko, so the API returns "{}" for that asset. I presumed that the price would then be taken from dexscreener but it appears as though it doesn't fail as I expected.

Config:

[[chains]]
name = "Provenance"
lcd-endpoint = "https://api.provenance.io"
base-denom = "nhash"
denoms = [
{ denom = "nhash", display-denom = "hash", coingecko-currency = "provenance", denom-coefficient = 10000000000, dex-screener-chain-id = "osmosis", dex-screener-pair-id = "693" },
]
bech-wallet-prefix = "pb"
validators = [
{ address = "pbvaloper1xnjupruwluzml987cljf225hyxfnzt6lnmacgv", consensus-address = "pbvalcons1thsssqk5lguk5295faxc9lstywg6ek5satchqm" }
]
[chains.queries]
validator = true
delegations = true
unbonds = true
self-delegation = true
validators = true
commission = true
rewards = true
balance = true
signing-info = true
slashing-params = true
staking-params = true

Return token price

Add metric with one label, chain, that will return the price of the token.
Consider removing the duplicate metrics with values in tokens/usd, as it may be calculated on client side.

Fix panic if the node is dead

1:13PM WRN Query returned bad HTTP code component=rpc status=500 url=https://api.sentinel.freak12techno.io/cosmos/staking/v1beta1/params
panic: runtime error: index out of range [-1]
goroutine 2934443 [running]:
main.(*Manager).GetValidatorRankAndTotalStake(0xc000ec6480, {{0xc0011be6f0, 0x8}, {0xc000f785a0, 0x25}, {0xc0011be710, 0x8}, {0x0, 0x0}, {0x0, ...}, ...}, ...)
	/home/monitoring/cosmos-validators-exporter/manager.go:410 +0x40a
main.(*Manager).GetAllValidators.func1.3()
	/home/monitoring/cosmos-validators-exporter/manager.go:148 +0xbe
created by main.(*Manager).GetAllValidators.func1
	/home/monitoring/cosmos-validators-exporter/manager.go:147 +0x8ce

Add total bonded metric

After that, metric with voting power % can be removed as it can be calculated as: your stake / total bonded

Fix panic when the node is not synced on querying delegations

12:51PM DBG Query is finished component=http duration=2.181219 url=https://api.jackal.quokkastake.io/cosmos/staking/v1beta1/validators/jklvaloper1ramqkapjkg0f58vypc43szaqp5juq0ly93t3hm/delegations?pagination.count_total=true&pagination.limit=1
panic: strconv.ParseInt: parsing "": invalid syntax
goroutine 876378 [running]:
main/pkg/utils.StrToInt64(...)
	/home/monitoring/cosmos-validators-exporter/pkg/utils/utils.go:29
main/pkg/queriers.(*DelegationsQuerier).GetMetrics.func1({0xc0011202c0, 0x31}, 0xc0022093e0?, {{0xc000fff8b0, 0x6}, {0xc000e7bf20, 0x21}, {0x0, 0x0}, {0xc000fff930, ...}, ...})
	/home/monitoring/cosmos-validators-exporter/pkg/queriers/delegations.go:72 +0x465
created by main/pkg/queriers.(*DelegationsQuerier).GetMetrics
	/home/monitoring/cosmos-validators-exporter/pkg/queriers/delegations.go:45 +0x2e5

Node response:

{
  "code": 2,
  "message": "runtime error: invalid memory address or nil pointer dereference: panic",
  "details": [
  ]
}

Remove old metrics that can be computed

  1. *_usd - can be computed as: (value in tokens) / on (chain, denom) denom coefficient * on (chain) price
  2. voting power percent - can be computed as: validator delegations / total delegations
  3. missed blocks % - can be computed as 1 - (missed blocks / blocks window)

Evmos + Fetch not compitable

goroutine 83 [running]:
cosmossdk.io/math.Int.Int64({0xc000236600?})
/home/runner/go/pkg/mod/cosmossdk.io/[email protected]/int.go:158 +0x86
main.(*RPC).GetSingleDelegation(0xc000236600, {0xc000fda880, 0x33}, {0xc000e26090, 0x2c})
/home/runner/work/cosmos-validators-exporter/cosmos-validators-exporter/tendermint.go:76 +0x18c
main.(*Manager).GetSelfDelegationsBalance(0xc0010dc6e0, {{0xc000fbf938, 0x5}, {0xc000e6cff0, 0x21}, {0xc000fbf960, 0x5}, {0xc000fbf968, 0x5}, 0xf4240, ...}, ...)
/home/runner/work/cosmos-validators-exporter/cosmos-validators-exporter/manager.go:297 +0x159
main.(*Manager).GetAllValidators.func1.4()
/home/runner/work/cosmos-validators-exporter/cosmos-validators-exporter/manager.go:116 +0x9d
created by main.(*Manager).GetAllValidators.func1
/home/runner/work/cosmos-validators-exporter/cosmos-validators-exporter/manager.go:115 +0x993
cosmos-validators-exporter.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
cosmos-validators-exporter.service: Failed with result 'exit-code'.
cosmos-validators-exporter.service: Scheduled restart job, restart counter is at 2.
Stopped Cosmos Validators Exporter.
Started Cosmos Validators Exporter.

Add base denom info

  1. return a metric with base denom coefficient to display denom and labels chain, base_denom, display_denom
  2. return values in base denom, no need to convert it as it can be done in query (except for price calculation)

BUG : Parsing error GOroutine

panic: strconv.ParseInt: parsing "": invalid syntax
goroutine 37 [running]:
main.StrToInt64(...)
/home/runner/work/cosmos-validators-exporter/cosmos-validators-exporter/utils.go:29
main.(*Manager).GetAllValidators.func1({0xc000560880, 0x34}, {{0xc000fea0f0, 0x6}, {0xc000d038f0, 0x23}, {0xc000fea110, 0x7}, {0xc000fea0f9, 0x4}, ...}, ...)
/home/runner/work/cosmos-validators-exporter/cosmos-validators-exporter/manager.go:237 +0x1d4b
created by main.(*Manager).GetAllValidators
/home/runner/work/cosmos-validators-exporter/cosmos-validators-exporter/manager.go:46 +0x450
cosmos-validators-exporter.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
cosmos-validators-exporter.service: Failed with result 'exit-code'.

Old version works without issues.

Add last validator stake metric

To see diff between your validator and the last one and how much tokens needed to kick you out. Should be 0 if active validators < active set size

HUAHUA chain not compitable

pt0e0gdwqxpf4lh886rgj3ht
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x38 pc=0x139ddc3]
goroutine 79 [running]:
main.(*RPC).GetSingleDelegation(0xc0005c8c80, {0xc00010c600, 0x37}, {0xc0005c29f0, 0x30})
/home/runner/work/cosmos-validators-exporter/cosmos-validators-exporter/tendermint.go:76 +0x183
main.(*Manager).GetSelfDelegationsBalance(0xc000e32840, {{0xc0005a82a0, 0x9}, {0xc000ec2840, 0x2b}, {0xc000ccd908, 0xf}, {0xc0005a82c0, 0x7}, 0xf4240, ...}, ...)
/home/runner/work/cosmos-validators-exporter/cosmos-validators-exporter/manager.go:297 +0x159
main.(*Manager).GetAllValidators.func1.4()
/home/runner/work/cosmos-validators-exporter/cosmos-validators-exporter/manager.go:116 +0x9d
created by main.(*Manager).GetAllValidators.func1
/home/runner/work/cosmos-validators-exporter/cosmos-validators-exporter/manager.go:115 +0x993

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.