Git Product home page Git Product logo

injective-price-oracle's Introduction

injective-price-oracle

Injective's Oracle with dynamic price feeds. Allows anyone to start their own pre-approved price submission process to the oracle module on the Injective Chain.

Getting Started

First, make sure your Go env is correctly configured, download a distribution from https://go.dev/dl/ the minimum required version is 1.17.

Clone this repo:

git clone [email protected]:InjectiveLabs/injective-price-oracle-ext.git
cd injective-price-oracle-ext
make install

After binary has been built, it should be available for usage in CLI:

$ injective-price-oracle version

Version dev (77a1017)
Compiled at 20220203-2207 using Go go1.17.3 (arm64)

Environment

Before starting the injective-price-oracle service, make sure you copy a template .env file and fill the correct values, especially for the Cosmos keys. You can use latest injectived release to manage the keyring dir. It's never recommended to use private key as plaintext in ORACLE_COSMOS_PK, except for testing.

More info on private keys: A Guide to Private Key Management (Oracle)

cp .env.example .env

Running with dynamic feeds via binary

This is an example that loads all dynamic feeds from examples/ dir! Make sure to specify correct path to your TOML dir.

$ injective-price-oracle start --dynamic-feeds examples


INFO[0000] using Cosmos Sender inj128jwakuw3wrq6ye7m4p64wrzc5rfl8tvwzc6s8
INFO[0000] waiting for GRPC services
INFO[0001] found 1 dynamic feed configs
INFO[0001] got 1 enabled price feeds                     svc=oracle
INFO[0001] initialized 1 price pullers                   svc=oracle
INFO[0001] starting pullers for 1 feeds                  svc=oracle
INFO[0007] PullPrice (pipeline run) done in 1.0506275s   dynamic=true provider=binance_v3 svc=oracle ticker=INJ/USDT
INFO[0014] sent Tx in 2.72982375s                        batch_size=1 hash=1D7D02BDBAEC200BD585E90215459E93C760A1317EFF9D83B822FA4F34AD6A03 svc=oracle timeout=true
INFO[0067] PullPrice (pipeline run) done in 314.4035ms   dynamic=true provider=binance_v3 svc=oracle ticker=INJ/USDT
INFO[0073] sent Tx in 1.706471708s                       batch_size=1 hash=6E3A6C8F7706DB0B0355C5691A628A56CD5A87BB14877D2F0D151178FCF2784A svc=oracle timeout=true
INFO[0128] PullPrice (pipeline run) done in 310.32875ms  dynamic=true provider=binance_v3 svc=oracle ticker=INJ/USDT
INFO[0133] sent Tx in 1.776902583s                       batch_size=1 hash=29D615079A891F25E5ADE167E78D478F8AA99CEEFED7DB47B3F5E71BFEDEB582 svc=oracle timeout=true

Running with dynamic feeds via docker-compose

  1. Docker-compose file
version: '3.8'
networks:
  injective:
    name: injective
services:
  injective-price-oracle:
    container_name: injective-price-oracle
    image: public.ecr.aws/l9h3g6c6/injective-price-oracle:prod
    build: ../../../injective-price-oracle/
    command: injective-price-oracle start --dynamic-feeds /root/oracle-feeds
    logging:
      driver: journald
    environment:
      # log config
      ORACLE_ENV: prod
      ORACLE_LOG_LEVEL: info
      # chain config
      ORACLE_SERVICE_WAIT_TIMEOUT: "1m"
      ORACLE_COSMOS_CHAIN_ID: injective-1
      ORACLE_COSMOS_GRPC: tcp://sentry0.injective.network:9900
      ORACLE_TENDERMINT_RPC: http://sentry0.injective.network:26657
      ORACLE_COSMOS_GAS_PRICES: 500000000inj
      ORACLE_DYNAMIC_FEEDS_DIR:
      # keyring config
      ORACLE_COSMOS_KEYRING: file
      ORACLE_COSMOS_KEYRING_DIR: /root/keyring-oracle
      ORACLE_COSMOS_KEYRING_APP: injectived
      ORACLE_COSMOS_FROM: oracle-user
      ORACLE_COSMOS_FROM_PASSPHRASE: 12345678
      ORACLE_COSMOS_PK:
      ORACLE_COSMOS_USE_LEDGER: "false"
      # You can pass variables from env here into specific integrations,
      # make sure to suport that in the source code.
      # ORACLE_BINANCE_URL=
      # statsd config
      ORACLE_STATSD_PREFIX: "inj-oracle"
      ORACLE_STATSD_ADDR: host.docker.internal:8125
      ORACLE_STATSD_STUCK_DUR: 5m
      ORACLE_STATSD_MOCKING: "false"
      ORACLE_STATSD_DISABLED: "false"
    networks:
      - injective
    volumes:
      - ~/keyring-oracle:/root/keyring-oracle
      - ~/docker-volume/oracle-feeds:/root/oracle-feeds
  1. Start and get logs
docker-compose up -d injective-price-oracle
docker logs injective-price-oracle

Adding new feeds

There are two ways to add new feeds.

Dynamic feeds (TOML)

Most preferred way is to create dynamic feeds using DOT Syntax, using the Chainlink innovation in this area (see Job Pipelines).

Check out this most simple example:

provider = "binance_v3"
ticker = "INJ/USDT"
pullInterval = "1m"
observationSource = """
   ticker [type=http method=GET url="https://api.binance.com/api/v3/ticker/price?symbol=INJUSDT"];
   parsePrice [type="jsonparse" path="price"]
   multiplyDecimals [type="multiply" times=1]

   ticker -> parsePrice -> multiplyDecimals
"""

Beautiful, isn't it? The observationSource provided in DOT Syntax, while the rest of the file is a TOML config. Place these configs under any names into a special dir and start the oracle referencing the dir with --dynamic-feeds <dir>.

See the full documentation on the supported Tasks that you can use.

List of supported pipeline tasks:

  • http - docs๐Ÿ”—
  • mean - docs๐Ÿ”—
  • median - docs๐Ÿ”—
  • mode - docs๐Ÿ”—
  • sum - docs๐Ÿ”—
  • multiply - docs๐Ÿ”—
  • divide - docs๐Ÿ”—
  • jsonparse - docs๐Ÿ”—
  • any - docs๐Ÿ”—
  • ethabiencode - docs
  • ethabiencode2 - docs๐Ÿ”—
  • ethabidecode - docs๐Ÿ”—
  • ethabidecodelog - docs๐Ÿ”—
  • merge - docs๐Ÿ”—
  • lowercase
  • uppercase

More can be added if needed.

List of config fields:

  • provider - name (or slug) of the used provider, used for logging purposes.
  • ticker - name of the ticker on the Injective Chain. Used for loading feeds for enabled tickers.
  • pullInterval time duration spec in Go-flavoured duration syntax. Cannot be negative or less than "1s". Valid time units are "ns", "us" (or "ยตs"), "ms", "s", "m", "h".
  • observationSource - pipeline spec in DOT Syntax

Notes on changes:

  • http task has been changed from the Chainlink's reference, to skip allowUnrestrictedNetworkAccess option, since TOMLs are trusted in this context. Added ability to specify additional HTTP headers, since some price fetching APIs require authorization โ€“ headerMap. Usage: headerMap="{\\"x-api-key\\": \\"foobar\\"}"

Probing dynamic feeds

During development sometimes one needs to evaluate if his TOML file is correct and the pipeline specification yields a correct result. To avoid running the whole E2E flow with chain, there is a simple stateless command - probe!

Probe does the following:

  • Loads TOML and parses the pipleine
  • Created a dynamic price feed, as if it was orchestrated in the oracle
  • Tries to pull the price once, using the pipeline
  • Prints the anwer or an error

Example:

$ injective-price-oracle probe examples/dynamic_binance.toml

INFO[0000] PullPrice (pipeline run) done in 530.560708ms  dynamic=true provider=binance_v3 svc=oracle ticker=INJ/USDT
INFO[0000] Answer: 4948000

Native Go code

Yes, you can also simply fork this repo and add own native implementations of the price feeds. There is a Binance example provided in feed_binance.go. Any complex feed can be added as long as the implementation follows this Go interface:

type PricePuller interface {
	Provider() FeedProvider
	ProviderName() string
	Symbol() string
	Interval() time.Duration

	// PullPrice method must be implemented in order to get a price
	// from external source, handled by PricePuller.
	PullPrice(ctx context.Context) (price decimal.Decimal, err error)
}

injective-price-oracle's People

Contributors

achilleas-kal avatar albertchon avatar nhannamsiu avatar vinhphuctadang avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

injective-price-oracle's Issues

problems compiling

list of things that I tried

Running on Ubuntu 22.10

  1. Run injective-price-oracle docker image
    Generated keys using latest injectived

trying to init relay but getting error -

injective-price-oracle |
time="2024-02-06T16:52:29Z" level=fatal msg="failed to init
Cosmos keyring" error="could not find an entry for the key
'oracle-user' in keybase: Bytes left over in
UnmarshalBinaryLengthPrefixed, should read 10 more bytes but have
186"

  1. Compile binary from source
  • cannot compile because of missing libwasmvm_musl.a
    copy over the libwasmvm_musl.a from Dockerfile
# github.com/CosmWasm/wasmvm/internal/api
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../../lib/libwasmvm_muslc.a(num_cpus-1c9defd37b809459.num_cpus.7dcba15d214e99a8-cgu.00.rcgu.o): in function `num_cpus::linux::init_cgroups':
num_cpus.7dcba15d214e99a8-cgu.00:(.text._ZN8num_cpus5linux12init_cgroups17h0f03e9fe176cff02E+0x1168): undefined reference to `ceil'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../../lib/libwasmvm_muslc.a(cranelift_codegen-5569a6a9805983e7.cranelift_codegen.a594a1881b33da17-cgu.13.rcgu.o): in function `cranelift_codegen::ir::immediates::Ieee32::ceil':
cranelift_codegen.a594a1881b33da17-cgu.13:(.text._ZN17cranelift_codegen2ir10immediates6Ieee324ceil17hcee661c5d0270a92E+0x7): undefined reference to `ceilf'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../../lib/libwasmvm_muslc.a(cranelift_codegen-5569a6a9805983e7.cranelift_codegen.a594a1881b33da17-cgu.13.rcgu.o): in function `cranelift_codegen::ir::immediates::Ieee32::floor':

  1. Compile docker image from source
    Use docker compose from github
    Docker image runs but
 โœ” Container injective-price-oracle  Created                                                                                                                                         0.0s 
Attaching to injective-price-oracle
injective-price-oracle  | time="2024-02-08T15:03:41Z" level=warning msg="metrics init failed, will retry in 1 min" error="unsupported agent type"
injective-price-oracle  | time="2024-02-08T15:03:41Z" level=info msg="using Injective Sender inj1ylclkvwwng3g0e8d5036w9xn9czv0xlf5qw0ns"
injective-price-oracle  | time="2024-02-08T15:03:41Z" level=info msg="successfully load server TLS cert"
injective-price-oracle  | time="2024-02-08T15:03:41Z" level=fatal msg="failed to connect to daemon, is injectived running?" endpoint="tcp://sentry0.injective.network:9900" error="failed to get initial account num and seq: rpc error: code = NotFound desc = rpc error: code = NotFound desc = account inj1ylclkvwwng3g0e8d5036w9xn9czv0xlf5qw0ns not found: key not found"
injective-price-oracle exited with code 1

  1. Copy injectived into the docker image

Tried to that but alpine and the injectived don't seem to work with each other


Questions:

  1. What is the preferred platform for builds
  2. What is the Dockerfile for the image in README
  3. Any other gotchas?

The service fails to broadcast the price feed when it's null on the chain

Hey @xlab I found an issue with the price oracle.

If the price feed hasn't been submitted before on-chain and the service starts you end up with the error below. I submitted the price feed once with sdk-python using the same relayer address, it was submitted normally with no issues and then the price started broadcasting txs just fine and updating the feed.

I didn't even touch the docker container in the yaml file neither stop the process after broadcasting the tx from sdk-python. The moment the price feed was on the chain from sdk-python it started submitting txs normally.

image

time="2022-03-22T09:51:57Z" level=error msg="failed to commit msg batch: null" error="4483E42CEB308CDC5A7C9B202E103F164FF1A4A54E4F1700CE310898561C696E: tx timed out" module=sdk-go size=1 svc=cosmosClient
time="2022-03-22T09:51:57Z" level=error msg="failed to SyncBroadcastMsg" batch_size=1 error="4483E42CEB308CDC5A7C9B202E103F164FF1A4A54E4F1700CE310898561C696E: tx timed out" svc=oracle timeout=true
time="2022-03-22T09:52:07Z" level=warning msg="metrics init failed, will retry in 1 min" error="statsd init failed: dial udp: lookup host.docker.internal: no such host"
time="2022-03-22T09:52:15Z" level=info msg="PullPrice (pipeline run) done in 81.165393ms" dynamic=true provider=nftbank-estimates-v2 svc=oracle ticker=BAYC/WETH
time="2022-03-22T09:52:57Z" level=error msg="failed to commit msg batch: null" error="4483E42CEB308CDC5A7C9B202E103F164FF1A4A54E4F1700CE310898561C696E: tx timed out" module=sdk-go size=1 svc=cosmosClient
time="2022-03-22T09:52:57Z" level=error msg="failed to SyncBroadcastMsg" batch_size=1 error="4483E42CEB308CDC5A7C9B202E103F164FF1A4A54E4F1700CE310898561C696E: tx timed out" svc=oracle timeout=true
time="2022-03-22T09:53:07Z" level=warning msg="metrics init failed, will retry in 1 min" error="statsd init failed: dial udp: lookup host.docker.internal: no such host"
time="2022-03-22T09:53:15Z" level=info msg="PullPrice (pipeline run) done in 79.507951ms" dynamic=true provider=nftbank-estimates-v2 svc=oracle ticker=BAYC/WETH
time="2022-03-22T09:53:18Z" level=error msg="set price Tx error: code: 11\ncodespace: sdk\ndata: ""\nevents:\n- attributes:\n - index: false\n key: YWNjX3NlcQ==\n value: aW5qMWN5OHJwNGUyY3ptZjducnN6N2VrcWZmeXp0dnVkOXZ5cm5kem12LzA=\n type: tx\n- attributes:\n - index: false\n key: c2lnbmF0dXJl\n value: RmZzUG5TYWY3UGZ6ejh3QWVDUytnR01nVDlBbHJRWHhCYWh3OXVVdEMrWk9wQUxQM1h4anAyVFd5U3g0ODhaTkpkaDZvdEZ0V3FmTnhwb0pXL1lUdndBPQ==\n type: tx\n- attributes:\n - index: false\n key: c3BlbmRlcg==\n value: aW5qMWN5OHJwNGUyY3ptZjducnN6N2VrcWZmeXp0dnVkOXZ5cm5kem12\n - index: false\n key: YW1vdW50\n value: NDQ2MDE1MDAwMDAwMDAwaW5q\n type: coin_spent\n- attributes:\n - index: false\n key: cmVjZWl2ZXI=\n value: aW5qMTd4cGZ2YWttMmFtZzk2MnlsczZmODR6M2tlbGw4YzVsNnM1eWU5\n - index: false\n key: YW1vdW50\n value: NDQ2MDE1MDAwMDAwMDAwaW5q\n type: coin_received\n- attributes:\n - index: false\n key: cmVjaXBpZW50\n value: aW5qMTd4cGZ2YWttMmFtZzk2MnlsczZmODR6M2tlbGw4YzVsNnM1eWU5\n - index: false\n key: c2VuZGVy\n value: aW5qMWN5OHJwNGUyY3ptZjducnN6N2VrcWZmeXp0dnVkOXZ5cm5kem12\n - index: false\n key: YW1vdW50\n value: NDQ2MDE1MDAwMDAwMDAwaW5q\n type: transfer\n- attributes:\n - index: false\n key: c2VuZGVy\n value: aW5qMWN5OHJwNGUyY3ptZjducnN6N2VrcWZmeXp0dnVkOXZ5cm5kem12\n type: message\n- attributes:\n - index: false\n key: ZmVl\n value: NDQ2MDE1MDAwMDAwMDAwaW5q\n type: tx\ngas_used: "89739"\ngas_wanted: "89203"\nheight: "36905"\ninfo: ""\nlogs: []\nraw_log: 'out of gas in location: Has; gasWanted: 89203, gasUsed: 89739: out of gas'\ntimestamp: ""\ntx: null\ntxhash: A52C6EC43D2F277CFA32AD8F9B5BA54D3F52A52ABC70CDB73F2F3155D7A85341\n" batch_size=1 err_code=11 hash=A52C6EC43D2F277CFA32AD8F9B5BA54D3F52A52ABC70CDB73F2F3155D7A85341 svc=oracle timeout=true
time="2022-03-22T09:54:08Z" level=warning msg="metrics init failed, will retry in 1 min" error="statsd init failed: dial udp: lookup host.docker.internal: no such host"
time="2022-03-22T09:54:15Z" level=info msg="PullPrice (pipeline run) done in 85.853405ms" dynamic=true provider=nftbank-estimates-v2 svc=oracle ticker=BAYC/WETH
time="2022-03-22T09:54:18Z" level=info msg="sent Tx in 921.88306ms" batch_size=1 hash=95B3026EE52E4B0AFB73E0960E687638273233F032438C4930E9B4DED49CCA63 svc=oracle timeout=true
time="2022-03-22T09:55:08Z" level=warning msg="metrics init failed, will retry in 1 min" error="statsd init failed: dial udp: lookup host.docker.internal: no such host"
time="2022-03-22T09:55:15Z" level=info msg="PullPrice (pipeline run) done in 87.577109ms" dynamic=true provider=nftbank-estimates-v2 svc=oracle ticker=BAYC/WETH
time="2022-03-22T09:55:19Z" level=info msg="sent Tx in 1.89983051s" batch_size=1 hash=C9F48D77C811E63BD7D6FC2CB3AEE646A32C91B0B99FB8033DE9B7962597CEAC svc=oracle timeout=true

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.