Git Product home page Git Product logo

tradingstrategy-ai / web3-ethereum-defi Goto Github PK

View Code? Open in Web Editor NEW
523.0 10.0 109.0 36.52 MB

A Python library for trading automation on DeFi, data research and integration. Supporting Uniswap, Aave, Chainlink, USDC and other protocols.

Home Page: https://tradingstrategy.ai

License: MIT License

Makefile 0.35% Python 91.34% Shell 0.76% Solidity 7.54% Dockerfile 0.02%
ethereum solidity evm pytest uniswap sushiswap pancakeswap binance-smart-chain polygon python

web3-ethereum-defi's Introduction

PyPI version

Automated test suite

Documentation Status

Web3-Ethereum-Defi

Web-Ethereum-DeFi (eth_defi) Python package provides high level modules for smart contracts, with prepackaged ABI files for DeFi protocol integration, wallet management, JSON-RPC providers and automated test suites. The package aims for robustness, high quality of the code and documentation.

Supported protocols include Uniswap, Aave, others

Use cases

Use cases for this package include

  • Trading and bots
  • Data research, extraction, transformation and loading
  • Portfolio management and accounting
  • System integrations and backends

Features

Features include e.g.

Web3-Ethereum-Defi supports

  • Uniswap (both v2 and v3)
  • Sushi
  • Aave
  • Enzyme Protocol
  • dHEDGE Protocol
  • More integrations to come
  • Built-in integration for over 600 smart contracts with precompiled Solidity ABI files

Read the full API documentation). For code examples please see below.

Prerequisites

To use this package you need to

Install

With pip:

pip install "web3-ethereum-defi[data]"

With poetry:

# Poetry version
poetry add -E data web3-ethereum-defi

With poetry - master Git branch:

git clone [email protected]:tradingstrategy-ai/web3-ethereum-defi.git
cd web3-ethereum-defi
poetry shell
poetry install --all-extras

Example code

See the tutorials section in the documentation for full code examples.

PancakeSwap swap example

import os
import time
from functools import lru_cache

from web3 import HTTPProvider, Web3

from eth_defi.abi import get_contract
from eth_defi.chain import install_chain_middleware
from eth_defi.event_reader.filter import Filter
from eth_defi.event_reader.logresult import decode_log
from eth_defi.event_reader.reader import read_events, LogResult
from eth_defi.uniswap_v2.pair import fetch_pair_details, PairDetails


QUOTE_TOKENS = ["BUSD", "USDC", "USDT"]


@lru_cache(maxsize=100)
def fetch_pair_details_cached(web3: Web3, pair_address: str) -> PairDetails:
    return fetch_pair_details(web3, pair_address)


def main():
    json_rpc_url = os.environ.get("JSON_RPC_BINANCE", "https://bsc-dataseed.binance.org/")
    web3 = Web3(HTTPProvider(json_rpc_url))
    web3.middleware_onion.clear()
    install_chain_middleware(web3)

    # Read the prepackaged ABI files and set up event filter
    # for any Uniswap v2 like pool on BNB Smart Chain (not just PancakeSwap).
    #
    # We use ABI files distributed by SushiSwap project.
    #
    Pair = get_contract(web3, "sushi/UniswapV2Pair.json")

    filter = Filter.create_filter(address=None, event_types=[Pair.events.Swap])

    latest_block = web3.eth.block_number

    # Keep reading events as they land
    while True:

        start = latest_block
        end = web3.eth.block_number

        evt: LogResult
        for evt in read_events(
            web3,
            start_block=start,
            end_block=end,
            filter=filter,
        ):

            decoded = decode_log(evt)

            # Swap() events are generated by UniswapV2Pool contracts
            pair = fetch_pair_details_cached(web3, decoded["address"])
            token0 = pair.token0
            token1 = pair.token1
            block_number = evt["blockNumber"]

            # Determine the human-readable order of token tickers
            if token0.symbol in QUOTE_TOKENS:
                base = token1  # token
                quote = token0  # stablecoin/BNB
                base_amount = decoded["args"]["amount1Out"] - decoded["args"]["amount1In"]
                quote_amount = decoded["args"]["amount0Out"] - decoded["args"]["amount0In"]
            else:
                base = token0  # stablecoin/BNB
                quote = token1  # token
                base_amount = decoded["args"]["amount0Out"] - decoded["args"]["amount0Out"]
                quote_amount = decoded["args"]["amount1Out"] - decoded["args"]["amount1Out"]

            # Calculate the price in Python Decimal class
            if base_amount and quote_amount:
                human_base_amount = base.convert_to_decimals(base_amount)
                human_quote_amount = quote.convert_to_decimals(quote_amount)
                price = human_quote_amount / human_base_amount

                if human_quote_amount > 0:
                    # We define selling when the stablecoin amount increases
                    # in the swap
                    direction = "sell"
                else:
                    direction = "buy"

                price = abs(price)

                print(f"Swap block:{block_number:,} tx:{evt['transactionHash']} {direction} price:{price:,.8f} {base.symbol}/{quote.symbol}")
            else:
                # Swap() event from DEX that is not Uniswap v2 compatible
                # print(f"Swap block:{block_number:,} tx:{evt['transactionHash']} could not decode")
                pass

        else:
            # No event detected between these blocks
            print(".")

        latest_block = end
        time.sleep(1)


if __name__ == "__main__":
    main()

How to use the library in your Python project

Add web3-ethereum-defi as a development dependency:

Using Poetry:

# Data optional dependencies include pandas and gql, needed to fetch Uniswap v3 data
poetry add -D "web3-ethereum-defi[data]"

Documentation

Development and contributing

Version history

Support

Social media

License

MIT.

Created by Trading Strategy.

web3-ethereum-defi's People

Contributors

alexthelion123 avatar dxganta avatar hieuh25 avatar kennu avatar kolvir73 avatar miohtama avatar omahs avatar plamut avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

web3-ethereum-defi's Issues

Three-way trades

For DeFi trading

  • Most of quote tokens are in the blockchain native token (ETH, BNB, MATIC)
  • Most of trading happens in US dollar and stablecoins (USDC, BUSD)

To trade more pairs, we need to be able to execute three-way trades and price estimations.

Update

  • Price estimation logic to handle trades through three pools e.g. USDC -> ETH -> AAVE
  • Buy and selll through three pools e.g. USDC -> ETH -> AAVE

Make sure the MEV slippage is still prevented by setting price change tolerance to high value e.g. 99%. This can be tested by preparing a trade, then going to some of the pool and changing its content by trading and after pool has been unbalanced, trying to broadcast the transaction. It should revert.

More selective number → string conversion in `fetch_aave_reserves_shapshot`

Issue

The fetch_aave_reserves_shapshot function converts all numeric values to strings. This conflicts with the type expectations of the @aave/math-utils module.

Details

The output from fetch_aave_reserves_shapshot is made available to frontend via the lending-reserve/details API. frontend needs to convert this data into human-readable USD values using @aave/math-utils.

When calling the formatReserveUSD function from the @aave package, the following error occurs:

Error: [BigNumber Error] Argument not a primitive number: 18

This is due to the decimals value being "18" (string) instead of 18 (number). The @aave/math-utils conversion functions fail when the wrong type is supplied.

Reviewing the ReserveData interface defined by @aave/math-utils, it appears the following properties should preserve the number type:

  • decimals
  • stableDebtLastUpdateTimestamp
  • lastUpdateTimestamp
  • eModeCategoryId
  • debtCeilingDecimals
  • eModeLtv
  • eModeLiquidationThreshold
  • eModeLiquidationBonus

Other numeric values are expected to be strings in order to retain precision.

Get Aave lending and borrow rates directly from on-chain

Aave has deployed its v3 version on Ethereum mainnet and Polygon.

See Ethereum markets - MetaMask required

image

The market has two sides

  • What lenders receive as a payment, called supply APR
  • What borrowers must pay for having a loan, called borrow APR

Business use case

  • Aave lending markets can be used to build a short position of tokens
  • By knowing how much borrowing costs, we know what is the cost of maintaining a short position

We need to be able to answer the questions

  • How much having a loan for a token e.g. ETH would have cost us historically

    • Opening amoun
    • Opening date
    • Closing date
    • Interest payments accumulated (how to calculate?)
  • How much having a loan for a token would cost us right now (current interest rate)

Note that the interest rate is paid in the pool native token. E.g. ETH will accrue ETH interest and needs to be converted to US Dollar in some point.

Task

  • Create a proof of concept Aave lending and borrow rate fetcher in Python
  • It must be a self-contained notebook similar to Uniswap v3 price example
  • Study how Aave v3 lending and borrow rates are reported. The rates are variable over time.
  • Make short README documentation of available events
  • Use a event reader Python module to read this data to a CSV file
  • Plot a graph using Plotly for the lending and borrow rate of one market, e.g. ETH
  • The graph matches what we have on the Aave website

References

Aave displays borrow rate on its main UI:

image

Aave is internally displaying this information on its website using RatesHistory API.

Aave internal MongoDB schema is here. It is using a cron job to update rates by reading them from Subgraph.

Aave Subgraph code is an undocumented mess.

The rate is probably determined by this event. See GithubSearch for ReserveDataUpdated.

Ankr specific timestamp reader

Block timestamps are needed for mass event ingestion with eth_getLogs. We need to map block number -> timestamp.

Currently we offer two ways to do it

  • extract_timestamp_json_rpc - the naive method
    • Calls eth_getBlockByNumber for the range of all blocks (start, end) we need to get events from
    • Very slow, because to read just one uint64 timestamp avg. eth_getBlockByNumber returns 8 MB data and we often need to read 100 - 1,000 blocks once
    • Also often breaks QuickNode, because their API throttling limits is low
    • However it works on any EVM node
  • GraphQL-based timestamp reading
    • Reads all timestamps for a block range in a single call... you can get 1,000s of timestamps with a single API request
    • Uses GoEthereum standard /grapql endpoint
    • Best for in-house hosted nodes
    • Does not work with any commercial SaaS node provider because they do not support /graphql

To be able to have commercial SaaS node providers as a fallback option for JSON-RPC, we need to be able to read block timestamps so that their nodes do not complain.

  • Commercial SaaS vendors provide their incompatible proprietary APIs for tasks like reading multiple block headers
  • This is exactly like of proprietary low-quality APIs you can think of

As first commercial SaaS vendor to pilot out we'd like to try Ankr
- ank_getBlocks proprietary API can read blocks in batch up to 20 blocks at a time
- Ankr claims to support up to API 1500 request/minute
- Ankr offer pay-as-you-go plan so it is easy get started with them

For the task

  • Create reader/ankr.py module
  • Create extract_timestamps_ankr_get_block method
  • Create has_ankr_support method in eth_defi.chain
  • Create AnkrReorganisationMonitor that is similar to GraphQLReorganisationMonitor - using the underlying extract_timestamps_ankr_get_block
  • Add a compatibility check in create_reorganisation_monitor that will automatically create AnkrReorganisationMonitor if we are connected to Ankr node
  • Create an integration test that take JSON_RPC_ANKR_PRIVATE as the environment variable input to test these against a live Ankr plan

Paint it black

Everything else but please let's have unlimited line lengths.

Handle chain reorg in event reader

Current the event reader can't handle chain reorg situation, so it's likely to fail here:

if extract_timestamps is not None:
timestamps = extract_timestamps(web3, start_block, end_block)
for log in logs:
block_hash = log["blockHash"]
block_number = int(log["blockNumber"], 16)
# Retrofit our information to the dict
event_signature = log["topics"][0]
log["context"] = context
log["event"] = filter.topics[event_signature]
try:
log["timestamp"] = timestamps[block_hash] if extract_timestamps else None
except KeyError as e:
raise RuntimeError(f"Timestamp missing for block number {block_number:,}, hash {block_hash}, our timestamp table has {len(timestamps)} blocks") from e

Something wierd with the log

I run the uniswap-v2-swaps-live.py, and a problem occurs to me.

The filtered log is:
{'address': '0x3e13f23bbc3fb4c7ea5bc5421344a163a0d00b0c', 'topics': ['0xd78ad95fa46c994b6551d0da85fc275fe613ce37657fb8d5e3d130840159d822', '0x00000000000000000000000007c4b115ddc6cc66526394629c22dda907795e57', '0x0000000000000000000000006fd80b18d73cd2f5f972b3c2829456cc16d6d543'], 'data': '0x00000000000000000000000000000000000000000000000002e75474d14644000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000002e75474d1464400', 'blockNumber': 17975651, 'transactionHash': '0x583f6a5890883dd69bc4713b00bf6697b0cac7a6c3445e589a7ad24ada6be595', 'transactionIndex': '0x47', 'blockHash': '0x081be90f9e290f9b670d9fa057185b1abe62da96c610ae08a566a97867a5e887', 'logIndex': '0x94', 'removed': False, 'context': None, 'event': <class 'web3._utils.datatypes.Swap'>, 'chunk_id': 17975650, 'timestamp': 1692772487}

And this should be a uniswap v2 swap log
'topics': ['0xd78ad95fa46c994b6551d0da85fc275fe613ce37657fb8d5e3d130840159d822'

But the weird thing is that the address is a ERC20 address:
'address': '0x3e13f23bbc3fb4c7ea5bc5421344a163a0d00b0c'

Then this script got an error, because it try to decode it as an uniswap v2 contract.

Why the address in this log is ERC20 contract address, not its pool address?

Support Uniswap v3 swap

Add new swap function to eth_defi/uniswap_v3 package, ideally the function would be similar as the one in uniswap_v2

Organize all contracts and submodules to their own holder

Currently we have

  • contracts
  • sushiswao
  • uniswap_v3/*

To make everything tidy and understandable, move everything under one common folder contracts

  • contracts/in-house
  • contracts/sushiswap
  • contracts/uniswap-v3-core
  • contracts/uniswap-v3-periheliaps

Error when loading last N blocks using JSONRPCReorganisationMonitor

How do you only load the last N blocks? This code wants to loads only last 5 blocks i believe, however it errors when adding new blocks.

    reorg_mon = JSONRPCReorganisationMonitor(web3, check_depth=30)

    reorg_mon.load_initial_block_headers(block_count=5)

    while True:
        try:
            # Figure out the next good unscanned block range,
            # and fetch block headers and timestamps for this block range
            chain_reorg_resolution = reorg_mon.update_chain()

            if chain_reorg_resolution.reorg_detected:
                logger.info(f"Chain reorganisation data updated: {chain_reorg_resolution}")

            # Read specified events in block range
            for log_result in read_events(
                web3,
                start_block=chain_reorg_resolution.latest_block_with_good_data + 1,
                end_block=chain_reorg_resolution.last_live_block,
                filter=filter,
                notify=None,
                chunk_size=100,
                context=token_cache,
                extract_timestamps=None,
                reorg_mon=reorg_mon,
            ):
               pass
INFO:eth_defi.event_reader.reorganisation_monitor:figure_reorganisation_and_new_blocks(), range 17,285,423 - 17,285,443, last block we have is 17,285,443, check depth is 20
ERROR: LoadError: Python: AssertionError: Blocks must be added in order. Last block we have: 17285443, the new record is: BlockHeader(block_number=17285423, block_hash='0x8d481922bd607150c9f3299004a113e44955327770ab04ed10de115e2172d6fe', timestamp=1684400615)
Python stacktrace:
 [1] add_block
   @ eth_defi.event_reader.reorganisation_monitor ~/Library/Caches/pypoetry/virtualenvs/cryptopy-5siZoxZ4-py3.10/lib/python3.10/site-packages/eth_defi/event_reader/reorganisation_monitor.py:324
 [2] figure_reorganisation_and_new_blocks
   @ eth_defi.event_reader.reorganisation_monitor ~/Library/Caches/pypoetry/virtualenvs/cryptopy-5siZoxZ4-py3.10/lib/python3.10/site-packages/eth_defi/event_reader/reorganisation_monitor.py:396
 [3] update_chain

Websocket support

Hi!
I was wondering whether web3-ethereum-defi supports websocket RPCs.
These would be useful to listen to events in a faster way than using HTTP polling.
For example, it would be great to use wss to listen to live swaps.
Thanks,
Cocco

fix eth_defi uniswap_v2 tests

Two tests, test_get_amount_in and test_get_amount_out, located in test_uniswap_v2_price.py in eth_defi are currently failing. This is because the tests require some argument called mocker which is not defined

how to use analyse_trade_by_hash()?

Below are my codes., but I don't know how to set uniswap_v2 = UniswapV2Deployment(pancakeV2ContractAddress, pancakeV2ContractAbi). How should I set? It gives me error:

TypeError: UniswapV2Deployment.init() missing 4 required positional arguments: 'weth', 'router', 'init_code_hash', and 'PairContract'

import json

from web3 import Web3
from eth_defi.uniswap_v2.analysis import analyse_trade_by_hash
from eth_defi.uniswap_v2.deployment import UniswapV2Deployment

BSC_RPC = "https://bsc-dataseed.binance.org:443"
web3 = Web3(Web3.HTTPProvider(BSC_RPC))

# Initialize contract ABI
with open("pancakeV2ContractAbi.json") as file:
    pancakeV2ContractAbi = json.load(file)
with open("pancakeV3ContractAbi.json") as file:
    pancakeV3ContractAbi = json.load(file)

# pancake contract
# Initialize contract addresses
pancakeV2ContractAddress = "0x10ed43c718714eb63d5aa57b78b54704e256024e"
pancakeV3ContractAddress = '0x13f4ea83d0bd40e75c8222255bc855a974568dd4'

tx_hash = '0x4c95f90c0d5faace72da410d7602271bd1f503373627e65cafc88a6c18cf095c'

uniswap_v2 = UniswapV2Deployment(pancakeV2ContractAddress, pancakeV2ContractAbi)

analysis = analyse_trade_by_hash(web3, uniswap_v2, tx_hash)
print("Swapped in", analysis.amount_in)
print("Swapped out", analysis.amount_out)

Avalanche Node

Install Avalanche node on mihailo, we will have to change the name.

autosummary docs to gitignore

Currently autosummary docs are generated, but they still get committed to the repo.

  • Add to .gitignore.
  • Remove from the existing git tree
  • Check that readthedocs will still correctly build the docs

Aave deposit function

  • Write a function that opens a loan position in Aave v3 by depositing any Aave v3 reserve token and receives aToken back
  • The function can live in eth_defi.aave_v3.loan module
    - The module should be modelled similar as e.g. Uniswap v2 swap_with_slippage_protection and deploy_trading_pair
    - Function name: deposit_in_aave
    - Inputs: HotWallet instance (assumed loaded with USDC), aave_deposit_address, token_address, amount
    - Outputs: tx_hash
  • The module must be sufficiently documented for autodoc according to the coding conventions
  • The code must be formatted to according to the coding conventions - there is is black that complains on open PRs if this is not the case
  • There must be integration test
    • Because there is no framework to set up Aave v3 smart contracts in Ethereum Tester environment, one must use Ganache mainnet fork for the test. Normally this is unpreferable, but creating Aave v3 test environment itself is large task.
    • There are some examples of using Ganache in tests in test_ganache and test_token_tax
    • The test can be called test_aave_deposit.py
    • A test case check that the 0) Hot wallet starts with USDC on Ethereum mainnet 1) deposit was correctly registered with the Aave reserves 2) you receive the corresponding Aave aUSDC token back in the wallet, with the correct amount
    • Because there is no automatic mechanism to fetch Aave reserves and addresses as a list, please use hardcoded and commented values as text fixtures for now for any Ethereum address, with links to their respective descriptions of what they are

Better tracking for Uniswap v3 price estimation

  • Currently estimate_sell_received_amount (and co) return raw amount

To track the slippage better, we should also return

  • Block number
  • Mid-price at the time of the reading

This allows us to post-mortem diagnose why slippage tolerance was exceeded

Investigate Uniswap v3 and getting historical price data in good format

  • Github and other searches to find Python (and JavaScript) open source examples for getting data out from Uniswap v3
  • Data structure how to store this data in the (SQL) database for historical queries

Question that needs to be answered:

  • Given (chain id, Uniswap v3 deployment address)
  • What is the historical price for buy/sell (timestamp, trading pair, token0 quantity in) -> token1 quantity out

Can be a single pool price at milestone 1, later expanded to cover auto routing and multi-hop trades.

Read token tax from a token

Some tokens with ponzinomis have a token tax. This is especially popular on wild west blockchains like BNB Chain and PancakeSwap.

Token tax is used to

  • Create deflanatory tokens
  • Create malicious honey pots for trading bots ("buy only") - effectively a very high tokex tax like 90% on sell

An example trading pair and token with token tax is ELEPHANT-BUSD.

  • Buy has 10% tax
  • Sell has 10% tax

image

honeypot.is is an independent service to check the token tax. It does this by (likely) running Ganache mainnet fork and simulates the transactions. There is no API to ask this information.

Step 1: Read the token tax with Python

To figure out the amount of different token taxes (buy, sell, transfer) one needs to run a simulated transaction in Ganache.

Here is a pseudo-Python code to do it:

from eth_account.signers.local import LocalAccount
from eth_typing import HexAddress
from web3 import Web3

from eth_defi.uniswap_v2.deployment import UniswapV2Deployment
from tradeexecutor.utils import dataclass


@dataclass
class TokenTaxInfo:
    """Different token taxes we figured out."""

    #: Token in the question
    base_token: HexAddress

    #: Which token we traded against it
    quote_token: HexAddress

    #: How much % we lost of the token on buy
    buy_tax: float

    #: How much % we lose the token when we transfer between addresses
    transfer_tax:float

    #: How much % we lose the token when we sold it
    sell_tax: float


def estimate_token_taxes(
        uniswap: UniswapV2Deployment,
        base_token: HexAddress,
        quote_token: HexAddress,
        buy_account: LocalAccount,
        sell_account: LocalAccount
    ) -> TokenTaxInfo:
    """Estimates different token taxes for a token by running Ganache simulations for it.

    :param uniswap:
        Uniswap deployment on a Ganache mainnet fork.
        Set up prior calling this function.
        See `ganache.py` and `test_ganache.py` for more details.

    :param base_token:
        The token of which tax properties we are figuring out.

    :param quote_token:
        Address of the quote token used for the trading pair. E.g. `BUDS`, `WBNB`
        Based on this information we can derive Uniswap trading pair address.

    :param buy_account:
        The account that does initial buy to measure the buy tax.
        This account must be loaded with gas money (ETH/BNB) and `quote_token`
        for a purchase.

    :param sell_account:
        The account that receives the token transfer and does the sell to measure the sell tax.
        This account must be loaded with gas money for the sell.

    :return:
        ToxTaxInfo tells us what we figure out about taxes.
        This can be later recorded to a database.
    """

    web3: Web3 = uniswap.web3

    # Figure out base_token/quote_token trading pair
    # Buy base_token with buy_account
    # Measure the loss as "buy tax"
    # Transfer tokens to sell_account
    # Measure the loss as "transfer tax"
    # Sell tokens
    # Measure the loss as "sell tax"

The TokenTaxInfo info can be then stored in a database or similar, like SQL, JSON file repo and so on.
It can be later retrieved when you want to trade tokens.

Step 2: Making taxed tokens easily tradeable with Python

Create a buy/sell_with_tax(token_tax_info: TokenTaxInfo, max_slippage) function that considers slippage and token tax and correctly handles trading these kinds of tokens.

  • Normally buying ELELPHANT-BUSD would revert
  • Normally selling a token tax token would fail with execution reverted: TransferHelper: TRANSFER_FROM_FAILED because there is a mismatch between allowance() and swap amount

Create four test based on BSC mainnet fork using the existing Ganache fixtures

  • Naive buy of ELEPHANT fails
  • Taxed buy of ELEPHANT success, we can calculate the taxed amount after the buy (update analyse_trade to return this value)
  • Naive sell of ELEPHANT fails
  • Taxed sell of ELEPHANT success, we can calculate the taxed amount after the sell
  • Comment how this can be also considered in three-way trade (BUSD-BNB-ELEPHANT)

BSC and "the method eth_feeHistory does not exist/is not available"

web3.py does not get BSC fee automatically

Full traceback:

root                                               ERROR    {'code': -32601, 'message': 'the method eth_feeHistory does not exist/is not available'}
Traceback (most recent call last):
  File "/root/.cache/pypoetry/virtualenvs/tradeexecutor-B9xoSur2-py3.9/lib/python3.9/site-packages/web3/eth.py", line 631, in max_priority_fee
    return self._max_priority_fee()
  File "/root/.cache/pypoetry/virtualenvs/tradeexecutor-B9xoSur2-py3.9/lib/python3.9/site-packages/web3/module.py", line 57, in caller
    result = w3.manager.request_blocking(method_str,
  File "/root/.cache/pypoetry/virtualenvs/tradeexecutor-B9xoSur2-py3.9/lib/python3.9/site-packages/web3/manager.py", line 198, in request_blocking
    return self.formatted_response(response,
  File "/root/.cache/pypoetry/virtualenvs/tradeexecutor-B9xoSur2-py3.9/lib/python3.9/site-packages/web3/manager.py", line 171, in formatted_response
    raise ValueError(response["error"])
ValueError: {'code': -32601, 'message': 'the method eth_maxPriorityFeePerGas does not exist/is not available'}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/root/trade-executor/trade-executor/tradeexecutor/cli/main.py", line 168, in start
    run_main_loop(
  File "/root/trade-executor/trade-executor/tradeexecutor/cli/loop.py", line 141, in run_main_loop
    runner.tick(ts, universe, state, debug_details)
  File "/root/trade-executor/trade-executor/tradeexecutor/strategy/runner.py", line 116, in tick
    self.execution_model.execute_trades(clock, state, approved_trades)
  File "/root/trade-executor/trade-executor/tradeexecutor/ethereum/uniswap_v2_execution.py", line 73, in execute_trades
    approvals = approve_tokens(
  File "/root/trade-executor/trade-executor/tradeexecutor/ethereum/execution.py", line 180, in approve_tokens
    tx = token.functions.approve(
  File "/root/.cache/pypoetry/virtualenvs/tradeexecutor-B9xoSur2-py3.9/lib/python3.9/site-packages/web3/contract.py", line 1079, in buildTransaction
    return build_transaction_for_function(
  File "/root/.cache/pypoetry/virtualenvs/tradeexecutor-B9xoSur2-py3.9/lib/python3.9/site-packages/web3/contract.py", line 1648, in build_transaction_for_function
    prepared_transaction = fill_transaction_defaults(web3, prepared_transaction)
  File "cytoolz/functoolz.pyx", line 250, in cytoolz.functoolz.curry.__call__
  File "/root/.cache/pypoetry/virtualenvs/tradeexecutor-B9xoSur2-py3.9/lib/python3.9/site-packages/web3/_utils/transactions.py", line 114, in fill_transaction_defaults
    default_val = default_getter(web3, transaction)
  File "/root/.cache/pypoetry/virtualenvs/tradeexecutor-B9xoSur2-py3.9/lib/python3.9/site-packages/web3/_utils/transactions.py", line 64, in <lambda>
    web3.eth.max_priority_fee + (2 * web3.eth.get_block('latest')['baseFeePerGas'])
  File "/root/.cache/pypoetry/virtualenvs/tradeexecutor-B9xoSur2-py3.9/lib/python3.9/site-packages/web3/eth.py", line 637, in max_priority_fee
    return fee_history_priority_fee(self)
  File "/root/.cache/pypoetry/virtualenvs/tradeexecutor-B9xoSur2-py3.9/lib/python3.9/site-packages/web3/_utils/fee_utils.py", line 45, in fee_history_priority_fee
    fee_history = eth.fee_history(*PRIORITY_FEE_HISTORY_PARAMS)  # type: ignore
  File "/root/.cache/pypoetry/virtualenvs/tradeexecutor-B9xoSur2-py3.9/lib/python3.9/site-packages/web3/eth.py", line 863, in fee_history
    return self._fee_history(block_count, newest_block, reward_percentiles)
  File "/root/.cache/pypoetry/virtualenvs/tradeexecutor-B9xoSur2-py3.9/lib/python3.9/site-packages/web3/module.py", line 57, in caller
    result = w3.manager.request_blocking(method_str,
  File "/root/.cache/pypoetry/virtualenvs/tradeexecutor-B9xoSur2-py3.9/lib/python3.9/site-packages/web3/manager.py", line 198, in request_blocking
    return self.formatted_response(response,
  File "/root/.cache/pypoetry/virtualenvs/tradeexecutor-B9xoSur2-py3.9/lib/python3.9/site-packages/web3/manager.py", line 171, in formatted_response
    raise ValueError(response["error"])
ValueError: {'code': -32601, 'message': 'the method eth_feeHistory does not exist/is not available'}

Most things do not work?

Hi,

I've tried working with this library but to me seems like most things do not work.

I'll just provide extra information

I am working on Windows 10
Python 3.10.6

Foremost installing this lib is very difficult on Windows. It is using ethash which is the problem. To install this you have to do some manual funky stuff found here: unable to install pyethash · Issue #131 · ethereum/ethash (github.com) for whoever encounters the same.

Then I tried to do this tutorial:
https://web3-ethereum-defi.readthedocs.io/tutorials/uniswap-v3-price-analysis.html

Just scraping the data already fails:

state = JSONFileScanState("/tmp/uniswap-v3-price-scan.json") fetch_events_to_csv(json_rpc_url, state, start_block=start_block, end_block=end_block)

the fetch_events_to_csv fails from the start because it needs the files
uniswap-v3-swap.csv
uniswap-v3-poolcreated.csv
uniswap-v3-burn.csv
uniswap-v3-mint.csv

Which I think should be created automatically if not existing. After creating them manually I still had difficulties. The program was not finding them in the /tmp folder which is the default folder according to the API.

It then also failed because it could not find "uniswap-v3-price-scan.json". Initially it at least detected there was no state, then afterwards it failed without state. Then I had to look into the code to actually see what is supposed to be in this file because I can't get the code to run without this state file.

Then I found out that uniswap-v3-price-scan.json just contains the block number...

After doing that, I finally get to scrape data which gets me into the following error:

File "C:\Python310\lib\site-packages\eth_defi\uniswap_v3\events.py", line 435, in fetch_events_to_csv raise RuntimeError(f"Could not decode {log_result}") from e RuntimeError: Could not decode {'address': '0x1f98431c8ad98523631ae4a59f267346ea31f984', 'blockHash': '0xe8228e3e736a42c7357d2ce6882a1662c588ce608897dd53c3053bcbefb4309a', 'blockNumber': 12369739, 'data': '0x000000000000000000000000000000000000000000000000000000000000003c0000000000000000000000001d42064fc4beb5f8aaf85f4617ae8b3b5b8bd801', 'logIndex': '0x18', 'removed': False, 'topics': ['0x783cca1c0412dd0d695e784568c96da2e9c22ff989357a2e8b1d9b2b4e6b7118', '0x0000000000000000000000001f9840a85d5af5bf1d1762f925bdaddc4201f984', '0x000000000000000000000000c02aaa39b223fe8d0a0e5c4f27ead9083c756cc2', '0x0000000000000000000000000000000000000000000000000000000000000bb8'], 'transactionHash': '0x37d8f4b1b371fde9e4b1942588d16a1cbf424b7c66e731ec915aca785ca2efcf', 'transactionIndex': '0x21', 'context': <eth_defi.uniswap_v3.events.TokenCache object at 0x000001DBE157D9F0>, 'event': <class 'web3._utils.datatypes.PoolCreated'>, 'chunk_id': 12369721, 'timestamp': 1620157956}

Get revert reason of any tx and especially for failed trades

Uniswap trade analyzer should be able to tell why the trade failed

  • Too much slippage
  • Some internal Uniswap error
  • (There should be no other reasons if the tokens are not scam tokens)

As a bonus, trade analyzer should able to tell if the trade was reverted because of slippage. Though not sure how we can pick up this from the transaction receipt, I believe is going to be quite hard. The ”real” EVM nodes do not store the revert reason for very long time (few hundreds of blocks) and one might need to replay the transaction.

https://snakecharmers.ethereum.org/web3py-revert-reason-parsing/

This would be a research task of doing some little trades with Ganache and see what kind of data we can get out of JSON-RPC for the revert reason -if any. And then do the transaction replay trick.

Aave position management code

This is an example task for those who look to work with the project in a professional role.

  • Write a function that opens a loan position in Aave v3 by depositing any Aave v3 reserve token and receives aToken back

This includes

  • The function can live in eth_defi.aave_v3.loan module
    - The module should be modelled similar as e.g. Uniswap v2 swap_with_slippage_protection and deploy_trading_pair
    - Function name: deposit_in_aave
    - Inputs: HotWallet instance (assumed loaded with USDC), aave_deposit_address, token_address, amount
    - Outputs: tx_hash
  • You must be able to demonstrate professional open source development best practices and behavior when submitting a Github pull request
  • The module must be sufficiently documented for autodoc according to the coding conventions
  • The code must be formatted to according to the coding conventions - there is is black that complains on open PRs if this is not the case
  • There must be integration test
    • See how the existing Aave functions are tested using pytest
    • If you cannot ramp up Aave environment in a unit test please use Anvil based Polygon mainnet fork for a test
    • The test can be called test_aave_deposit.py
    • A test case check that the 0) Hot wallet starts with USDC on Ethereum mainnet 1) deposit was correctly registered with the Aave reserves 2) you receive the corresponding Aave aUSDC token back in the wallet, with the correct amount
    • Because there is no automatic mechanism to fetch Aave reserves and addresses as a list, please use hardcoded and commented values as text fixtures for now for any Ethereum address, with links to their respective descriptions of what they are

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.