rainshowerlabs / sothis Goto Github PK

Describe the bug

Cannot use keyboard to interrupt sothis process. Instead, I have to manually kill the process.

Separate issues: cannot kill anvil process with keyboard interrupt, anvil blocks not increasing after the sothis track mode instance is started.

To Reproduce

Steps to reproduce the behavior:

1. Run anvil: anvil --fork-url https://eth-mainnet.alchemyapi.io/v2/xxxx --fork-block-number 17499990 --cups 2000
2. Run sothis historic replay instance: sothis --source_rpc https://mainnet.gateway.tenderly.co/xxxx --replay_rpc http://127.0.0.1:8545 --terminal_block 17500100
3. Run sothis track mode instance: sothis --mode track --source_rpc http://127.0.0.1:8545 --contract_address 0x88e6a0c2ddd26feeb64f039a2c41296fcb3f5640 --storage_slot 0 --filename uniswap-slot0.json --path ./
4. Try to use keyboard interrupt on the sothis track mode instance. Does not work, must kill process manually.
5. Try to use keyboard interrupt on the anvil process. Does not work, must kill process manually.
6. Anvil block number not increasing anymore like it was before the sothis track mode instance was started.

I did not have this issue with track mode on a live node (current block), I have only observed the issue using a local node.

Expected behavior

Can use keyboard interrupt to kill process. Anvil block numbers should increase.

Screenshots

N/A

Setup (please complete the following information):

• OS: Ubuntu 22.04
• Source node: Alchemy
• Replay node: anvil

Additional context

I am troubleshooting this error related to binding to a local port

sothis --source_rpc https://eth-mainnet.alchemyapi.io/v2/xxxx --replay_rpc http://127.0.0.1:8545 --terminal_block 17500000

Replaying in historic mode...
Error: JsonSerializationFailed("error sending request for url (http://127.0.0.1:8545/): error trying to connect: tcp connect error: Connection refused (os error 111)")

Is your feature request related to a problem? Please describe.

The block number and storage slot values are in hex. A flag to convert these values to decimals would save a post-processing step

Describe the solution you'd like

A --decimal flag or similar so the JSON (or CSV) output file is in decimal

Describe alternatives you've considered

Post-processing can handle this outside of sothis, so maybe it is out of scope

Additional context

N/A

Describe the bug
Using a public RPC that's running erigon, not geth, results in errors.

To Reproduce
Steps to reproduce the behavior:

1. Run sothis with an RPC using erigon using this list. For example sothis --mode fast_track --source_rpc https://eth-mainnet.public.blastapi.io --contract_address 0xbebc44782c7db0a1a60cb6fe97d0b483032ff1c7 --storage_slot 80084422859880547211683076133703299733277748156566366325829078699459944778999 --origin_block 16870000 --terminal_block 16910000 --query_interval 100 --filename tripool_usdc.json
2. Observe an error such as Error: RequestFailed("\"finding bebc44782c7db0a1a60cb6fe97d0b483032ff1c7 in the changeset 17939419: account changes purged for block 17939419\"")
   Using the RPC https://rpc.builder0x69.io instead returns the error Error: JsonDeserializationFailed("error decoding response body: expected value at line 1 column 1")

Note: blutgang does not give an error in most of these cases, only sothis encounters an error.

Expected behavior
No errors. Or consider checking what server software is running and returning an error to the user if the result is not geth.

Is your feature request related to a problem? Please describe.

Instead of querying a state variable at every block, it would be great to add a tag to choose a block interval to allow a user to query every X blocks. This is useful if a user wants a summary of data over a longer timespan and does not want to wait for data from every block.

Describe the solution you'd like

Add a --block_interval flag to choose which blocks to query data from.

Describe alternatives you've considered

None

Additional context

Is your feature request related to a problem? Please describe.

fast_track reads state variables. Some state variables are not as easy to read as uint256 variables. For example, consider price data provided by Chainlink. It can be requested with latestAnswer(), but the source data is stored in a uint32 -> Transmission mapping. If sothis can read from mappings, it will be easy to pull historic token price data directly from the blockchain without any data brokers.

Describe the solution you'd like

Read data from mappings. A new flag will be needed to identify which item in the mapping to read from. Handling of custom structs will be needed, likely with another flag which passes the struct layout/ABI.

Describe alternatives you've considered

None

Additional context

None

Track the history of state variables without replaying. The idea is to just querry the historical state.

This kinda works in the 0.4.0 dev branch but is broken. Needs cleanup

Is your feature request related to a problem? Please describe.
--storage_slot only supports decimal storage slot values

Describe the solution you'd like
--storage_slot could support and autodetect a hex value if the value starts with 0x

Describe alternatives you've considered
Update CLI arg description to state requirement for decimal storage slot value

Describe the bug
Getting an error when I use the --decimal flag

To Reproduce
Steps to reproduce the behavior:

1. Run: sothis --mode fast_track --source_rpc https://eth.llamarpc.com --contract_address 0xbebc44782c7db0a1a60cb6fe97d0b483032ff1c7 --storage_slot 80084422859880547211683076133703299733277748156566366325829078699459944778998 --origin_block 17750000 --terminal_block 17753000 --query_interval 100 --decimal --filename tripool_dai.json
2. When all the data has been queried from the specified blocks, this error message will appear when attempting to write the received data to the output file. No output file is created:
   Writing to file: ./tripool_dai.json thread 'main' panicked at 'called Result::unwrap()on anErrvalue: ParseIntError { kind: PosOverflow }', /home/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/sothis-0.5.0/src/tracker/common.rs:70:37 note: run withRUST_BACKTRACE=1 environment variable to display a backtrace
3. Run the same command as step 1 without the --decimal flag and observe no error

Expected behavior
No error. Output file contains data with decimal conversion.

Sothis is designed to be used to test tooling and protocols against mainnet conditions and their interactions. When setting up an enviroment to test such protocols, our local chain may have high rates of entropy and therefor, high rates of transaction failure.

One way to mitigate this without restarting the fork is to automatically attempt to recover. We can store which accounts have high failiure rates, and restore their storage/balances to the mainnet values at the current block height we're replaying at.

There might be better solutions to this problem, so feedback is appreciated.

When sending raw transactions, we are replaying them as legacy txs. if the tx type and signatures dont match, hardhat doesnt accept the tx. We'd need to get the transaction, construct properly, and then send it as raw for hardhat to play nice.

Not that difficult, but very tedious.

Is your feature request related to a problem? Please describe.

Saving to an output file uses JSON format. Allowing CSV output can make it easier to analyze the data with certain tools, like spreadsheets.

Describe the solution you'd like

A flag to toggle the output format between JSON and CSV.

Describe alternatives you've considered

An external tool to convert JSON data to CSV data

Additional context

N/A

Is your feature request related to a problem? Please describe.
Migrating away from using static vars would make sothis much more portable

Describe the solution you'd like
Making things portable should be the number one priority for the upcoming release. this would probably mean having arguments explicitly passed.

Describe alternatives you've considered
¯_(ツ)_/¯

Additional context

Describe the bug
Same pattern as in #25

• --storage_slot and --calldata have the same description
• --origin_block and --query_interval have the same description

To Reproduce
Steps to reproduce the behavior:

1. sothis --help

Expected behavior
Unique description for every CLI arg

The 3 tracking modes preform identically and only have minor changes between each other. They should be refactored so they use common functions/libs.

Add proper, concise, useful docs besides the readme