rubicondefi / rubi-py Goto Github PK
View Code? Open in Web Editor NEWRubicon Python SDK
License: Apache License 2.0
Rubicon Python SDK
License: Apache License 2.0
today: we require the user to pass in a web3 object in order to create the Rubicon class. we should just ask the user to input an endpoint instead. input is error
๐ we will allow the rubi class, and possibly other contract classes(?), to be created just from a node endpoint. we should determine if the connection is HTTP or WS. in the future, we should have WS-specific functionality that is enabled.
Experiencing an issue with "insufficient funds". It looks like the numbers I'm inputting in the add_pair() allowance are larger than the minimum fee values outlined here: https://docs.rubicon.finance/protocol/rubicon-market/min-order-sizes
I have at least 50% of each of the values that I am using in the add pair functions.
import logging as log
import os
from _decimal import Decimal
from multiprocessing import Queue
from dotenv import load_dotenv
from rubi import Client
# load from env file
load_dotenv(".env")
# set logging config
log.basicConfig(level=log.INFO)
# set the env variables
http_node_url = os.getenv("HTTP_NODE_URL")
wallet = os.getenv("WALLET")
key = os.getenv("KEY")
queue = Queue()
# create client
client = Client.from_http_node_url(
http_node_url=http_node_url,
# custom_token_addresses_file="custom_token_addresses.yaml",
wallet=wallet,
key=key,
message_queue=queue
)
client.add_pair(
pair_name="WETH/USDC",
base_asset_allowance=Decimal(.005),
quote_asset_allowance=Decimal(100)
)
and here is the traceback that I get:
Traceback (most recent call last):
File "/Users/perryscott/Desktop/rubicon/./rubi_test2.py", line 32, in <module>
client.add_pair(
File "/Users/perryscott/Desktop/rubicon/rubi_env/lib/python3.10/site-packages/rubi/client.py", line 159, in add_pair
self.update_pair_allowance(
File "/Users/perryscott/Desktop/rubicon/rubi_env/lib/python3.10/site-packages/rubi/client.py", line 194, in update_pair_allowance
self._update_asset_allowance(
File "/Users/perryscott/Desktop/rubicon/rubi_env/lib/python3.10/site-packages/rubi/client.py", line 603, in _update_asset_allowance
asset.approve(
File "/Users/perryscott/Desktop/rubicon/rubi_env/lib/python3.10/site-packages/rubi/contracts/erc20.py", line 203, in approve
return self._default_transaction_handler(
File "/Users/perryscott/Desktop/rubicon/rubi_env/lib/python3.10/site-packages/rubi/contracts/base_contract.py", line 220, in _default_transaction_handler
self.w3.eth.send_raw_transaction(signed_txn.rawTransaction)
File "/Users/perryscott/Desktop/rubicon/rubi_env/lib/python3.10/site-packages/web3/eth/eth.py", line 372, in send_raw_transaction
return self._send_raw_transaction(transaction)
File "/Users/perryscott/Desktop/rubicon/rubi_env/lib/python3.10/site-packages/web3/module.py", line 68, in caller
result = w3.manager.request_blocking(
File "/Users/perryscott/Desktop/rubicon/rubi_env/lib/python3.10/site-packages/web3/manager.py", line 232, in request_blocking
return self.formatted_response(
File "/Users/perryscott/Desktop/rubicon/rubi_env/lib/python3.10/site-packages/web3/manager.py", line 205, in formatted_response
raise ValueError(response["error"])
ValueError: {'code': -32000, 'message': 'insufficient funds for gas * price + value'}
Hey guys,
I'm having some trouble with any functions that alter existing limit orders, whether that be an update or a cancel. Sometimes the functions work well and do their prescribed function, at other times I get a txn failure that looks something like this from ethersscan:
Fail with error 'Offer can not be cancelled because user is not owner, and market is open, and offer sells required a ...'
Keep in mind that this error is the same for both quote updates and cancellations. I'm sourcing my order limit ID's directly from my active offers using the sql query API.
headers = {'Content-Type': 'application/json'}
response = requests.post(url, headers=headers, data=json.dumps({'query': query}))
I'm using the client.batch_update_limit_orders()
, but I've seen the same issue using client.market.batch_requote()
and client.batch_cancel_limit_orders()
. I'm not sure if it matters, but oftentimes I'm only passing in a single order into these functions.
today: we either utilize the gas value provided by the user or default to a value of 300000 when signing transactions.
soon: we should be dynamically setting gas based on estimation. web3.py has a function to deal with this, but upon trying to implement this function i received values that were significantly underestimated. worth digging back into to see if it was, most likely, an issue in my implementation.
today: we utilize the class objects defined in rubi/contracts/helper/rolodex.py
to pass contract address data, abi data, and other chain relevant information throughout the package depending upon which chain_id the node provided is on.
soon -> it would be great if we:
to keep a running list of potential improvements:
rubi/rubi/contracts/aid.py
functions that involve passing in array valuesHey Rubicon,
I'm experiencing an issue with WEI being rounded on the backend. The reason I have come to conclusion is that when I place an order with a very small spread that's anything below ~10,000 ints of each other, the orders complete each other. For example:
Order 1 - Buy:
WETH = 8000000000000000
DAI = 149999999999999999500
Order 2 - Sell:
WETH = 8000000000000000
DAI = 150000000000000000500
I've made up these numbers but they're in the ballpark of orders I've seen. Let's call this an "int spread" of 1,000, for a WETH order of 0.1 size. One of these orders will be a Offer and then the next will be a Buy. This will happen until the "int spread" is > ~10,000. I'm not sure exactly how price is being calculated but there seems to be some rounding issues that cause the price calculation to be slightly altered. It's a very nit-picky bug report, but it felt worth reporting and it is causing some issues on my end. I can DM someone a tx hash if that will help as well.
Note: I'm using the client.market.batch_offer() function and passing these int values in.
currently, we have wired up a majority of our data work to the RubiconMarketLight subgraph. for a variety of reason we are going to need to switch to the RubiconV2
subgraph, and make some changes to how we handle a couple of things:
The user should be able to create a new Transaction object and estimate how much gas it will cost to execute the transaction.
Ideally, less information is required to retrieve this information. I'm ill-informed on the variables that go into gas cost. If it's possible, maybe the user only needs to provide the following to a function:
To receive an estimate of the gas cost.
web3.py does not have a nonce manager like ethers does, it would be great if this were supported out of the box. there are some good thoughts on this topic in this article ๐งโ๐ซ and I have a variety of my own on the subject ๐งโ๐
i believe this should be the next area of any major investment in the repo ๐ and will be a must-have for any future open-source bot implementations ๐ part of me wants to work to enable this within the context of the web3 repo itself, but as it's of need today, i think the best course of action will be to internalize it to this repo, and later any learnings and work can be transferred to the web3 repo more broadly. apes together strong ๐ฆ
I'm having an issue with the client.add_pair() function. Here is what the code looks like so far, borrowed entirely from the quickstart guide.
import os
from dotenv import load_dotenv
from decimal import Decimal
from rubi import Client
# load from env file
load_dotenv(".env")
# set the env variables
http_node_url = os.getenv("HTTP_NODE_URL")
wallet = os.getenv("WALLET")
key = os.getenv("KEY")
# instantiate the client
client = Client.from_http_node_url(
http_node_url=http_node_url,
wallet=wallet,
key=key
)
# add the WETH/USDC pair to the client
# the base asset is WETH and the quote asset is USDC
client.add_pair(
pair_name="WETH/USDC",
base_asset_allowance=Decimal("1"),
quote_asset_allowance=Decimal("10000")
)
and the resulting error:
Traceback (most recent call last):
File "/Users/perryscott/Desktop/rubicon/./rubi_test.py", line 25, in <module>
client.add_pair(
File "/Users/perryscott/Desktop/rubicon/rubi_env/lib/python3.10/site-packages/rubi/client.py", line 159, in add_pair
self.update_pair_allowance(
File "/Users/perryscott/Desktop/rubicon/rubi_env/lib/python3.10/site-packages/rubi/client.py", line 194, in update_pair_allowance
self._update_asset_allowance(
File "/Users/perryscott/Desktop/rubicon/rubi_env/lib/python3.10/site-packages/rubi/client.py", line 603, in _update_asset_allowance
asset.approve(
File "/Users/perryscott/Desktop/rubicon/rubi_env/lib/python3.10/site-packages/rubi/contracts/erc20.py", line 203, in approve
return self._default_transaction_handler(
File "/Users/perryscott/Desktop/rubicon/rubi_env/lib/python3.10/site-packages/rubi/contracts/base_contract.py", line 222, in _default_transaction_handler
return self._wait_for_transaction_receipt(transaction=signed_txn)
File "/Users/perryscott/Desktop/rubicon/rubi_env/lib/python3.10/site-packages/rubi/contracts/base_contract.py", line 267, in _wait_for_transaction_receipt
result = TransactionReceipt.from_tx_receipt(
File "/Users/perryscott/Desktop/rubicon/rubi_env/lib/python3.10/site-packages/rubi/contracts/contract_types/transaction_reciept.py", line 56, in from_tx_receipt
l1_fee_scalar=None if tx_receipt.get("l1FeeScalar") is None else int(tx_receipt.get("l1FeeScalar"), 16)
ValueError: invalid literal for int() with base 16: '0.684'
I've also tried removing the quotations in the Decimal() function as well as other pair combinations and values. Let me know if there's something I'm missing. Thanks.
Also a heads up, in the docs quickstart section the 'from decimal import Decimal' import is missing and an extra 'client = ' found it's way into the Client() declaration.
today: the repo has very mundane error handling that could be improved by utilizing w3.eth.wait_for_transaction_receipt
throughout any write call interaction. this should be optional and probably defined at initialization, as it has impacts on the speed of which orders can be processed. ideally, a nonce manager solves many of these problems for the users.
soon: for sure improved error handling utilizing w3.eth.wait_for_transaction_receipt
๐งโ๐ the most pressing use case would be handling token approval errors for the user as these are only identified through transaction failure, not on signing or sending ๐ฅ
we currently lack nested filtering throughout the repo, it would be great to add this in for obvious use cases, for example on the AidData
class allowing to filter get_aid_history
by token would be ๐ฏ
i know this is supported in subgrounds, need to go dig around more and find an example or see whats going on underneath so i can utilize the proper syntax. this would be a good place to add in an example to that repo to help out others who are experiencing the same problem ๐ซ
this is a really interesting error i am getting that i am unsure on how to proceed with, so putting it here for now. on the hosted service the query below executes as expected. from rubicon's
indexer, the query fails. if i had to guess, this can be resolved with a graph-node
update:
`
subgraph = "https://api.rubicon.finance/subgraphs/name/RubiconV2_Optimism_Mainnet_Dev"
#subgraph = "https://api.thegraph.com/subgraphs/name/denverbaumgartner/rubiconv2-optimism-mainnet"
sg = Subgrounds()
data = sg.load_subgraph(subgraph)
Take = data.Take
takes = data.Query.takes(
first=100,
where={"offer_": {"from_address": "0xde4de639d699a66f860814468298639c49e4f778"}}
)
fields = [
takes.id
]
df = sg.query_df(fields)
`
currently, we expect that the result of our queries are not empty, and then proceed to attempting to transform the data. this causes problems when the dataframe actually is empty, and we should handle scenarios like this. an example where this can occur is rubi.data.market_aid_optimism_processing.aid_fill_tracking(aid = aid, asset = asset, quote = quote)
today: the repo only passes, expects, and utilizes the full uint representation of erc20, erc721, etc. token values. most people tend to like seeing these values converted to human-readable format at some point or another.
soon: at a bare minimum we will add in additional support to the ERC20 class for dealing with integer values. throughout the repo itself i'm not sure we should really do must as we should encourage best practice of dealing with values as integers to avoid precision errors or mistakes. within data aggregation & reporting ๐ etc. this will be an obvious part of data cleaning & prep
today: there are several compromises made around these functions for the sake of viability today. these include:
web3._utils.events.get_event_data
eth_abi.decode
, or even the need to decode the id itselfsoon: open to any suggestions ๐ฆ ๐ฎ
from a bug report submitted by @soheiyagi in the public discord
occasionally, when attempting to construct an OrderBook
object, when building the BookSide
using from_rubicon_offers
we encounter an IndexError
such as follows:
if levels and levels[-i].price == price:... IndexError: list index out of range
looking into this function it appears that we should be utilizing levels[-1]
instead of levels[-i]
. i believe the issue we are encountering is a result of offers existing at the same level of the book, which only occasionally happens. furthermore, i believe we need to sort the offers
based on price if we want to use this current approach in order to correctly place multiple offers within the same BookLevel
. alternatively, we could utilize a dictionary such as is done here to keep track of price and at which level an offer should be inserted.
i may be misinterpreting something here, would love to know your thoughts @adamamsmith ๐
today: we are signing transactions utilizing gas
and gasPrice
. this is a dated method because of the London upgrade and should be updated.
soon: we should sign transactions utilizing maxFeePerGas
and maxPriorityFeePerGas
. this is also a mere matter of what worked today, but will be improved upon very ๐ ๐
today: throughout the repository almost every contract class has a function called get_log_xyz_hash that returns an event signature. today these values are hardcoded.
soon: it would be great if these values were generated from the contract .sol files upon initialization of the class, or even within rolodex.py
I am placing an order with rubi-py, but currently TransactionReceipt does not return information such as whether my order was confirmed or failed.
It seems that there is no other rubi-py method, so currently there is no choice but to get information from the subgraph, but I think it would be good to be able to complete it with rubi-py as much as possible when launching a small bot.
I'm not sure if it's better to implement it with another rubi-py method or add it to TransactionReceipt.
Thanks for developing!
when using a pattern as follows: client.start_event_poller("WETH/USDC", event_type=EmitOfferEvent, filters={"maker": client.wallet})
ANY offer from the client.wallet
is reported back as an offer for the given pair, even if it is not for that pair. i'm not sure if this is an error during filter creation for the node or if this is something we need to handle client side instead. will try to investigate further, this limits the ability for users to effectively run multiple pairs from a single client if they are utilizing the event_poller
.
it would be great to support open-telemetry out of the box and have some useful tooling implemented or available throughout the library. this is a lower-priority, but high-impact enhancement that should most likely be implemented in support of examples.
today: the market aid class is initialized from a contract address that is passed in by the user. because the factory can tie a user's signing address to all of the market aid instances that it controls, it would be nice if in the setup we had the option for a user to either select a market-aid instance that they would like to deploy to, or to create a new one.
soon: we should improve rubi/rubi/contracts/aid.py
, and the repo as a whole, to include something like the following:
current state: today, we hardcode the contract abis and bytecodes in order to interact with the protocol.
desired state: soon, it would be great if we compiled the protocol's .sol files to get this data. useful library.
Hey guys,
I'm trying to create a Arbitrum client using the following function:
client = Client.from_http_node_url(
http_node_url=os.getenv("HTTP_NODE_URL"),
wallet= os.getenv("WALLET"),
key=os.getenv("KEY"),
message_queue=queue
)
I've tried several arbitrum endpoints such as :
All of the node url's I've gotten from these websites yield the same errors:
https://indulgent-old-borough.arbitrum-mainnet.discover.quiknode.pro/perry's_api_key/
Traceback (most recent call last):
File "/home/ec2-user/rubi-volume-bot/./app.py", line 114, in <module>
client = get_client(queue=my_queue, pair=token)
File "/home/ec2-user/rubi-volume-bot/utils.py", line 73, in get_client
client = Client.from_http_node_url(
File "/home/ec2-user/.local/lib/python3.10/site-packages/rubi/client.py", line 105, in from_http_node_url
network = Network.from_config(
File "/home/ec2-user/.local/lib/python3.10/site-packages/rubi/network/network.py", line 125, in from_config
network_name = NetworkId(w3.eth.chain_id).name.lower()
File "/usr/local/lib/python3.10/enum.py", line 385, in __call__
return cls.__new__(cls, value)
File "/usr/local/lib/python3.10/enum.py", line 710, in __new__
raise ve_exc
ValueError: 42161 is not a valid NetworkId
Is there a preferred provider I should be using to do this?
when i originally switched the allowance argument to be optional in client.add_pair
i did not also make the ERC20
class initialization of the base
and quote
assets not require a wallet/key. when an allowance is not provided, we should not initialize the ERC20
classes with write
permissions.
today: when placing match market-making trades there a self-trade check to make sure orders don't auto-match on the contract side but not on the client side. it would be really not nice ๐ if someone were to be in a situation where they accidentally submitted a lot of transactions that fail but still spend gas ๐ผ
soon: we should add checks client side to make sure that offers aren't submitted that can self match. for a lot of these additional safety checks I think we should give the user the option to opt-out if they want to (speed?). a lot of other paths to improve on the speed front before we get to that point IMO. additionally, i think we should have some way to monitor for bad fail-state scenarios and cut the cord if needed. some type of safety module? maybe this is baked into the nonce manager ๐ง
Now: the client can only interact with the chain using http requests.
Soon: add a websocket implementation that allows the client to listen to updates from a node instead of having to poll for them.
we should add an additional pr-title word that bypasses the deployment of the package on merge
. we can call it skip
, tertiary
, or something of that sort. would be nice for when we need to update docs, the bots, etc.
I am experiencing an issue with the start_event_poller stopping after extended usage.
Replicating this is difficult, as the problem usually happens at a somewhat random interval, but the behavior usually goes something like this:
Replicate:
# Here is the exact way I'm initializing the event poller
# The problem is not local to a certain pair type
# problem occured on versions 2.0.5 and 2.0.6
client.start_event_poller(pair_string, event_type=EmitOfferEvent, filters={"maker": client.wallet}, poll_time=poll_time)
client.start_event_poller(pair_string, event_type=EmitTakeEvent, filters={"maker": client.wallet}, poll_time=poll_time)
client.start_event_poller(pair_string, event_type=EmitCancelEvent, filters={"maker": client.wallet}, poll_time=poll_time)
client.start_event_poller(pair_string, event_type=EmitDeleteEvent, filters={"maker": client.wallet}, poll_time=poll_time)
Let me know if there is any other info I can provide to help.
today: rubi/rubi/contracts/helper/erc20.py
is set up to be fed off of an implementation of the ERC20 abi that is utilized in the Rubicon Protocol. the class instance utilizes a decimals call that is optional and not present on some ERC20 implementations.
soon:: we should identify the best ERC20 template to utilize withing the repo and update the erc20.py file.
currently, we do not report back the block number in a query, rather we just collect the data as is. by expanding each query to include a _meta_
field that contains block number, we can provide more insight to user's regarding how delayed the data they are working with is.
Open source bot using: RubiArbitrageur using contract 0x80130D182ED6D3A99C06166d939E9f0bA864DFC8
today: the repo does not provide the user with any information regarding the fee they incur when taking liquidity off the book. an example of this is the swap
function in rubi/rubi/contracts/router.py
where the function must be provided with an expected_market_fee_bps
value. it would be great if this was set as a class variable at initialization.
soon: we should work to avoid hardcoding the fee throughout the repo when needed and work to incorporate more reporting & information regarding fees throughout the repo.
in addressing issue #21 its clear that we are going to start to enter scenarios where we want the aggregate data from multiple chains at the same time. these queries should most likely be executed in parallel to save time and then be merged upon receipt. in the name of going fast and only building what is needed right now, i am putting this off till a later time. i think it will require a bit of reworking the existing functionality i am using today, but it shouldn't be too bad. will attempt to avoid creating tech debt for this scenario ๐
currently, we utilize a ^ pattern throughout our dependencies to utilize the most recent version of packages. this is poor practice and can lead to dependency conflicts. we should statically lock dependencies to known versions that work with one another to avoid this scenario going forward.
currently, we do not have any type of error handling for failed api calls. i think ideally, we have a common fallback pattern that we utilize throughout our queries to handled networking issues. we should provide the user the option to override these fallbacks if they do not want the system to retry, but rather raise an error instead.
today: we do not utilize any multi-call functionality throughout the repo, and instead place repeat read calls throughout the repo to obtain data directly from the chain ๐ช
soon: we should utilize a multi-call functionality, most likely adding this dependency, to group any and all calls throughout the repo ๐จ
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.