cowprotocol / dune-bridge Goto Github PK
View Code? Open in Web Editor NEWLicense: Other
License: Other
This iterative async block of code could probably be a batch request.
for hash in uninitialized_app_data_hashes.iter() {
download_referral_from_ipfs_and_store_in_referral_store(db.clone(), *hash).await?;
}
cf referral_maintenance.rs
Original issue gnosis/gp-dune-bridge#59 by @bh2smith
We have testplans, how we are currently testing the code:
E.g. here is one example: #6
It would be great to automatized this test and put it as an e2e test
Here for example:
dune-bridge/pysrc/fetch/dune.py
Lines 39 to 40 in a53f3cd
If the sync has already been deployed, this will require a schema change on Dune's side.
This is not a hard requirement, but could make data verification much easier from the User's perspective.
It is pretty easy to implement now if think we may want it.
more context in slack
The following query should eliminate the need for the entire python portion of this project and also any component of the Rust project responsible for fetching and parsing App Data
The API part of this project would now only refresh this query (responsibly) say once per hours and be able to serve this data to the user account page on CoW.Fi
Observe the current end point is:
https://api.cow.fi/affiliate/api/v1/profile/0x8540f80fab2afcae8d8fd6b1557b1cf943a0999b
which can be compared with any of the trader addresses returned by the query:
Note that, while working on this I have discovered what appears to be a bug in the existing API service/referral query.
Namely, that for 0x8540f80fab2afcae8d8fd6b1557b1cf943a0999b
(the address in the link above) the new query returns non zero referral count and volume. I went to look at the trade history of these referrals and found that the traders first trade did, in fact, use this wallet as the referred on their first trade. So I can not understand why we would be neglecting this as a valid referral:
cc @gentrexha and @josojo as something we should plan for in the beginning of January.
Currently this project is using the solvable_orders
api endpoint
dune-bridge/src/referral_maintenance.rs
Lines 173 to 192 in 4c37f08
It should be using the get_auction
https://protocol-rinkeby.dev.gnosisdev.com/api/#/default/get_api_v1_auction endpoint.
Our cron-jobs updating Dune data are currently failing with:
Dune API Request failed with errors [{'extensions': {'path': '$', 'code': 'validation-failed'}, 'message': 'query is not allowed'}]
It would be good to investigate why the "query is not allowed". If this is an intermittent issue, then we need to add some sort of retry mechanism because the alerts are quite noisy.
Currently we get the "block_from", but opening a file in our own persisted storage, however this value can also be obtained by querying the table in dune itself. The table does not yet exist, but once it does we could experiment with fetching this value from one, or both places...
dune-bridge/pysrc/fetch/dune.py
Lines 94 to 104 in a53f3cd
Currently we are updating a single table in Dune. This means if we run the app data script in both production and staging the two different versions of the query will disagree with each other.
Proposed Solution
append a parameter to the table name
dune_user_generated.gnosis_protocol_v2_app_data_{environment}
where we pass staging/prod as parameter names.
Several pieces introduced in #37 can be refined, generalized and moved into the dependency package dune-cli
In particular:
There are a few reasons why these have not yet been moved into the client outlined here. But, in essence, it is because they need to be refined a bit more before we can make then serve as a useful general purpose. For example, it isn't clear that either of QueryData or DuneFetcher are all that beneficial -- perhaps its better to just extend the Query and DuneClient classes to include the additional fields added in these "extended classes". Would love to open up a discussion about this (maybe in the Dune Client).
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.