evan-kim2028 / datastreams Goto Github PK
View Code? Open in Web Editor NEWLicense: MIT License
License: MIT License
datastreamer
currently runs as a (limited) proof of concept data flow path. There is still a lot of friction that remains between datastreamer
and streamer. A refactor/enchancement of datastreamer
is needed to expand functionality
DataStreams/src/datastreams/streamer.py
Line 167 in 587b276
This method name was edited to runQuery by mistake. Needs to be changed
streamer.py
has a lot of unused comments. Needs a refactor as well as design considerations with respect to datastreams.py
Replace the if statements with match
from functools. See here for explanation of match
https://gist.github.com/Evan-Kim2028/df57d5528c7a35fb40cc308eba3dfead
The schema query for sub.Query.liquidityPools
has more than 16 available columns to query from, but only shows 16 by default. For example, if inputTokenBalances
is not explicitly mentioned, then it will not appear in the query results.
Current proposed idea for this solution is to make a FieldPath + string function that binds string columns to an existing fieldpath.
Example:
FieldPath = sub.Query.liquidityPools
String = 'inputTokenBalances`
Function Output: --> sub.Query.liquidityPools.inputTokenBalances
as a single FieldPath object
File ~/.local/lib/python3.10/site-packages/requests/models.py:971, in Response.json(self, **kwargs)
970 try:
--> 971 return complexjson.loads(self.text, **kwargs)
972 except JSONDecodeError as e:
973 # Catch JSON-related errors and raise as requests.JSONDecodeError
974 # This aliases json.JSONDecodeError and simplejson.JSONDecodeError
File ~/.local/lib/python3.10/site-packages/simplejson/__init__.py:525, in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, use_decimal, **kw)
521 if (cls is None and encoding is None and object_hook is None and
522 parse_int is None and parse_float is None and
523 parse_constant is None and object_pairs_hook is None
524 and not use_decimal and not kw):
--> 525 return _default_decoder.decode(s)
526 if cls is None:
File ~/.local/lib/python3.10/site-packages/simplejson/decoder.py:370, in JSONDecoder.decode(self, s, _w, _PY3)
369 s = str(s, self.encoding)
--> 370 obj, end = self.raw_decode(s)
371 end = _w(s, end).end()
File ~/.local/lib/python3.10/site-packages/simplejson/decoder.py:400, in JSONDecoder.raw_decode(self, s, idx, _w, _PY3)
399 idx += 3
--> 400 return self.scan_once(s, idx=_w(s, idx).end())
...
---> 89 raise PaginationError(exn.args[0], strategy)
91 return data
93 except SkipPagination:
PaginationError: Expecting value: line 1 column 1 (char 0)```
currently fixed at version 0.0
Problem: Streamer
is becoming too monolithic to update over time. There are too many moving pieces to get a concise understanding of how all of the functions work together.
Solution: Break Streamer
up into multiple classes. Separate query functionality from schema information. This will more clearly delineate functional dependencies that aren't so clear in Streamer
right now.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.