Comments (4)
Weird. Thanks for telling me. This is surely an issue.
from asyncio-redis.
Just to be clear, I have not confirmed this behavior. I was trying to wrap my head around asyncio in general and this library in particular, wondering how this code would run, and this is my assumption after looking through the code.
Presumably the Redis protocol doesn't support retrieving values from a "get" query without consuming the full response from the previous query, so I thought maybe the library would use a second connection while streaming a multi-bulk response, but I concluded it wasn't, so I just wanted to bring it up as a topic.
I'm not sure what the right behavior is.
from asyncio-redis.
(ignore this comment, it was not really a reply to your question.)
Apparently, there's an issue that if the multi bulk reply will contain 500,000 items, we already create that many futures. Better would be to use a queue for that. We should be able to support at least streaming per IP packet. We don't have to wait before all the IP packets arrive, before we return a SetReply.
Further, in the case of sets, redis doesn't seem to support streaming that well. Try to run smembers
from redis-cli for such a huge set. It doesn't stream like keys
does. I'm not yet sure about this.
from asyncio-redis.
Oh, your are right, these calls will be pipelined. This means that the get() will be blocking untill the whole smembers answer has been received. It's best to use a connection pool for this.
Connection pooling docs:
http://asyncio-redis.readthedocs.org/en/latest/#connection-pooling
Some example of how the streaming works:
from asyncio-redis.
Related Issues (20)
- ZRangeReply do not have aslist method
- Using context managers HOT 1
- How to release connection from subscribe HOT 1
- Broken on python 3.7.0 HOT 4
- The right way to share the returned connection.
- Unable to turn off auto_reconnect
- Deprecation warning for formatargspec in Python3.7 HOT 1
- Function `asyncio_redis.Connection.create()` hangs if connection can't be established HOT 2
- Python 3.8 DeprecationWarnings; drop support for legacy Python HOT 3
- Future and roadmap of asyncio-redis HOT 5
- Difference between aioredis?
- Release new version? HOT 4
- Update documentation HOT 1
- SPOP [count] argument.
- intersphinx looks for implementation modules
- asyncio_redis should wait for server restart
- Adding python types HOT 1
- Missing ignore_subscribe_messages equivalent HOT 1
- Proposing a PR to fix a few small typos
- Async Class Constructors HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from asyncio-redis.