pyo3 / tokio Goto Github PK
View Code? Open in Web Editor NEWAsyncio event loop written in Rust language
License: Other
Asyncio event loop written in Rust language
License: Other
async-tokio already uses tokio-signal crate for ctrl-c handling
should be easy to extend to all signals
I implemented create_server() method before added async getaddrinfo(), so right now
create_server() resolves addresses synchronously.
loop.current_task()
was never a part of asyncio API.
Starting from 3.7 third party loops can use dedicated calls for custom task implementation:
asyncio._register_task
, ``asyncio._unregister_task,
asyncio._enter_task`, `asyncio._leave_task`.
The API calls are started from underscore to emphasize these very low level position (should be used by libraries like tokio only) but they are a part of of official API.
I just had a quick try with tokio, I'm currently getting a slightly speed up (~5%) compared to asyncio loop, but significantly less quick than uvloop. Is that to be expected at this stage?
Also your readme refer to tokio.EventLoopPolicy()
which is wrong, as of pypi v0.1.0 the policy is called tokio.TokioLoopPolicy()
does not work at the moment
tokio-tls is already included and ssl is supported for "open_connection()" api.
current implementation is based on asyncio.unix_events, we need native rust implementation
As the title says.
Same as like this: https://github.com/MagicStack/uvloop
https://github.com/PyO3/tokio/blob/master/tokio/__init__.py#L9
import asyncio
import tokio
asyncio.set_event_loop_policy(tokio.EventLoopPolicy())
loop = tokio.new_event_loop()
asyncio.set_event_loop(loop)
class Empty(object):
pass
# This not work!
loop.xyz = Empty()
If I add a signal handler for SIGINT
in pure asyncio and uvloop implementation, a ctrl-c
is handled by the handler in contrast to tokio, where still a KeyboardInterrupt
is raised.
I use Python 3.6.3 and tokio bbf9be4.
When I send SIGINT
via kill -s INT <pid>
tokio raises also a KeyboardInterrupt
in contrast to asyncio and uvloop.
It seems that SIGINT
handler is not recognized at all...
Proposed header:
# Copied from the uvloop project. If you add a new unittest here,
# please consider contributing it to the uvloop project.
#
# Portions copyright (c) 2015-present MagicStack Inc. http://magic.io
As opposed to asyncio, which relies on mock tests a lot, uvloop contains a good number of functional unittests. They cross-test uvloop and asyncio to ensure full compatibility.
While working on uvloop
I've iterated on the following three different approaches to how get_extra_info('socket')
should be implemented:
Wrap the underlying socket FD into a Python socket object: socket.socket(fileno=FD)
. The downside of this approach is that Python will try to control the FD. For instance, when the socket object is GCed, Python will try to close the FD, which isn't acceptable for uvloop and asyncio-tokio. Moreover, beginning with Python 3.6 there is a race to who closes the socket first, and Python expects to win, otherwise it will raise an IOError
in socket.close()
.
Duplicate the FD before wrapping (currently implemented in uvloop). This fixes (1), but has its own downsides: FDs are a limited resource and now we need 2x of them; FDs can stay alive longer than necessary (if the transport is closed, but its duplicated sockets are alive).
Return a socket-like object. Essentially, the object should implement all socket.socket
methods and have all of its attributes (to be duck-type compatible), but the implementation of socket.close()
and socket.shutdown()
will be empty (or will raise an error).
The third option is the one I want to implement in uvloop, and I believe the one we need to implement for asyncio-tokio.
we need this for some options, like TCP_NODELAY, TCP_CORK, SO_KEEPALIVE
should we provide python module with async-tokio
? at the moment async-tokio is rust crate, to run event loop aiohttp-tokio is required.
running 51 tests
test test_compression_unknown ... ok
test test_compression_deflate ... ok
test test_compression_gzip ... ok
test test_conn_close ... ok
test test_conn_close_and_upgrade ... ok
test test_conn_close_and_upgrade_order ... ok
test test_conn_close_1_0 ... ok
test test_conn_default_1_0 ... ok
test test_conn_default_1_1 ... ok
test test_conn_keep_alive_1_0 ... ok
test test_conn_other_1_0 ... ok
test test_conn_other_1_1 ... ok
test test_headers_content_length_err_1 ... ok
test test_conn_upgrade ... ok
test test_headers_content_length_err_2 ... ok
test test_headers_multi_feed ... ok
test test_http_payload_parser_length ... ok
test test_http_request_bad_status_line ... ok
test test_http_request_chunked_payload ... ok
test test_http_request_chunked_payload_and_next_message ... ok
test test_http_request_chunked_payload_chunks ... ok
test test_http_request_parser_bad_method ... ok
test test_http_request_parser_bad_version ... ok
test test_http_request_parser_non_utf8 ... ok
test test_http_request_parser_two_slashes ... ok
test test_http_request_parser_utf8 ... ok
test test_http_request_upgrade ... ok
test test_http_request_websocket ... ok
test test_invalid_header ... ok
test test_invalid_name ... ok
test test_parse_body ... ok
test test_parse_chunked_payload_chunk_extension ... ok
test test_parse_chunked_payload_size_error ... ok
test test_parse_delayed ... ok
test test_parse_eof_payload ... ok
test test_parse_headers_max_multi ... ok
test test_parse_length_payload ... ok
test test_parse_headers_multi ... FAILED
test test_parse_length_payload_eof ... ok
test test_parse_no_length_payload ... ok
test test_request_chunked ... ok
test test_request_chunked_partial ... ok
test test_request_simple ... ok
test test_request_simple_10 ... ok
test test_request_simple_with_prefix_cr ... ok
test test_request_simple_with_prefix_crlf ... ok
test test_special_headers_partial ... ok
test test_http_request_max_status_line ... ok
test test_max_header_value_size ... ok
test test_max_header_value_size_continuation ... ok
test test_max_header_name_size ... ok
failures:
---- test_parse_headers_multi stdout ----
thread 'test_parse_headers_multi' panicked at 'assertion failed: `(left == right)` (left: `"c2=cookie2"`, right: `"c1=cookie1"`)', tests/test_http_decoder.rs:260
note: Run with `RUST_BACKTRACE=1` for a backtrace.
failures:
test_parse_headers_multi
test result: FAILED. 50 passed; 1 failed; 0 ignored; 0 measured
error: test failed, to rerun pass '--test test_http_decoder'
Hey, this is great. It's exactly what I need for a project I'm working on. Is there any documentation or examples on how to use it in a project using Rust code with tokio that calls python with asyncjo/aiohttp? Thanks much.
We now use new python binding https://github.com/PyO3/PyO3
it is based on rust-cpython, but whole class generation has been significantly redesigned
now code looks more like rust and not like some kind of python:
#[py:class]
pub struct Socket {
connected: bool
}
#[py:proto]
impl PyIterProtocol for Socket {
fn __iter__(&self, py: Python) -> PyResult<PyObject> {
}
}
#[py:proto]
impl PyAsyncProtocol for Socket {
fn __await__(&self, py: Python) -> PyResult<PyObject> {
}
}
most of class functionality of already implemented, everything else should be completed within next week
asyncio's future ans task are not allowed at the moment.
If I register a signal handler within a task to cancel itself, tokio failes with:
AttributeError: 'NoneType' object has no attribute 'cancel'
async def test_tokio_failes():
current_task = asyncio.Task.current_task()
asyncio.get_event_loop().add_signal_handler(signal.SIGINT, current_task.cancel)
def main():
# ...
loop.run_until_complete(test_tokio_failes())
exception_handler api is implemented, but rust level logging is not defined.
should we use python logging? or rust "log" or "slog" crates and separately define logging configuration api.
what classes should support weakrefs? PyHandle, PyTimerHandler needs to support for sure. what about Future and Task objects?
what type PyFuture
should resolve to?
currently, Future::Item
is PyResult<PyObject>
and Future::Err
is unsync::oneshot::Cancelled
future handling code looks like:
PyFuture::new(...).map(|res: PyResult<PyObject>| {
match res {
Ok(obj) => "do something with result (PyObject)"
Err(err) => "do something with py exception"
}
}).map_err(|err| "cancelled");
other option is to use PyObject
as Future::Item
and PyErr
as Future::Err
:
PyFuture::new(...).map(|res: PyObject| {
"do something with result (PyObject)"
}).map_err(|err| {
"do something with py exception"
);
I like second option, but there would be no explicit cancelation state
problem described here aio-libs/aiohttp#1883
uvloop works fine
current implementation is based on asyncio.unix_events, we need native rust pipes support
I noticed exceptions does not contain right traceback.
I'd suggest picking Apache 2.0
(tokio) rocklobster:~/python/tokio $ python setup.py build
running build
running build_py
running egg_info
writing tokio.egg-info/PKG-INFO
writing dependency_links to tokio.egg-info/dependency_links.txt
error[E0308]: mismatched typesio.egg-info/top_level.txt
--> /home/name/.cargo/git/checkouts/pyo3-a22e69bc62b9f0fd/68937db/src/objects/string.rs:54:21
|> /home/name/.cargo/git/checkouts/pyo3-a22e69bc62b9f0fd/68937db/src/objects/string.rs:53:21
= note: expected type `*const u8`tr() as *const i8)) pyo3/python3 pyo3/extension-module --release -- --crate-type cderror: aborting due to 2 previous errors^^^^^^^^^^^^^ expected u8, found i8
error: cargo failed with code: 101^^^^^^^^^^^^^^^^^^^^^ expected u8, found i8
To learn more, run the command again with --verbose.
Python 3.6.3
rustc 1.23.0-nightly (827cb0d61 2017-11-26)
aarch64
looks like a solvable issue, just haven't hit the source yet.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.