Git Product home page Git Product logo

tokio's Introduction

Asyncio event loop based on tokio-rs (WIP) Build Status Join the dev chat at https://gitter.im/PyO3/Lobby

async-tokio is a drop-in replacement of the built-in asyncio event loop. async-tokio is implemented in rust and uses tokio-rs under the hood and PyO3 python binding.

Using tokio loop

You can create an instance of the loop manually, using:

import tokio

policy = tokio.EventLoopPolicy()
asyncio.set_event_loop_policy(policy)

Development of tokio loop

To build tokio loop, you'll need rust nightly and Python 3.6. The best way is to create a virtual env, so that you'll have python commands pointing to the correct tools.

  1. git clone [email protected]:PyO3/tokio.git
  2. cd tokio
  3. make build
  4. make test

Status

At the moment async-tokio works on unix like systems only. Supported api:

  • time api (call_at, call_later)
  • sockets api (sock_xxx methods)
  • tcp (client/server)
  • unix domain socket
  • dns
  • pipes
  • subprocess
  • signals
  • executors

UDP support is missing.

License

async-tokio is offered under the Apache 2.0 licenses.

tokio's People

Contributors

fafhrd91 avatar messense avatar popravich avatar pyup-bot avatar wmanley avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

tokio's Issues

documentation error: TokioLoopPolicy not EventLoopPolicy

I just had a quick try with tokio, I'm currently getting a slightly speed up (~5%) compared to asyncio loop, but significantly less quick than uvloop. Is that to be expected at this stage?

Also your readme refer to tokio.EventLoopPolicy() which is wrong, as of pypi v0.1.0 the policy is called tokio.TokioLoopPolicy()

New python binding implementation

We now use new python binding https://github.com/PyO3/PyO3
it is based on rust-cpython, but whole class generation has been significantly redesigned
now code looks more like rust and not like some kind of python:

#[py:class]
pub struct Socket {
  connected: bool
}

#[py:proto]
impl PyIterProtocol for Socket {
  fn __iter__(&self, py: Python) -> PyResult<PyObject> {
  }
}

#[py:proto]
impl PyAsyncProtocol for Socket {
  fn __await__(&self, py: Python) -> PyResult<PyObject> {
  }
}

most of class functionality of already implemented, everything else should be completed within next week

Better documentation for rust-python interop

Hey, this is great. It's exactly what I need for a project I'm working on. Is there any documentation or examples on how to use it in a project using Rust code with tokio that calls python with asyncjo/aiohttp? Thanks much.

Transport.get_extra_info('socket') should return a socket-like object

While working on uvloop I've iterated on the following three different approaches to how get_extra_info('socket') should be implemented:

  1. Wrap the underlying socket FD into a Python socket object: socket.socket(fileno=FD). The downside of this approach is that Python will try to control the FD. For instance, when the socket object is GCed, Python will try to close the FD, which isn't acceptable for uvloop and asyncio-tokio. Moreover, beginning with Python 3.6 there is a race to who closes the socket first, and Python expects to win, otherwise it will raise an IOError in socket.close().

  2. Duplicate the FD before wrapping (currently implemented in uvloop). This fixes (1), but has its own downsides: FDs are a limited resource and now we need 2x of them; FDs can stay alive longer than necessary (if the transport is closed, but its duplicated sockets are alive).

  3. Return a socket-like object. Essentially, the object should implement all socket.socket methods and have all of its attributes (to be duck-type compatible), but the implementation of socket.close() and socket.shutdown() will be empty (or will raise an error).

The third option is the one I want to implement in uvloop, and I believe the one we need to implement for asyncio-tokio.

add __weakref__ slot support

what classes should support weakrefs? PyHandle, PyTimerHandler needs to support for sure. what about Future and Task objects?

How to distribute?

should we provide python module with async-tokio? at the moment async-tokio is rust crate, to run event loop aiohttp-tokio is required.

impl future::Future for PyFuture

what type PyFuture should resolve to?

currently, Future::Item is PyResult<PyObject> and Future::Err is unsync::oneshot::Cancelled

future handling code looks like:

PyFuture::new(...).map(|res: PyResult<PyObject>| {
    match res {
        Ok(obj) => "do something with result (PyObject)"
        Err(err) => "do something with py exception"
    }
}).map_err(|err| "cancelled");

other option is to use PyObject as Future::Item and PyErr as Future::Err:

PyFuture::new(...).map(|res: PyObject| {
    "do something with result (PyObject)"
}).map_err(|err| {
    "do something with py exception"
);

I like second option, but there would be no explicit cancelation state

Add copyright headers to tests

Proposed header:

# Copied from the uvloop project.  If you add a new unittest here,
# please consider contributing it to the uvloop project.
#
# Portions copyright (c) 2015-present MagicStack Inc.  http://magic.io

add signal handling

async-tokio already uses tokio-signal crate for ctrl-c handling
should be easy to extend to all signals

building issues

(tokio) rocklobster:~/python/tokio $ python setup.py build
running build
running build_py
running egg_info
writing tokio.egg-info/PKG-INFO
writing dependency_links to tokio.egg-info/dependency_links.txt
error[E0308]: mismatched typesio.egg-info/top_level.txt
  --> /home/name/.cargo/git/checkouts/pyo3-a22e69bc62b9f0fd/68937db/src/objects/string.rs:54:21
   |> /home/name/.cargo/git/checkouts/pyo3-a22e69bc62b9f0fd/68937db/src/objects/string.rs:53:21
   = note: expected type `*const u8`tr() as *const i8)) pyo3/python3 pyo3/extension-module --release -- --crate-type cderror: aborting due to 2 previous errors^^^^^^^^^^^^^ expected u8, found i8
error: cargo failed with code: 101^^^^^^^^^^^^^^^^^^^^^ expected u8, found i8
To learn more, run the command again with --verbose.

Python 3.6.3
rustc 1.23.0-nightly (827cb0d61 2017-11-26)
aarch64

looks like a solvable issue, just haven't hit the source yet.

Replace loop.current_task() with officially supported API

loop.current_task() was never a part of asyncio API.
Starting from 3.7 third party loops can use dedicated calls for custom task implementation:
asyncio._register_task, ``asyncio._unregister_task, asyncio._enter_task`, `asyncio._leave_task`.

The API calls are started from underscore to emphasize these very low level position (should be used by libraries like tokio only) but they are a part of of official API.

test_parse_headers_multi is broken

running 51 tests
test test_compression_unknown ... ok
test test_compression_deflate ... ok
test test_compression_gzip ... ok
test test_conn_close ... ok
test test_conn_close_and_upgrade ... ok
test test_conn_close_and_upgrade_order ... ok
test test_conn_close_1_0 ... ok
test test_conn_default_1_0 ... ok
test test_conn_default_1_1 ... ok
test test_conn_keep_alive_1_0 ... ok
test test_conn_other_1_0 ... ok
test test_conn_other_1_1 ... ok
test test_headers_content_length_err_1 ... ok
test test_conn_upgrade ... ok
test test_headers_content_length_err_2 ... ok
test test_headers_multi_feed ... ok
test test_http_payload_parser_length ... ok
test test_http_request_bad_status_line ... ok
test test_http_request_chunked_payload ... ok
test test_http_request_chunked_payload_and_next_message ... ok
test test_http_request_chunked_payload_chunks ... ok
test test_http_request_parser_bad_method ... ok
test test_http_request_parser_bad_version ... ok
test test_http_request_parser_non_utf8 ... ok
test test_http_request_parser_two_slashes ... ok
test test_http_request_parser_utf8 ... ok
test test_http_request_upgrade ... ok
test test_http_request_websocket ... ok
test test_invalid_header ... ok
test test_invalid_name ... ok
test test_parse_body ... ok
test test_parse_chunked_payload_chunk_extension ... ok
test test_parse_chunked_payload_size_error ... ok
test test_parse_delayed ... ok
test test_parse_eof_payload ... ok
test test_parse_headers_max_multi ... ok
test test_parse_length_payload ... ok
test test_parse_headers_multi ... FAILED
test test_parse_length_payload_eof ... ok
test test_parse_no_length_payload ... ok
test test_request_chunked ... ok
test test_request_chunked_partial ... ok
test test_request_simple ... ok
test test_request_simple_10 ... ok
test test_request_simple_with_prefix_cr ... ok
test test_request_simple_with_prefix_crlf ... ok
test test_special_headers_partial ... ok
test test_http_request_max_status_line ... ok
test test_max_header_value_size ... ok
test test_max_header_value_size_continuation ... ok
test test_max_header_name_size ... ok

failures:

---- test_parse_headers_multi stdout ----
	thread 'test_parse_headers_multi' panicked at 'assertion failed: `(left == right)` (left: `"c2=cookie2"`, right: `"c1=cookie1"`)', tests/test_http_decoder.rs:260
note: Run with `RUST_BACKTRACE=1` for a backtrace.


failures:
    test_parse_headers_multi

test result: FAILED. 50 passed; 1 failed; 0 ignored; 0 measured

error: test failed, to rerun pass '--test test_http_decoder'

asyncio.Task.current_task() returns None

If I register a signal handler within a task to cancel itself, tokio failes with:
AttributeError: 'NoneType' object has no attribute 'cancel'

async def test_tokio_failes():
    current_task = asyncio.Task.current_task()
    asyncio.get_event_loop().add_signal_handler(signal.SIGINT, current_task.cancel)

def main():
    # ...
    loop.run_until_complete(test_tokio_failes())

add native PIPES api

current implementation is based on asyncio.unix_events, we need native rust pipes support

need to decide how to do logging

exception_handler api is implemented, but rust level logging is not defined.
should we use python logging? or rust "log" or "slog" crates and separately define logging configuration api.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.