Git Product home page Git Product logo

duet's Introduction

duet

A simple future-based async library for python

Duet takes inspiration from the amazing trio library and the structured concurrency approach to async programming that it uses. However, duet differs from trio in two major ways:

  • Instead of a full-blown implementation of asynchronous IO, duet relies on the Future interface for parallelism, and provides a way to run async/await coroutines around those Futures. This is useful if you are using an API that returns futures, such as RPC libraries like gRPC. The standard Future interface does not implement __await__ directly, so Future instances must be wrapped in duet.AwaitableFuture.

  • duet is re-entrant. At the top level, you run async code by calling duet.run(foo). Inside foo suppose you call a function that has not yet been fully refactored to be asynchronous, but itself calls duet.run(bar). Most async libraries, including trio and asyncio, will raise an exception if you try to "re-enter" the event loop in this way, but duet allows it. We have found that this can simplify the process of refactoring code to be asynchronous because you don't have to completely separate the sync and async parts of your codebase all at once.

Installation

Install from pypi:

pip install duet

Note

duet is not an official Google project.

duet's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

duet's Issues

Add tests of typing plugin for duet.sync

I'm not sure how to do this, but it'd be really nice to have tests that the typing plugin for duet.sync works properly. This could avoid breakages like the one fixed by #36. I've looked around in the mypy docs for some tips about testing type annotations, but haven't found much, so we might have to roll our own scheme.

Problems with repeated timeouts in the same task

There seems to be an issue with repeated interrupts in the same task. I discovered this when trying to sleep in a loop, but doing two back-to-back sleeps also fails:

await duet.sleep(1)
await duet.sleep(1)

The first sleep succeeds, but the second sleep causes duet to block indefinitely. I haven't diagnosed the root cause here, but it is probably a confusion with edge- versus level-triggering of interrupts. Basically, we need to allow a given task to be interrupted more than once as long as interrupts are handled and don't cause the task to exit.

Should CancelledError derive from BaseException?

Currently when a task is cancelled we use concurrent.futures.CancelledError, which derives from Exception, rather than BaseException. The problem with this is that code that catches Exception will also capture CancelledError and may thus inadvertently swallow the cancellation. We might want to use an exception type that derives from BaseException instead, so that it will propagate by default unless the user does something to explicitly catch it.

Add deadline/timeout support

It'd be very useful to have support for deadlines and timeouts in duet (deadlines are absolute points in time while timeouts are relative to now). In what follows, I will say "deadlines" to refer to both. Here's a sketch of how the API could work:

async with duet.deadline(time.time() + 1):
    await some_future
    # raises TimeoutError and cancels some_future if it does not complete in 1 sec

async with duet.timeout(1):
    await some_future
    # raises TimeoutError and cancels some_future if it does not complete in 1 sec

Deadlines are scoped, and any async tasks spawned within the scope or any subscopes will inherit the same deadline. For convenience, we will allow specifying a deadline or timeout directly on duet.new_scope via duet.new_scope(deadline=...) or duet.new_scope(timeout=...):

async with duet.new_scope(deadline=when) as scope:
    ...

# equivalent to:
async with duet.deadline(when):
    async with duet.new_scope() as scope:
        ...

Implementation of this would involve storing the current (optional) deadline somewhere and using it in Scheduler.tick to avoid blocking forever waiting for tasks to become ready (e.g. the Condition.wait call in ReadySet.get_all).

Add support for python 3.6

We'd like to be able to use duet in python3.6 since it's still used by colab and internally at google (see quantumlib/Cirq#4009). Among other things, this will require adding back in support for AsyncContextManager and adding a 3.6-only dependency on contextvars. There may be some other changes required as well.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.