mitogen.core.Router._loop_once() passes sets to select.select() about mitogen HOT 9 CLOSED

Hrm annoying, I'd call it a docs bug, since behaviour in CPython (and PyPy) going back forever is that it just takes an iterable. Does MyPy have some kind of override facility? Converting the sets to lists on every loop would be wasteful, just to match basically a doc bug (the structure passed to select is inherently useless and must internally be iterated to transform it into an fdset anyway).

from mitogen.

We could potentially try just replacing the sets with lists, in the usual case they're only going to contain <5 entries anyway, and the code might even end up faster. The question would be after I get async connect finished, whether scanning the lists for add/removal slows down with a master that has hundreds of connections

from mitogen.

I don't think so.. once a connection is established to a context, it is never removed from the readers set (so there is never any scan of the list until a disconnection occurs).

The writers set only contains contexts that currently have data buffered, and in the typical case it'll simply be empty. I'm not sure about the worst case, though

from mitogen.

mind.. blown.. select() is converting the set into a list internally (via PySequence_Fast()). So there's a real chance swapping the set for a list will improve things

from mitogen.

THank you for looking. FTR changing the arguments to sorted(self...) didn't make any difference to unit test failures.

Does MyPy have some kind of override facility?

Yes, I can add a # type: ignore on a per-line basis

from mitogen.

FTR I've push the MyPy trial to https://github.com/moreati/mitogen/tree/mypy-try. I'm not proposing to do anything more with it. I think I've learned what I can.

from mitogen.

What's your verdict on MyPy? I'm not completely averse to the idea of it, especially since the annotations are just comments, and don't forget you really did find a bug :).. maybe we could merge it

from mitogen.

What's your verdict on MyPy?

On a per bug found basis it was an expensive exercise. As a learning exercise for me it was useful. As a way of documenting the APIs in-situ I found it very handy. I don't know of a way to feed MyPy annotations into doctools, so there would be some duplicated effort and or chance of discrepancies there.

I'm not completely averse to the idea of it, especially since the annotations are just comments, and don't forget you really did find a bug :).. maybe we could merge it

Not all the changes are comments, e.g. I sprinked a few asserts, and changed a base class or two

moreati@5df74fc#diff-60eabbf230c5d9137e81bf0f41ed37bbR104
moreati@5df74fc#diff-3540c3617f4566023f000914cebe2d71L317

Edit: I also had to delete the vendored pkgutil to stop an import loop

from mitogen.

Going to close this and open a separate ticket to keep track of your MyPy stuff :)

from mitogen.