Python 3 support (PEP 3333) about bjoern HOT 18 CLOSED

@jonashaag I picked up and made it working for python3 ... not sure if this is clean enough to be merged but comments/suggestions are welcome. :-)

#104

from bjoern.

Wait, Bjoern doesn't support Python 3, in 2017, and this issue is still open? Wow. I guess this is not a viable option for a project :(

from bjoern.

• Waitress
• CherryPy
• gunicorn
• Werkzeug
• meinheld

With some of these servers you can pass an already opened socket, but in most cases it's so convoluted you'll regret even thinking about it.

To be fair uWSGI probably supports it in some way, but the documentation is killing me.

from bjoern.

Came to post same. I'm willing but have literally zero experience with C extensions. If I can be of any help let me know. If you can point me to a decent tool for or document describing porting extensions from 2 to 3 I'll pour over it and try my luck.

from bjoern.

Shouldn't be too hard as there's not much that needs to be ported.

As PyString_foo is gone in Py3, we need to use a macro for the native string type (WSGI dict key/values).

Also see http://rhodesmill.org/brandon/2008/porting-a-c-extension-module-to-python-30/ for general Py2-to-Py3 porting help.

If you have any questions feel free to write an e-mail :)

from bjoern.

For the record, here's the PEP 333 to PEP 3333 diff: http://paste.pocoo.org/show/320496/

from bjoern.

Apologies but I'm not going to be able to accomplish this on my own. Hope something here can be of help to anyone. I simply don't know C.

In my fork's current state code compiles such that vPy2 appears to remain unaffected (working) and vPy3 can at least be initialized. I receive a segfault upon making the first request, according to this backtrace.

http://docs.python.org/py3k/howto/cporting.html describes changes to:

• longs/ints
• strings
• modules

Longs/Ints

There is one line with two instances of a need to go from PyInt_FromLong to PyLong_FromLong in request.c.

Strings

These defines should cover all instances of PyString.

Module initialization

http://docs.python.org/py3k/c-api/module.html provides greater detail.

cStringIO is deprecated

I believe the cStringIO in request.c should be replaced with standard PyBytes.

I used memcpy in place of one cString function and simply did not know how to handle wsgi.input. A call to PycString_IMPORT has just been removed without replacement.

from bjoern.

Module initialization: Whoa that's ugly :-( They could at least have provided a routine hiding the truth from us :-(

Strings: You should use PyUnicode for header items in Py3 and PyString in Py2, not PyBytes in Py3. Did you read the PEP 3333 diff?

cStringIO: Good idea, actually I'm not sure why I use cStringIO at all (because the content length is known so a static string object should suffice). But cStringIO has to be turned into a file-like before calling the WSGI application, so either we use the Py3 cStringIO replacement (whatever it is) or we roll out our own wrapper (I could do that).

from bjoern.

btw the segfault is because the passed argument is a NULL pointer simply because there's no request body.

from bjoern.

Module initialization: Whoa that's ugly :-( They could at least have provided a routine hiding the truth from us :-(

Yeah it really doesn't help the < 1KLOC either. I actually just removed much of the boilerplate and it doesn't look nearly as bad.

Strings: You should use PyUnicode for header items in Py3 and PyString in Py2, not PyBytes in Py3. Did you read the PEP 3333 diff?

Yeah, not quite well enough. I actually used the official diff and got lost in the yellow, almost all of which refer to bytestrings. You're right. I just replaced all appropriate code with PyBytes and PyUnicode and reversed defined them to PyString for Py2x. Should the environ and/or the headers be handled something like this? When using PyUnicode_FromString on HTTP/1.1 400 Bad Request telnet spits garbage but PyBytes_FromString works fine..

Google fails to find cStringIO and Python3 referenced together with minor exception. Here's one. I just removed it entirely

That was the wrong segfault, sorry. I had already at least traced that one to its source. This one complains about wsgi_app not being callable which leads me to think I'm close to a runnable server. If I can get that figured out I'll be able to better work out encoding issues.

from bjoern.

I think you're getting screwed up the native string/unicode string thing at little bit, so here are some tips:

Use two header files, py2.h and py3.h, #defining all used PyStr_* routines to the Python version's native data type (str, = PyString on Py2 and PyUnicode on Py3). Then define PyBytes_* macros to PyString on Py2 (because Py2's str is Py3's bytes). PyUnicode is not used at all using Python 2 as PEP 3333 emphasizes.

The response must always be PyBytes: On Python 2, you don't have to do any encoding because PyString is bytes whereas on Python 3 I'm not sure whether the WSGI application may return PyUnicode. If that's allowed, that unicode string must be encoded somehow; I don't know which encoding to chose, though.

Your segfault is very weird. I guess you messed up refcounts somewhere else... don't worry everybody does :)

The current cStringIO replacement (using memcpy) is broken because every call to on_body overrides the previously received data. You have to keep track of the total amount of memcpyed data (extend the bjparser struct). Also remove the body = FromString(body) (line 203) because it's unnecessary and does a complete body copy.

Thank you very much for your effort!

from bjoern.

Hey Angelo, any news on Py3 support?

from bjoern.

Ping! Any plans to revive this anytime soon? If not, I might take a stab at it.

from bjoern.

Nope, no plans. There is a port by someone else https://github.com/isaiah/bjoern-py3 but I've never tested it, plus it adds too much code (bytesio object).

from bjoern.

An old issue revival to let you know that bjoern is still the only WSGI server with a SO_REUSEPORT implementation. Too bad we can't use it.

from bjoern.

Are you sure? This is a standard feature every server should implement

from bjoern.

I found this project too late :-( All this time I dreaming about lightweight Python web-server, without any dances around nginx. Do you really have no plans about Python3?

from bjoern.

No, but it shouldn't be too complicated to implement, so feel free if you have some experience with CPython C API (or are willing to learn it)!

from bjoern.