Keepalive not working? about bjoern HOT 5 CLOSED

Thanks for the report! Can you please compile bjoern with debug output (make debug) and run your tests again?

from bjoern.

Hi Jonas, I was actually doing that as you replied :) Anyway, after further investigation, this seems to be an issue of standards implementation (and I am certainly no expert). But this is what I've observed (with the help of debug mode and tcpdumps):

Curl (by default) sends an HTTP 1.1 request, but does not set a 'Connection: Keep-Alive' header, so bjoern closes the connection (confirmed by debug messages). Curl is designed to reuse connections, so it obviously expects them to be left open (when using HTTP1.1) without sending a the keep-alive header. I think keep-alive is supposed to be default in 1.1, so I'm not sure if this request header should be mandatory? Used pycurl to manually set this header and connections are reused successfully.

ApacheBench sends HTTP 1.0 requests with the keep-alive header (required by 1.0), but expects a "Connection: keep-alive" response header, and hangs on the connection until it gets one (confirmed bjoern keeps connection alive until it times out). So the apachebench scenario is solved by explicitly setting the response header, which you can obviously do in the wsgi app code, e.g. my app changes to:

import bjoern

def application(environ, start_response):
    status = '200 OK'
    output = 'Pong!'

    response_headers = [('Content-type', 'text/plain'),
                        ('Content-Length', str(len(output))), 
                        ('Connection', 'keep-alive')]
    start_response(status, response_headers)
    return [output]

bjoern.listen(application, '192.168.1.190', 8081)
bjoern.run()

I guess the question is, does the implementation in Bjoern need to change to meet the relevant standard? (RFC 2616?)

from bjoern.

Also, just confirmed the same issue exists when using httperf as the client (with http 1.1 and keep-alives). httperf does not consider setting the response header and so hits a connection reset error when bjoern closes. Command to test:

httperf --hog --timeout=5 --client=0/1 --server=192.168.1.190 --port=8081 --uri=/ --rate=50000 --send-buffer=4096 --recv-buffer=16384 --num-conns=1 --num-calls=3 --http-version="1.1"

^attempts 3 requests on the same socket.

from bjoern.

Wow, thanks for the detailed analysis!

I think that most HTTP/1.1 implementations indeed default to Keep-Alive and if you don't want to use it you have to return Connection: close which - IIRC - bjoern currently doesn't do. Could you test the curl thing with manually settings Connection: close in the WSGI app?

(Also I agree that HTTP connection should be completely abstracted away from WSGI apps, i.e. no fiddling with the Connection header should be necessary. The server should take care of that.)

from bjoern.

If I set "connection: close" then Curl works as expected, not attempting to reuse the socket. I noticed from earlier packet captures that bjoern was not erroneously setting 'Connection: close'.

The main issue seems to be that HTTP 1.1 connections aren't treated as persistent by default, even though the code in http-parser.c http_should_keep_alive suggests it will be, and the parser flags bitmask seems to be tested ok in bjoern/wsgi.c, so I can't spot the issue. Tests from http-parser/test.c seem to return ok too...

from bjoern.