T2 instances inherently rely on excess CPU cycles and therefore do not provide for a reliable performance baseline upon which benchmarks can be run accurately.

Considering running the benchmarks on a dedicated private instance.

Starlite is gaining in popularity due to it's appealing featureset. Please include it.

As stated here http://flask.pocoo.org/docs/0.11/api/#flask.json.jsonify flask pre-processes the json response with "pretty print" if the header X-Requested-With: XMLHttpRequest was not used in the request so, the "encode a object to json" test requests should have that header to make things fair.

Hi ! I would like to add Vibora to the benchmarks list. Are you open to a pull request ? :)

How about adding Masonite framework in the list?
Masonite Framework

Hi @klen 👋

I think the pipeline is not running anymore because GitHub limits it when a repository doesn't receive changes for a period of time. The last one was https://github.com/klen/py-frameworks-bench/actions/runs/1982085780 👀

Would you mind checking it? If you don't have time, @adriangb or myself can also help here. 🙏

Would be really helpful to add this web server

• https://github.com/squeaky-pl/japronto

and also if this benchmark coincide with your results

• https://medium.com/@squeaky_pl/million-requests-per-second-with-python-95c137af319

Thanks.

just want to see the real performance of sanic~

Since you're varying the ORM based on the http framework the benchmarks seem vary misleading. Some were using pee_wee, some sqlalchemy, and django was using django. Couldn't you make the benchmarks all use pee_wee so you're bench-marking the web delivery part of the framework in isolation of ORM performance and features?

Hi,

I'd like to fork the codebase to do some improvements and use it to verify some other behavior, but I noticed this project has no license.

can you add a license? If there's no strong preference, MIT would be simplest, but really any license that lets me use the code in a derivative work is sufficient.

Thanks for your time!

The benchmark doesn't include cherrypy

do you have any plan to add this ?

Some framework use simplejson as default(flask), some use built-in json module(tornado).

May I suggest "test setup" or "methodology?"

Cheers.

If I didn't include the --provider=virtualbox vagrant tried to use libvirt, with this it went along.

The ORM benchmark seems to state the 50% time for AIOHttp incorrectly - the value listed in the table is actually the min (as shown on the graph). The value for 50% is rather 902.72. Screenshot below for details:

Please add robyn which uses rust under the hood.

https://github.com/sansyrox/robyn
https://robyn.tech/

The benchmark is missing the number of requests/s. I ran a similar test as the JSON one, and yes flask is faster than aiohttp with those wrk settings. However, aiohttp handles more than 11 times more requests as flask. 4098 vs 359 in my case.

The django database connection doesn't have CON_MAX_AGE set, which means that it will make a new database connection every time (the default behaviour), rather than keeping the TCP/IP connection open, which can be very slow.

Is empty response faster than json/remote/message?

maybe help.

I'm getting a TASK [bench.setup git]

which fails because it Failed to find required executable git.

I did a vagrant ssh and apt-get install git and then it passed, only to fail on make in TASK [bench.setup command].

Would like to see the performance of these two compared to Flask itself.
Quart: https://pgjones.gitlab.io/quart/
Klein: http://klein.readthedocs.io/en/latest/index.html

Which framework is best for what? Could you give some hints to a manager (or six year old kid) please ;-)

I wanted to point out the discussion at https://news.ycombinator.com/item?id=9724999 in case it wasn't on your radar.

Several frameworks's apps are written in a manner that does not reflect the usage suggested in their docs.

For example, the FastAPI documentation suggests using File() to have files extracted from forms and injected: https://fastapi.tiangolo.com/tutorial/request-files/. But in the benchmarks the Request.form is used just like in Starlette. This is obviously going to be faster, but then it's not using any of the features the framework provides on top of Starlette. If that's what users were doing, they'd be using Starlette.

I suggest that the apps get restructured to better reflect the documented usage of each framework. That is much more useful to users (and framework developers) and would help highlight issues like tiangolo/fastapi#4187

Проще показать, внимание на скриншоты.

gunicorn -k uvicorn.workers.UvicornWorker -b 0.0.0.0:8080 app:app — самый тормозной вариант, не понимаю что происходит, и не хочется разбираться, но на каждый запрос принтится лого, отсюда и результат.

gunicorn -k sanic.worker.GunicornWorker -b 0.0.0.0:8080 app:app — более адекватный вариант, опять же, разбираться лень, но принтится access_log, что влияет на результат, как отключить, я сходу не нашёл.

sanic --host 0.0.0.0 --port=8080 --workers=1 app:app — дефолтный и наиболее корректный способ запуска, дающий результат на уровне blacksheep.

И, собственно, blacksheep, для сравнения:

can you measure web2py too on your benchmarks?

It would be awesome if the Python's Web Framework Benchmarks were updated to reflect the current state of affairs (2018/2019).