Comments (7)
For environment specific libraries like this one, I recommend re-running tests per environment. It would likely require a fairly large text matrix though.
For example, for ReactPy we run each part of our code through IOPS tests. The concept is the same, although we focus on network IO which means we need a live web browser.
Since 99% percent of users only care about read/write performance, your test result matrix might look something like this:
Read
This test repeatedly reads 5MB of binary data from a file, and measures how many reads occurred within 60 seconds (higher is better).
1 Worker | 500 Workers | 1 Worker (Linux) | 500 Workers (Linux) | |
---|---|---|---|---|
aiofile |
x | x | x | x |
aiofiles |
x | x | x | x |
stdlib | x | x | x | x |
Write
This test repeatedly writes 5MB of binary data to a file, and measures how many writes occurred within 60 seconds (higher is better).
1 Worker | 500 Workers | 1 Worker (Linux) | 500 Workers (Linux) | |
---|---|---|---|---|
aiofile |
x | x | x | x |
aiofiles |
x | x | x | x |
stdlib | x | x | x | x |
Since I probably don't need to show you how to create the test cases themselves, I'll give some top-level tips.
I recommend reading/writing binary data in order to minimize variables. Also, you should use asyncio.gather
to spawn workers. For example...
import asyncio
async def read_binary_test():
...
async def main():
tasks = [read_test() for _ in range(500)]
await asyncio.gather(*tasks)
if __name__ == "__main__":
asyncio.run(main())
The write test should make sure to store BINARY_DATA_5MB
as a global to avoid reconstructing it every time, which would impact test results
I arbitrarily chose 5MB as a value that gives Python enough "downtime" to actually do something else while IO is being executed.
The test cases should be written to stop recording results after 60 seconds has elapsed.
from aiofile.
The performance actually depends on what underlaying implementation of caio
is currently used in the benchmark environment. The linux_aio
-based implementation, of course, only for linux this have a better performance.
The thread_aio
and python_aio
implementations should not show much improvement over aiofiles in the general case, but should be slightly better on POSIX-compatible OSes.
I have no ideas how to make this benchmark fair enough, if you have any share it with me, preferably the code.
from aiofile.
This would be quite valuable.
When starting new project I'm not even sure if I should bother with asyncio .
According to #18 (comment) performance before new implementation was atrocious (i.e. you will be better of having blocking stdlib write/reads most of the time).
#18 (comment) looks like a huge improvement have been achieved, but does not compare against stdlib, which is quite crucial.
As for which interface need the benchmark the most - even doing it for just for linux would be good enough, since most of the applications in which performance is critical, tend to be hosted on servers, which tend to be Linux-based.
I tried rerunning test from the aforementioned comment, but the results look bad. I hope this benchmark is somewhat botched ( I did not try to debug it, just changed it a little to produce table automatically): https://gist.github.com/rooterkyberian/a2c12fc6269c86bcf4e199149eb6b9ec .
Results I got:
Python version: 3.11.6
Platform: Linux-6.5.0-25-generic-x86_64-with-glibc2.35
aiofiles version: 23.2.1
aiofile version: 3.8.8
uvloop version: 0.19.0
aiofile default context <caio.linux_aio_asyncio.AsyncioContext object at 0x7e2eb3ac0f50>
iterations | sync | async executor 'dumb' | async executor w/ coroutines | async multiple executors | async aiofiles | async aiofile | aiofiles@uvloop | aiofile@uvloop |
---|---|---|---|---|---|---|---|---|
1000 | 0.006 | 0.01 | 0.007 | 0.01 | 0.748 | 0.303 | 0.455 | 0.293 |
10000 | 0.051 | 0.059 | 0.059 | 0.061 | 7.558 | 3.09 | 4.653 | 2.709 |
100000 | 0.481 | 0.49 | 0.666 | 0.518 | 74.853 | 190.855 | 47.254 | 185.326 |
1e+06 | 4.676 | 4.651 | 4.85 | 5.199 | 746.184 |
I left it running more than an hour after last result and still nothing, so it seems like either benchmarking script is broken or aiofile (caio backend) is.
from aiofile.
Related Issues (20)
- Can't install on Amazon Linux HOT 2
- Function not implemented HOT 1
- async_open does not create file if file does not exist in mode 'a+' HOT 1
- Add support for StringIO and BytesIO HOT 1
- Can I use aiofile.async_open without a with statement? HOT 4
- Race condition in `AIOFile.open()` HOT 4
- Some sort of changelog / release notes?
- async_open doesn't fully mimic the behavior of Python file objects HOT 5
- Manual context management example on readme has a bug HOT 1
- How to work with NFS? HOT 1
- Tag the source HOT 1
- AIOFile context manager looses data HOT 3
- aiofile LineReader does a read for every line in spite of having multiple lines in CHUNK_SIZE HOT 2
- Which of the methods are coroutine-safe? HOT 2
- Why is fsync calling fdsync HOT 2
- No flush API and no way to set unbuffered writes in binary mode HOT 3
- Unable to call close() twice
- Memory leak during import
- TextFileWrapper.read reads more than requested
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from aiofile.