Comments (4)
When I designed pyperf, I didn't want to run tests in more than 1 process for different reasons:
- A single CPU may benefit from Turbo Boost (even if pyperf suggests to disable it)
- If you only isolate a single physical CPU core, multiple processes will race to grab CPU time
- Intel CPU now have complex rules to select the CPU frequency: C-state, P-state, Turbo Boost, etc.
If you are confident that multiple processes will have similar or also the same performance, you can run a single process with computes tons of values in the same process: -p 1 -n 50, for example. Linux ASLR makes performance harder to measure.
Code placement became more critical for L1-instruction cache performance. See the BOLT compiler to see how it matters on nowadays Intel CPUs.
Well, said differently: you have to make a compromise between your "time budget" (how long all benchmarks will take on the wall clock) and the accuracy of results (get "reproducible" and "stable" benchmarks results). pyperf default configuration is designed for best accuracy. There is the --fast
option which is a small trade-off towards shorter time budget: https://pyperf.readthedocs.io/en/latest/runner.html#loop-iterations
I also suggest you to have a look at my old articles: https://vstinner.github.io/category/benchmark.html
These days, I am away from pyperf and I let others like @corona10 considers changing pyperf ;-)
from pyperf.
Well, I am on the same side with @vstinner
We can provide such an option as optional until we know the use case of such an option.
I am conservative in providing such options.
Our biggest user is pyperformance, but I am sure that they want accuracy more than fast execution.
from pyperf.
Thanks for the reply. I am thinking if we can make pyperf to run load test as well with ability to generate request with certain qps. Another thought about that, or another project for that?
from pyperf.
if we can make pyperf to run load test as well with ability to generate request with certain qps.
If you are considering load tests for specific tests, why are you considering using pyperf instead of load testing tools?
I think that pyperf measures the performance of python code rather than measuring stress-test for external services.
from pyperf.
Related Issues (20)
- Increase default running time per process? HOT 7
- Having trouble while using pypy: pyperf is much slower than timeit HOT 4
- Waiting for runner's bench_func and bench_command functions to complete instead of receiving outputs individually? HOT 5
- Allow for warmup in PEP659 HOT 7
- Pyperf API examples fail when running from a REPL HOT 2
- Propagating PYTHONPATH in tests HOT 2
- Support custom loop factory in bench_async_func
- Instability and slowness after using `pyperf system tune` HOT 2
- Display additional information through table.
- Programmatically define output file for bench_func HOT 1
- bench_func returning None for all but the first benchmark
- How to programmatically get the output for timeit() or bench_func()? HOT 9
- Add Heirarchical Performance Testing (HPT) technique to `compare_to`? HOT 3
- 2.6.2: sphinx warnings `reference target not found`
- Disable pyperf for free-threaded build for a while HOT 1
- Include `CONFIG_ARGS` in Metadata HOT 3
- Benchmark with disabled GIL build and -Xgil=1 isn't enabling GIL HOT 8
- `warmups` not a supported parameter for Runner anymore HOT 1
- bench_command doesn't collect pystats
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pyperf.