Comments (3)
I do ongoing PGO research on different applications - all results are available at https://github.com/zamazan4ik/awesome-pgo . I performed some PGO benchmarks on the rust_serialization_benchmark
too and want to share my results here.
Test environment
- Fedora 39
- Linux kernel 6.7.6
- AMD Ryzen 9 5900x
- 48 Gib RAM
- SSD Samsung 980 Pro 2 Tib
- Compiler - Rustc 1.76
- Repo version:
master
branch on commitce821970017832f43d00bc5110462ff1a2c38e17
- Disabled Turbo boost for improving consistency across runs
Benchmark
Release benchmarks are done with tasket -c 0 cargo +nightly bench
, PGO training phase with tasket -c 0 cargo +nightly pgo bench
, PGO optimization with tasket -c 0 cargo +nightly pgo optimize bench
. taskset -c 0
is used for better benchmark consistency. All PGO-related routines are done with cargo-pgo. All benchmarks are done on the same machine, with the same hardware/software during runs, with the same background "noise" (as much as I can guarantee, of course).
Results
Here are the results:
- Release: https://gist.github.com/zamazan4ik/292d71316488e3e09bcb56f4e499f42e
- PGO-optimized: https://gist.github.com/zamazan4ik/5cfb7a8acbe37ca3e876ba1a2468f5d5
- (just for reference) PGO instrumented: https://gist.github.com/zamazan4ik/84ff553619aae1faab7a40fa33944d86
At least in the provided by project benchmarks, there are measurable improvements in many cases. However, also there are some regressions.
Look at autofdo with perf record.
I recommend starting with the regular PGO via instrumentation. AutoFDO is used for sampling the PGO approach. Starting with the instrumentation is generally a better idea since it has wider platform support and can be easily enabled for the project (compared to the sampling-based PGO).
from rust_serialization_benchmark.
Great job presenting the results of PGO! In general, it seems like PGO increases average performance by ~5% but introduces noise. Not necessarily from run to run, but from benchmark to benchmark and version to version. Might make more sense to average PGO results over all datasets (and just live with the fact that there are only 4, so the average isn't totally immune to noise). Could also average over all crates, and just report a single PGO result. The goal is to give users an accurate answer for 1) which crate 2) whether to try PGO.
from rust_serialization_benchmark.
Look at autofdo with perf record
.
from rust_serialization_benchmark.
Related Issues (20)
- Add validated/unvalidated benchmarks HOT 3
- Add memory and code size benchmarks
- Benchmark `abomonation` deserialization using `.clone()` HOT 1
- Flatbuffers serialization could be faster HOT 1
- Add support for simd-json HOT 1
- Benchmark including I/O HOT 3
- Add support for bebop HOT 2
- Compress using `zstd` instead of `zlib` HOT 1
- Bincode deserialization could be faster HOT 2
- Cannot compile on linux HOT 10
- Join rosetta-rs? HOT 1
- Add ciborium HOT 1
- use simd-json-derive for simd-json benchmarks
- Framework hinting standard HOT 1
- Add SBE (Simple Binary Encoding)
- Help with adding ε-serde HOT 20
- Add databuf.rs
- Can't add new framework on MacOS HOT 4
- Apache Avro is not well supported
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rust_serialization_benchmark.