Git Product home page Git Product logo

pprofutils's Introduction

documentation ci test status

pprofutils

pprofutils is a swiss army knife for pprof files. You can use it as a command line utility or as a free web service.

Install

pprofutils requires Go 1.16 and can be installed like this:

go install github.com/felixge/pprofutils/v2/cmd/pprofutils@latest

Alternatively you can use it as a free web service hosted at https://pprof.to.

Utilities

anon

Takes a pprof profile and anonymizes it by replacing pkg, file and function names with human readable hashes. The whitelist can be used to prevent certain packages from being anonymized.

TODO: Ignore all stdlib packages by default and maybe also popular OSS libs.

The input and output file default to "-" which means stdin or stdout.

Use anon utility via cli

pprofutils anon [-whitelist=<regex>] <input file> <output file>

FLAGS:
  -whitelist=^runtime;^net;^encoding Semicolon separated pkg name regex list

Use anon utility via web service

curl --data-binary @<input file> 'pprof.to/anon?whitelist=^runtime;^net;^encoding' > <output file>

Example 1: Anonymize a CPU profile

pprofutils anon examples/anon.in.pprof examples/anon.out.pprof
# or
curl --data-binary @examples/anon.in.pprof pprof.to/anon > examples/anon.out.pprof

Converts the profile examples/anon.in.pprof that looks like this:

Into a new profile examples/anon.out.pprof that looks like this:

avg

Takes a block or mutex profile and creates a profile that contains the average time per contention by dividing the nanoseconds or value in the profile by the sample count value.

TODO: Support memory profiles.

The input and output file default to "-" which means stdin or stdout.

Use avg utility via cli

pprofutils avg <input file> <output file>

Use avg utility via web service

curl --data-binary @<input file> 'pprof.to/avg' > <output file>

Example 1: Convert block profile to avg time

pprofutils avg examples/avg.in.pprof examples/avg.out.pprof
# or
curl --data-binary @examples/avg.in.pprof pprof.to/avg > examples/avg.out.pprof

Converts the profile examples/avg.in.pprof that looks like this:

Into a new profile examples/avg.out.pprof that looks like this:

folded

Converts pprof to Brendan Gregg's folded text format and vice versa. The input format is automatically detected and used to determine the output format.

The input and output file default to "-" which means stdin or stdout.

Use folded utility via cli

pprofutils folded [-headers] [-line_numbers] <input file> <output file>

FLAGS:
  -headers=false Add header column for each sample type
  -line_numbers=false Add line numbers to the name of each frame

Use folded utility via web service

curl --data-binary @<input file> 'pprof.to/folded?headers=false&line_numbers=false' > <output file>

Example 1: Convert folded text to pprof

pprofutils folded examples/folded.in.txt examples/folded.out.pprof
# or
curl --data-binary @examples/folded.in.txt pprof.to/folded > examples/folded.out.pprof

Converts examples/folded.in.txt with the following content:

main;foo 5
main;foo;bar 3
main;foobar 4

Into a new profile examples/folded.out.pprof that looks like this:

Example 2: Convert pprof to folded text

pprofutils folded examples/folded.in.pprof examples/folded.out.txt
# or
curl --data-binary @examples/folded.in.pprof pprof.to/folded > examples/folded.out.txt

Converts the profile examples/folded.in.pprof that looks like this:

Into a new folded text file examples/folded.out.txt that looks like this:

main;foo 5
main;foobar 4
main;foo;bar 3

heapage

Adds virtual frames showing the average allocation lifetime for Go memory allocations.

The input and output file default to "-" which means stdin or stdout.

Use heapage utility via cli

pprofutils heapage -period=<period> <input file> <output file>

FLAGS:
  -period=10s The time period covered by the heap profile.

Use heapage utility via web service

curl --data-binary @<input file> 'pprof.to/heapage?period=10s' > <output file>

Example 1: Calculate Avg Inuse Object Age

pprofutils heapage examples/heapage.in.pprof examples/heapage.out.pprof
# or
curl --data-binary @examples/heapage.in.pprof pprof.to/heapage > examples/heapage.out.pprof

Converts the profile examples/heapage.in.pprof that looks like this:

Into a new profile examples/heapage.out.pprof that looks like this:

jemalloc

Converts jemalloc heap profile to pprof format.

The input and output file default to "-" which means stdin or stdout.

Use jemalloc utility via cli

pprofutils jemalloc <input file> <output file>

Use jemalloc utility via web service

curl --data-binary @<input file> 'pprof.to/jemalloc' > <output file>

json

Converts from pprof to json and vice vera. The input format is automatically detected and used to determine the output format.

The input and output file default to "-" which means stdin or stdout.

Use json utility via cli

pprofutils json <input file> <output file>

Use json utility via web service

curl --data-binary @<input file> 'pprof.to/json' > <output file>

Example 1: Convert pprof to json

pprofutils json examples/json.in.pprof examples/json.out.json
# or
curl --data-binary @examples/json.in.pprof pprof.to/json > examples/json.out.json

See examples/json.in.pprof and examples/json.out.json for more details.

Example 2: Convert json to pprof

pprofutils json examples/json.in.json examples/json.out.pprof
# or
curl --data-binary @examples/json.in.json pprof.to/json > examples/json.out.pprof

See examples/json.in.json and examples/json.out.pprof for more details.

labelframes

DEPRECATED: Use pprof -tagleaf or -tagroot instead.

Adds virtual root frames for the each value of the selected pprof label. This is useful to visualize label values in a flamegraph.

The input and output file default to "-" which means stdin or stdout.

Use labelframes utility via cli

pprofutils labelframes -label=<label> <input file> <output file>

FLAGS:
  -label=mylabel The label key to turn into virtual frames.

Use labelframes utility via web service

curl --data-binary @<input file> 'pprof.to/labelframes?label=mylabel' > <output file>

Example 1: Add root frames for pprof label values

pprofutils labelframes examples/labelframes.in.pprof examples/labelframes.out.pprof
# or
curl --data-binary @examples/labelframes.in.pprof pprof.to/labelframes > examples/labelframes.out.pprof

Converts the profile examples/labelframes.in.pprof that looks like this:

Into a new profile examples/labelframes.out.pprof that looks like this:

raw

Converts pprof to the same text format as go tool pprof -raw.

The input and output file default to "-" which means stdin or stdout.

Use raw utility via cli

pprofutils raw <input file> <output file>

Use raw utility via web service

curl --data-binary @<input file> 'pprof.to/raw' > <output file>

Example 1: Convert pprof to raw

pprofutils raw examples/raw.in.pprof examples/raw.out.txt
# or
curl --data-binary @examples/raw.in.pprof pprof.to/raw > examples/raw.out.txt

See examples/raw.in.pprof and examples/raw.out.txt for more details.

Use Cases

Convert linux perf profiles to pprof

Convert a Linux perf.data profile to pprof, via Brendan Gregg's stackcollapse-perf.pl script:

perf script | stackcollapse-perf.pl | pprofutils folded > perf.pprof

License

pprofutils is licensed under the MIT License.

pprofutils's People

Contributors

alexjf avatar dqminh avatar felixge avatar mhansen avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

pprofutils's Issues

"Bad line" error on an Android simpleperf trace

Came across this trying to convert an android simpleperf trace to pprof:

I'll note the full repo steps here in case they're useful; they probably aren't though; the error message is probably sufficient:

$ git clone https://android.googlesource.com/platform/system/extras
$ cd extras/simpleperf/scripts
$ ./app_profiler.py -p com.google.android.apps.maps -r "-f 4000 -g -e cpu-clock --trace-offcpu --duration 60" --activity com.google.android.maps.MapsActivity --skip_collect_binaries
$ ./report_sample.py ~/simpleperf/perf.data | ~/FlameGraph/stackcollapse-perf.pl | ~/go/bin/text2pprof

Gives this error:

bad line: 6: "Binder:23281_1;__start_thread;__pthread_start;thread_data_t::trampoline;android::AndroidRuntime::javaThreadShell;android::Thread::_threadLoop;android::PoolThread::threadLoop;android::IPCThreadState::joinThreadPool;android::IPCThreadState::executeCommand;android::BBinder::transact;JavaBBinder::onTransact;_JNIEnv::CallBooleanMethod;art::JNI<false>::CallBooleanMethodV;art::JValue art::InvokeVirtualOrInterfaceWithVarArgs<art::ArtMethod*> 6"

This happens on line 6... let's look at the first 10 lines:

./report_sample.py ~/simpleperf/perf.data | ~/FlameGraph/stackcollapse-perf.pl | head
Binder:23281_1;__start_thread;__pthread_start;thread_data_t::trampoline;android::AndroidRuntime::javaThreadShell;android::Thread::_threadLoop;android::PoolThread::threadLoop;android::IPCThreadState::joinThreadPool 11
Binder:23281_1;__start_thread;__pthread_start;thread_data_t::trampoline;android::AndroidRuntime::javaThreadShell;android::Thread::_threadLoop;android::PoolThread::threadLoop;android::IPCThreadState::joinThreadPool;@plt 6
Binder:23281_1;__start_thread;__pthread_start;thread_data_t::trampoline;android::AndroidRuntime::javaThreadShell;android::Thread::_threadLoop;android::PoolThread::threadLoop;android::IPCThreadState::joinThreadPool;@plt;work_pending;do_notify_resume;__schedule;__schedule 3
Binder:23281_1;__start_thread;__pthread_start;thread_data_t::trampoline;android::AndroidRuntime::javaThreadShell;android::Thread::_threadLoop;android::PoolThread::threadLoop;android::IPCThreadState::joinThreadPool;android::IPCThreadState::executeCommand 10
Binder:23281_1;__start_thread;__pthread_start;thread_data_t::trampoline;android::AndroidRuntime::javaThreadShell;android::Thread::_threadLoop;android::PoolThread::threadLoop;android::IPCThreadState::joinThreadPool;android::IPCThreadState::executeCommand;android::BBinder::transact 1
Binder:23281_1;__start_thread;__pthread_start;thread_data_t::trampoline;android::AndroidRuntime::javaThreadShell;android::Thread::_threadLoop;android::PoolThread::threadLoop;android::IPCThreadState::joinThreadPool;android::IPCThreadState::executeCommand;android::BBinder::transact;JavaBBinder::onTransact;_JNIEnv::CallBooleanMethod;art::JNI<false>::CallBooleanMethodV 2
Binder:23281_1;__start_thread;__pthread_start;thread_data_t::trampoline;android::AndroidRuntime::javaThreadShell;android::Thread::_threadLoop;android::PoolThread::threadLoop;android::IPCThreadState::joinThreadPool;android::IPCThreadState::executeCommand;android::BBinder::transact;JavaBBinder::onTransact;_JNIEnv::CallBooleanMethod;art::JNI<false>::CallBooleanMethodV;art::JValue art::InvokeVirtualOrInterfaceWithVarArgs<art::ArtMethod*> 6
Binder:23281_1;__start_thread;__pthread_start;thread_data_t::trampoline;android::AndroidRuntime::javaThreadShell;android::Thread::_threadLoop;android::PoolThread::threadLoop;android::IPCThreadState::joinThreadPool;android::IPCThreadState::executeCommand;android::BBinder::transact;JavaBBinder::onTransact;_JNIEnv::CallBooleanMethod;art::JNI<false>::CallBooleanMethodV;art::JValue art::InvokeVirtualOrInterfaceWithVarArgs<art::ArtMethod*>;art_quick_invoke_stub;android.os.Binder.execTransact 2
Binder:23281_1;__start_thread;__pthread_start;thread_data_t::trampoline;android::AndroidRuntime::javaThreadShell;android::Thread::_threadLoop;android::PoolThread::threadLoop;android::IPCThreadState::joinThreadPool;android::IPCThreadState::executeCommand;android::BBinder::transact;JavaBBinder::onTransact;_JNIEnv::CallBooleanMethod;art::JNI<false>::CallBooleanMethodV;art::JValue art::InvokeVirtualOrInterfaceWithVarArgs<art::ArtMethod*>;art_quick_invoke_stub;android.os.Binder.execTransact;android.os.Binder.execTransactInternal 2
Binder:23281_1;__start_thread;__pthread_start;thread_data_t::trampoline;android::AndroidRuntime::javaThreadShell;android::Thread::_threadLoop;android::PoolThread::threadLoop;android::IPCThreadState::joinThreadPool;android::IPCThreadState::executeCommand;android::BBinder::transact;JavaBBinder::onTransact;_JNIEnv::CallBooleanMethod;art::JNI<false>::CallBooleanMethodV;art::JValue art::InvokeVirtualOrInterfaceWithVarArgs<art::ArtMethod*>;art_quick_invoke_stub;android.os.Binder.execTransact;android.os.Binder.execTransactInternal;ExecuteNterpImpl 1

At a guess, I'd say the problem is probably the space in between JValue art.

It's very possible that I'm holding this wrong and there shouldn't be spaces in these lines according to the format? However perhaps there is a way to 'greedily' consume forward till we get to the numbers, perhaps?

Unknown Errror with JSON File

I am attempting to convert the below JSON into a PPROF file. However, pprofutils returns the below error message:

input format is neither pprof nor json

Is there a way to identify what in the file is causing this error? I have used multiple json validators to confirm the format is valid.

single_stack_trace_ex.txt

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.