This program provides a lightweight real-time view of system calls and traces python/java/php/ruby function calls
It uses eBPF (linux only) to trace and report stats on syscalls and functions/methods calls made by python/java/php/ruby. By default it traces every system calls for every processes. It then prints the info in a top-like manner.
You can also trace your application calls by selecting its pid within the tool.
- display number, rate and latency of system calls and functions/methods calls.
- top like output.
- increase / decrease refresh rate
- sort stats (pid, process name, total count, rate)
- reset stats
- increase / decrease refresh rate
- filtering at the command line:
- on the function name
- on PIDs
- on process names
- filtering in the tool
- filter dynamically on process name, command line, pid, system call, or function.
- batch mode (for your scripts)
- trace userspace application functions.
- integration with graphing tools
- details stats on a given function
The usage output brings most of the information
$ sudo ./calltop.py -h
usage: calltop.py [-h] [-e SYSCALL] [-i INTERVAL] [-p PID] [-c COMM]
[--no-latency] [-b]
It prints realtime view of the Linux syscalls but also languages method calls.
It uses eBPF to do the tracing. So it is working only on Linux.
optional arguments:
-h, --help show this help message and exit
-e SYSCALL, --syscall SYSCALL
-e open,read,write,sendto. Used to trace ONLY specific
syscalls. It uses kprobe. Without this option
TRACEPOINT are used to get all syscalls.
-i INTERVAL, --interval INTERVAL
Set the interval in sec : -i 0.5
-p PID, --pid PID Filter on pids : --pid 10001,10002,10003
-c COMM, --comm COMM Filter on comm : --comm nginx,memcache,redis
--no-latency Do not display latency of the functions you trace. It
saves a few nanoseconds per call.
-b, --batch Print output in batch mode
Then when the tool is running you can :
- filter on process name : [f] key. Type the filter and press ENTER
- The filter should look like this :
pid:1234sys:read,comm:nginxsys:bpf,comm:calltopfn:print_body,comm:calltop,pid:1234
- The filter should look like this :
- trace function/method call from python/java/php/ruby in your app : [t] key. Type pid of the process you want to trace and press ENTER to validate. It attaches USDT to this pid.
- reset the datas : 'z' key
- print the command line : 'c' key
- sort processes with the arrow key (right and left key)
- you can sort on pid, process name, rate and total call number.
- you can also revert the sort (increasing or decreasing order) by pressing R keys.
- sort the stats within each process. You can select to sort it on :
- function name
- latency
- call/s
- Total
This tool is written in python and just need the bcc packages. See section below.
It requires the following packages:
$ dpkg -l | grep -e bpfcc
ii bpfcc-tools 0.12.0-2 all tools for BPF Compiler Collection (BCC)
ii libbpfcc 0.12.0-2 amd64 shared library for BPF Compiler Collection (BCC)
ii python-bpfcc 0.12.0-2 all Python wrappers for BPF Compiler Collection (BCC)
ii python3-bpfcc 0.12.0-2 all Python 3 wrappers for BPF Compiler Collection (BCC)You will need to install the above packages. It is already packaged in the major Linux distributions. The packages name may change according to the distribs. The minimum version is 0.12.0. If your distribution is not packaging v0.12+, you can follow the detailed installation guide of the iovisor/bcc project.
You can find Dockerfiles. This will help you to test the tool. Because of the container, you will be limited to syscall tracing. Python tracing is not supported in container because we can't see the host processes).
To build and run this container use these command lines. You need to be root and add the privileged flag to run it.
docker build . -t calltop --file [email protected]
sudo docker run -it -v /sys:/sys -v /lib/modules:/lib/modules -v /usr/src/:/usr/src --privileged calltopHave fun !
Developped by Emilien Gobillot
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent