egobillot / calltop Goto Github PK
View Code? Open in Web Editor NEWcalltop is a tracing tool. It provides a dynamic real-time view of system calls on Linux. It traces also python, java, php and ruby function calls.
License: Apache License 2.0
calltop is a tracing tool. It provides a dynamic real-time view of system calls on Linux. It traces also python, java, php and ruby function calls.
License: Apache License 2.0
Is your feature request related to a problem? Please describe.
Tracing python is cool, tracing more language is super cool 👍
Describe the solution you'd like
Describe the bug
Interactive mode provides a way to sort based on different criteria.
But its seems very limited and seems to work in a quite confusing way.
Expected behavior
It should be possible to change and choose ascending or descending sort.
Additional information
When sorting by "total count" or by "Call/s", resulting order does not seem to be very logic.
It seems to me that process are first ordered by the total calls (all functions summed up), which is value that is not shown ! Maybe you should add a "process summary" line for each process ?
Even though the order seems wrong to me (?). I would expect that, for each process, lines are ordered by descending call/s or total, but it is apparently not the case.
So it is unclear whether sort applies to processes, functions for each process, or both (somehow).
pid filtering have to done in eBPF
The PID filtering at the command line is done in user space. In order to reduce the impact, filtering in eBPF is a better solution.
eBPF filtering
the idea is to used the pid(s) in the eBPF and if it does not match then return 0;
As describes in the docstring monotonic_time() returns time in nanoseconds.
The time returned is in second.
Please fix it.
Is your feature request related to a problem? Please describe.
We have the cumulated number of call, but not the cumulated time.
Describe the solution you'd like
One more column
Is your feature request related to a problem? Please describe.
if pid is selected with the option -p pid at the command line, then attach an usdt directly.
Is your feature request related to a problem? Please describe.
For certains process like python, the tool only print python which is not usefull to clearly identify the application.
Describe the solution you'd like
2 solutions :
Use the pid to collect the command line in /proc/PID/cmdline :
Note that you will need to replace \NUL char by SPACE. see the output below
cat /proc/3285/cmdline| tr '\0' ' '
/usr/lib/firefox/firefox -contentproc -childID 3 -isForBrowser -prefsLen 7169 -prefMapSize 215831 -parentBuildID 20200403064753 -greomni /usr/lib/firefox/omni.ja -appomni /usr/lib/firefox/browser/omni.ja -appdir /usr/lib/firefox/browser 3081 true tab
Some key like [s] or [e] are not documented, and help menu would solve this. Bottom line is not adapted.
Describe the bug
The debug CLI option (-d) does nothing. Accoridng to inline documentation (-h), it outputs 'eBPF code'
Environment
Describe the bug
Does not support terminal without color support
To Reproduce
Steps to reproduce the behavior:
xterm -cm
Traceback (most recent call last): File "./calltop.py", line 821, in <module> display.printHeader("Exiting...") NameError: name 'display' is not defined
Expected behavior
No error
Environment
Additional context
terminal does not support colors.
please use black and white in this case
Describe the bug
python tracing reports wrong values :
To choose the columns on wich the sort is done, there are < and > keys.
It is also natural too add right and left keys.
Describe the bug
when we use these options -e read,write
, this is useless and resource consuming to attach a kretprobe to the exit of syscall.
Metrics precision is the most important
The call rate computation is a bit too naïve. It suffers from delay in the main loop, and it happens when there are a lot of data in the bpf map.
Today call_rate = call_nb / static_sampling_interv ==> rps = stSysStats.cntPerIntvl / self.refreshIntvl
Get a more accurate sampling interval per call
The interval should be the time between 2 reads access. If the main loop is slow down, the sampling interval will be bigger and will reflect the right call rate.
Do it in user space
Avoid doing it in the ebpf code. The fewer instructions, the less the tracing impact will be.
Display the minimum / maximum latency for every stats collected during the interval.
Computing the percentile 95 or percentile 5 is a bit more tricky since it has to be done in the eBPF and has to be memory and time efficient.
when you scroll down or up and you want to comme back to the top or bottom, it is a bit boring to move back.
Imagine you scroll down a lot, if you want to come back to the top, you will habe to scroll up (a lot too).
Solution.
One shortkey to go at the first line, one to go at the last one.
Describe the bug
In interactive mode, pressing the reset key (z) does not reset the dynamic filter on processes
To Reproduce
Steps to reproduce the behavior:
Expected behavior
3. reset action also resets the dynamic filter on process name.
Environment
Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
The interactive view does not allow to identify whether a filter is in place or not.
Especially annoying as reset action does not clear the filter (but that's in a separate bug report)
Describe the solution you'd like
The fact that a filter is active is explicit and visible, and possible the filter rule as well.
Maybe added tothe bottom line.
Describe alternatives you've considered
Alternative ideas to display it :
Describe the bug
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
Execution does not fail.
Or this dependency is explained in documentation.
Environment
Describe the bug
ebpf code could be refactorized.
I add this bug to not forget this behaviour.
BUG : Trace calltop itself, so use USDT for python and check the first values reported. Those values can be complety wrong. This is a race condition. What is surprising is that it last only ~1 second and then everything goes fine ...
It would be great to filter on process name
just enter the process name or a subpart of it. And display only process name that matched
The solution expected
Like htop F4 filter, or something similar.
Is your feature request related to a problem? Please describe.
I think users want by default the average latency per function. So we should remove the flag -l for latency and make it the default behaviour
Describe the solution you'd like
calltop.py
will display latency metrics.
calltop.py --no-latency
won't display latency metrics
Describe alternatives you've considered
Do we need the --no-latency
flags ? the only reason to keep it, is for performance, as we need a second eBPF attached on return.
The exception below is often related to non-root privilege
Traceback (most recent call last):
File "./calltop.py", line 910, in
main(display)
File "./calltop.py", line 892, in main
b = create_and_load_bpf(syscall_list, latency)
File "./calltop.py", line 762, in create_and_load_bpf
b = BPF(text=prog)
File "/usr/lib/python2.7/dist-packages/bcc/init.py", line 321, in init
raise Exception("Failed to compile BPF text")
Exception: Failed to compile BPF text
Describe the solution you'd like
A more explicit message would be helpful :
It fails compiling and load the eBPF. Do you have root access ?
With ubuntu 20.04 python3 package comes with the dtrace support that allows the tracing with usdt (ebpf). That not the case for the previous ubuntu version. Other linux distributions may have the same problem. It's important the describe how to build python with dtrace enable (--with-dtrace flag)
A wiki pages would do the job.
No longer compatible with python3.x
All is in the title.
Describe the bug
the 'u' key is used to enter the pid of the process you want to trace
To Reproduce
Expected behavior
Please change the doc accordingly
Describe the bug
When we do not want to collect latency, the latency should not be displayed.
To Reproduce
python3 ./calltop.py --no-latency
Expected behavior
2 solutions:
Description
The path to load ebpf.c does not work when the script is call from outside the directory.
To Reproduce
sudo ./calltop/calltop.py -p 6413
Traceback (most recent call last):
File "./calltop/calltop.py", line 844, in <module>
main(display)
File "./calltop/calltop.py", line 826, in main
b = create_and_load_bpf(syscall_list, latency)
File "./calltop/calltop.py", line 662, in create_and_load_bpf
with open('ebpf.c', 'r') as ebpf_code:
IOError: [Errno 2] No such file or directory: 'ebpf.c'
Is your feature request related to a problem? Please describe.
Sometimes you want to read an information but it is refresh to quickly. Pressing the space key stops screen refresh. A second press on space key stop pause mode.
Describe the solution you'd like
Describe the bug
Running the --help fails. Note that other commands work nicely.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
Help message is provided
Environment
Add the possibility to filter dynamically on different criteria:
and a combinaison of this criteria, for example pid is 1234 and syscall is read
Description
When using batch mode, output includes log messages in addition to expected data.
It could confuse output parsing and make integration more difficult.
I've noticed erorr messages and more informal messages as well
To Reproduce
Steps to reproduce the behavior:
Expected behavior
Output only contains data, each batch separated with an empty line.
Not error or other information messages.
Error messages should be sent to stderr.
Actual behaviour
Error/info messages at the beggining of the file:
head /tmp/calltop.out
Failed to attach to kprobe b'__x64_sys_all'
b'Collecting first data ...'
Pid Process name Function latency(us) Call/s Total
569 chromium read 0 1 1
569 chromium poll 0 3 3
Environment
Additional context
Maybe the best solution would be to add an option parameter for batch output, to be able to send it to a file.
And at least, to redirect all errors and info messages to stderr (?)
Describe the bug
latency is wrong when using the -e read,write
option
add batch mode
Add option -b to enable a batch mode. It is useful to record output to text file.
expected output
Keep the same output but print the new stat to stdout at the given rate.
Describe the bug
It is apparently possible to dynamically filter while using calltop in interactive mode.
But how to use it is hard to guess ! I discovered (by error) that if was the F key.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
2. Key mapped for filtering is listed in the bottom line
Actual behaviour
2. Key mapped for filtering is listed in the bottom line, as other keys mapped to other actions
3. Nothing about key mapping
4. Nothing about key mapping
Environment
Additional context
I may not be relevant to document everywhere, but at least once, and in a consistent way wrt other mapped keys.
Description
calltop.py works with python2.7.
python3-bpfcc make it possible to use python3 with bcc.
To Reproduce
python3 calltop.py
screen is empty; nothing is reported.
Expected behavior
works like with python2.7
Environment
bpfcc-tools 0.8.0-4
bpftrace 0.8+git60-gccac69c2239b-1
libbpfcc 0.8.0-4
python-bpfcc 0.8.0-4
python3-bpfcc 0.8.0-4
Additional context
install python 3 for bcc
sudo apt-get install python3-bpfcc̀
An inactive stats related to a process is a stats not updated after a given amount of time. In order to free the ebpf map entries, we clear the inactive stats as it is less likely to have concurrency of this data.
This design is a workaround to avoid non atomicity of operations on eBPF maps. The less we touch the map, the better is the atomicity (but not guaranteed)
So far the inactivity threshold is set a 5 secs. This is a bit too much.
Let's try 1 second; 1 second is already an eternity.
It will put more pressure on old stats in map:
Header Line is no longer visible when scrolling down
Header line is only visible on the first screen, but as soon as we scroll down, it disappears. this is not convenient.
Solution
Always print the header line.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.