Hi,

I'd like to tell people about flameprof in talk I'm giving, but it's failing for me:

$ python -m cProfile -o out.prof benchmarks/serialization.py
$ flameprof out.prof > prof.svg
Warning: flameprof can't find proper roots, root cumtime is 0.0 but sum tottime is 1.6221999999999899

I'm specifically running the benchmark from https://github.com/itamarst/eliot

Hey, this is not really an issue. But I wanted to let you know that I started using this and it's very useful, thanks for writing it! :)

I'm using it as a library by importing render and rendering to a StringIO, and then embedding the resulting SVG.

Hi, I'm attempting to benchmark a project, and I'm getting a really weird result. There are call paths in the flame graph that cannot really be part of the program. Here's a minimal, self-contained example of my problem:

import cProfile
import subprocess


def a():
    s = 0
    for i in range(1000):
        s += i
    return s


def b():
    s = 0
    for i in range(1000):
        s += i ** 2
    return s


def dispatch(func):
    return func()


def a_caller():
    return dispatch(a)


def b_caller():
    return dispatch(b)


def to_profile():
    return a_caller() + b_caller()


stats_file = 'test.prof'
flame_file = 'test.svg'
cProfile.run('to_profile()', stats_file)
subprocess.check_call(f'flameprof {stats_file} > {flame_file}', shell=True)

In this program, the function a_caller() calls a(), and the function b_caller() calls b(). There is no path for a_caller() to call b(), although the function dispatch() will appear on both stacks.

This is what the profile looks like:

The profile reports that a_caller() spends most of its time inside of b(). Removing dispatch() and calling a() or b() directly provides the correct result.

This is a problem for me because any numpy operation I do goes through a single numpy function, <built-in method numpy.core._multiarray_umath.implement_array_function>. For example, in this part of the trace, I'm calling np.repeat(). As far as I know, that shouldn't invoke cumsum(), hstack(), or delete().

Any ideas on how to improve this, or things I might be doing wrong? Is this just an intrinsic limitation of cProfile?

when i try to render the svg file by myself, i got exception like this

  File "profmiddleware.py", line 37, in post_prof
    flameprof.render(pr.stats, flameprof.get_out(svg_file_name))
  File "/home/cpys/anaconda3/lib/python3.8/site-packages/flameprof.py", line 305, in render
    blocks, bblocks, maxw = prepare(funcs, calls, threshold=threshold)
  File "/home/cpys/anaconda3/lib/python3.8/site-packages/flameprof.py", line 232, in prepare
    _calc_back((f for f in funcs if f != 'root'), 0, None, 0, set(), 0)
  File "/home/cpys/anaconda3/lib/python3.8/site-packages/flameprof.py", line 225, in _calc_back
    _calc_back(func['called'], level+1, name, origin, visited | {key}, ttt)
  File "/home/cpys/anaconda3/lib/python3.8/site-packages/flameprof.py", line 225, in _calc_back
    _calc_back(func['called'], level+1, name, origin, visited | {key}, ttt)
  File "/home/cpys/anaconda3/lib/python3.8/site-packages/flameprof.py", line 225, in _calc_back
    _calc_back(func['called'], level+1, name, origin, visited | {key}, ttt)
  [Previous line repeated 2 more times]
  File "/home/cpys/anaconda3/lib/python3.8/site-packages/flameprof.py", line 199, in _calc_back
    factor = pw / sum(calls[(r, to)][3] for r in names)
ZeroDivisionError: float division by zero

flameprof.render(pr.stats, flameprof.get_out(svg_file_name))

Description

pip install flameprof requires that python be the correct version of the executable. For those of us running python3 where python still points to python2, this causes a problem.

Steps to reproduce

1. On macOS 10.15 running python3 from homebrew, run pip3 install --user flameprof
2. $ cat $(which flameprof) will output:

   #!/bin/sh
   exec python -m flameprof "$@"
   

3. python --version will output Python 2.7.17

Proposed solution

The first way I can think of to solve this is to write the contents of flameprof with an explicit version numbered interpreter, IE: python2 or python3, as determined by the version running when installed. That seems fragile, but less fragile than what exists today.

When using cprofile as a CLI, there can be lots of intermediate or ancillary noise in the profile. The ability to filter stacks become very useful then.

It's doable using —format=log and flamegraph.pl as the flamegraph log is a list of stacks so a simple grep works, but that means not getting the improvements of flameprof's "native" SVG (especially getting the reverse graph alongside the normal one).

While I do love the feature to export the data as an SVG file, a more interactive way would be to load the data into speedscope. I would love to build a small parser, which parses a cProfile dump into a speedscope json file following its schema.

Could you maybe explain me what parts in the code are mainly doing the parsing so I could implement it myself?

small bars are invisible, and cannot be seen even zoomed in.

If you ever have a little free time—

I'd like to use the SVG-generating code not for pstats, but for my own datastructures (see itamarst/eliot#220).

My original thought was to generate Pstats instances and then use your code, but the Pstats class internals don't guarantee stability it seems?

More broadly, there are a bunch of Python projects that currently rely on flamegraph.pl and might benefit from pure-Python flamegraph SVG generating code, separate from the pstats-specific code.

I understand that the last two numbers may be tottime and cumtime corresponding to the profile, and the first number may be ncalls, so what is the second number 0?

def test_recursion(num):
    if num < 0:
        return
    print('test_recursion', num)
    time.sleep(0.1)
    test_recursion(num-1)
    test_recursion(num-3)


def main():
    test_recursion(5)


if __name__ == "__main__":
    main()