fabianp / yep Goto Github PK

Profiler for Python extensions

Home Page: http://pypi.python.org/pypi/yep

Python 100.00%

yep's Introduction

The Python Extension Profiler

extension_profiler is a tool to profile compiled code (C/C++/Fortran) from the Python interpreter. It uses the google-perftools CPU profiler and depends on pprof (google-pprof on Debian) for visualization.

Install

This package depends on google-perftools (package google-perftools in Debian). Once you have fulfilled these depencies you can install it with easy_install:

$ easy_install -U yep

or download the package, uncompress and execute the standard command:

$ python setup.py install

However, since the package one-file distribution, you can also just put yep.py somewhere in your $PYTHONPATH.

Usage

There are various ways to use the profiler. The simplest is add this module as argument to the Python interpreter when running your script and add flag -v to visualize the result:

$ python -m yep -v -- my_script.py [arg] ... ...

This will create a file my_script.py.prof that can be analyzed with pprof. Execute python -m yep to get the full list of options.

It is also possible to manually start/stop the profiler from inside Python code:

>>> import yep
>>> yep.start('file_name.prof')
>>> # do your computations
>>> yep.stop()

This will create a file_name.prof to be analized with pperf.

Development

Git repository can be found here:

https://github.com/fabianp/yep

Bugs

Visualize the result with -v will only work on UNIX. An option --prof= would also be nice.

Misc

Author: Fabian Pedregosa <[email protected]>

License

yep's People

Contributors

Stargazers

Watchers

Forkers

ogrisel guniorobot camillescott pczerkas imclab zhouzhaoze glemaitre juampamuc lcytzk casotto saisai

yep's Issues

Errors when invoking yep

$ python -myep -c profile.py # or -c -- profile.py, same result
Using local file /usr/bin/python.
Using local file /tmp/tmpjhe2lx83.
substr outside of string at /usr/bin/pprof line 3618.
Use of uninitialized value in string eq at /usr/bin/pprof line 3618.
substr outside of string at /usr/bin/pprof line 3620.
Use of uninitialized value in string eq at /usr/bin/pprof line 3620.
/tmp/tmpjhe2lx83: header size >= 2**16

$ python -myep -v profile.py
Using local file /usr/bin/python.
Using local file profile.py.prof.
substr outside of string at /usr/bin/pprof line 3618.
Use of uninitialized value in string eq at /usr/bin/pprof line 3618.
substr outside of string at /usr/bin/pprof line 3620.
Use of uninitialized value in string eq at /usr/bin/pprof line 3620.
profile.py.prof: header size >= 2**16

$ python -myep -v -- profile.py
Using local file /usr/bin/python.
Gathering CPU profile from http://profile.py.prof/pprof/profile?seconds=30 for 30 seconds to
  /home/antony/pprof/python.1430301840.profile.py.prof
Be patient...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (7) Failed to connect to profile.py.prof port 80: Connexion refusée
Failed to get profile: curl 'http://profile.py.prof/pprof/profile?seconds=30' > /home/antony/pprof/.tmp.python.1430301840.profile.py.prof: No such file or directory.

argument problem

When I run the command: "python -m yep -v mycode.py usps 1.0 -1"
The following appears: in fact -1 is one of my argument.

Usage: python -m yep [options] scriptfile [arg] ...

This will create a file scriptfile.prof that can be analyzed with
pprof (google-pprof on Debian-based systems).

yep.py: error: no such option: -1

What is pperf? Very little data is collected...

OK, first, there's a problem in this code:

    if sys.version_info >= (3, 0):
        if file_name is None:
            file_name = b'out.prof'
        status = libprofiler.ProfilerStart(bytes(file_name, 'ASCII'))

b'out.prof' should be 'out.prof' (not an asci string). Converting an asci string in this way will result in an error.

Second: what is pperf? I couldn't find such program in gperftools package (Arch name for Debian google-perftools). There's only pprof.

This aside, when I try running yep.start() ... yep.stop() very little information is collected. When I then try:

pprof $(which python) ./out.pprof

It does enter interactive mode, but doing top100 gives just 10 records, which are selected kind of at random...

Unable to find libprofiler (m1 macOS)

I am trying to use yep in my Macbook.

Actually, I wanted to use gperftools.

I installed google-perftools using brew.

% brew install google-perftools
% pkg-config --libs libprofiler
-L/opt/homebrew/Cellar/gperftools/2.9.1_1/lib -lprofiler

When I tried to install yep by typing "python setup.py install", an error message appeared.

error: ImportError: Unable to find libprofiler, please make sure google-perftools is installed on your system

I thought that location of lib profiler in my computer could raise the error to install yep.

I also tried to run following: ctypes.util.find_library('profiler')

It could not find any libprofiler.

How can I fix my problem?

Thank you for reading.

Link on PyPI dead

https://pypi.python.org/pypi/yep still refers to https://github.com/fabianp/yep, but apparently the package was moved here.

Use trace function for finer profiling

Use this:

http://docs.python.org/library/sys.html#sys.settrace

in combination with the 'c_call' event to activate/deactivate the profiler.