Git Product home page Git Product logo

pptop's Introduction

ppTOP

Project home: https://pptop.bma.ai/

ppTOP is open, extensible Python injector/profiler/analyzer.

The main idea is to inject a custom code inside Python process (already running or going to be launched) and analyze its behavior with no any code modifications.

Say no to "prints" and garbage in debug logs - now you have ppTOP. A modern MRI scanner for Python.

Installation

  pip3 install pptop
  • Python: 3.5+ is required, as ppTOP uses asyncio
  • Can be injected into any Python version (tested: 2.7+)
  • Injection into running processes requires gdb (present in almost all Linux distributions)
  • Only Linux systems are currently supported

Features

ppTOP can be injected into any running Python process

asciicast

or you can load Python program manually and profile its launch

asciicast

To launch a program, press Ctrl+L or specify -w sec param in command line args to start it automatically after the specified delay.

Data from connected process is collected in real-time and displayed in table-based console UI, which can be easily extended with custom plugins.

All data tables can be scrolled, filtered, new data collection can be paused. In case of problems, any plugin can be re-injected at any time.

Usage

To start ppTOP, type

    pptop

and then select Python process you want to inject to from the list.

Alternatively, you can start it with

    pptop PID
    # or
    pptop PID-FILE

and specify the process from the command line.

If you want to analyze program startup, just type

    pptop /path/to/program.py

The program will be loaded in waiting state, press Ctrl+L when you are ready.

To get a help for the current plugin, press F1 to display module documentation.

Configuration

Plugins and keyboard shortcuts are configured by default in ~/.pptop/pptop.yml file (created automatically at first launch).

Standard plugins

  • asyncio asyncio loop monitor
  • atasker atasker monitor
  • env view process OS environment variables
  • log inject into all Python loggers and collect log messages
  • malloc trace object memory allocations
  • neotasker neotasker monitor
  • open_files view process open files
  • script_runner launch a custom scripts inside process
  • threads view process threads
  • vars variable/function watcher
  • yappi yappi profiler plugin

Most of plugins contain 2 parts of code: one collects data inside profiling process, other display it in ppTOP UI. For the profiling process all plug-ins are invisible, safe and unloadable (at least they try their best :)

Console mode

With "`" button, remote Python console can be opened. It has no full functionality like a standard Python console, but supports all frequently used features, like importing modules, defining functions and variables, executing functions and loops etc.

asciicast

Console has own remote globals, which are preserved until ppTOP quit and injected server is terminated.

Grab stdout/stderr

If launched with "-g" option, ppTOP will grab stdout/stderr of injected process and print it to local console. You can view local console without entering console mode, by pressing Ctrl+O.

Documentation

Configuration, troubleshooting, advanced usage, plugin development: https://pptop.bma.ai/doc/

TODO

  • More plugins
  • Advanced filtering
  • Data snapshots
  • Step-by-step debugger
  • JSON API, web interface
  • Charts

Enjoy! :)

p.s. Code in master can be completely broken, install with pip only.

pptop's People

Contributors

divi255 avatar sgnn7 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

Forkers

sgnn7

pptop's Issues

Problems attaching to an already-running (`No symbol "__pptop_start_injection" in current context`)

Setup:

  • Docker for Mac
  • Target: Running container with datadog-agent (Python 3.8 embedded process):
docker run --rm -d \
           --name dd-agent \
           --net host \
           -e DD_API_KEY="fake" \
           -v /proc/:/host/proc/:ro \
           -v /sys/fs/cgroup/:/host/sys/fs/cgroup:ro \
           -v /var/run/docker.sock:/var/run/docker.sock:ro \
           -v /opt/datadog-agent/run:/opt/datadog-agent/run:rw \
           datadog/agent

Attachment container (some flags aren't needed - was using this for debugging):

$ docker run --rm -it \
           --name="devenv" \
           --pid="container:dd-agent" \
           --cap-add="SYS_PTRACE" \
           --cap-add="SYS_ADMIN" \
           --privileged \
           --security-opt="seccomp=unconfined" \
           --security-opt="apparmor=unconfined" \
           ubuntu:21.10

Running:

$ apt update && apt install -y gdb python3-pip
$ pip3 install pptop

$ agent_pid=$(ps -ef | grep 'agent run$' | awk '{print $2}')

$ 
pptop $agent_pid
b'No symbol "__pptop_start_injection" in current context.\n'

musl/alpine support

Hey there! Wonderful tool, love it, been looking for something like this for years.

I had to make a slight tweak to make it work with alpine / musl libc (which is fairly common in docker containers).

The call to __libc_dlopen_mode() is glibc specific (as far as i can see, a workaround for the fact that libdl is not always loaded?), with musl you should just do dlopen().

I don't have a clean patch for this (that doesn't break glibc and tries to detect musl somehow), but wanted to post the issue to not forget.

Cheers

Awesome project!

Nice work, thanks so much for making this!

(out of curiosity, any hope for macOS support someday? does it rely on strace/can it use dtrace instead on mac? feel free to close this issue)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.