Git Product home page Git Product logo

nvidia-htop's Introduction

nvidia-htop

A tool for enriching the output of nvidia-smi.

CI PyPI version

Install

pip3 install nvidia-htop

Yes, this tool has been on PyPi since 2021! Enjoy the super-easy way to install it.

Usage

nvidia-htop.py [-l [length]]
  print GPU utilization with usernames and CPU stats for each GPU-utilizing process

  -l|--command-length [length]     Print longer part of the commandline. If `length'
                                   is provided, use it as the commandline length,
                                   otherwise print first 100 characters.
  -c|--color                       Colorize the output (green - free GPU, yellow -
                                   moderately used GPU, red - fully used GPU)
  -u|--user USER[,USER]            Limit the list of processes to selected users
                                   (comma-separated).
  -i|--id ID[,ID[,ID]]             Limit the command to selected GPU IDs (comma-separated).

Note: for backward compatibility, nvidia-smi | nvidia-htop.py [-l [length]] is also supported.

Note: running inside a container (docker, singularity, ...), nvidia-smi can only see processes running in the container.

Note: To periodically check the output of nvidia-htop, use the watch utility: watch nividia-htop.py. To get colored output, you have to pass option -c to both watch and nvidia-htop, e.g. watch -c nvidia-htop.py -c.

Example output

$ nvidia-htop.py -l
Mon May 21 15:06:35 2018
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 390.25                 Driver Version: 390.25                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 108...  Off  | 00000000:04:00.0 Off |                  N/A |
| 53%   75C    P2   174W / 250W |  10807MiB / 11178MiB |     97%      Default |
+-------------------------------+----------------------+----------------------+
|   1  GeForce GTX 108...  Off  | 00000000:05:00.0 Off |                  N/A |
| 66%   82C    P2   220W / 250W |  10783MiB / 11178MiB |    100%      Default |
+-------------------------------+----------------------+----------------------+
|   2  GeForce GTX 108...  Off  | 00000000:08:00.0 Off |                  N/A |
| 45%   67C    P2    85W / 250W |  10793MiB / 11178MiB |     51%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
|  GPU   PID     USER    GPU MEM  %MEM  %CPU  COMMAND                                                                                               |
|    0  1032 anonymou   10781MiB   308   3.7  python train_image_classifier.py --train_dir=/mnt/xxxxxxxx/xxxxxxxx/xxxxxxxx/xxxxxxx/xxxxxxxxxxxxxxx  |
|    1 11021 cannotte   10765MiB   114   1.5  python3 ./train.py --flagfile /xxxxxxxx/xxxxxxxx/xxxxxxxx/xxxxxxxxx/xx/xxxxxxxxxxxxxxx                |
|    2 25544 nevermin   10775MiB   108   2.0  python -m xxxxxxxxxxxxxxxxxxxxxxxxxxxxx                                                               |
+-----------------------------------------------------------------------------+

Screenshot with output coloring

Screenshot

nvidia-htop's People

Contributors

bobholamovic avatar budel avatar dexteritas avatar keineahnung2345 avatar pcudrano avatar peci1 avatar raytroop avatar tomix1024 avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

dangmanhtruong1995 doriankodelja muks14x parita keisks hujilin1229 iamjohnrain wikipedia2008 dengdan neosun100 junya-takayama wavefrontshaping rinhz chongruo romanrudensky shunk031 david-lindner baekis1185 ikhwansong jackie840129 antreasantoniou simon5u bobholamovic asanakoy sssomani liushaoweihua nrasadi aiwithshekhar anican andyjzhao goingmyway anotherkaizhang pouyiouk stardust2602 cemde yimian rookielxy janychan kashu7100 collector-m richylyq icodein stjordanis o7s8r6 danieljuravski k-nick rayanht erik-koynov flow79 k11m1 cemberk thomasbad zhpinkman chengzegang iq-scm tomix1024 pcudrano rit628

nvidia-htop's Issues

What do the colors mean?

I'm using the tool with colored output. What do the colors mean?

I'm getting the impression that they may refer to the amount of memory being used, but my interpretation seems inconsistent (i.e., some GPU's with 95%+ usage are red, but some others are white).

issue install on proxmox 6.4

Hi
Currently tried installing but when i run im getting this issue

nvidia-htop.py
Traceback (most recent call last):
  File "/usr/local/bin/nvidia-htop.py", line 57, in <module>
    ps_call = subprocess.run('nvidia-smi', stdout=subprocess.PIPE, stderr=subprocess.PIPE)
  File "/usr/lib/python3.7/subprocess.py", line 472, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/lib/python3.7/subprocess.py", line 775, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.7/subprocess.py", line 1522, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'nvidia-smi': 'nvidia-smi'


 sudo lshw -C display
  *-display
       description: VGA compatible controller
       product: G98 [GeForce 9300 GS]
       vendor: NVIDIA Corporation
       physical id: 0
       bus info: pci@0000:03:00.0
       version: a1
       width: 64 bits
       clock: 33MHz
       capabilities: pm msi pciexpress vga_controller bus_master cap_list rom
       configuration: driver=nouveau latency=0
       resources: irq:40 memory:f6000000-f6ffffff memory:e0000000-efffffff memory:f4000000-f5ffffff ioport:bc80(size=128) memory:c0000-dffff

subprocess AttributeError: 'module' object has no attribute 'run'

(base) [xx@L01]$ nvidia-htop.py
Traceback (most recent call last):
File "/home/zgong/nfs/program/anaconda2/bin/nvidia-htop.py", line 51, in
processes = subprocess.run('nvidia-smi', stdout=subprocess.PIPE)
AttributeError: 'module' object has no attribute 'run'

Failed to align when lengths of PID are greater than 5.

The application is very convenient but I ran into alignment issues when the PID is too long.

+-----------------------------------------------------------------------------+
|  GPU   PID     USER    GPU MEM  %CPU  %MEM      TIME  COMMAND               |
|    0  3753     root       4MiB   0.0   0.0  18:13:36  /usr/lib/xorg/Xorg v  |
|    0 164425     xxxx   10647MiB   106   0.3     35:59  python main.py -i 6   |
|    1  3753     root       4MiB   0.0   0.0  18:13:36  /usr/lib/xorg/Xorg v  |
|    1 161143     xxxx   10923MiB   100   0.6     59:46  python main.py -i 1   |
|    2  3753     root       4MiB   0.0   0.0  18:13:36  /usr/lib/xorg/Xorg v  |
|    2 161102     xxxx   10731MiB   100   0.6  01:00:02  python main.py -i 1   |
|    3  3753     root       4MiB   0.0   0.0  18:13:36  /usr/lib/xorg/Xorg v  |
|    3 160197     xxxx   10195MiB   141   0.6  01:00:49  python main.py -i 1   |
|    4  3753     root       4MiB   0.0   0.0  18:13:36  /usr/lib/xorg/Xorg v  |
|    4 160481     xxxx   11099MiB   101   0.4  01:00:40  python main.py -i 3   |
|    5  3753     root       4MiB   0.0   0.0  18:13:36  /usr/lib/xorg/Xorg v  |
|    5 159486     xxxx   10761MiB   101   0.4  01:01:59  python main.py -i 3   |
|    6  3753     root       4MiB   0.0   0.0  18:13:36  /usr/lib/xorg/Xorg v  |
|    6 159424     xxxx   10623MiB  98.2   0.5  01:02:30  python main.py -i 5   |
|    7  3753     root       4MiB   0.0   0.0  18:13:36  /usr/lib/xorg/Xorg v  |
|    7 159383     xxxx   10311MiB  98.7   0.5  01:02:53  python main.py -i 5   |
+-----------------------------------------------------------------------------+

command not found?

I'm probably using this wrong, but I get the above error when I try the following:

  1. Place nvidia-htop.py in my current directory.
  2. Call nvidia-smi | nvidia-htop.py -l

Calling nvidia-smi on its own works just fine. Any suggestions?

error: list of process IDs must follow -p

This application seems great but I ran into an issue when trying to use it. I pip installed it per the directions but when I ran nvidia-htop.py -l it printed out the normal nvidia-smi output but then it ran into an error: error: list of process IDs must follow -p. Am I doing something wrong?

Add a LICENSE (e.g. MIT)

Great work! I really like your little tool. However, it would be great, if you could add an open source LICENSE (e.g. MIT) to allow using it also in a professional environment.

Theoretically, without a license we would not be allowed to use your code even privately or modify it and do a pull request.

IndexError: pid.append(line[2])

Unexpected IndexError when calling nvidia-htop.py. Error persists regardless the passed arguments/options.

nvidia-htop.py --color -l 30
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 450.57       Driver Version: 450.57       CUDA Version: 11.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  GeForce RTX 2070    Off  | 00000000:01:00.0 Off |                  N/A |
|  0%   45C    P2    27W / 175W |    749MiB /  7981MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   1  GeForce GTX 750 Ti  Off  | 00000000:04:00.0  On |                  N/A |
| 33%   33C    P0     2W /  46W |    725MiB /  2002MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
Traceback (most recent call last):
  File "/home/tmp/.local/bin/nvidia-htop.py", line 127, in <module>
    pid.append(line[2])
IndexError: list index out of range

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.