Comments (4)
@FelixMildon Hi, nvitop
provides APIs to query the device and process status, see https://github.com/XuehaiPan/nvitop#device.
from nvitop import Device, GpuProcess
all_devices = Device.all()
all_gpu_processes = []
for device in all_devices:
all_gpu_processes.extend(device.processes().values())
all_device_snapshots = [device.as_snapshot() for device in all_devices]
all_gpu_process_snapshots = [process.as_snapshot() for process in all_processes]
I'm going to add tensorboard
integration this weekend (see #18 (comment)), which will introduce more user-friendly interfaces.
from nvitop.
@XuehaiPan I'm really unsure how to use NVItop to log processes and gpu-utalisation (from nvidia-smi). Logging these in intervals into a CSV would be really useful for University of Aucklands GPUs.
from nvitop.
Hi @FelixMildon, I implement a new metric collector in PR #21, which can collect the resource consumption for GPUs and processes.
import datetime
import time
import pandas as pd
from nvitop import ResourceMetricCollector
collector = ResourceMetricCollector(root_pids={1}, interval=2.0) # log all devices and all GPU processes
df = pd.DataFrame()
with collector(tag='resources'):
for _ in range(12):
# Do something
time.sleep(5)
metrics = collector.collect()
df_metrics = pd.DataFrame.from_records(metrics, index=[len(df)])
df = pd.concat([df, df_metrics], ignore_index=True)
# Flush to CSV file ...
df.insert(0, 'time', df['resources/timestamp'].map(datetime.datetime.fromtimestamp))
df.to_csv('results.csv', index=False)
See also #18.
from nvitop.
Close as resolved by PR #21.
from nvitop.
Related Issues (20)
- [Feature Request] Add shortcut that shows parent process's name for selected process HOT 1
- [Question] Memory bandwidth utilization of GPUs? HOT 4
- [BUG] Monitor mode displays nothing under Python 3.12 in Windows 11 HOT 8
- [Feature Request] It is recommended to change the dependency from nvidia-ml-py to pynvml HOT 1
- [BUG][exporter] Process metrics still exist when the process is gone HOT 5
- [BUG] curser error init display on ubuntu 22.10 HOT 1
- [Question] How to log GPU performance to `wandb` HOT 2
- [Feature Request] add io stat for disk and process like glances HOT 2
- [BUG] Pytorch lightning callback HOT 3
- [Feature Request] Show real-time bandwidth under monitor mode
- [Question] ERROR: Failed to initialize `curses` (setupterm: could not find terminfo database) HOT 4
- [Question] How snapshot could be used HOT 2
- Installation: which step to follow? HOT 1
- [BUG] Prometheus connection refused HOT 4
- [Question] Grafana Dashboard Example
- [Feature Request] can you support word wrap for COMMAND information HOT 1
- [Question] How to count the total amount of one user's GPU memory on all graphics cards by using nvitop? HOT 2
- [BUG] ERROR: Failed to initialize `curses` (curs_set() returned ERR) in tmux environment HOT 5
- [BUG] Display issue when running in Docker environment HOT 5
- [BUG] `nvitop.Device.from_cuda_visible_devices()` not detecting GPU HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nvitop.