Git Product home page Git Product logo

Comments (4)

XuehaiPan avatar XuehaiPan commented on July 17, 2024

@FelixMildon Hi, nvitop provides APIs to query the device and process status, see https://github.com/XuehaiPan/nvitop#device.

from nvitop import Device, GpuProcess

all_devices = Device.all()
all_gpu_processes = []
for device in all_devices:
    all_gpu_processes.extend(device.processes().values())

all_device_snapshots = [device.as_snapshot() for device in all_devices]
all_gpu_process_snapshots = [process.as_snapshot() for process in all_processes]

I'm going to add tensorboard integration this weekend (see #18 (comment)), which will introduce more user-friendly interfaces.

from nvitop.

FelixMildon avatar FelixMildon commented on July 17, 2024

@XuehaiPan I'm really unsure how to use NVItop to log processes and gpu-utalisation (from nvidia-smi). Logging these in intervals into a CSV would be really useful for University of Aucklands GPUs.

from nvitop.

XuehaiPan avatar XuehaiPan commented on July 17, 2024

Hi @FelixMildon, I implement a new metric collector in PR #21, which can collect the resource consumption for GPUs and processes.

import datetime
import time

import pandas as pd

from nvitop import ResourceMetricCollector

collector = ResourceMetricCollector(root_pids={1}, interval=2.0)  # log all devices and all GPU processes
df = pd.DataFrame()

with collector(tag='resources'):
    for _ in range(12):
        # Do something
        time.sleep(5)

        metrics = collector.collect()
        df_metrics = pd.DataFrame.from_records(metrics, index=[len(df)])
        df = pd.concat([df, df_metrics], ignore_index=True)
        # Flush to CSV file ...

df.insert(0, 'time', df['resources/timestamp'].map(datetime.datetime.fromtimestamp))
df.to_csv('results.csv', index=False)

Screen Shot

See also #18.

from nvitop.

XuehaiPan avatar XuehaiPan commented on July 17, 2024

Close as resolved by PR #21.

from nvitop.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.