a0s / nvidia-smi-exporter Goto Github PK

View Code? Open in Web Editor NEW

27.0 3.0 1.0 67 KB

Nvidia-smi Prometheus exporter with respecting of GPU-UUID

Home Page: https://github.com/a0s/nvidia-smi-exporter

License: MIT License

Ruby 97.52% Dockerfile 2.48%

mining docker nvidia-smi prometheus-exporter nvidia

nvidia-smi-exporter's Introduction

Nvidia SMI Exporter

Another one nvidia-smi data exporter for Prometheus.

> curl localhost:9454/ping
OK

> curl localhost:9454/metrics
nvidia_smi_attached_gpus 2
nvidia_smi_display_mode{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 1
nvidia_smi_display_active{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 0
nvidia_smi_persistence_mode{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 1
nvidia_smi_accounting_mode{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 0
nvidia_smi_accounting_mode_buffer_size{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 4000
nvidia_smi_minor_number{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 0
nvidia_smi_multigpu_board{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 0
nvidia_smi_inforom_version_oem_object{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 1.1
nvidia_smi_pci_pci_sub_system_id{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 37021458
nvidia_smi_pci_pci_gpu_link_info_pcie_gen_max_link_gen{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 3
nvidia_smi_pci_pci_gpu_link_info_pcie_gen_current_link_gen{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 3
nvidia_smi_pci_pci_gpu_link_info_link_widths_max_link_width{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 16
nvidia_smi_pci_pci_gpu_link_info_link_widths_current_link_width{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 1
nvidia_smi_pci_replay_counter{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 65535
nvidia_smi_pci_replay_rollover_counter{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 0
nvidia_smi_pci_tx_util_bytes_per_second{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 11264000
nvidia_smi_pci_rx_util_bytes_per_second{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 31744000
nvidia_smi_fan_speed_ratio{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 0.53
nvidia_smi_performance_state{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 2
nvidia_smi_clocks_throttle_reasons_clocks_throttle_reason_gpu_idle{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 0
nvidia_smi_clocks_throttle_reasons_clocks_throttle_reason_applications_clocks_setting{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 0
nvidia_smi_clocks_throttle_reasons_clocks_throttle_reason_sw_power_cap{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 1
nvidia_smi_clocks_throttle_reasons_clocks_throttle_reason_hw_slowdown{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 0
nvidia_smi_clocks_throttle_reasons_clocks_throttle_reason_hw_thermal_slowdown{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 0
nvidia_smi_clocks_throttle_reasons_clocks_throttle_reason_hw_power_brake_slowdown{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 0
nvidia_smi_clocks_throttle_reasons_clocks_throttle_reason_sync_boost{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 0
nvidia_smi_clocks_throttle_reasons_clocks_throttle_reason_sw_thermal_slowdown{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 0
nvidia_smi_clocks_throttle_reasons_clocks_throttle_reason_display_clocks_setting{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 0
nvidia_smi_fb_memory_usage_total_bytes{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 8510242816
nvidia_smi_fb_memory_usage_used_bytes{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 3040870400
nvidia_smi_fb_memory_usage_free_bytes{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 5469372416
nvidia_smi_bar1_memory_usage_total_bytes{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 268435456
nvidia_smi_bar1_memory_usage_used_bytes{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 5242880
nvidia_smi_bar1_memory_usage_free_bytes{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 263192576
nvidia_smi_compute_mode{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 0
nvidia_smi_utilization_gpu_util_ratio{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 1.0
nvidia_smi_utilization_memory_util_ratio{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 1.0
nvidia_smi_utilization_encoder_util_ratio{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 0.0
nvidia_smi_utilization_decoder_util_ratio{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 0.0
nvidia_smi_encoder_stats_session_count{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 0
nvidia_smi_encoder_stats_average_fps{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 0
nvidia_smi_encoder_stats_average_latency{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 0
nvidia_smi_fbc_stats_session_count{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 0
nvidia_smi_fbc_stats_average_fps{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 0
nvidia_smi_fbc_stats_average_latency{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 0
nvidia_smi_temperature_gpu_temp_celsius{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 67.0
nvidia_smi_temperature_gpu_temp_max_threshold_celsius{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 96.0
nvidia_smi_temperature_gpu_temp_slow_threshold_celsius{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 93.0
nvidia_smi_temperature_gpu_target_temperature_celsius{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 83.0
nvidia_smi_supported_gpu_target_temp_gpu_target_temp_min_celsius{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 60.0
nvidia_smi_supported_gpu_target_temp_gpu_target_temp_max_celsius{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 92.0
nvidia_smi_power_readings_power_state{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 2
nvidia_smi_power_readings_power_management{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 1
nvidia_smi_power_readings_power_draw_watts{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 211.86
nvidia_smi_power_readings_power_limit_watts{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 216.0
nvidia_smi_power_readings_default_power_limit_watts{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 200.0
nvidia_smi_power_readings_enforced_power_limit_watts{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 216.0
nvidia_smi_power_readings_min_power_limit_watts{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 100.0
nvidia_smi_power_readings_max_power_limit_watts{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 216.0
nvidia_smi_clocks_graphics_clock_hz{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 1784000000.0
nvidia_smi_clocks_sm_clock_hz{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 1784000000.0
nvidia_smi_clocks_mem_clock_hz{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 5514000000.0
nvidia_smi_clocks_video_clock_hz{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 1594000000.0
nvidia_smi_max_clocks_graphics_clock_hz{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 1999000000.0
nvidia_smi_max_clocks_sm_clock_hz{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 1999000000.0
nvidia_smi_max_clocks_mem_clock_hz{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 5005000000.0
nvidia_smi_max_clocks_video_clock_hz{uuid="683c6e09-3969-31a5-b7ca-cc88252b7fce"} 1708000000.0

Caveats

In XML mode (see below) the exporter skips all implicit arrays like this:

<supported_clocks>
  <supported_mem_clock>
    <value>3504 MHz</value>
    <supported_graphics_clock>1721 MHz</supported_graphics_clock> <== will skip all supported_graphics_clock
    <supported_graphics_clock>1708 MHz</supported_graphics_clock>
    <supported_graphics_clock>1695 MHz</supported_graphics_clock>
    <supported_graphics_clock>1683 MHz</supported_graphics_clock>
    <supported_graphics_clock>1670 MHz</supported_graphics_clock>
    <supported_graphics_clock>1657 MHz</supported_graphics_clock>
    <supported_graphics_clock>1645 MHz</supported_graphics_clock>
    <supported_graphics_clock>1632 MHz</supported_graphics_clock>

Also, it skips all empty and "N/A" values (and their keys accordingly).

Prometheus is not supporting string values, thus exporter converts some strings to numbers:

Disabled, Not Active, No, Not Supported and Unsupported will convert to 0
Enabled, Active, Yes, Supported will convert to 1
P0..P15 will convert to 0..15
1x, 2x, 4x, 8x, 16x, 32x (pci lines) will convert to 1,2,4,8,16,32
Default, Exclusive_Thread, Prohibited, Exclusive_Process (_compute_mode) will convert to 0..3

Other not integer/float strings will eliminate.

Run in Docker

nvidia-smi requires using the same versions of packages (libnvidia-compute-460 and nvidia-utils-460) inside the container and outside (on the host).

Get driver version on the host:

> dpkg --list | grep nvidia-driver-460
ii  nvidia-driver-460                    460.27.04-0ubuntu1                    amd64        NVIDIA driver metapackage

Build container with appropriate version of driver:
```
> docker build . --tag nvidia-smi-exporter --build-arg NVIDIA_VERSION=460.27.04-0ubuntu1
```
All available versions can be found in the Cuda repository

Run with --privileged flag (not recommended due to security)

> docker run --rm --privileged -p 9454:9454 nvidia-smi-exporter

or enumerating (map inside container) all devices explicitly

> docker run --rm \
  --device /dev/nvidiactl:/dev/nvidiactl \
  --device /dev/nvidia0:/dev/nvidia0 \
  --device /dev/nvidia1:/dev/nvidia1 \
  -p 9454:9454 nvidia-smi-exporter

Parameters

There are several environment variables to setting runtime parameters:

NVIDIA_SMI_EXPORTER_BINARY - path to nvidia-smi executive binary, default: $(which nvidia-smi)
NVIDIA_SMI_EXPORTER_PORT - a port where the server will be started, default: 9454
NVIDIA_SMI_EXPORTER_HOST - a host where the server will be started, default: 0.0.0.0
NVIDIA_SMI_EXPORTER_NAME_PREFIX - prefix for every parameter name in output, default: nvidia_smi_
NVIDIA_SMI_EXPORTER_SOURCE - source to processing - xml or csv, default: xml.

When NVIDIA_SMI_EXPORTER_SOURCE == csv:

NVIDIA_SMI_EXPORTER_QUERY - comma-separated list to query parameters from nvidia-smi, default: clocks.current.graphics,clocks.current.memory,clocks.current.sm,clocks.current.video,clocks.max.graphics,clocks.max.memory,clocks.max.sm,fan.speed,memory.total,memory.used,power.draw,power.limit,temperature.gpu,utilization.gpu,utilization.memory

nvidia-smi-exporter's People

Contributors

Stargazers

Watchers

Forkers

pint1022

nvidia-smi-exporter's Issues

Switch to XML output for significantly more data

I would love some of the information available only with xml output e.g. processes. Thoughts?

Command:

nvidia-smi -q -x

My Output:

<?xml version="1.0" ?>
<!DOCTYPE nvidia_smi_log SYSTEM "nvsmi_device_v10.dtd">
<nvidia_smi_log>
        <timestamp>Fri Sep 27 11:00:44 2019</timestamp>
        <driver_version>418.56</driver_version>
        <cuda_version>10.1</cuda_version>
        <attached_gpus>1</attached_gpus>
        <gpu id="00000000:04:00.0">
                <product_name>Quadro P2000</product_name>
                <product_brand>Quadro</product_brand>
                <display_mode>Disabled</display_mode>
                <display_active>Disabled</display_active>
                <persistence_mode>Disabled</persistence_mode>
                <accounting_mode>Disabled</accounting_mode>
                <accounting_mode_buffer_size>4000</accounting_mode_buffer_size>
                <driver_model>
                        <current_dm>N/A</current_dm>
                        <pending_dm>N/A</pending_dm>
                </driver_model>
                <serial>0324217004816</serial>
                <uuid>GPU-1ffe36ef-ac55-60ad-f9cf-623871e7736d</uuid>
                <minor_number>0</minor_number>
                <vbios_version>86.06.3F.00.0D</vbios_version>
                <multigpu_board>No</multigpu_board>
                <board_id>0x400</board_id>
                <gpu_part_number>900-5G410-1700-000</gpu_part_number>
                <inforom_version>
                        <img_version>G410.0502.00.02</img_version>
                        <oem_object>1.1</oem_object>
                        <ecc_object>N/A</ecc_object>
                        <pwr_object>N/A</pwr_object>
                </inforom_version>
                <gpu_operation_mode>
                        <current_gom>N/A</current_gom>
                        <pending_gom>N/A</pending_gom>
                </gpu_operation_mode>
                <gpu_virtualization_mode>
                        <virtualization_mode>None</virtualization_mode>
                </gpu_virtualization_mode>
                <ibmnpu>
                        <relaxed_ordering_mode>N/A</relaxed_ordering_mode>
                </ibmnpu>
                <pci>
                        <pci_bus>04</pci_bus>
                        <pci_device>00</pci_device>
                        <pci_domain>0000</pci_domain>
                        <pci_device_id>1C3010DE</pci_device_id>
                        <pci_bus_id>00000000:04:00.0</pci_bus_id>
                        <pci_sub_system_id>11B310DE</pci_sub_system_id>
                        <pci_gpu_link_info>
                                <pcie_gen>
                                        <max_link_gen>3</max_link_gen>
                                        <current_link_gen>3</current_link_gen>
                                </pcie_gen>
                                <link_widths>
                                        <max_link_width>16x</max_link_width>
                                        <current_link_width>16x</current_link_width>
                                </link_widths>
                        </pci_gpu_link_info>
                        <pci_bridge_chip>
                                <bridge_chip_type>N/A</bridge_chip_type>
                                <bridge_chip_fw>N/A</bridge_chip_fw>
                        </pci_bridge_chip>
                        <replay_counter>0</replay_counter>
                        <replay_rollover_counter>0</replay_rollover_counter>
                        <tx_util>0 KB/s</tx_util>
                        <rx_util>0 KB/s</rx_util>
                </pci>
                <fan_speed>61 %</fan_speed>
                <performance_state>P0</performance_state>
                <clocks_throttle_reasons>
                        <clocks_throttle_reason_gpu_idle>Not Active</clocks_throttle_reason_gpu_idle>
                        <clocks_throttle_reason_applications_clocks_setting>Not Active</clocks_throttle_reason_applications_clocks_setting>
                        <clocks_throttle_reason_sw_power_cap>Not Active</clocks_throttle_reason_sw_power_cap>
                        <clocks_throttle_reason_hw_slowdown>Not Active</clocks_throttle_reason_hw_slowdown>
                        <clocks_throttle_reason_hw_thermal_slowdown>Not Active</clocks_throttle_reason_hw_thermal_slowdown>
                        <clocks_throttle_reason_hw_power_brake_slowdown>Not Active</clocks_throttle_reason_hw_power_brake_slowdown>
                        <clocks_throttle_reason_sync_boost>Not Active</clocks_throttle_reason_sync_boost>
                        <clocks_throttle_reason_sw_thermal_slowdown>Not Active</clocks_throttle_reason_sw_thermal_slowdown>
                        <clocks_throttle_reason_display_clocks_setting>Not Active</clocks_throttle_reason_display_clocks_setting>
                </clocks_throttle_reasons>
                <fb_memory_usage>
                        <total>5059 MiB</total>
                        <used>319 MiB</used>
                        <free>4740 MiB</free>
                </fb_memory_usage>
                <bar1_memory_usage>
                        <total>256 MiB</total>
                        <used>2 MiB</used>
                        <free>254 MiB</free>
                </bar1_memory_usage>
                <compute_mode>Default</compute_mode>
                <utilization>
                        <gpu_util>0 %</gpu_util>
                        <memory_util>0 %</memory_util>
                        <encoder_util>0 %</encoder_util>
                        <decoder_util>0 %</decoder_util>
                </utilization>
                <encoder_stats>
                        <session_count>0</session_count>
                        <average_fps>0</average_fps>
                        <average_latency>0</average_latency>
                </encoder_stats>
                <fbc_stats>
                        <session_count>0</session_count>
                        <average_fps>0</average_fps>
                        <average_latency>0</average_latency>
                </fbc_stats>
                <ecc_mode>
                        <current_ecc>N/A</current_ecc>
                        <pending_ecc>N/A</pending_ecc>
                </ecc_mode>
                <ecc_errors>
                        <volatile>
                                <single_bit>
                                        <device_memory>N/A</device_memory>
                                        <register_file>N/A</register_file>
                                        <l1_cache>N/A</l1_cache>
                                        <l2_cache>N/A</l2_cache>
                                        <texture_memory>N/A</texture_memory>
                                        <texture_shm>N/A</texture_shm>
                                        <cbu>N/A</cbu>
                                        <total>N/A</total>
                                </single_bit>
                                <double_bit>
                                        <device_memory>N/A</device_memory>
                                        <register_file>N/A</register_file>
                                        <l1_cache>N/A</l1_cache>
                                        <l2_cache>N/A</l2_cache>
                                        <texture_memory>N/A</texture_memory>
                                        <texture_shm>N/A</texture_shm>
                                        <cbu>N/A</cbu>
                                        <total>N/A</total>
                                </double_bit>
                        </volatile>
                        <aggregate>
                                <single_bit>
                                        <device_memory>N/A</device_memory>
                                        <register_file>N/A</register_file>
                                        <l1_cache>N/A</l1_cache>
                                        <l2_cache>N/A</l2_cache>
                                        <texture_memory>N/A</texture_memory>
                                        <texture_shm>N/A</texture_shm>
                                        <cbu>N/A</cbu>
                                        <total>N/A</total>
                                </single_bit>
                                <double_bit>
                                        <device_memory>N/A</device_memory>
                                        <register_file>N/A</register_file>
                                        <l1_cache>N/A</l1_cache>
                                        <l2_cache>N/A</l2_cache>
                                        <texture_memory>N/A</texture_memory>
                                        <texture_shm>N/A</texture_shm>
                                        <cbu>N/A</cbu>
                                        <total>N/A</total>
                                </double_bit>
                        </aggregate>
                </ecc_errors>
                <retired_pages>
                        <multiple_single_bit_retirement>
                                <retired_count>N/A</retired_count>
                                <retired_pagelist>N/A</retired_pagelist>
                        </multiple_single_bit_retirement>
                        <double_bit_retirement>
                                <retired_count>N/A</retired_count>
                                <retired_pagelist>N/A</retired_pagelist>
                        </double_bit_retirement>
                        <pending_retirement>N/A</pending_retirement>
                </retired_pages>
                <temperature>
                        <gpu_temp>62 C</gpu_temp>
                        <gpu_temp_max_threshold>104 C</gpu_temp_max_threshold>
                        <gpu_temp_slow_threshold>101 C</gpu_temp_slow_threshold>
                        <gpu_temp_max_gpu_threshold>N/A</gpu_temp_max_gpu_threshold>
                        <memory_temp>N/A</memory_temp>
                        <gpu_temp_max_mem_threshold>N/A</gpu_temp_max_mem_threshold>
                </temperature>
                <power_readings>
                        <power_state>P0</power_state>
                        <power_management>Supported</power_management>
                        <power_draw>20.32 W</power_draw>
                        <power_limit>75.00 W</power_limit>
                        <default_power_limit>75.00 W</default_power_limit>
                        <enforced_power_limit>75.00 W</enforced_power_limit>
                        <min_power_limit>75.00 W</min_power_limit>
                        <max_power_limit>75.00 W</max_power_limit>
                </power_readings>
                <clocks>
                        <graphics_clock>1215 MHz</graphics_clock>
                        <sm_clock>1215 MHz</sm_clock>
                        <mem_clock>3499 MHz</mem_clock>
                        <video_clock>1101 MHz</video_clock>
                </clocks>
                <applications_clocks>
                        <graphics_clock>1075 MHz</graphics_clock>
                        <mem_clock>3504 MHz</mem_clock>
                </applications_clocks>
                <default_applications_clocks>
                        <graphics_clock>1075 MHz</graphics_clock>
                        <mem_clock>3504 MHz</mem_clock>
                </default_applications_clocks>
                <max_clocks>
                        <graphics_clock>1721 MHz</graphics_clock>
                        <sm_clock>1721 MHz</sm_clock>
                        <mem_clock>3504 MHz</mem_clock>
                        <video_clock>1556 MHz</video_clock>
                </max_clocks>
                <max_customer_boost_clocks>
                        <graphics_clock>1721 MHz</graphics_clock>
                </max_customer_boost_clocks>
                <clock_policy>
                        <auto_boost>N/A</auto_boost>
                        <auto_boost_default>N/A</auto_boost_default>
                </clock_policy>
                <supported_clocks>
                        <supported_mem_clock>
                                <value>3504 MHz</value>
                                <supported_graphics_clock>1721 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1708 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1695 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1683 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1670 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1657 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1645 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1632 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1620 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1607 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1594 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1582 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1569 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1556 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1544 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1531 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1518 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1506 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1493 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1480 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1468 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1455 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1442 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1430 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1417 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1404 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1392 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1379 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1366 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1354 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1341 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1328 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1316 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1303 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1290 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1278 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1265 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1252 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1240 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1227 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1215 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1202 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1189 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1177 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1164 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1151 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1139 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1126 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1113 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1101 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1088 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1075 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1063 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1050 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1037 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1025 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1012 MHz</supported_graphics_clock>
                                <supported_graphics_clock>999 MHz</supported_graphics_clock>
                                <supported_graphics_clock>987 MHz</supported_graphics_clock>
                                <supported_graphics_clock>974 MHz</supported_graphics_clock>
                                <supported_graphics_clock>961 MHz</supported_graphics_clock>
                                <supported_graphics_clock>949 MHz</supported_graphics_clock>
                                <supported_graphics_clock>936 MHz</supported_graphics_clock>
                                <supported_graphics_clock>923 MHz</supported_graphics_clock>
                                <supported_graphics_clock>911 MHz</supported_graphics_clock>
                                <supported_graphics_clock>898 MHz</supported_graphics_clock>
                                <supported_graphics_clock>885 MHz</supported_graphics_clock>
                                <supported_graphics_clock>873 MHz</supported_graphics_clock>
                                <supported_graphics_clock>860 MHz</supported_graphics_clock>
                                <supported_graphics_clock>847 MHz</supported_graphics_clock>
                                <supported_graphics_clock>835 MHz</supported_graphics_clock>
                                <supported_graphics_clock>822 MHz</supported_graphics_clock>
                                <supported_graphics_clock>810 MHz</supported_graphics_clock>
                                <supported_graphics_clock>797 MHz</supported_graphics_clock>
                                <supported_graphics_clock>784 MHz</supported_graphics_clock>
                                <supported_graphics_clock>772 MHz</supported_graphics_clock>
                                <supported_graphics_clock>759 MHz</supported_graphics_clock>
                                <supported_graphics_clock>746 MHz</supported_graphics_clock>
                                <supported_graphics_clock>734 MHz</supported_graphics_clock>
                                <supported_graphics_clock>721 MHz</supported_graphics_clock>
                                <supported_graphics_clock>708 MHz</supported_graphics_clock>
                                <supported_graphics_clock>696 MHz</supported_graphics_clock>
                                <supported_graphics_clock>683 MHz</supported_graphics_clock>
                                <supported_graphics_clock>670 MHz</supported_graphics_clock>
                                <supported_graphics_clock>658 MHz</supported_graphics_clock>
                                <supported_graphics_clock>645 MHz</supported_graphics_clock>
                                <supported_graphics_clock>632 MHz</supported_graphics_clock>
                                <supported_graphics_clock>620 MHz</supported_graphics_clock>
                                <supported_graphics_clock>607 MHz</supported_graphics_clock>
                                <supported_graphics_clock>594 MHz</supported_graphics_clock>
                                <supported_graphics_clock>582 MHz</supported_graphics_clock>
                                <supported_graphics_clock>569 MHz</supported_graphics_clock>
                                <supported_graphics_clock>556 MHz</supported_graphics_clock>
                                <supported_graphics_clock>544 MHz</supported_graphics_clock>
                                <supported_graphics_clock>531 MHz</supported_graphics_clock>
                                <supported_graphics_clock>518 MHz</supported_graphics_clock>
                                <supported_graphics_clock>506 MHz</supported_graphics_clock>
                                <supported_graphics_clock>493 MHz</supported_graphics_clock>
                                <supported_graphics_clock>480 MHz</supported_graphics_clock>
                                <supported_graphics_clock>468 MHz</supported_graphics_clock>
                                <supported_graphics_clock>455 MHz</supported_graphics_clock>
                                <supported_graphics_clock>442 MHz</supported_graphics_clock>
                                <supported_graphics_clock>430 MHz</supported_graphics_clock>
                                <supported_graphics_clock>417 MHz</supported_graphics_clock>
                                <supported_graphics_clock>405 MHz</supported_graphics_clock>
                                <supported_graphics_clock>392 MHz</supported_graphics_clock>
                                <supported_graphics_clock>379 MHz</supported_graphics_clock>
                                <supported_graphics_clock>367 MHz</supported_graphics_clock>
                                <supported_graphics_clock>354 MHz</supported_graphics_clock>
                                <supported_graphics_clock>341 MHz</supported_graphics_clock>
                                <supported_graphics_clock>329 MHz</supported_graphics_clock>
                                <supported_graphics_clock>316 MHz</supported_graphics_clock>
                                <supported_graphics_clock>303 MHz</supported_graphics_clock>
                                <supported_graphics_clock>291 MHz</supported_graphics_clock>
                                <supported_graphics_clock>278 MHz</supported_graphics_clock>
                                <supported_graphics_clock>265 MHz</supported_graphics_clock>
                                <supported_graphics_clock>253 MHz</supported_graphics_clock>
                                <supported_graphics_clock>240 MHz</supported_graphics_clock>
                                <supported_graphics_clock>227 MHz</supported_graphics_clock>
                                <supported_graphics_clock>215 MHz</supported_graphics_clock>
                                <supported_graphics_clock>202 MHz</supported_graphics_clock>
                                <supported_graphics_clock>189 MHz</supported_graphics_clock>
                                <supported_graphics_clock>177 MHz</supported_graphics_clock>
                                <supported_graphics_clock>164 MHz</supported_graphics_clock>
                                <supported_graphics_clock>151 MHz</supported_graphics_clock>
                                <supported_graphics_clock>139 MHz</supported_graphics_clock>
                        </supported_mem_clock>
                        <supported_mem_clock>
                                <value>810 MHz</value>
                                <supported_graphics_clock>1721 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1708 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1695 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1683 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1670 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1657 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1645 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1632 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1620 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1607 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1594 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1582 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1569 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1556 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1544 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1531 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1518 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1506 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1493 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1480 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1468 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1455 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1442 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1430 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1417 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1404 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1392 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1379 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1366 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1354 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1341 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1328 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1316 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1303 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1290 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1278 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1265 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1252 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1240 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1227 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1215 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1202 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1189 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1177 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1164 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1151 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1139 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1126 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1113 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1101 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1088 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1075 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1063 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1050 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1037 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1025 MHz</supported_graphics_clock>
                                <supported_graphics_clock>1012 MHz</supported_graphics_clock>
                                <supported_graphics_clock>999 MHz</supported_graphics_clock>
                                <supported_graphics_clock>987 MHz</supported_graphics_clock>
                                <supported_graphics_clock>974 MHz</supported_graphics_clock>
                                <supported_graphics_clock>961 MHz</supported_graphics_clock>
                                <supported_graphics_clock>949 MHz</supported_graphics_clock>
                                <supported_graphics_clock>936 MHz</supported_graphics_clock>
                                <supported_graphics_clock>923 MHz</supported_graphics_clock>
                                <supported_graphics_clock>911 MHz</supported_graphics_clock>
                                <supported_graphics_clock>898 MHz</supported_graphics_clock>
                                <supported_graphics_clock>885 MHz</supported_graphics_clock>
                                <supported_graphics_clock>873 MHz</supported_graphics_clock>
                                <supported_graphics_clock>860 MHz</supported_graphics_clock>
                                <supported_graphics_clock>847 MHz</supported_graphics_clock>
                                <supported_graphics_clock>835 MHz</supported_graphics_clock>
                                <supported_graphics_clock>822 MHz</supported_graphics_clock>
                                <supported_graphics_clock>810 MHz</supported_graphics_clock>
                                <supported_graphics_clock>797 MHz</supported_graphics_clock>
                                <supported_graphics_clock>784 MHz</supported_graphics_clock>
                                <supported_graphics_clock>772 MHz</supported_graphics_clock>
                                <supported_graphics_clock>759 MHz</supported_graphics_clock>
                                <supported_graphics_clock>746 MHz</supported_graphics_clock>
                                <supported_graphics_clock>734 MHz</supported_graphics_clock>
                                <supported_graphics_clock>721 MHz</supported_graphics_clock>
                                <supported_graphics_clock>708 MHz</supported_graphics_clock>
                                <supported_graphics_clock>696 MHz</supported_graphics_clock>
                                <supported_graphics_clock>683 MHz</supported_graphics_clock>
                                <supported_graphics_clock>670 MHz</supported_graphics_clock>
                                <supported_graphics_clock>658 MHz</supported_graphics_clock>
                                <supported_graphics_clock>645 MHz</supported_graphics_clock>
                                <supported_graphics_clock>632 MHz</supported_graphics_clock>
                                <supported_graphics_clock>620 MHz</supported_graphics_clock>
                                <supported_graphics_clock>607 MHz</supported_graphics_clock>
                                <supported_graphics_clock>594 MHz</supported_graphics_clock>
                                <supported_graphics_clock>582 MHz</supported_graphics_clock>
                                <supported_graphics_clock>569 MHz</supported_graphics_clock>
                                <supported_graphics_clock>556 MHz</supported_graphics_clock>
                                <supported_graphics_clock>544 MHz</supported_graphics_clock>
                                <supported_graphics_clock>531 MHz</supported_graphics_clock>
                                <supported_graphics_clock>518 MHz</supported_graphics_clock>
                                <supported_graphics_clock>506 MHz</supported_graphics_clock>
                                <supported_graphics_clock>493 MHz</supported_graphics_clock>
                                <supported_graphics_clock>480 MHz</supported_graphics_clock>
                                <supported_graphics_clock>468 MHz</supported_graphics_clock>
                                <supported_graphics_clock>455 MHz</supported_graphics_clock>
                                <supported_graphics_clock>442 MHz</supported_graphics_clock>
                                <supported_graphics_clock>430 MHz</supported_graphics_clock>
                                <supported_graphics_clock>417 MHz</supported_graphics_clock>
                                <supported_graphics_clock>405 MHz</supported_graphics_clock>
                                <supported_graphics_clock>392 MHz</supported_graphics_clock>
                                <supported_graphics_clock>379 MHz</supported_graphics_clock>
                                <supported_graphics_clock>367 MHz</supported_graphics_clock>
                                <supported_graphics_clock>354 MHz</supported_graphics_clock>
                                <supported_graphics_clock>341 MHz</supported_graphics_clock>
                                <supported_graphics_clock>329 MHz</supported_graphics_clock>
                                <supported_graphics_clock>316 MHz</supported_graphics_clock>
                                <supported_graphics_clock>303 MHz</supported_graphics_clock>
                                <supported_graphics_clock>291 MHz</supported_graphics_clock>
                                <supported_graphics_clock>278 MHz</supported_graphics_clock>
                                <supported_graphics_clock>265 MHz</supported_graphics_clock>
                                <supported_graphics_clock>253 MHz</supported_graphics_clock>
                                <supported_graphics_clock>240 MHz</supported_graphics_clock>
                                <supported_graphics_clock>227 MHz</supported_graphics_clock>
                                <supported_graphics_clock>215 MHz</supported_graphics_clock>
                                <supported_graphics_clock>202 MHz</supported_graphics_clock>
                                <supported_graphics_clock>189 MHz</supported_graphics_clock>
                                <supported_graphics_clock>177 MHz</supported_graphics_clock>
                                <supported_graphics_clock>164 MHz</supported_graphics_clock>
                                <supported_graphics_clock>151 MHz</supported_graphics_clock>
                                <supported_graphics_clock>139 MHz</supported_graphics_clock>
                        </supported_mem_clock>
                        <supported_mem_clock>
                                <value>405 MHz</value>
                                <supported_graphics_clock>607 MHz</supported_graphics_clock>
                                <supported_graphics_clock>594 MHz</supported_graphics_clock>
                                <supported_graphics_clock>582 MHz</supported_graphics_clock>
                                <supported_graphics_clock>569 MHz</supported_graphics_clock>
                                <supported_graphics_clock>556 MHz</supported_graphics_clock>
                                <supported_graphics_clock>544 MHz</supported_graphics_clock>
                                <supported_graphics_clock>531 MHz</supported_graphics_clock>
                                <supported_graphics_clock>518 MHz</supported_graphics_clock>
                                <supported_graphics_clock>506 MHz</supported_graphics_clock>
                                <supported_graphics_clock>493 MHz</supported_graphics_clock>
                                <supported_graphics_clock>480 MHz</supported_graphics_clock>
                                <supported_graphics_clock>468 MHz</supported_graphics_clock>
                                <supported_graphics_clock>455 MHz</supported_graphics_clock>
                                <supported_graphics_clock>442 MHz</supported_graphics_clock>
                                <supported_graphics_clock>430 MHz</supported_graphics_clock>
                                <supported_graphics_clock>417 MHz</supported_graphics_clock>
                                <supported_graphics_clock>405 MHz</supported_graphics_clock>
                                <supported_graphics_clock>392 MHz</supported_graphics_clock>
                                <supported_graphics_clock>379 MHz</supported_graphics_clock>
                                <supported_graphics_clock>367 MHz</supported_graphics_clock>
                                <supported_graphics_clock>354 MHz</supported_graphics_clock>
                                <supported_graphics_clock>341 MHz</supported_graphics_clock>
                                <supported_graphics_clock>329 MHz</supported_graphics_clock>
                                <supported_graphics_clock>316 MHz</supported_graphics_clock>
                                <supported_graphics_clock>303 MHz</supported_graphics_clock>
                                <supported_graphics_clock>291 MHz</supported_graphics_clock>
                                <supported_graphics_clock>278 MHz</supported_graphics_clock>
                                <supported_graphics_clock>265 MHz</supported_graphics_clock>
                                <supported_graphics_clock>253 MHz</supported_graphics_clock>
                                <supported_graphics_clock>240 MHz</supported_graphics_clock>
                                <supported_graphics_clock>227 MHz</supported_graphics_clock>
                                <supported_graphics_clock>215 MHz</supported_graphics_clock>
                                <supported_graphics_clock>202 MHz</supported_graphics_clock>
                                <supported_graphics_clock>189 MHz</supported_graphics_clock>
                                <supported_graphics_clock>177 MHz</supported_graphics_clock>
                                <supported_graphics_clock>164 MHz</supported_graphics_clock>
                                <supported_graphics_clock>151 MHz</supported_graphics_clock>
                                <supported_graphics_clock>139 MHz</supported_graphics_clock>
                        </supported_mem_clock>
                </supported_clocks>
                <processes>
                        <process_info>
                                <pid>7134</pid>
                                <type>C</type>
                                <process_name>/usr/lib/plexmediaserver/Plex Transcoder</process_name>
                                <used_memory>309 MiB</used_memory>
                        </process_info>
                </processes>
                <accounted_processes>
                </accounted_processes>
        </gpu>

</nvidia_smi_log>

Release Windows Executable

Hi,

Great project!

Since nvidia-smi is a binary that is also available on Windows, this project can potentially run without any issues on a Windows system. It would be great to have a Windows binary (using ruby-packer or something) or an installer, so that regular gaming PCs etc can have their metrics exported.

Sharing Nvidia GPUs with Docker for Windows is still experimental, and there shouldn't be a need to run Docker at all.

Would you consider providing pre-built executables on releases? Not only for Windows but for other OSes as well?

Data Type for Prometheus

Hi,

I am trying out your gpu-exporter and I have the following issue :

I am running on k8s and have made a little helm chart with a daemonset as well as a service.

Trying from an outside container or within the container I successfully access the metrics at http://nvidia-smi-exporter:9454/metrics

I put the following job in my prometheus configuration :

      - job_name: nvidia_gpus
        static_configs:
          - targets:
            - nvidia-smi-exporter:9454

I am getting :


Endpoint | State | Labels | Last Scrape | Scrape Duration | Error
-- | -- | -- | -- | -- | --
http://nvidia-smi-exporter:9454/metrics | DOWN | instance="nvidia-smi-exporter:9454"job="nvidia_gpus" | 16.146s | 486.794ms | strconv.ParseFloat: parsing "455.32.00": invalid syntax

Am I missing something?

Thanks
Marius

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.