Git Product home page Git Product logo

iostat-tool's Introduction

iostat-tool

parse and visualize iostat output

Requirements

  • Matplotlib
  • Support output of iostat for Linux only
    • iostat command needs -t argument to get timestamp, and -ymxt 1 arguments are recommended
$ man iostat
  -t  Print the time for each report displayed. The timestamp format may depend on the value of the S_TIME_FORMAT environment variable (see below).

How to install

for user

$ pip install iostat-tool

for developer

Create venv to install iostat-tool.

$ mkdir venvs
$ python3 -m venv venvs/venv
$ source venvs/venv/bin/activate
(venv) $

Install iostat-tool package from the repository.

(venv) $ git clone https://github.com/t2y/iostat-tool.git
(venv) $ cd iostat-tool
(venv) $ python setup.py develop

Base CLI options

Confirm iostat-cli works as below.

(venv) $ iostat-cli --help
usage: iostat-cli [-h] [--backend BACKEND] [--data DATA]
                  [--fig-output FIGOUTPUT] [--fig-size FIGSIZE]
                  [--output OUTPUT] [--disks DISKS [DISKS ...]]
                  [--since SINCE] [--until UNTIL] [-v] [--version]
                  {csv,monitor,plot} ...

positional arguments:
  {csv,monitor,plot}

optional arguments:
  -h, --help            show this help message and exit
  --backend BACKEND     set backend for matplotlib, use TkAgg to monitor in
                        the foreground
  --data DATA           set path to iostat output file
  --fig-output FIGOUTPUT
                        set path to save graph
  --fig-size FIGSIZE    set figure size
  --output OUTPUT       set path to save output of iostat
  --disks DISKS [DISKS ...]
                        set disk name in iostat
  --since SINCE         set since datetime, format: yyyymmddHHMISS
  --until UNTIL         set until datetime, format: yyyymmddHHMISS
  -v, --verbose         set verbose mode
  --version             show program version

Sub Commands

csv

Create csv/tsv file from output of iostat.

(venv) $ iostat-cli csv --help
usage: iostat-cli csv [-h] [--dialect {excel,excel-tab,unix}]
                      [--separator {comma,tab}]

optional arguments:
  -h, --help            show this help message and exit
  --dialect {excel,excel-tab,unix}
                        set dialect for csv writer, default is excel
  --separator {comma,tab}
                        set separator, default is comma

plot

Create image file rendered by matplotlib from output of iostat.

(venv) $ iostat-cli plot --help
usage: iostat-cli plot [-h] [--plot-type {plotter,scatter}]
                       [--subplots {io_rqm,iops,io_transfer,%util,avgrq-sz,avgqu-sz,await,svctm} [{io_rqm,iops,io_transfer,%util,avgrq-sz,avgqu-sz,await,svctm} ...]]
                       [--vlines VLINES [VLINES ...]]

optional arguments:
  -h, --help            show this help message and exit
  --plot-type {plotter,scatter}
                        set plot type ("plotter" by default)
  --subplots {io_rqm,iops,io_transfer,%util,avgrq-sz,avgqu-sz,await,svctm} [{io_rqm,iops,io_transfer,%util,avgrq-sz,avgqu-sz,await,svctm} ...]
                        set subplots to show
  --vlines VLINES [VLINES ...]
                        set vertical line, format: yyyymmddHHMISS
  --x-datetime-format X_DATETIME_FORMAT
                        set datetime format for devices x-axis
  --title TITLE         set title for graph
  --without-cpu         don't plot CPU data
  --cpu-only            plot only CPU data

monitor

Monitor and logging output of iostat command.

(venv) $ iostat-cli monitor --help
usage: iostat-cli monitor [-h] [--iostat-args IOSTAT_ARGS]
                          [--max-queue-size MAX_QUEUE_SIZE]

optional arguments:
  -h, --help            show this help message and exit
  --iostat-args IOSTAT_ARGS
                        set arguments for iostat
  --max-queue-size MAX_QUEUE_SIZE
                        set queue size to read iostat output

How to use

This is sample image rendered by matplotlib.

there is already an iostat.ouput as data file

plot

  • show all subplots of /dev/sda and cpu average
(venv) $ iostat-cli --data tests/fixtures/sample_iostat.output --disk sda --fig-output my-iostat.png plot
(venv) $ file my-iostat.png
my-iostat.png: PNG image data, 1800 x 1400, 8-bit/color RGBA, non-interlaced
  • show any subplots of /dev/sda and cpu average
    • filter io_rqm and iops with --subplots
(venv) $ iostat-cli --data tests/fixtures/sample_iostat.output --disk sda --fig-output my-iostat.png \
  plot --subplots io_rqm iops
  • show any range of date time
    • filter since 2018-06-13 14:11:00 with --since
    • filter until 2018-06-13 14:11:30 with --until
(venv) $ iostat-cli --data tests/fixtures/sample_iostat.output --disk sda --fig-output my-iostat.png \
  --since 20180613141100 --until 20180613141130 plot --subplots await svctm
  • show vertical lines into graph
    • 2018-06-13 14:11:10 and 2018-06-13 14:11:20 with --vlines
(venv) $ iostat-cli --data tests/fixtures/sample_iostat.output --disk sda --fig-output my-iostat.png \
  --since 20180613141100 --until 20180613141130 plot --subplots await svctm --vlines 20180613141110 20180613141120
  • show only CPU-related data
(venv) $ iostat-cli --data tests/fixtures/sample_iostat.output --disk sda --fig-output my-iostat.png \
  plot --cpu-only
  • show only one subplot, without CPU information
(venv) $ iostat-cli --data tests/fixtures/sample_iostat.output --disk sda --fig-output my-iostat.png \
  plot --subplots await --without-cpu
  • generate graph with custom title
(venv) $ iostat-cli --data tests/fixtures/sample_iostat.output --disk sda --fig-output my-iostat.png \
  plot --title 'my custom test'

csv

  • output 2 csv files (iostat_cpu.csv and iostat_devices.csv)
(venv) $ iostat-cli --data tests/fixtures/sample_iostat.output --output iostat.csv csv
(venv) $ ls iostat_*.csv
iostat_cpu.csv		iostat_devices.csv
(venv) $ head -n 3 iostat_*.csv
==> iostat_cpu.csv <==
datetime,%user,%nice,%system,%iowait,%steal,%idle
2018-06-13 14:10:50,0.47,0.0,0.24,0.18,0.0,99.11
2018-06-13 14:10:51,3.07,0.0,0.66,0.09,0.0,96.18

==> iostat_devices.csv <==
datetime,device,rrqm/s,wrqm/s,r/s,w/s,rMB/s,wMB/s,avgrq-sz,avgqu-sz,await,r_await,w_await,svctm,%util
2018-06-13 14:10:50,sdd,0.07,45.88,1.57,0.59,0.08,0.18,246.55,0.26,121.04,1.28,436.94,2.07,0.45
2018-06-13 14:10:50,sdh,0.07,45.78,1.59,0.6,0.08,0.18,245.64,0.22,101.97,1.17,367.51,1.89,0.41

run iostat and logging the output

  • monitor iostat command running
    • almost the same as iostat -yxmt 1 | tee my-iostat.log
(venv) $ iostat-cli --output my-iostat.log --fig-output my-scatter.png monitor --iostat-args "-yxmt 1"
...
06/13/2018 10:11:07 PM
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.94    0.00    0.03    0.00    0.00   99.03

Device:         rrqm/s   wrqm/s     r/s     w/s    rMB/s    wMB/s avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
sda               0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00    0.00    0.00   0.00   0.00

06/18/2018 10:11:08 PM
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.16    0.00    0.06    0.00    0.00   99.78
...

(Ctrl + C) # stop to run iostat
(venv) $ file my-iostat.log my-scatter.png
my-iostat.log:  ASCII text
my-scatter.png: PNG image data, 1800 x 1300, 8-bit/color RGBA, non-interlaced

NOTE: Saving my-scatter.png is experimental feature when io-stat terminated, so it might fails to save the figure.

iostat-tool's People

Contributors

alexott avatar raharper avatar t2y avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

iostat-tool's Issues

ValueError: x and y must have same first dimension, but have shapes (1,) and (306,)

Hi,
i try to use iostat-cli (version 0.3.0) but i only get this stacktrace:

iostat-cli --data 5.19_SDCIT_patch_nooptimize_download_success.iostat.log --disk mmcblk1 --fig-output success.png plot
Traceback (most recent call last):
File "/home/stefanw/.local/bin/iostat-cli", line 8, in
sys.exit(main())
File "/home/stefanw/.local/lib/python3.10/site-packages/iostat/main.py", line 217, in main
plotter.plot()
File "/home/stefanw/.local/lib/python3.10/site-packages/iostat/plotter.py", line 202, in plot
self.plot_device(datetime_data)
File "/home/stefanw/.local/lib/python3.10/site-packages/iostat/plotter.py", line 188, in plot_device
self.subplots[name].plot(x, values, label=column)
File "/home/stefanw/.local/lib/python3.10/site-packages/matplotlib/axes/_axes.py", line 1635, in plot
lines = [*self._get_lines(*args, data=data, **kwargs)]
File "/home/stefanw/.local/lib/python3.10/site-packages/matplotlib/axes/_base.py", line 312, in call
yield from self._plot_args(this, kwargs)
File "/home/stefanw/.local/lib/python3.10/site-packages/matplotlib/axes/_base.py", line 498, in _plot_args
raise ValueError(f"x and y must have same first dimension, but "
ValueError: x and y must have same first dimension, but have shapes (1,) and (306,)

What's wrong here?
Thanks in advance

5.19_SDCIT_patch_nooptimize_download_success.iostat.log

ValueError: time data '08/08/23 21:49:22' does not match format

File "/home/sdp/dpaul/venvs/venv/lib/python3.8/site-packages/iostat/utils.py", line 27, in parse_datetime
return datetime.strptime(s, fmt)
File "/usr/local/lib/python3.8/_strptime.py", line 568, in _strptime_datetime
tt, fraction, gmtoff_fraction = _strptime(data_string, format)
File "/usr/local/lib/python3.8/_strptime.py", line 349, in _strptime
raise ValueError("time data %r does not match format %r" %
ValueError: time data '08/08/23 21:49:22' does not match format '%m/%d/%y %I:%M:%S'

Any clue how to resolve them ? thanks

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.