Comments (1)
For anybody striking this issue, here is a snippet that permits the ingestion of dstat csv log data into pandas/jupyter without errors brought on by the variable column width issue.
The column alignment in this code snippet is specific to the the output produced by dstat run with the -Tcdilmnprsy --proc-count
options.
import pandas as pd
dstat_csv_file = '/tmp/dstat_Tcdilmnprsy_proc_dstat_master.csv'
column_names = ['timestamp',
'cpu_usr', 'cpu_sys', 'cpu_idl', 'cpu_wai', 'cpu_hiq', 'cpu_siq',
'dsk_read', 'dsk_writ',
'intp_a', 'intp_b', 'intp_c',
'la_1m', 'la_5m', 'la_15m',
'mem_used', 'mem_buff', 'mem_cach', 'mem_free',
'net_recv', 'net_send',
'proc_run', 'proc_blk', 'proc_new',
'io_read', 'io_writ',
'swap_used', 'swap_free',
'sys_int', 'sys_csw',
'proc_total']
df = pd.read_csv(dstat_csv_file, skiprows=6, usecols=range(0,31)) # usecols used to slice a consistent data set and avoid csv variable column width issue.
df.columns = column_names # Setting column names here rather than with the 'read_csv' 'names' option as read_csv errors when variable column widths seen.
# See https://github.com/dagwieers/dstat/issues/154
df['timestamp'] = pd.to_datetime(df['timestamp'],unit='s') # epoch to datetime format
df['timestamp'] = df['timestamp'].values.astype('datetime64[s]') # Remove millisecond precision
df.set_index('timestamp', inplace=True)
df.head(2)
from dstat.
Related Issues (20)
- Incorrect unit in top-mem csv
- Horizontal layout, maybe with bar plots
- Plugin path computed incorrectly when dstat is symlinked
- CSV output to stdout
- output statics in mgb or mgB ?
- [wishlist] ability to change the "blue" color in the header (blue is too dark, can't read) HOT 2
- Add way to supress missed tick output
- Feature request : Total Bytes cumulated on delay X HOT 3
- Infiniband Problem
- Incorrect numbers with raid configuration
- Feature request : extended memory statistics : Huge Pages and Commited pages
- I would like to participate in dstat development and responding to issues. HOT 25
- what's the unit of io?
- MariaDB modules HOT 3
- Fix for freespace plugin errors never merged
- Release a version of dstat with Python 3 support
- Master branch throws an error when you try and use a delay
- linux 2.6, dstat --mem not working HOT 1
- So long and thanks for all the fish.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dstat.