Comments (4)
I found the pattern, if I run 4K IO tests on it, it will hang like this.
intel@intel:/mnt/data1$ sudo iotrace --start-tracing --devices /dev/nvme1n1 --time 45 --size 1024
{
"timestamp": "2020-06-03 04:02.805",
"system": "iotrace",
"severity": "Information",
"trace": "Trace collecting has started"
}
{
"timestamp": "2020-06-03 04:03.894",
"system": "iotrace",
"severity": "Information",
"trace": "Trace collecting has completed"
}
^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C
^C^C^C
^C^C^C
from standalone-linux-io-tracer.
Could you post some additional info about the environment -
Version of iotrace and octf (post results of iotrace -V)
Kernel version (uname -r)
OS (Fedora/Ubuntu/Centos version)
Any kernel logs around the time of the tracing
from standalone-linux-io-tracer.
my OS is centOS7
kernel is 3.10
I used to check dmesg, no info related. I will get iotrace -V next time.
I use the minio to run the 4K small objects. then I use iotrace, then it hang there.
it can stable reproduce the issue. after setup minio, then run below benchmark test.
intel@intel:~/wayne/s3-benchmark$ sudo ./s3-benchmark -a minioadmin -b jeff-s3-benchmark -s minioadmin -u http://127.0.0.1:9000 -t 24 -d 20 -z 4k
S3 benchmark program v3.0
Parameters: url=http://127.0.0.1:9000, bucket=jeff-s3-benchmark, duration=20, threads=24, loops=1, size=4k
Wed, 03 Jun 2020 10:08:09 GMT Loop 1: PUT time 20.0 secs, objects = 158201, speed = 30.9MB/sec, 7908.9 operations/sec.
Wed, 03 Jun 2020 10:08:29 GMT Loop 1: GET time 20.0 secs, objects = 119104, speed = 23.3MB/sec, 5954.0 operations/sec.
Wed, 03 Jun 2020 10:08:49 GMT Loop 1: DELETE time 19.8 secs, 7999.9 operations/sec.
Benchmark completed.
from standalone-linux-io-tracer.
Thanks for the information, I am able to consistently reproduce it on my end on the current master so no extra information is necessary for now
from standalone-linux-io-tracer.
Related Issues (20)
- `count` in trace stats is confusing. HOT 1
- Traces description is inconsistent between device statistics and FS statistics HOT 1
- In trace statistics average latency has wrong value
- Missing '99.99' latency percentile
- Source package doesn't contain VERSION
- Invalid discard section content
- Source package contains .git directory HOT 1
- IO trace doesn't show LBA if LBA equals 0
- Unsupported operands in test_lba_histogram
- No parse_json attribute in test_files_privileges
- Kernel panic when when running iotrace on RHEL7.6 from RPM built on RHEL7.7
- package doesn't contain NOTICE and LICENSE file
- Tracing doesn't handle situation when IO is splitted by Linux block layer
- Name of generated packages should contain kernel version
- build will fail on ubuntu20.04 HOT 1
- io trace reports missing device model and need units in columns
- Add units to CSV columns HOT 1
- ubuntu 20.04 compile incomplete definition of type 'struct folio' HOT 4
- build error on kernel6.3 in fedoraOS HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from standalone-linux-io-tracer.