waico / skab Goto Github PK
View Code? Open in Web Editor NEWSKAB - Skoltech Anomaly Benchmark. Time-series data for evaluating Anomaly Detection algorithms.
License: GNU General Public License v3.0
SKAB - Skoltech Anomaly Benchmark. Time-series data for evaluating Anomaly Detection algorithms.
License: GNU General Public License v3.0
Hi,
The data in valve1
is labeled 0-15, and in valve2
is labeled 0-3. The README.md
file, however states that the experiment labels should be reversed, and that names should be 1-based.
Thanks
Hi, I have some questions about the column: 'changepoint'.
According to my last post(thanks Kaster! you're really helpful!) : https://www.kaggle.com/datasets/yuriykatser/skoltech-anomaly-benchmark-skab/discussion/325533, the changepoint should indicate the status changing between 'normal' and 'anomaly': i.e. 'normal' to 'anomaly' or 'anomaly' to 'normal'.
However, when I checked the data, I found something unexpected.
I merged all .csv file under data/valve1/ and sort them by the column: datetime. There are 63 records with changepoint = 1, but only 32 indicate a status switching.
Hello,
I have some concerns regarding the notebooks provided.
Why is the training made with files having anomalies instead of the free anomaly csv file and then testing on the other files?
Also the results are taking in count the whole file including the training samples but I know that results need to be done on only data that is unknown to the model. Am I missing something?
Thank you.
We need to add calculating F1-score because FAR and MAR metrics are quite misleading.
Good Morning :)
I wanna know when are you gonna upload the English version of EDA on kaggle?
Also, where can I get upcoming update to v1.0? I want to get more anomaly and anomaly-free dataset :)
Can u plz share more valve datasets?
Thanks for your great work!
Have a great day.
The function Anomaly_detection is not defined in ArimeFD.ipynb
Hi!
Any idea when we could expect 1.0 version?
Hello, I have a question about this snippet of code (the loop helps to find the anomalies):
# data i is an anomaly if samples [(i - timesteps + 1) to (i)] are anomalies
anomalous_data = cnn_residuals > (3/2 * UCL)
anomalous_data_indices = []
for data_idx in range(N_STEPS - 1, len(X) - N_STEPS + 1):
if np.all(anomalous_data[data_idx - N_STEPS + 1 : data_idx]):
anomalous_data_indices.append(data_idx)
comment says that given a point i
, if all samples between i-timesteps+1
and i
are anomalies then the point i
is anomaly.
This loop starts from N_STEPS - 1
which is (lets say N_STEPS=10
) from 9 and ends to len(X) - N_STEPS + 1
which is (lets say len(X)=100
) the number 91.
Then the if
statement going to check (for each iteration) the samples from data_idx - N_STEPS + 1
to data_idx
So
iteration 1: all samples from 9-10+1
which is 0 to 9 (not inclusive)
iteration 2: all samples from 10-10+1
which is 1 to 10 (not inclusive)
....[going at the end].....
iteration 82: all samples from 10-10+1
which is 81 to 90 (not inclusive)
In other words:
We check i=9
with samples 0 to 8 (inclusive)
We check i=10
with samples 1 to 9 (inclusive)
....[going at the end].....
We check i=90
with samples 81 to 89 (inclusive)
Question
I think we are missing i=92, 93, 94,....,99. Am I wrong? And why?
If am not wrong shouldn't the for data_idx in range(N_STEPS - 1, len(X) - N_STEPS + 1)
be replaced with for data_idx in range(N_STEPS - 1, len(X)):
in order to iterate until the end?
Thank you in advance
Hello :)
Where can I get evaluating python code file in your repository?
SKAB/notebooks/autoencoder.ipynb
Line 24 in 9b9ad7b
Thank you.
Need to add a file to the notebooks folder with package dependencies to provide reproducibility.
One of the files contains only 104 points before the first anomaly is introduced, which means that it is not possible to train on the first 400 points as suggested.
All the files are irregularly sampled with some intervals 2 seconds instead of 1 second. Three files contain gaps of greater than 50 seconds. Two more have multiple gaps of 3, 4, and 5 seconds throughout.
As such, the data is not really usable for a ML study.
Hi,
I try to run the ArimaFD notebook but I'm not able to load the MeshLoader package.
---> from other import MeshLoader
ModuleNotFoundError: No module named 'other'
Did I miss something ?
thank you
Need to add a README file to the notebooks folder with the information regarding anomaly detection algorithms.
Thanks for this very interesting project. Could you provide a few more details about the sensors and their location within your test bed? I am particularly interested in the location of the two accelerometers.
Many thanks.
... has gone missing from the repo
In real technical systems, there are the following tasks of the diagnostic system: early detection of faults (anomalies), finding the root causes, and predicting the development of faults. The last task gives the greatest economic effects since it allows you to determine the time to failure, which makes it possible to optimize MRO. SCAB's capabilities allow it to be used as a predictive benchmark. Now work is underway to bring the idea to life.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.