Mutivariate anomalie detection. about adtk HOT 13 OPEN

@abhimanyu3 If we assume a series is the reason for anomaly, then I would apply a univariate detector to each series independently.

A multivariate detector is for the case where the anomaly is due to the relationship between series changes. In that case, it's hard to say which series "causes" the anomaly because the anomaly is caused by those series jointly.

from adtk.

@tailaiw Thanks a lot for your response. I have to find sudden peaks and drops in my multivariate time series data so I am using the PersistAD method on the df. Shall I use it on each column or even if I am using it on df it's the same thing?

Also, where I can find details like what is C in the PersistAD so that I can take a holistic decision in tuning.

Do you recommend any other method for finding sudden peaks and drop or persistAD is good.

from adtk.

hi there, do either of you know where you can find the formulae used in the PersistAD? I haven't been able to find it in the code.
Thanks a million guys

from adtk.

@abhimanyu3 and @ivanokeeffe PersistAD is implemented as a pipeline of DoubleRollingAggregate transformer and InterQuartileRangeAD detector. You may refer to the pipe_ attribute of a PersistAD object for more details.

The parameter c is the same one used by the internal InterQuartileRangeAD which controls the "normal range". InterQuartileRangeAD is a very classic simple outlier detection method. The value "c" is usually 1.5 or 3, although the user may specify according to the problem to solve.

from adtk.

from adtk.

@ivanokeeffe Hey! what kind of outlier you are trying to detect. Is it sudden peak and drops??

from adtk.

from adtk.

Does anyone know how to get the intermediate output for the PersistAD also?

from adtk.

Running a pipe object (adtk.pipeline or adtk.pipenet) with option return_intermediate=True will return the results of all steps of the pipe, instead of only the last one.

As mentioned above, like many other models in ADTK, PersistAD is internally implemented as a pipe of transformers and detectors. Attribute pipe_ points to the internal pipe object. So if we want the intermediate results, the easiest way is probably calling it as follows:

my_model = PersistAD()
my_model.pipe_.fit_detect(s, return_intermediate=True) # instead of my_model.fit_detect(s) which is equivalent to my_model.pipe_.fit_detect(s, return_intermediate=False)

from adtk.

@ivanokeeffe Hey! Are you also applying seasonality check in this. I mean by editing the pipeline?

from adtk.

from adtk.

from adtk.

Give it a read :- https://www.quora.com/How-do-you-identify-seasonality-in-a-time-series-data

Let me know if you will be able to do it. @ivanokeeffe

Did you get the maths behind the persistAD?

from adtk.