Conducted a thorough analysis of the recent research trend in anomaly detection. There are several machine learning methods reported to have a very high detection rate of 98% while keeping the false alarm rate at 1%. However, when we look at the state of the art IDS solutions and commercial tools, there is no evidence of using anomaly detection approaches, and practitioners still think that it is an immature technology. To find the reason of this contrast, lots of research was done done in anomaly detection and considered various aspects such as learning and detection approaches, training data sets, testing data sets, and evaluation methods.

The task is to build network intrusion detection system to detect anamolies and attacks in the network. There are two problems. 1. Binomial Classification: Activity is normal or attack 2. Multinomial classification: Activity is normal or DOS or PROBE or R2L or U2R Please note that, currently the dependent variable (target variable) is not definied explicitly. However, we can use attack variable to define the target variable as required.

This data is KDDCUP’99 data set, which is widely used as one of the few publicly available data sets for network-based anomaly detection systems. For more about data: http://www.unb.ca/cic/datasets/nsl.html