Real-World Anomaly Detection in Surveillance Videos

Python 100.00%

anomaly-detection-in-surveillance-videos's Introduction

Real-world Anomaly Detection in Surveillance Videos

This repository provides the implementation for the paper 'Real-world Anomaly Detection in Surveillance Videos' by Waqas Sultani, Chen Chen, Mubarak Shah.

Abstract

The project aims to detect anomolous activities in surveillance videos. A pre-trained 3-D convolution network was used to generate input feature vectors and using multiple instance learning an artificial neural network was trained for classification.

Prerequisites

Dataset:

UCF-Crime (http://crcv.ucf.edu/cchen/UCF_Crimes.tar.gz) courtesy of Waqas Sultani. It is the original dataset used for the aforementioned paper.

Tools:

Caffe, Facebook/C3D-1.0 (https://github.com/facebook/C3D), Tensorflow, Python

Implementation Details

PREPROCESSING:

Resize each video frame to 240*320 pixels and fix frame rate at 30fps.

FEATURE EXTRACTION:

C3D features for every 16-frame video clip followed by l2 normalization. To obtain features for a video segment, we take the average of all 16-frame clip features within that segment.

TRAINING:

We input these features (4096D) to a 3-layer FC neural network. The first FC layer has 512 units followed by 32 units and 1 unit FC layers. Using MIL we try to generate higher anomaly score for anomalous videos than normal videos.

Acknowledgments

This project was only possible due the work done by Waqas Sultani, and his help during the course of this project.
We are very gratefull to Dr. Rama Krishna Sai Gorthi, our academic advisor for the project.
The inspiration behing the project was to look into the techniques for anomoly detection in videos and exploit such techniqes to develop a real time automated moderator for surveillance.

Contributers

Citation

Sultani, Waqas, Chen Chen, and Mubarak Shah. "Real-world Anomaly Detection in Surveillance Videos." Center for Research in Computer Vision (CRCV), University of Central Florida (UCF) (2018).

anomaly-detection-in-surveillance-videos's People

Contributors

Stargazers

Watchers

anomaly-detection-in-surveillance-videos's Issues

Can you tell me some details about the environment of implementation

When I test this code, I should some details about the environments.
For example, the operating system, the versions of python.
And what library are you use, Keras or others? And the version of them.

Sequence of execution of the following steps

Hello
I'm new to programming and I am unable to understand the execution steps to be followed to see the output . Please guide the sequence of execution in order to see the output

Originally posted by @Shiny128 in #3 (comment)

About the result

Thanks for your nice work. I want to know the final result comparing with the original paper.
Look forwards for your reply.

Extracted Features

Since the training can be done on the extracted features alone. Are the extracted features available online? Or if you are having them, do you mind uploading them. Also, what is the approximate size of the dataset after feature extraction?

About download datasets

I try to download UCF-Crime datasets,but it is forbidden,"You don't have permission to access /cchen/UCF_Crimes.tar.gz on this server".Can I download the datasets in another way?

How to implement this project?

Hi! I'm new to video processing. Would you please give me more details about how to run this code?

Thank you so much for your help!!!!

Dataset

Who found this data set?can you send it .

How to run the project?

Can you elaborate on the steps to run the project and dependencies needed.

How to run this on a live video to test accuracy ?

Code is missing

The difference of model between model.ckpt.meta and the code

I check the trained model in file model.ckpt.meta using tensorboard. But the model is different from the one in your code and paper.

The model in your code is FC - ReLU - Dropout - FC - Dropout - FC - Sigmoid.
But the trained model is FC - ReLU - Dropout - FC - Sigmoid - Dropout - FC.

how did you convert video into vectors. I guess the code for that is missing.

Hello

Hello!! You did great work on anomaly detection. Could you please provide me with training annotation that you used in this code and kindly also tell me that how you label frames in your code. Waiting for your reply

Not able to start a tensor flow session

Seems like a file is missing.
If you can help me with that, it would be great

Error in anomaly_NN.py

Hai,

Which kind of error is this and how can resolve it ?

Thanks

Feature Extraction

Hope you are doing fine.
I'm stuck at the feature extraction part. The whole 16 frame/32 segments thing is confusing me. could you please elaborate a bit more on that?

More specifically, if we are supposed to take 32 individual segments that are 16 frames each. from which part of the video will we take those segments from?

any help regarding this will be really appreciated.

input_feature_vector_gen.py issue

The script in unable to convert strings to float of my fc6-1 feature files in the following line:

norm = normalize(np.array(map(float,open(os.path.join(dirpath, fl)).readlines()[0].strip().split(','))).reshape(1,-1),norm='l2')

The error is specifically in the map function, it states, cannot convert strings to float.
Any help will be appreciated!

The implementation of loss function

The loss function in your paper is

However, you implement it in the following way.

https://github.com/abhay97ps/Anomaly-Detection-in-Surveillance-Videos/blob/4be07aba4a8dda7cea882f33782595622d248d3e/Anomaly_NN.py#L51

https://github.com/abhay97ps/Anomaly-Detection-in-Surveillance-Videos/blob/4be07aba4a8dda7cea882f33782595622d248d3e/Anomaly_NN.py#L66-L77

The is implemented as a placeholder extra and calculated using python float type. Won't it make it impossible to back propagate gradient to extra? And the optimizer only minimizes the .

Shouldn't the be implemented in tensorflow functions rather than pure python?

What is the difference between this git and this git : https://github.com/WaqasSultani/AnomalyDetectionCVPR2018?

Thanks.

The display of `epoch_loss`

The epoch_loss initialized to 0 and never updated after then, the output of the program is always as follow. The loss is always 0.

Epoch 0 completed out of 10 loss: 0
Epoch 1 completed out of 10 loss: 0
Epoch 2 completed out of 10 loss: 0
Epoch 3 completed out of 10 loss: 0
Epoch 4 completed out of 10 loss: 0
Epoch 5 completed out of 10 loss: 0
Epoch 6 completed out of 10 loss: 0
Epoch 7 completed out of 10 loss: 0
Epoch 8 completed out of 10 loss: 0
Epoch 9 completed out of 10 loss: 0

https://github.com/abhay97ps/Anomaly-Detection-in-Surveillance-Videos/blob/4be07aba4a8dda7cea882f33782595622d248d3e/Anomaly_NN.py#L58-L79

How to implement the project??

Can you provide details on how to implement your project??

vantage-vision-vv / anomaly-detection-in-surveillance-videos Goto Github PK