Git Product home page Git Product logo

anomaly_detection_in_stockpricedata's Introduction

Detecting anomaly in stock price data using autoencoder architecture (unsupervised learning)

The problem is to detect anomalous behavior in stock price data collected over the years, using an unsupervised learning method, that is using the autoencoder architecture

Data Collection and Visualization

  1. I downloaded the stock price data of GE from yahoo finance website
  2. The data is read as a pandas dataframe, it has almost 15000 observations and 7 features
  3. Used seaborn to visualize the data, that is, time vs the closing price
  4. Divide the data into train and test sets

Data pre-processing

  1. Here the data 'Last' column is separated out as a new data frame and is converted to a numpy array
  2. The data is then scaled (I have used StandardScaler, that standardizes features by removing the mean and scaling to unit variance)
  3. I used a sequence size of 60 here and then converted the observations to sequences with features and labels for both train and test sets

Model building

  1. I used autoencoder model architecture
  2. Model is compiled using the default values and mean absolute error loss is calculated
  3. The model is trained with the data and the label with a batch_size of 32 and 50 epochs

Plotting

  1. The training and validation losses are plotted using matplotlib
  2. We know that anomaly is detected where the reconstruction error is large and we can define a value beyond which we call it an anomaly
  3. Looking at the MAE in training prediction
  4. Gathering all details in a dataframe and using seaborn to do a lineplot of the same
  5. Also plotted the anomalies as colored dots on the data

anomaly_detection_in_stockpricedata's People

Contributors

poulomi-p avatar

Stargazers

 avatar

Watchers

 avatar

anomaly_detection_in_stockpricedata's Issues

HI!

I have run the code in the Anomaly_detection_in_stockpricedata.py, but something wrong happened.I don't know how to fix it due to i'm a beginner,could you please tell me how to solve this problem?

ValueError: Expected 2D array, got 1D array instead:
array=[1.74866959 1.73545175 1.74322702 ... 0.46299787 0.43388762 0.43146177].
Reshape your data either using array.reshape(-1, 1) if your data has a single feature or array.reshape(1, -1) if it contains a single sample.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.