Git Product home page Git Product logo

philipxjm / deep-convolution-stock-technical-analysis Goto Github PK

View Code? Open in Web Editor NEW
586.0 60.0 174.0 34.94 MB

Uses Deep Convolutional Neural Networks (CNNs) to model the stock market using technical analysis. Predicts the future trend of stock selections.

License: Apache License 2.0

Python 100.00%
stock-price-prediction convolutional-neural-networks neural-network technical-analysis stock-market

deep-convolution-stock-technical-analysis's Introduction

Neural Stock Market Prediction

Uses Deep Convolutional Neural Networks (CNNs) to model the stock market using technical analysis. Predicts the future trend of stock selections.

How does it work?

Convolutional neural networks are designed to recognize complex patterns and features in images. It works by dividing an image up into multiple overlapping perceptive fields and running a myriad of trainable filters through them, capturing basic features and patterns. This process is repeated several times, and as the filtered image is ran through more filters, deeper and more meaningful features are extracted and quantified. For example, to recognize an image of a car we might have several filters that are sensitive to wheels, or windows, or exhaust pipes, or licence plates... and all of the results of these filters are gathered and quantified into a final classifier.

CNN

OK, that's great, but how does this tie in to stock analysis? Here we introduce the study of technical analysis. I'll let Investopedia's words describe it: "Technical analysis is a trading tool employed to evaluate securities and attempt to forecast their future movement by analyzing statistics gathered from trading activity, such as price movement and volume. Unlike fundamental analysts who attempt to evaluate a security's intrinsic value, technical analysts focus on charts of price movement and various analytical tools to evaluate a security's strength or weakness and forecast future price changes." In other words, technical analysis focuses on the movement patterns and trading behaviors of stock selections to pinpoint a stock's future trend. Wait a minute, if technical analysis works by analysing the movement patterns of stocks, we can use CNN to model this analytical technique!

For example, we would have some filters that are sensitive to shortterm uptrends, and they will be combined by fully connected layers to be sensitive to longterm uptrends. The same goes for some complex patterns such as shortterm floats, or an overall downward trend capture.

As previously mentioned, CNN works by stacking several filters on top of each other to form complex feature-sensitive filters; if we were to treat stock data as images, we can apply CNN to it and extract useful and deep information. How do we go about this?

Instead of convolving a 2D image, we convolved a 1D image, since stock data is linear and is represented as an 1D tensor.

def conv1d(input, output_dim,
           conv_w=9, conv_s=2,
           padding="SAME", name="conv1d",
           stddev=0.02, bias=False):
  with tf.variable_scope(name):
    w = tf.get_variable('w', [conv_w, input.get_shape().as_list()[-1], output_dim],
      initializer=tf.truncated_normal_initializer(stddev=stddev))
    c = tf.nn.conv1d(input, w, conv_s, padding=padding)

    if bias:
      b = tf.get_variable('b', [output_dim], initializer=tf.constant_initializer(0.0))
      return c + b

    return c

Also, the input images is in the shape [batch_size, 128, 5], the moving-window (the length of data we will be looking at in one batch) the five channels being [Open, High, Low, Close, Volume], all information I deemed important for technical analysis.

After several convolutional layers and batchnorms later, we arrive at a tensor sized [batch_size, 2, 1024], which we then run through several softmax layers and finally a sigmoid activation to result in a tensor sized [batch_size, 2], with two values, one representing the bullish confidence, and the other one the bearish confidence.

Materials for Consideration

Name Link
Historical Data https://quantquote.com/historical-stock-data
Description of Technical Analysis http://www.investopedia.com/terms/t/technicalanalysis.asp
Berkeley paper on ANN-based analysis http://www.cs.berkeley.edu/~akar/IITK_website/EE671/report_stock.pdf

Data Format

19991118,0,42.2076,46.382,37.4581,39.1928,43981812.87

Date Time Open High Low Close Volume
19991118 0 42.2076 46.382 37.4581 39.1928 43981812.87

Usage

The trained model is proprietary, but you are absolutely welcome to train your own using my code.

You must have python 3.5+ and tensorflow installed, tensorflow-gpu highly recommended as the training requires a lot of computational power.

pip install tensorflow-gpu

git clone https://github.com/philipxjm/Convolutional-Neural-Stock-Market-Technical-Analyser.git

cd Convolutional-Neural-Stock-Market-Technical-Analyser

python stock_model.py

Of course, you have to tinker with the hyper parameters, archeteture of the encoder, and the dataset setup if you want to achieve good results. Good luck and make some money.

deep-convolution-stock-technical-analysis's People

Contributors

philipxjm avatar pkage avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

deep-convolution-stock-technical-analysis's Issues

About dropout

Hi
@ line 103, may I know if it's typo? since the keep_prob will be 0 when dropout is 1, it's strange.

      if dropout > 0.0:
        output = tf.nn.dropout(output, keep_prob=1 - dropout)

about trained model

I have already trained and saved checkpoints, but can you show me how to use the checkpoint to use the trained model? Sorry I am new to python and CNN

typo README

In the README "python stock-model.py" should be "python stock_model.py"

Training time

Great article and code.

Many thanks.

Just wondering though - do you have any thoughts as to how best to reduce the training time. I do not have a GPU btw.

There are logic error in codes.

I am very glad to find somebody focusing on Stock Market with DL technique. Before finding this github, I also pay some time on this and beat with some problems, this code gives me a good reference.
Unfortunately, I find a fatal bug that indicates this github's CNN codes cannot run in reality. That is

# shuffling the data
perm = np.arange(labels_set.shape[0])
np.random.shuffle(perm)
stocks_set = stocks_set[perm]
labels_set = labels_set[perm]

The reason for that, in reality, stock stream comes with time order, this shuffle operation will break this rule.
If you do not believe, you can comment these codes and run it, you would find your test accuracy dramatically decrease from 70% down to 50%. That means it can only behave well on train set but always sucks on test set.

TL;DR, stock price/index prediction by CNN is nearly a hard (even impossible) task, because it is essentially event-driven not data-driven (price-driven). The codes here belongs to data-driven (price-driven) methods.
By the way, some researchers now pay much more attention on event-driven method, i.e., they grabbed event data of daily news and then use Deep Learning model to analysis it. But this method is relatively complicated.

tensorflow command removal due to upgrade

I'm self-studying to do some technical analysis. One outdate tf command appears and I can't fix it. The command is tensorflow.contrib.learn.python.learn.base.datasets. I think tensorflow upgrade their version and delete contrib. I can't find any alternative online. How can I fix this issue?

New complemetary tool

My name is Luis, I'm a big-data machine-learning developer, I'm a fan of your work, and I usually check your updates.

I was afraid that my savings would be eaten by inflation. I have created a powerful tool that based on past technical patterns (volatility, moving averages, statistics, trends, candlesticks, support and resistance, stock index indicators).
All the ones you know (RSI, MACD, STOCH, Bolinger Bands, SMA, DEMARK, Japanese candlesticks, ichimoku, fibonacci, williansR, balance of power, murrey math, etc) and more than 200 others.

The tool creates prediction models of correct trading points (buy signal and sell signal, every stock is good traded in time and direction).
For this I have used big data tools like pandas python, stock market libraries like: tablib, TAcharts ,pandas_ta... For data collection and calculation.
And powerful machine-learning libraries such as: Sklearn.RandomForest , Sklearn.GradientBoosting, XGBoost, Google TensorFlow and Google TensorFlow LSTM.

With the models trained with the selection of the best technical indicators, the tool is able to predict trading points (where to buy, where to sell) and send real-time alerts to Telegram or Mail. The points are calculated based on the learning of the correct trading points of the last 2 years (including the change to bear market after the rate hike).

I think it could be useful to you, to improve, I would like to share it with you, and if you are interested in improving and collaborating I am also willing, and if not file it in the box.

Tutorial For Loading Model/Getting Predictions?

I know this isn't really an issue, but I thought this might be the best way to ask the question. Can anyone point me to a good tutorial for writing something to use the keras checkpoint file this outputs in order get a prediction?

Thanks!

Indicators As Features

What if, instead of simply using the price history, we included indicators such as RSI, MACD, Polarized Fractal Efficiency, etc? Should that not improve the accuracy? I'd like to investigate ways of incorporating such extra features. Any thoughts on accomplishing this?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.