llsourcell / reinforcement_learning_for_stock_prediction Goto Github PK

D:\Anaconda\envs\tf-gpu\python.exe D:/GitHub/Reinforcement_Learning_for_Stock_Prediction/stock_predict.py
Traceback (most recent call last):
File "D:/GitHub/Reinforcement_Learning_for_Stock_Prediction/stock_predict.py", line 120, in get_prices
stock_prices = np.load(cache_filename)
File "D:\Anaconda\envs\tf-gpu\lib\site-packages\numpy\lib\npyio.py", line 372, in load
fid = open(file, "rb")
FileNotFoundError: [Errno 2] No such file or directory: 'resource/stock_prices.npy'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "D:\Anaconda\envs\tf-gpu\lib\site-packages\yahoo_finance_init_.py", line 120, in _request
_, results = response['query']['results'].popitem()
AttributeError: 'NoneType' object has no attribute 'popitem'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "D:\Anaconda\envs\tf-gpu\lib\site-packages\yahoo_finance_init_.py", line 123, in _request
raise YQLQueryError(response['error']['description'])
KeyError: 'error'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "D:/GitHub/Reinforcement_Learning_for_Stock_Prediction/stock_predict.py", line 130, in
prices = get_prices('MSFT', '2000-07-01', '2017-07-01', 'resource/stock_prices.npy')
File "D:/GitHub/Reinforcement_Learning_for_Stock_Prediction/stock_predict.py", line 122, in get_prices
share = Share(share_symbol)
File "D:\Anaconda\envs\tf-gpu\lib\site-packages\yahoo_finance_init_.py", line 178, in init
self.refresh()
File "D:\Anaconda\envs\tf-gpu\lib\site-packages\yahoo_finance_init_.py", line 142, in refresh
self.data_set = self.fetch()
File "D:\Anaconda\envs\tf-gpu\lib\site-packages\yahoo_finance_init.py", line 181, in _fetch
data = super(Share, self).fetch()
File "D:\Anaconda\envs\tf-gpu\lib\site-packages\yahoo_finance_init.py", line 134, in _fetch
data = self.request(query)
File "D:\Anaconda\envs\tf-gpu\lib\site-packages\yahoo_finance_init.py", line 125, in _request
raise YQLResponseMalformedError()
yahoo_finance.YQLResponseMalformedError: Response malformed.

Process finished with exit code 1

flake8 testing of https://github.com/llSourcell/Reinforcement_Learning_for_Stock_Prediction on Python 3.6.3

$ flake8 . --count --select=E901,E999,F821,F822,F823 --show-source --statistics

./evaluate.py:9:50: E999 SyntaxError: invalid syntax
	print "Usage: python evaluate.py [stock] [model]"
                                                 ^
./functions.py:27:11: F821 undefined name 'xrange'
	for i in xrange(n - 1):
          ^
./train.py:6:59: E999 SyntaxError: invalid syntax
	print "Usage: python train.py [stock] [window] [episodes]"
                                                          ^
./agent/agent.py:47:12: F821 undefined name 'xrange'
		for i in xrange(l - batch_size + 1, l):
           ^
2     E999 SyntaxError: invalid syntax
2     F821 undefined name 'xrange'
4

I am getting 0 buy and 0 sell action with the code availabe
If i update my first action to buy the i get only single sell action( in total 1 buy and 1 sell)

Can u guide me what to modify in my code

Hi,
Thank you for this repo.
I was wondering what's the meaning of:

def getState(data, t, n): d = t - n + 1 # print(data[d:t + 1], "data") block = data[d:t + 1] if d >= 0 else -d * [data[0]] + data[0:t + 1] # pad with t0 res = [] for i in range(n - 1): res.append(sigmoid(block[i + 1] - block[i])) return np.array([res])

Does the state have to be transformed like this?

I am looking at turning this into a multivariable model (with inputs outside price), so getting clarification on this function would be great.

Thanks

When memory deque max length is not equal or much greater than the batch size the training seems to be significantly slowed down. I added a constants file and used the same value for batch_size = BATCH_SIZE in train.py and self.memory = deque(maxlen=BATCH_SIZE) in agent.py.

After training a model, when I try to run the evaluation.py, I get the following error:
"TypeError: Unexpected keyword argument passed to optimizer: amsgrad"

When looking at Stackoverflow, it seems to be an error tied to saving and loading a Keras model if it was saved with on version of Keras and loaded with another. However, I did the 2 steps right after each other, and there was no new update during that time.

I haven't changed anything to the code, so I was wondering if any of you had any idea how I might fix it?

Hi Siraj, are you familiar with forking on github? Is there any reason why you just simply didn't fork this https://github.com/edwardhdlu/q-trader
BTW this link to the original repo is broken in your readme (missing 'r' character at the end of the url)

It is a bit strange but in the testing process, the model always predicts the same action.
I watched more carefully what was happening and I saw that the output of the model was always the same one (independently of inputs I am giving to the model).

For example,
[0.6701656 0.67643714 0.71922743]
[0.6701656 0.67643714 0.71922743]
[0.6701656 0.67643714 0.71922743]
[0.6701656 0.67643714 0.71922743]
[0.6701656 0.67643714 0.71922743]
[0.6701656 0.67643714 0.71922743]

Has anyone experienced the same problem?
Many thanks.
Luis.

Hi,

is python 3 required?
What about other dependencies?
E.g. 'ImportError: No module named agent.agent'

Thanks for helping!

Hey, could someone help me with my problem? What am i overseeing? :)

stock_name, window_size, episode_count = sys.argv[1], int(sys.argv[2]), int(sys.argv[3])

IndexError: list index out of range

Kernel died, restarting

1. In the README, you say:

• mkdir model
  I think this should say:
• mkdir models
  Minor point but might as well correct it!

2. To run the train you may need to put 'python train.py ^GSPC 10 1000' (add the .py), just like for the evaluate.py. Again a minor point but might as well change it.
3. For Python 3 compatibility, simply change all the 'print ...' statements to 'print(...)' and the 'xrange' to 'range' in all the .py files and everything should work. Maybe worth noting in the README.
4. Can you be clearer if there are any changes from the original code by edwardhdlu - if not, why have you copied rather than forked?
   Thanks

please let me know, how you build the diagrams , what are the parameters you set in x axis and y axis .

I ran the code and it is running fine but when i try to evaluate the results by running evaluate.py there is no model created inside my directory. Please help me with it.

After each episode, I expect agent learnt something from previous, but I plot the reward of 1000 Episode, the reward doesn't go up. Is it mean the agent can't learn from the dataset ?
I used EURUSD 2010 - 2015 total 1567 records with 3 window size

Hello,

I am trying your model, however it only takes as input 1 feature (close). I would like to include additional features into the states. But it is not as straight forward as simply using an array apparently. Is there a quick solution you might know to incorporate additional features into your model?

I'm trying to implement this in tensorflow js but i'm running into some issue with these lines in agent.py:

target_f = self.model.predict(state)
target_f[0][action] = target
self.model.fit(state, target_f, epochs=1, verbose=0)

What does target_f means? final target?

What is target[action] ? Why are we setting target_f to the predict and then assingning it something else?

thanks,
Tiago

Using TensorFlow backend.
Usage: python train.py [stock] [window] [episodes]
Backend TkAgg is interactive backend. Turning interactive mode on.

I'm trying to use this model to make it run on other dataset as well.
I have created another dataframe with the exact same numbers of columns (same name as well).
But when I try to run it I always get this error:
File "/Users/Desktop/Reinforcement_Learning_for_Stock_Prediction-master/functions.py", line 20, in sigmoid
return 1 / (1 + math.exp(-x))
OverflowError: math range error

I tried everything, round all the columns to no float. Just don't understand what can be the problem

Hi,
I want to ask is it okay to add LSTM layer when building a _model?
Would it make the results better?
Or it not needed at all, just need the fully connected network.

Thanks.

the reason why this code is able to predict peaks and troughs very well is because it uses the behavior at time T to buy and sell at time T. And the input data of T is obtained from the difference between the closing price of T on that day and the closing price of T-1. Therefore, future information is included. If in doubt, do two things: 1. Move the input window forward by one day, 2. Write a backtest program to test it.

I train to 200 episode. when I run evaluate.py with code python evaluate.py ^GSPC_2011 model_ep1000 .it always show output like this. How to use evaluate.py

 Reloaded modules: agent.agent, functions
 --------------------------------
 ^GSPC_2011 Total Profit: $0.00
 --------------------------------

Goood

The training works fine, but evaluate always gives profit=0, where is the model.fit?

Seems like the model is only compiled, but not fitted...

I like this code