andrewrporter / yahoo-historical Goto Github PK

View Code? Open in Web Editor NEW

99.0 99.0 40.0 59 KB

Downloads historical EOD (end of day) prices from yahoo finance

License: MIT License

Python 100.00%

finance historical-data pandas pandas-dataframe yahoo yahoo-api yahoo-finance

yahoo-historical's People

Contributors

Stargazers

Watchers

yahoo-historical's Issues

Syntax Error

Tried installing with pip and then by running the running the python code directly.
In both cases I get a SyntaxError: on line 13 position 53 (pointing to "interval").
You perhaps have a naming conflict?

is it possible to get exchange rate historical data

Hi Andrew
thanks for sharing this
do you know if it is possible to get historical data for "exchange rate"
I tried "EUR=" code but gave this error
"name='B', domain=None, path=None"

Automated nightly tests to catch API errors

Preferably through circle CI, can send emails upon failure

Download to csv

Hi (yes I am quite new to Python...) - while your code works very well - when I try to download long time series to csv files I always get it truncated btween the first 30 and the last 30 records. Any intentions to implement a 'download' function into your code? /Suggestions? That will help. Thx

Foreign Exchange tickers throw an error when downloading historical data

Hi Andrew,

example:
#l_tickers = ["^DJI","^IXIC","STM.MI"] these works
l_tickers = ["EURGBP=X","EURUSD=X"] # these not

API is not working

from yahoo_historical import Fetcher

data = Fetcher("AAPL", [2007,1,1], [2017,1,1])
print(data.get_historical())

The above code return the following.

Empty DataFrame
Columns: [Forbidden]
Index: []

Trying to download share information result with KeyError

Hi.

About a week ago, I download your tool (pip install yada yada) and it worked fine with the BEZQ.TA share. Today it failed downloading the stock info

The head of the file reads:

from pandas_datareader import data as pdr

import fix_yahoo_finance as yf
import datetime
import os

and the relevant part of the code reads

    # download dataframe
    stock_name = 'BEZQ.TA'
    start = datetime.datetime(2017,1,1)
    end = datetime.datetime(2018,1,1)

I saw in #8 that you expect this to be a wrong ticker, but I the code also failed with the error:

  File "stock_getter.py", line 68, in <module>
    download_and_store(s)
  File "stock_getter.py", line 37, in download_and_store
    data = pdr.get_data_yahoo(stock_name, start, end)
  File "/usr/local/lib/python2.7/dist-packages/fix_yahoo_finance/__init__.py", line 169, in download
    actions=actions, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/fix_yahoo_finance/__init__.py", line 329, in download_chunk
    get_yahoo_crumb()
  File "/usr/local/lib/python2.7/dist-packages/fix_yahoo_finance/__init__.py", line 63, in get_yahoo_crumb
    _YAHOO_COOKIE = res.cookies['B']
  File "/home/yotam/.local/lib/python2.7/site-packages/requests/cookies.py", line 329, in __getitem__
    return self._find_no_duplicates(name)
  File "/home/yotam/.local/lib/python2.7/site-packages/requests/cookies.py", line 400, in _find_no_duplicates
    raise KeyError('name=%r, domain=%r, path=%r' % (name, domain, path))
KeyError: "name='B', domain=None, path=None"

Sometimes I get instead of this an empty stock. How can I solve this?

Setup init.py

Add fetch.Fetch to init.py file so the Fetch can be imported from yahoo_historical module

TypeError: type object argument after * must be an iterable, not float

I tested the code in Python3.7:

from yahoo_historical import Fetcher
import datetime
import time

# create unix timestamp representing January 1st, 2007
timestamp = time.mktime(datetime.datetime(2007, 1, 1).timetuple())

data = Fetcher("AAPL", timestamp)
print(data.get_historical())

And I got:

TypeError Traceback (most recent call last)
/var/folders/5z/nfxgl_ps6qlf5tvrct964q980000gn/T/ipykernel_77052/2760101674.py in
6 timestamp = time.mktime(datetime.datetime(2007, 1, 1).timetuple())
7
----> 8 data = Fetcher("AAPL", timestamp)
9 print(data.get_historical())

/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/yahoo_historical/fetch.py in init(self, ticker, start, end, interval)
20 self.ticker = ticker.upper()
21 self.interval = interval
---> 22 self.start = int(cal.timegm(dt.datetime(*start).timetuple()))
23
24 if end is not None:

TypeError: type object argument after * must be an iterable, not float

Getting null values in dataframe

Doing:

data = Fetcher("ALV.F", [2016,9,27], [2017,9,27])

and getting:

           Date        Open        High         Low       Close   Adj Close  \
0    2016-09-26        null        null        null        null        null

"FutureWarning: .ix is deprecated" is raised from pandas

This is raised since recently when using Fetcher:

/home/.../venv3/lib/python3.7/site-packages/yahoo_historical/fetch.py:66: FutureWarning: 
.ix is deprecated. Please use
.loc for label based indexing or
.iloc for positional indexing

See the documentation here:
http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#ix-indexer-is-deprecated
  return self.getHistorical().ix[:,[0,4]]
/home/.../venv3/lib/python3.7/site-packages/pandas/core/indexing.py:822: FutureWarning: 
.ix is deprecated. Please use
.loc for label based indexing or
.iloc for positional indexing

See the documentation here:
http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#ix-indexer-is-deprecated
  retval = getattr(retval, self.name)._getitem_axis(key, axis=i)

Dates before 1970

I am looking at historic data for S&P 500 and Yahoo has it starting from 1950. time.mktime cannot handle dates before 1970-01-01. This is a rare case, but I think a very important one. The solution is to use calendar.timegm, which can handle everything, in your fetch.py file in init function.

Here is my proposed solution. It works for me using your original date format of (YYYY,MM,DD).

import calendar as cal

class Fetcher:
    def __init__(self, ticker, start, end=None, interval="1d"):
        self.start = int(cal.timegm(dt.datetime(*start).timetuple()))
        if end is not None:
            self.end = int(cal.timegm(dt.datetime(*start).timetuple()))
        else:
            self.end = int(time.time())

AttributeError: 'datetime.datetime' object has no attribute 'timestamp()'

Encountered when importing the module on python 2.7.12

Specify historical data interval

local variable 'crumb' referenced before assignment

I'm using Fetcher to download daily data, for some symbols, I got this error:

local variable 'crumb' referenced before assignment

Getting ParserError when running Fetcher on a list of stocks

When I run the Fetcher with a list of stocks, I keep getting this ParserError but it's not consistent. It just happens randomly. I tried putting an exemption and attempt the same stock again in a loop and it would sometimes work after a few tries, but later it would indefinitely get stuck on one of the stocks.

ParserError                               Traceback (most recent call last)
<ipython-input-15-6c36ca50f58c> in <module>
      2 
      3 print('Current time before download:', datetime.datetime.now())
----> 4 get_data(all_stocks,unqualified_stock_list,stock_list_idx=0)
      5 print('Current time after download:', datetime.datetime.now())

<ipython-input-13-f19ec7bb5b8b> in get_data(stock_list, unqualified_stock_list, save_limit, stock_list_idx)
      9         stock = stock_list[stock_list_idx]
     10         try:
---> 11             stock_data = Fetcher(stock, [stock_start_y,stock_start_m,stock_start_d],[stock_end_y,stock_end_m,stock_end_d]).getHistorical().values
     12         except KeyError:
     13             print('KeyError... Restarting Loop...')

~\Anaconda3\envs\tfgpu\lib\site-packages\yahoo_historical\fetch.py in getHistorical(self)
     49             DeprecationWarning,
     50         )
---> 51         return self._get("history")
     52 
     53     def getDividends(self):

~\Anaconda3\envs\tfgpu\lib\site-packages\yahoo_historical\fetch.py in _get(self, events)
     34         data = requests.get(url, headers={"User-agent": ""})
     35         content = StringIO(data.content.decode("utf-8"))
---> 36         return pd.read_csv(content, sep=",")
     37 
     38     def getData(self, events):

~\Anaconda3\envs\tfgpu\lib\site-packages\pandas\io\parsers.py in parser_f(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, squeeze, prefix, mangle_dupe_cols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, skipfooter, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, dayfirst, cache_dates, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, doublequote, escapechar, comment, encoding, dialect, error_bad_lines, warn_bad_lines, delim_whitespace, low_memory, memory_map, float_precision)
    674         )
    675 
--> 676         return _read(filepath_or_buffer, kwds)
    677 
    678     parser_f.__name__ = name

~\Anaconda3\envs\tfgpu\lib\site-packages\pandas\io\parsers.py in _read(filepath_or_buffer, kwds)
    452 
    453     try:
--> 454         data = parser.read(nrows)
    455     finally:
    456         parser.close()

~\Anaconda3\envs\tfgpu\lib\site-packages\pandas\io\parsers.py in read(self, nrows)
   1131     def read(self, nrows=None):
   1132         nrows = _validate_integer("nrows", nrows)
-> 1133         ret = self._engine.read(nrows)
   1134 
   1135         # May alter columns / col_dict

~\Anaconda3\envs\tfgpu\lib\site-packages\pandas\io\parsers.py in read(self, nrows)
   2035     def read(self, nrows=None):
   2036         try:
-> 2037             data = self._reader.read(nrows)
   2038         except StopIteration:
   2039             if self._first_chunk:

pandas\_libs\parsers.pyx in pandas._libs.parsers.TextReader.read()

pandas\_libs\parsers.pyx in pandas._libs.parsers.TextReader._read_low_memory()

pandas\_libs\parsers.pyx in pandas._libs.parsers.TextReader._read_rows()

pandas\_libs\parsers.pyx in pandas._libs.parsers.TextReader._tokenize_rows()

pandas\_libs\parsers.pyx in pandas._libs.parsers.raise_parser_error()

ParserError: Error tokenizing data. C error: Expected 1 fields in line 4, saw 2

andrewrporter / yahoo-historical Goto Github PK

yahoo-historical's People

Contributors

Stargazers

Watchers

Forkers

yahoo-historical's Issues

Recommend Projects

Recommend Topics

Recommend Org