chrisspen / django-sec Goto Github PK
View Code? Open in Web Editor NEWThis project forked from lukerosiak/pysec
Parse XBRL filings from the SEC's EDGAR in Python
This project forked from lukerosiak/pysec
Parse XBRL filings from the SEC's EDGAR in Python
I have been trying to extract attributes from 10-K filings but I dont see many balance sheet fields (like assets) extracted even though I can see those fields in /django_sec/fixtures/normalized_fields.csv. How can I extract the balance sheet fields?
I noticed that sec_import_index hang when downloading years 2016,2015,2014,2013 qtr2 etc. Strange thing is it will work for qrt 1 of 2013...
`C:\Users\Math\math_sec>python manage.py sec_import_index --start-year=2013 --end-year=2016
Opening index file /tmp/django_sec\company_2013_1.zip.
Found 475 prior ciks.
Processing record 303583 of 303583 (100.00%).
73537 new companies found.
301393 new indexes found.
Downloading ftp://ftp.sec.gov/edgar/full-index/2013/QTR2/company.zip.
Opening index file /tmp/django_sec\company_2013_2.zip.
Traceback (most recent call last):
File "manage.py", line 22, in
execute_from_command_line(sys.argv)
File "C:\Python27\lib\site-packages\django\core\management_init_.py", line 367, in execute_from_command_line
utility.execute()
File "C:\Python27\lib\site-packages\django\core\management_init_.py", line 359, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "C:\Python27\lib\site-packages\django\core\management\base.py", line 294, in run_from_argv
self.execute(*args, **cmd_options)
File "C:\Python27\lib\site-packages\django\core\management\base.py", line 345, in execute
output = self.handle(*args, **options)
File "C:\Python27\lib\site-packages\django_sec\management\commands\sec_import_index.py", line 111, in handle
max_lines=max_lines)
File "C:\Python27\lib\site-packages\django_sec\management\commands\sec_import_index.py", line 163, in get_filing_list
zipf = ZipFile(fn)
File "C:\Python27\lib\zipfile.py", line 770, in init
self._RealGetContents()
File "C:\Python27\lib\zipfile.py", line 811, in _RealGetContents
raise BadZipfile, "File is not a zip file"
zipfile.BadZipfile: File is not a zip file`
Hello there,
I think there is another bug potentially, I only have marked one company for download but when I run:
python manage.py sec_import_attrs --start-year=2016 --end-year=2017 --form=10-Q,10-K
Running process: {'start_year': '2016', 'show_pending': False, 'force': False, 'verbose': False, 'settings': None, 'cik': None, 'pythonpath': None, 'verbosity': 1, 'traceback': None, 'multi': 0, 'forms': '10-Q,10-K', 'no_color': False, 'end_year': '2017', 'quarter': None}
total_count: 7114
None [ ] 0 of ? ?% eta=?: 7114 total rows.
None [ ] 1 of 7114 0% eta=2016-05-05 05:14:06.056550+00:00: Processing index edgar/data/1006045/0001564590-16-015763.txt.
It's parsing every company in the database.
Is the use of WGET nessesary (maybe for the parallel processing)? I have rewritten to use 'requests' module instead. I am on windows and don´t want to install WGET.
SEC has closed the FTP download option with end of 2016. The index download uses this option. This means that 'sec_import_index.py' line 129 should be:
url = 'https://www.sec.gov/Archives/edgar/full-index/%d/QTR%d/company.zip' % (year, quarter)
Hello,
I can run the server and see the webpage, but I can not use the sec_import_index.
Here is the error I get :
C:\Users\Math\math_sec>python manage.py sec_import_index --start-year=2015 --end-year=2016 Traceback (most recent call last): File "manage.py", line 22, in <module> execute_from_command_line(sys.argv) File "C:\Python27\lib\site-packages\django\core\management\__init__.py", line 367, in execute_from_command_line utility.execute() File "C:\Python27\lib\site-packages\django\core\management\__init__.py", line 359, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "C:\Python27\lib\site-packages\django\core\management\__init__.py", line 208, in fetch_command klass = load_command_class(app_name, subcommand) File "C:\Python27\lib\site-packages\django\core\management\__init__.py", line 40, in load_command_class module = import_module('%s.management.commands.%s' % (app_name, name)) File "C:\Python27\lib\importlib\__init__.py", line 37, in import_module __import__(name) File "C:\Python27\lib\site-packages\django_sec\management\commands\sec_import_index.py", line 12, in <module> from django.core.management.base import NoArgsCommand ImportError: cannot import name NoArgsCommand
Any way to fix this
Hello,
I think this is another foreign key issue introduced in Django 1.9.
If you go on : /admin/django_sec/company/
Put a company name say Verdier I get:
FieldError at /admin/django_sec/company/
Cannot resolve keyword '_ticker' into field. Choices are: attributes, cik, filings, load, max_date, min_date, name
Request Method: GET
Request URL: http://localhost:8000/admin/django_sec/company/?q=Verdier
Django Version: 1.9
Exception Type: FieldError
Exception Value:
Cannot resolve keyword '_ticker' into field. Choices are: attributes, cik, filings, load, max_date, min_date, name
Exception Location: /usr/local/lib/python2.7/dist-packages/django/db/models/sql/query.py in names_to_path, line 1330
Python Executable: /usr/bin/python
Python Version: 2.7.9
Python Path:
['/home/pi/Documents/DjangoSec',
'/usr/lib/python2.7',
'/usr/lib/python2.7/plat-arm-linux-gnueabihf',
'/usr/lib/python2.7/lib-tk',
'/usr/lib/python2.7/lib-old',
'/usr/lib/python2.7/lib-dynload',
'/usr/local/lib/python2.7/dist-packages',
'/usr/lib/python2.7/dist-packages',
'/usr/lib/python2.7/dist-packages/gtk-2.0',
'/usr/lib/pymodules/python2.7']
Server time: Mon, 2 May 2016 22:05:30 +0000
I had to alter: https://github.com/chrisspen/django-sec/blob/master/django_sec/migrations/0001_initial.py#L63
By removing the seemingly erroneous 'b' preceding the 'ticker' string.
When using the Company search, Django throws an exception, as _ticker is activated for search but isn't contained in the company object. I fixed the issue in the file i was working on by commenting out line 211 in the admin.py file.
any way you can put up a better explanation of how to use this package?
Hello Chriss,
I am getting an invalid syntax now when I install from pip, it seems to be just some extra tabs, it doesn't affect the build but best to remove them .
Found existing installation: django-sec 0.3.3
Uninstalling django-sec:
Successfully uninstalled django-sec
Running setup.py install for django-sec
File "/usr/local/lib/python2.7/dist-packages/django_sec/management/commands/sec_mark_units.py", line 75^ SyntaxError: invalid syntax warning: no files found matching '*' under directory 'chroniker/tests/fixtures'
Successfully installed django-sec
Cleaning up...
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.