openkratio / proyecto-colibri Goto Github PK
View Code? Open in Web Editor NEWLas votaciones del Congreso de los Diputados de España (via API)
License: GNU General Public License v3.0
proyecto-colibri's People
Contributors
Stargazers
Watchers
Forkers
jneight andresgutgon didacrios pablocastellano escalant3 penyaskito kikecastillo bienvenidosaez emijrp joseenrique maxiwebdevproyecto-colibri's Issues
Cannot scrape votes
Scrapy shows some errors of this kind when running scrapy crawl votes:
2015-08-08 20:17:55+0200 [votes] ERROR: Spider must return Request, BaseItem or None, got 'Voting' in <GET http://www.congreso.es/votaciones/OpenData?sesion=274&votacion=2&legislatura=10>
2015-08-08 20:17:55+0200 [votes] DEBUG: Crawled (200) <GET http://www.congreso.es/votaciones/OpenData?sesion=274&votacion=1&legislatura=10> (referer: http://www.congreso.es/portal/page/portal/Congreso/Congreso/Actualidad/Votaciones?_piref73_10908194_73_9536063_9536063.next_page=%2Fwc%2FaccesoHistoricoVotaciones&fechaSeleccionada=2015%2F06%2F23)
2015-08-08 20:17:56+0200 [votes] ERROR: Spider must return Request, BaseItem or None, got 'Voting' in <GET http://www.congreso.es/votaciones/OpenData?sesion=274&votacion=1&legislatura=10>
2015-08-08 20:17:56+0200 [votes] DEBUG: Crawled (200) <GET http://www.congreso.es/votaciones/OpenData?sesion=276&votacion=10&legislatura=10> (referer: http://www.congreso.es/portal/page/portal/Congreso/Congreso/Actualidad/Votaciones?_piref73_10908194_73_9536063_9536063.next_page=%2Fwc%2FaccesoHistoricoVotaciones&fechaSeleccionada=2015%2F06%2F25)
Swagger don't detect custom fields.
Swagger documentation has problems on detect custom fields like session in vote resource.
Problems executing first crawler: scrapy crawl members
I hope someone can help me. I reinstall all project to prepare an environment for developing a new aggregation functionality and I found this issue.
Traceback (most recent call last):
File "/home/edipotrebol/env/colibri/bin/scrapy", line 4, in <module>
execute()
File "/home/edipotrebol/env/colibri/local/lib/python2.7/site-packages/scrapy/cmdline.py", line 109, in execute
settings = get_project_settings()
File "/home/edipotrebol/env/colibri/local/lib/python2.7/site-packages/scrapy/utils/project.py", line 60, in get_project_settings
settings.setmodule(settings_module_path, priority='project')
File "/home/edipotrebol/env/colibri/local/lib/python2.7/site-packages/scrapy/settings/__init__.py", line 109, in setmodule
module = import_module(module)
File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/home/edipotrebol/dev/proyecto-colibri/scrap/settings.py", line 12, in <module>
from django.core.management import setup_environ
ImportError: cannot import name setup_environ
Activate send mails on exceptions.
Added utf-8 like default format in responses.
The json response doesn't have default encoding.
maybe with charset='utf-8' could be resolve it.
Cannot scrape initiatives
Scrapy shows some errors of this kind when running scrapy crawl initiatives:
File "/usr/local/lib/python2.7/dist-packages/amqp/transport.py", line 294, in create_transport
return TCPTransport(host, connect_timeout)
File "/usr/local/lib/python2.7/dist-packages/amqp/transport.py", line 95, in __init__
raise socket.error(last_err)
socket.error: [Errno 111] Connection refused
2015-08-08 20:20:33+0200 [initiatives] DEBUG: Crawled (200) <GET http://www.congreso.es/portal/page/portal/Congreso/Congreso/Iniciativas/Indice%20de%20Iniciativas?BASE=IW10&CMD=VERLST&DOCORDER=FIFO&DOCS=801-825&FMT=INITXLUS.fmt&OPDEF=Y&QUERY=%28I%29.ACIN1.+%26+%28219%29.SINI.&_piref73_1335503_73_1335500_1335500.next_page=%2Fwc%2FservidorCGI> (referer: http://www.congreso.es/portal/page/portal/Congreso/Congreso/Iniciativas/Indice%20de%20Iniciativas?BASE=IW10&CMD=VERLST&DOCORDER=FIFO&DOCS=751-775&FMT=INITXLUS.fmt&OPDEF=Y&QUERY=%28I%29.ACIN1.+%26+%28219%29.SINI.&_piref73_1335503_73_1335500_1335500.next_page=%2Fwc%2FservidorCGI)
Sort by (optional)
Hi!
It would be awesome if some API calls could be sorted by field (e.g. date).
Example:
# send with params:
# sort=date
http://proyectocolibri.es/api/v1/session/
filter by session in api voting resource
The filter session don't working.
It need to add a relationship Session-Voting to the resource Voting and create the resource Session.
Session field on voting resource don't work
If the request include session parameters the response get zero elements.
Would it be possible to add a license to the proyect to know how we can use and/or contribute to the code?
Generate doc
Install sphinx, genrate documentation and add this to readthedocs.org
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.