Http error when running tweets-client.py about politwoops-tweet-collector HOT 7 CLOSED

It's nice to see our code find uses beyond our own needs. Are you applying the politwoops code to another set of legislators, another country, a totally different set of users? Always curious to see where our code ends up.

The username should not be @ prefixed, so the second stack trace is simply a case of the twitter API refusing authentication with an @ prefixed username -- you can ignore that one.

The first stack-trace shows twitter returning a 406 status code. The error codes documentation says:

Returned by the Search API when an invalid format is specified in the request.

That's obviously vague, but I think it suggests a compatibility issue between the tweetstream library version you are using and the current twitter API. Could you copy/paste the output of pip freeze here? The paths in the stack trace suggest you are either not using a virtualenv or using a virtualenv that includes the site packages. Is that right?

from politwoops-tweet-collector.

Yes I am applying politwoops code to Greek politicians. Here is the output of pip freeze

Logbook==0.3
MySQL-python==1.2.4b4
PyYAML==3.10
anyjson==0.3.3
argparse==1.2.1
beanstalkc==0.2.0
boto==2.5.2
distribute==0.6.28
ipdb==0.7
pytz==2012f
requests==0.14.0
tweetstream==1.1.1
virtualenv==1.8.2
wsgiref==0.1.2

The tweetstream version is the one mentioned in your requirements.txt. I think that the version of virtualenv includes the site packages. How it can be solved?

from politwoops-tweet-collector.

Your package info looks proper. Could you run tweets-client.py with the --log-level=debug and copy/paste the output here?

from politwoops-tweet-collector.

[2012-10-13 08:44] DEBUG: tweets-client.py: Starting tweets-client.py
[2012-10-13 08:44] INFO: tweets-client.py: Initiating beanstalk connection. Queueing tweets to taKrymena...
[2012-10-13 08:44] DEBUG: tweets-client.py: Setting up stream ...
[2012-10-13 08:44] DEBUG: tweets-client.py: Loading track plugin: tweetsclient.mysql_track - MySQLTrackPlugin
[2012-10-13 08:44] DEBUG: tweets-client.py: Initializing a users stream of tweets.
[2012-10-13 08:44] DEBUG: tweetsclient.mysql_track: Making DB connection
[2012-10-13 08:44] DEBUG: tweetsclient.mysql_track: Executing query: SELECT twitter_id FROM politicians WHERE status=1
[2012-10-13 08:44] DEBUG: tweets-client.py: ['OlevN']
[2012-10-13 08:44] DEBUG: tweets-client.py: Done setting up stream ...
Traceback (most recent call last):
File "./bin/tweets-client.py", line 149, in
sys.exit(main(args))
File "./bin/tweets-client.py", line 132, in main
return app.run()
File "./bin/tweets-client.py", line 101, in run
for tweet in stream:
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tweetstream/streamclasses.py", line 165, in iter
self._init_conn()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tweetstream/streamclasses.py", line 99, in _init_conn
self._conn = opener.open(req)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 406, in open
response = meth(req, response)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 519, in http_response
'http', request, response, code, msg, hdrs)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 438, in error
result = self._call_chain(_args)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 378, in _call_chain
result = func(_args)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 890, in http_error_401
url, req, headers)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 865, in http_error_auth_reqed
response = self.retry_http_basic_auth(host, req, realm)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 878, in retry_http_basic_auth
return self.parent.open(req, timeout=req.timeout)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 406, in open
response = meth(req, response)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 519, in http_response
'http', request, response, code, msg, hdrs)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 444, in error
return self._call_chain(_args)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 378, in _call_chain
result = func(_args)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 527, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 406: Not Acceptable

from politwoops-tweet-collector.

Hello dvogel

I finally managed to find a solution. It was my own fault. Don't waste any time on that. Thank you.

from politwoops-tweet-collector.

@AxGa is there a chance that you could share your problem and solution? I hit exactly the same issue…

from politwoops-tweet-collector.

ah, right, actually importing the initial CSV file with Twitter handles helps a lot…

from politwoops-tweet-collector.