Get_tweets gives the same page about twitter-scraper HOT 6 CLOSED

Finally got it to work! See changes in #150

Brief summary of how I fixed it by modifying tweets.py:

Change r = session.get(url, headers=headers) to r = session.get(url+'&max_position', headers=headers). This allows response json to return min_position parameter which will then be used as the max_position parameter in the next session.get

r_json = r.json()

Change r = session.get(url, params={'max_position': last_tweet}, headers=headers) to r = session.get(url, params={'max_position': r_json['min_position']}, headers=headers).

This gets rid of twitter pages repeating on search query.

from twitter-scraper.

First at all, thanks. Looks like "max_position" parameter not supported for hashtag pages. I couldn't search it yet. Helps are welcome!

from twitter-scraper.

Yes, look like scrolling mechanism not working on hashtag pages.

>>> liste = [] 
... from twitter_scraper import get_tweets 
... for tweet in get_tweets('#mock', 3): 
...     liste.append(tweet['tweetId'])                                                                                                                                                                               

>>> liste.count(liste[0])                                                                                                                                                                                            
3 # first tweet's id repeats 3 times in our scraped tweets


>>> liste = [] 
... from twitter_scraper import get_tweets 
... for tweet in get_tweets('bugraisguzar', 3): 
...     liste.append(tweet['tweetId'])                                                                                                                                                                               

>>> liste.count(liste[0])                                                                                                                                                                                            
1 # first tweet's id is uniq as it should be

We will work on it, help needed

from twitter-scraper.

Got the same error. Does it have to do with the way it's queried and the resulting URL? maybe the pagination works different from User profiles to Hashtags?
Any thoughts? would be awesome to make it work!
Ps. by the way, great repo!

from twitter-scraper.

did anyone have any progress? stuck at the same issue :/

from twitter-scraper.

Would this pagination method in search applicable to digging all tweets from a user?

from twitter-scraper.