Add "is retweet" field to the return about twitter-scraper HOT 4 CLOSED

yeah, that would be a nice feature

from twitter-scraper.

send a PR!

from twitter-scraper.

I was tinkering with this script and here's what i found.

For retweets you can add:

try:
    retweetId = tweet.find('.js-stream-tweet')[0].attrs['data-retweet-id']
except:
    retweetId = None

If it's None - it's not a retweet.

You can also check if a tweet is pinned or not:

try:
    pinned = tweet.find('.user-pinned')[0]
    pinned = True
except:
    pinned = False

Also it's better to have a url for a tweet to get a proper tweet id to figure out unique tweet id:

href = tweet.find('.js-permalink')[0].attrs['href']
hrefTweet = href.split( '/' )[-1]

["tweetId"] is not a unique id of a tweet in case of a retweet or a tweet that's part of a tweet chain (reply tweet?).
If retweetId is present then it's a unique id of a tweet.
If not then href holds a unique id of a tweet.

realTweetId = tweetId
if ( retweetId != None ):
    realTweetId = retweetId
else:
    realTweetId = hrefTweet

Using info from before we can determine if a tweet is a reply tweet or not:

tweetChain = False
if ( hrefTweet != tweetId ):
    tweetChain = True

Then add this information to tweet dictionary:

tweets.append( { 'realTweetId': realTweetId, 'tweetId': tweetId, 'retweetId': retweetId,
                 'href': href, 'pinned': pinned, "tweetChain": tweetChain,

Hope that helps!

from twitter-scraper.

Just added this field. Please check #58 and thanks to @brachna for detailed research

from twitter-scraper.