Git Product home page Git Product logo

twint's Introduction

TWINT - Twitter Intelligence Tool

Version Build Status Python 3.5|3.6 GitHub license

No authentication. No API. No limits.

Formerly known as Tweep, Twint is an advanced Twitter scraping tool written in Python that allows for scraping Tweets from Twitter profiles without using Twitter's API.

Twint utilizes Twitter's search operators to let you scrape Tweets from specific users, scrape Tweets relating to certain topics, hashtags & trends, or sort out sensitive information from Tweets like e-mail and phone numbers. I find this very useful, and you can get really creative with it too.

Twint also makes special queries to Twitter allowing you to also scrape a Twitter user's followers, Tweets a user has liked, and who they follow without any authentication, API, Selenium, or browser emulation.

tl;dr Benefits

Some of the benefits of using Twint vs Twitter API:

  • Can fetch almost all Tweets (Twitter API limits to last 3200 Tweets only)
  • Fast initial setup
  • Can be used anonymously and without Twitter sign up
  • No rate limitations

Requirements

  • Python 3.5/3.6
  • pip3 install -r requirements.txt

Basic Examples and Combos.

A few simple examples to help you understand the basics:

  • python3 twint.py -u username - Scrape all the Tweets from user's timeline.
  • python3 twint.py -u username -s pineapple - Scrape all Tweets from the user's timeline containing pineapple.
  • python3 twint.py -s pineapple - Collect every Tweet containing pineapple from everyone's Tweets.
  • python3 twint.py -u username --year 2014 - Collect Tweets that were tweeted before 2014.
  • python3 twint.py -u username --since 2015-12-20 - Collect Tweets that were tweeted since 2015-12-20.
  • python3 twint.py -u username -o file.txt - Scrape Tweets and save to file.txt.
  • python3 twint.py -u username -o file.csv --csv - Scrape Tweets and save as a csv file.
  • python3 twint.py -u username --fruit - Show Tweets with low-hanging fruit.
  • python3 twint.py -s "Donald Trump" --verified --users - List verified users that Tweet about Donald Trump.
  • python3 twint.py -g="48.880048,2.385939,1km" -o file.csv --csv - Scrape Tweets from a radius of 1km around a place in Paris and export them to a csv file.
  • python3 twint.py -u username -es localhost:9200 - Output Tweets to Elasticsearch
  • python3 twint.py -u username -o file.json --json - Scrape Tweets and save as a json file.
  • python3 twint.py -u username --database tweets.db - Save Tweets to a SQLite database.
  • python3 twint.py -u username --followers - Scrape a Twitter user's followers.
  • python3 twint.py -u username --following - Scrape who a Twitter user follows.
  • python3 twint.py -u username --favorites - Collect all the Tweets a user has favorited.

More detail about the commands and options are located in the wiki

Example String

955511208597184512 2018-01-22 18:43:19 GMT <now> pineapples are the best fruit

Storing Options

  • Write to file.
  • CSV
  • JSON
  • SQLite
  • Elasticsearch

Elasticsearch Setup

Details on setting up Elasticsearch with Twint is located in the wiki.

Thanks

Thanks to @hpiedcoq & @pielco11 for contributing several features!

Contact

Shout me out on Twitter: @now

If you have problems or have suggestions don't hesitate to open an issue or ask about it directly.

twint's People

Contributors

haccer avatar pielco11 avatar hpiedcoq avatar 4sakura avatar prabod avatar gdelavald avatar sshinol avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.