Git Product home page Git Product logo

lxngoddess5321 / ultimate-facebook-scraper Goto Github PK

View Code? Open in Web Editor NEW

This project forked from harismuneer/ultimate-social-scrapers

0.0 0.0 0.0 26.73 MB

๐Ÿค– A bot which scrapes almost everything about a Facebook user's profile including all public posts/statuses available on the user's timeline, uploaded photos, tagged photos, videos, friends list and their profile photos (including Followers, Following, Work Friends, College Friends etc).

License: MIT License

Python 100.00%

ultimate-facebook-scraper's Introduction

Ultimate Facebook Scraper (UFS)

Tooling that automates your social media interactions to collect posts, photos, videos, friends, followers and much more on Facebook.





Featured by Top Security Blogs and OSINT Researchers


BlackArch Hakin9 Kali Linux Tutorials Security Online OSINT

Kitploit Kali Tools Journalist ToolBox OneHack


2nd Spot in Top Trending Python Repositories on GitHub

UFS trended among top Python repositories on GitHub for 3 consecutive weeks ๐ŸŒŸ

Trending on GitHub


News Updates ๐Ÿ†

Features ๐Ÿš€

A bot which scrapes almost everything about a user's Facebook profile including:

  • uploaded photos
  • tagged photos
  • videos
  • friends list and their profile photos (including Followers, Following, Work Friends, College Friends etc)
  • and all public posts/statuses available on the user's timeline
  • NEW: Now you can scrape Facebook group posts.

Data is scraped in an organized format to be used for educational/research purposes by researchers. This scraper does not use Facebook's Graph API meaning there are no rate limiting issues.

This tool is being used by thousands of developers weekly and we are pretty amazed at this response! Thank you!๐ŸŽ‰

For citing/referencing this tool for your research, check the 'Citation' section below.

Note ๐Ÿค

This tool uses xpaths of 'divs' to extract data. Since Facebook updates its site frequently, the 'divs' get changed. Consequently, we have to update the divs accordingly to correctly scrape data.

The developers of this tool have devoted time and effort in developing, and maintaining this tool for a long time. In order to keep this amazing tool alive, we need support from you geeks.

The code is intuitive and easy to understand, so you can update the relevant xpaths in the code if you find data is not being scraped from profiles. Facebook has most likely updated their site, so please generate a pull request. Much appreciated!

Sample

Screenshot


Usage ๐Ÿ”ง

Installation ๐Ÿ’ป

You will need to:

git clone https://github.com/harismuneer/Ultimate-Facebook-Scraper.git
cd Ultimate-Facebook-Scraper

# Install Python requirements
pip install -e .

The code is multi-platform and is tested on both Windows and Linux. Chrome driver is automatically downloaded using the chromedriver_manager package.

How to Run

  • Fill your Facebook credentials into credentials.yaml
  • Edit the input.txt file and add profile, groups and individual group posts links as you want in the following format with each link on a new line:

Make sure the link only contains the username or id number at the end and not any other stuff. Make sure its in the format mentioned above.

Run the ultimate-facebook-scraper command ! ๐Ÿš€

python scraper/scraper.py

Note: There are two modes to download Friends Profile Pics and the user's Photos: Large Size and Small Size. By default they are set to Small Sized Pics because its really quick while Large Size Mode takes time depending on the number of pictures to download.

You can personalize your scrapping needs using the command line arguments:

python scraper/scraper.py \
    --uploaded_photos True \
    --friends_photos True \
    --friends_small_size True \
    --photos_small_size True \
    --total_scrolls 2500 \
    --scroll_time 8

Note that those are the default values so no need to write them down if you're just testing or are okay with them.


Citation ๐Ÿ“š

If you use this tool for your research, then kindly cite it. Click the above badge for more information regarding the complete citation for this tool and diffferent citation formats like IEEE, APA etc.


Important Message โš ๏ธ

This tool is for research purposes only. Hence, the developers of this tool won't be responsible for any misuse of data collected using this tool. Used by many researchers and open source intelligence (OSINT) analysts.

This tool will not works if your account was set up with 2FA. You must disable it before using.


Authors ๐Ÿ‘‹

You can get in touch with us on our LinkedIn Profiles:

Haris Muneer

LinkedIn Link

You can also follow my GitHub Profile to stay updated about my latest projects: GitHub Follow

Hassaan Elahi

LinkedIn Link

You can also follow my GitHub Profile to stay updated about my latest projects: GitHub Follow

If you liked the repo then please support it by giving it a star โญ!

For Future ๐Ÿ”ฎ

Shoutout to geeks willing to contribute to this project. Please have a look at the UFS kanban board for a list of things to be done.

There are a lot of features that can be added to this tool like adding support for pages, groups, comments etc! Please contribute :)

Contributions Welcome โœจ

forthebadge

If you find any bug in the code or have any improvements in mind then feel free to generate a pull request.

Note: We use Black to lint Python files. Please use it in order to have a valid pull request ๐Ÿ˜‰

Issues ๐Ÿ”จ

GitHub Issues

If you face any issue, you can create a new issue in the Issues Tab and I will be glad to help you out.

License ๐Ÿ“„

MIT

Copyright (c) 2018-present, harismuneer, Hassaan-Elahi

ultimate-facebook-scraper's People

Contributors

harismuneer avatar hassaan-elahi avatar skynewz avatar yeouchiou avatar aayaffe avatar guylewin avatar elyesmanai avatar scottvr avatar dahlitzflorian avatar frostflame avatar imgbotapp avatar szepnapot avatar holygame avatar compscikai avatar wagnernoise avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.