k9team3 / facebook_scraper Goto Github PK

#Facebook Scraper ###A web scraper for Facebook utilizing Node.js

####Introduction The purpose of this project is to grab information concerning name, age, gender, hometown, and highschool from profile that are members in a facebook group. The information will be used to crossreference University of Minnesota databases. As it stands today, June 15, 2016, Facebook_Scraper is functional, but the software liscence for the finished product was purchased. The client asked me not to upload the finished code on Github.

Facebook_Scraper is implemented through a Node.js based webserver and a Google Chrome extension. It is built this way because extracting information from a Facebook web page is trivial, but outputting the information to a file is not. The file output aspect is what generated a need for a webserver. In a nutshell, Facebook_Scraper will fetch the profile URLs from a Facebook group member page, iterate through the collected URLs, and post them to a local webserver. The webserver then writes the profile information to a file in XML format.

####Installation

1. You'll need to be running linux with both npm and nodejs installed. Both are available using apt-get. You will also need Google Chrome.
2. Clone this repo to wherever. That path will be referred to as "scraper_path" in the following steps.
3. In Chrome load the unpacked extension located in scraper_path/extension.
4. In the scraper_path directory, run the command npm install. This will install the express and body-parser node modules
5. Next, run nodejs server.js
   You should see a message indicating the server is listening on localhost:8080. You may change the listening port as needed.
6. Navigate to a Facebook group page and click the facebook_Scraper extension button. In the drop down menu click "Get Profile Urls". Once the Urls have been collected click the extension again and click "Scrape all Profiles". In scraper_path, there will be a new file called face_Book_Profiles.xml that will hold from that page. You may do this on as many pages as needed, and all profile information will be appended to face_Book_Profiles.xml.