To use the crawler, follow the steps as given below:
-
Make sure your Internet connection is stable.
-
Run the following commands on your terminal to setup your work environment in a directory of your preference.
git clone https://github.com/rifatrakib/eco-friendly-crawler.git
cd eco-friendly-crawler
virtualenv venv
venv\Scripts\activate.bat
pip install -r requirements.txt
-
Create a folder named
data
in your project folder/directory. Or runmkdir data
in your terminal. -
Run
main.py
using your terminal with the commandpython main.py
After the execution has completed, you should find two files with .csv
and .xlsx
extensions in the data
folder which contains the necessary data.