This is a crawl designed for kickstarter.
itemadapter==0.1.0
Scrapy==2.3.0
pymongo==3.11.0
requests==2.24.0
selenium==3.141.0
pandas==1.1.0
kick==1.1.0
Python 3.8 MongoDB MongoDB Compass(optional)
First, please instal Python, MongoDB and MongoDB Compass(optional)
Create an vitural env by virtualenv
Active the virtualenv and install all the requiremtns.
Clone and run scrapy crawl kick
to start the crawler.