my8100 Goto Github PK

awesome-crawler

A collection of awesome web crawler,spider in different languages

awesome-flask

A curated list of awesome Flask resources and plugins

awesome-python

A curated list of awesome Python frameworks, libraries, software and resources

awesome-python-applications

💿 Free software that works great, and also happens to be open-source Python.

awesome-python-cn

Python资源大全中文版，包括：Web框架、网络爬虫、模板引擎、数据库、数据可视化、图片处理等，由伯乐在线持续更新。

awesome-scrapy

A curated list of awesome packages, articles, and other cool resources from the Scrapy community.

awesome-web-scraping

List of libraries, tools and APIs for web scraping and data processing.

files

Docs and files for ScrapydWeb, Scrapyd, Scrapy, and other projects

logparser

A tool for parsing Scrapy log files periodically and incrementally, extending the HTTP JSON API of Scrapyd.

notes

Keep on reading

public-test

public-test

python-algorithms

All Algorithms implemented in Python

queuelib

Collection of persistent (disk-based) queues

scrapy-redis

Redis-based components for Scrapy.

scrapyd

[PR #326] Native support for basic auth :lock: `pip install -U git+https://github.com/scrapy/scrapyd.git`, then add `username = yourusername` and `password = yourpassword` in the scrapyd.conf file. DOCS :point_right:

scrapyd-circleci

scrapyd-cluster-on-heroku

Set up free and scalable Scrapyd cluster for distributed web-crawling with just a few clicks. DEMO :point_right:

scrapyd-cluster-on-heroku-scrapyd-app

How to set up Scrapyd cluster on Heroku

scrapyd-cluster-on-heroku-scrapyd-app-basic-auth

scrapyd-cluster-on-heroku-scrapyd-app-basic-auth

scrapyd-cluster-on-heroku-scrapyd-app-git

How to set up Scrapyd cluster on Heroku

scrapyd-cluster-on-heroku-scrapydweb-app

How to set up Scrapyd cluster on Heroku

scrapyd-cluster-on-heroku-scrapydweb-app-git

How to set up Scrapyd cluster on Heroku

scrapydweb

Web app for Scrapyd cluster management, Scrapy log analysis & visualization, Auto packaging, Timer tasks, Monitor & Alert, and Mobile UI. DEMO :point_right:

temp

temp