This project forked from pujinxiao/qiantuwang
scrapy千图网高清图实战
实现：千图网上高清图片的爬取
程序运行20小时，爬取大约162000张图片，一共49G,存入百度云。链接：http://pan.baidu.com/s/1hsolxNe 密码：y0ut
一、scrapy图片爬虫构建思路
1.分析网站
2.选择爬取方式与策略
3.创建爬虫项目 → 定义items.py
4.编写爬虫文件
5.编写pipelines与setting
6.调试
二、千图网难点（http://www.58pic.com/）
1.要爬取全站的图片
2.要爬取高清的图片------找出高清地址即可
3要有相应的反爬虫机制------如模拟浏览器，不记录cookie等，只要相应注释去掉即可COOKIES_ENABLED = False
三、散点知识
1.from scrapy.http import Request 是回调函数用Request(url=...,callback=...)
2.xpath的//表示提取所有符合的节点
如果本项目对你有用请给我一颗star，万分感谢。
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.