Git Product home page Git Product logo

dou_ban_spider's Introduction

#Dou Ban Top 100 Spider

  • douban_spider.py you can use it to get top 100 movie name which is listed by douban website

##1. Install

git clone [email protected]:Andrew-liu/Dou_Ban_Spider.git

##2. Use It simply

$ douban_spider.py
$ threading_douban.py  #多线程爬取豆瓣Top250

##3. Example Output

        ###############################
            一个简单的豆瓣电影前100爬虫
            Author: Andrew_liu
            Version: 0.0.1
            Date: 2014-12-04
        ###############################

豆瓣电影爬虫准备就绪, 准备爬取数据...
Top1 肖申克的救赎
Top2 这个杀手不太冷
Top3 阿甘正传
Top4 霸王别姬
Top5 美丽人生
Top6 海上钢琴师
Top7 辛德勒的名单
Top8 千与千寻
Top9 机器人总动员
...
Top97 驯龙高手
Top98 真爱至上
Top99 致命ID
Top100 超脱
豆瓣爬虫爬取结束...

##4. Something

To Do List :

  • [ ]添加Master/Slave模式, 使爬取数据有序
  • [ ]学习Scapy

Done :

  • [x]实现了简单的多线程爬虫
  • [x]学习了Python中的编码问题

更多详细内容请查看Python-豆瓣电影抓站小结

dou_ban_spider's People

Contributors

andrew-liu avatar

Watchers

James Cloos avatar avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.