Git Product home page Git Product logo

weibospider's Introduction

Build Status Python PyPI

Weibo Spider

本程序可以连续爬取一个多个新浪微博用户(如胡歌迪丽热巴郭碧婷)的数据,并将结果信息写入文件数据库。写入信息几乎包括用户微博的所有数据,包括用户信息微博信息两大类。因为内容太多,这里不再赘述,详细内容见获取到的字段。如果只需要用户信息,可以通过设置实现只爬取微博用户信息的功能。本程序需设置cookie来获取微博访问权限,后面会讲解如何获取cookie。如果不想设置cookie,可以使用免cookie版,二者功能类似。

注意事项

  1. user_id不能为爬虫微博的user_id。因为要爬微博信息,必须先登录到某个微博账号,此账号我们姑且称为爬虫微博。爬虫微博访问自己的页面和访问其他用户的页面,得到的网页格式不同,所以无法爬取自己的微博信息;如果想要爬取爬虫微博内容,可以参考获取自身微博信息
  2. cookie有期限限制,大约三个月。若提示cookie错误或已过期,需要重新更新cookie。

weibospider's People

Contributors

1190200610 avatar bluerthanever avatar caixiangyue avatar chenleilei avatar codermino avatar dataabc avatar dependabot[bot] avatar duangan1 avatar eggachecat avatar huaiwens avatar jerrylaikr avatar kimiding avatar linbuxiao avatar minami9 avatar mksp2015 avatar moqimoqidea avatar mtuwei avatar schaepher avatar songzy12 avatar windlively avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.