Git Product home page Git Product logo

tiring-spiders's Introduction

Tired-Spiders 反反爬解决方法Demo

写在前面

写爬虫的时候遇到对方站点的反爬是家常便饭,所以我打算开始将平时遇到的觉得有代表性的放上来,并简要的说一下解决思路。

我将反爬总的分为了三类:

  • 频度反爬之代理IP

    FreeProxyPool:这个项目提供了几个站点的免费代理IP抓取程序,以及代理IP池调用接口

  • Cookies反爬

    Fuck-login:有些站点需要携带Cookies才能进行访问,有的甚至要求需要登陆之后才能正常访问。这个项目提供了部分站点的登陆脚本程序示例,会长期更新和维护

  • 数据的加密、混淆、下毒等

    此类反爬也是我写当前这个项目的目标,这个项目的demo中我会记录一些站点的反爬解决方案,包括js逆向安卓逆向绕过反爬数据接口来源


请求时参数加密


响应数据下毒或字体加密

  • 实习僧

响应内容加密

tiring-spiders's People

Contributors

imleowoo avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.