Git Product home page Git Product logo

proxy-scraper-checker's Introduction

proxy-scraper-checker

CI

Screenshot

HTTP, SOCKS4, SOCKS5 proxies scraper and checker.

  • Asynchronous.
  • Uses regex to search for proxies (ip:port format) on a web page, allowing proxies to be extracted even from json without making changes to the code.
  • It is possible to specify the URL to which to send a request to check the proxy.
  • Can sort proxies by speed.
  • Supports determining the geolocation of the proxy exit node.
  • Can determine if the proxy is anonymous.

You can get proxies obtained using this script in monosans/proxy-list.

Installation and usage

Desktop

  • Download and unpack the archive with the program.
  • Edit config.ini to your preference.
  • Install Python (minimum required version is 3.7).
  • Run the script that installs dependencies and starts proxy-scraper-checker:
    • On Windows run start.cmd
    • On Unix-like operating systems run start.sh

Termux

To use proxy-scraper-checker in Termux, knowledge of the Unix command-line interface is required.

  • Download Termux from F-Droid. Don't download it from Google Play.
  • Run the following command (it will automatically update Termux packages, install Python, and download and install proxy-scraper-checker):
    bash <(curl -fsSL 'https://raw.githubusercontent.com/monosans/proxy-scraper-checker/main/install-termux.sh')
  • Edit ~/proxy-scraper-checker/config.ini to your preference using a text editor (vim/nano).
  • To run proxy-scraper-checker use the following command:
    cd ~/proxy-scraper-checker && sh start-termux.sh

Folders description

When the script finishes running, the following folders will be created (this behavior can be changed in the config):

  • proxies - proxies with any anonymity level.
  • proxies_anonymous - anonymous proxies.
  • proxies_geolocation - same as proxies, but includes exit-node's geolocation.
  • proxies_geolocation_anonymous - same as proxies_anonymous, but includes exit-node's geolocation.

Geolocation format is ip:port|Country|Region|City.

License

MIT

proxy-scraper-checker's People

Contributors

monosans avatar pre-commit-ci[bot] avatar dependabot[bot] avatar quartzwarrior avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.