Node.js module for getting proxies from publicly available proxy lists.

• blackhatworld - Specific forum threads are scraped:
  • 100-scrapebox-proxies
• freeproxylist
• freeproxylists
• gatherproxy
• hidemyass
• incloak
• proxies24 - http, https, socks
• proxydb
• proxylisten
• sockslist

Proxy lists that require an API key:

• bitproxies
• kingproxies

Missing a proxy list that you think should be here? Open an issue to suggest it be added as a source. Or you can add a new source and create a pull request to have it added to this module.

If you wish to use this module as a CLI tool, install it globally via npm:

npm install -g proxy-lists

Otherwise, you can add it to your existing node application like this:

npm install proxy-lists --save

This will install proxy-lists and add it to your application's package.json file.

• Command-line interface
• API
  • getProxies
    • Proxy Object
  • getProxiesFromSource
  • addSource
  • listSources

This section assumes that you have proxy-lists installed globally and that it is available on your current user's PATH.

To view the help screen for the CLI tool:

proxy-lists --help

To view the help screen for the getProxies command:

proxy-lists getProxies --help

To output the proxies in .txt format:

proxy-lists getProxies --output-format="txt"

To output proxies to STDOUT:

proxy-lists getProxies --stdout

To output proxies to a different file than proxies.txt:

proxy-lists getProxies --output-file="somefile.txt"

To get proxies from specific sources:

proxy-lists getProxies --sources-white-list="hidemyass,sockslist"

To get proxies from specific countries:

proxy-lists getProxies --countries="us,ca"

To get proxies with specific protocols:

proxy-lists getProxies --protocols="http,https"

To get only anonymous and elite proxies:

proxy-lists getProxies --anonymity-levels="anonymous,elite"

The output of the getProxies command is written to a new file (proxies.txt) in your current working directory.

These are the public methods of the ProxyLists module that allow you to get proxies, add custom proxy sources, and list existing sources.

getProxies([options])

Gets proxies from all available proxy lists.

Usage:

var ProxyLists = require('proxy-lists');

var options = {
	countries: ['us', 'ca']
};

// `gettingProxies` is an event emitter object.
var gettingProxies = ProxyLists.getProxies(options);

gettingProxies.on('data', function(proxies) {
	// Received some proxies.
});

gettingProxies.on('error', function(error) {
	// Some error has occurred.
	console.error(error);
});

gettingProxies.once('end', function() {
	// Done getting proxies.
});

Sample proxies:

[
	{
		ipAddress: '123.123.2.42',
		port: 8080,
		country: 'us',
		source: 'superproxies'
	},
	{
		ipAddress: '234.221.233.142',
		port: 3128,
		country: 'cz',
		protocols: ['https'],
		source: 'someproxysource'
	},
	{
		ipAddress: '234.221.233.142',
		port: 3128,
		country: 'cz',
		anonymityLevel: 'elite',
		protocols: ['https'],
		source: 'anotherproxysource'
	}
]

All available options:

var options = {
	/*
		Get proxies for the specified countries.

		To get all proxies, regardless of country, set this option to NULL.

		See:
		https://en.wikipedia.org/wiki/ISO_3166-1

		Only USA and Canada:
		['us', 'ca']
	*/
	countries: null,

	/*
		Get proxies that use the specified protocols.

		To get all proxies, regardless of protocol, set this option to NULL.
	*/
	protocols: ['http', 'https'],

	/*
		Anonymity level.

		To get all proxies, regardless of anonymity level, set this option to NULL.
	*/
	anonymityLevels: ['anonymous', 'elite'],

	/*
		Include proxy sources by name.

		Only 'freeproxylists':
		['freeproxylists']
	*/
	sourcesWhiteList: null,

	/*
		Exclude proxy sources by name.

		All proxy sources except 'freeproxylists':
		['freeproxylists']
	*/
	sourcesBlackList: null,

	/*
		Set to TRUE to have all asynchronous operations run in series.
	*/
	series: false,

	/*
		Load GeoIp data for these types of IP addresses. Default is only ipv4.

		To include both ipv4 and ipv6:
		['ipv4', 'ipv6']
	*/
	ipTypes: ['ipv4']
};

The proxy object has the following properties:

• ipAddress - string The IP address of the proxy.
• port - integer The port number of the proxy.
• country - string Alpha-2 country code of the country in which the proxy is geo-located.
• source - string The name of the proxy list from which the proxy was gathered.
• protocols - optional array An array of protocols that the proxy supports. May contain one or more of the following:
  • http - The proxy uses HTTP.
  • https - The proxy uses HTTPS.
  • socks5 - The proxy server uses the socks5 protocol.
  • socks4 - The proxy server uses the socks4 protocol.
• tunnel - optional boolean Whether or not the proxy supports tunneling to HTTPS target URLs.
• anonymityLevel - optional string The anonymity level of the proxy. Can be any one of the following:
  • transparent - The proxy does not hide the requester's IP address.
  • anonymous - The proxy hides the requester's IP address, but adds headers to the forwarded request that make it clear that the request was made using a proxy.
  • elite - The proxy hides the requester's IP address and does not add any proxy-related headers to the request.

The attributes marked as optional above might not be given for all proxies. Some proxy lists are missing this information.

It's important to note that this module does NOT verify all of the information provided by the proxy lists from which the proxies are gathered. If you need to check that proxies work, verify their anonymity level, whether or not they support tunneling; use proxy-verifier.

getProxiesFromSource(name, [options])

Gets proxies from a specific proxy list.

Usage:

var ProxyLists = require('proxy-lists');

var options = {
	anonymityLevels: ['elite']
};

// `gettingProxies` is an event emitter object.
var gettingProxies = ProxyLists.getProxiesFromSource('freeproxylists', options);

gettingProxies.on('data', function(proxies) {
	// Received some proxies.
});

gettingProxies.on('error', function(error) {
	// Some error has occurred.
	console.error(error);
});

gettingProxies.once('end', function() {
	// Done getting proxies.
});

All available options:

var options = {
	/*
		Get proxies for the specified countries.

		To get all proxies, regardless of country, set this option to NULL.

		See:
		https://en.wikipedia.org/wiki/ISO_3166-1

		Only USA and Canada:
		['us', 'ca']
	*/
	countries: null,

	/*
		Get proxies that use the specified protocols.

		To get all proxies, regardless of protocol, set this option to NULL.
	*/
	protocols: ['http', 'https'],

	/*
		Anonymity level.

		To get all proxies, regardless of anonymity level, set this option to NULL.
	*/
	anonymityLevels: ['anonymous', 'elite'],

	/*
		Set to TRUE to have all asynchronous operations run in series.
	*/
	series: false,

	/*
		Load GeoIp data for these types of IP addresses. Default is only ipv4.

		To include both ipv4 and ipv6:
		['ipv4', 'ipv6']
	*/
	ipTypes: ['ipv4']
};

addSource(name, source)

Add a custom proxy source to the list of available proxies. The new proxy source will be used in addition to the existing sources, when calling getProxies().

Usage:

// Core nodejs module.
// See https://nodejs.org/api/events.html
var EventEmitter = require('events').EventEmitter || require('events');

var ProxyLists = require('proxy-lists');

ProxyLists.addSource('my-custom-source', {
	homeUrl: 'https://somewhere.com',
	getProxies: function(options) {

		var emitter = new EventEmitter();

		// When an error occurs, use the 'error' event.
		// The 'error' event can be emitted more than once.
		emitter.emit('error', new Error('Something bad happened!'));

		// When proxies are ready, use the 'data' event.
		// The 'data' event can be emitted more than once.
		emitter.emit('data', proxies);

		// When done getting proxies, emit the 'end' event.
		// The 'end' event should be emitted once.
		emitter.emit('end');

		// Must return an event emitter.
		return emitter;
	}
});

Your proxy source is required to return the following for each proxy: ipAddress, port. See Proxy Object above for more information.

Please consider sharing your custom proxy sources by creating a pull request to have them added to this module so that others can use them too.

listSources([options])

Get list of all available proxy sources.

Usage:

var ProxyLists = require('proxy-lists');

var sources = ProxyLists.listSources();

Sample sources:

[
	{
		name: 'freeproxylists',
		homeUrl: 'http://www.freeproxylists.com'
	},
	{
		name: 'hidemyass',
		homeUrl: 'http://proxylist.hidemyass.com/'
	}
]

All available options:

var options = {
	/*
		Include proxy sources by name.

		Only 'freeproxylists':
		['freeproxylists']
	*/
	sourcesWhiteList: null,

	/*
		Exclude proxy sources by name.

		All proxy sources except 'freeproxylists':
		['freeproxylists']
	*/
	sourcesBlackList: null
};

There are a number of ways you can contribute:

• Improve or correct the documentation - All the documentation is in this readme.md file. If you see a mistake, or think something should be clarified or expanded upon, please submit a pull request
• Report a bug - Please review existing issues before submitting a new one; to avoid duplicates. If you can't find an issue that relates to the bug you've found, please create a new one.
• Request a feature - Again, please review the existing issues before posting a feature request. If you can't find an existing one that covers your feature idea, please create a new one.
• Fix a bug - Have a look at the existing issues for the project. If there's a bug in there that you'd like to tackle, please feel free to do so. I would ask that when fixing a bug, that you first create a failing test that proves the bug. Then to fix the bug, make the test pass. This should hopefully ensure that the bug never creeps into the project again. After you've done all that, you can submit a pull request with your changes.

To run all tests:

grunt test

To run only unit tests:

grunt test:unit

To run only code-style checks:

grunt test:code-style

• v1.11.1:
  • Fixed #46
  • Removed source (maxiproxies) because it no longer exists.
• v1.11.0:
  • Added new source (blackhatworld).
  • Fix for #43
• v1.10.0:
  • Added new source (maxiproxies).
  • Removed source (proxyspy) because it is no longer working.
  • Fix for #42
  • If using your own custom sources:
    • Proxy sources are now only required to provide ipAddress and port; all other fields are optional and should be provided only if known.
• v1.9.0:
  • Fixes for proxydb.
  • Removed source (proxyocean) because it no longer exists.
  • Added support for ipv6 addresses.
• v1.8.0:
  • Added --stdout option to CLI utility.
  • Fixed issues: #35, #37
• v1.7.1:
  • Fixed issues: #28, #29
• v1.7.0:
  • Now performing geo-ip look-up for all proxies
  • More proxy sources: gatherproxy.com, incloak.com, proxydb.net
• v1.6.0:
  • Added command-line interface.
  • Fixes for source (kingproxies).
• v1.5.1:
  • Fixes for source (hidemyass).
  • Removed geo-ip lookups from source (proxies24).
• v1.5.0:
  • Added series option to ProxyLists.getProxies() and ProxyLists.getProxiesFromSource().
• v1.4.0:
  • isValidProxy no longer checks the proxy.country attribute.
  • ProxyLists.getProxies(), ProxyLists.getProxiesFromSource(), and getProxies() for all sources now using event emitter interface.
• v1.3.0:
  • Removed attribute proxy.protocol in favor of proxy.protocols (an array of all supported protocols).
  • Renamed attribute proxy.ip_address to proxy.ipAddress for consistency.
  • Added attribute proxy.source.