touv / node-fetch-with-proxy Goto Github PK

View Code? Open in Web Editor NEW

10.0 3.0 5.0 448 KB

Wrap isomorphic-fetch to enable a proxy use.

JavaScript 100.00%

node-fetch-with-proxy's Introduction

Use fetch behind a proxy

A wrapper of fetch that can be use behind a proxy.

It detect standard environment variables (HTTP_PROXY, https_proxy, etc.) to choose and use the proxy.

Unlike many similar packages, this one does not use tunnel in HTTP, like request, like browsers.

Example

import fetch from 'fetch-with-proxy';

const url = 'https://nodejs.org/';
fetch(url)
      .then((response) => response.text());
      .then(console.log)
      .catch(console.error)

Installation

With npm:

$ npm install fetch-with-proxy

Tests

Use mocha to run the tests.

$ mocha test

Environment variables

HTTP_PROXY
http_proxy
HTTPS_PROXY
https_proxy
ALL_PROXY
NO_PROXY

API Documentation

see https://github.com/matthew-andrews/isomorphic-fetch

Related projects

License

MIT/X11

node-fetch-with-proxy's People

Contributors

Stargazers

Watchers

Forkers

tcione jsplink 0leg0 frankszll

node-fetch-with-proxy's Issues

Make fetch package a dependency injection

Hi, instead of hardcoding this package to use isomorphic-fetch, is it possible to pass that dependency in? I'd like to use this package with cross-fetch, which is recommended over isomorphic-fetch for React apps.

Report correct hostname in proxy requests

When testing fetch-with-proxy in my development environment using a local proxy, it seems that all proxied requests have their Host header set to the proxy itself, instead of the final destination server. This causes errors in my environment, and seems to be contrary to the behavior exhibited by other proxy-capable tools like curl:

> curl -x http://localhost:8080 http://www.google.com/
GET http://www.google.com/ HTTP/1.1
Host: www.google.com
User-Agent: curl/7.54.0
Accept: */*
Proxy-Connection: Keep-Alive

I assume that target.hostname needs to be set to the proxy in order for the fetch request to work properly, but doing so seems to provide the wrong headers value for Host.

I've resolved my issue by recording and restoring the target.hostname as follows.

    // https://github.com/request/request/blob/b12a6245d9acdb1e13c6486d427801e123fdafae/lib/tunnel.js#L124-L130
    if (method.startsWith('httpOver')) {
        const targetHostname = target.hostname;
        target.path = target.protocol
            .concat('//')
            .concat(target.host)
            .concat(target.path);
        target.port = proxy.port;
        target.host = proxy.host;
        target.hostname = proxy.hostname;
        target.auth = proxy.auth;
        return crossFetch(target, {
            ...options,
            headers: { ...options.headers, 'Host': targetHostname },
        });

No proxy detected/used + Authentication

const fetch = require("fetch-with-proxy");

=> TypeError: fetch is not a function

using

const fetch = require("fetch-with-proxy").default;

=> Works but doesnt use proxy even when set.

my code:

const fetch = require("fetch-with-proxy").default;
process.env.HTTPS_PROXY = "myproxyprovider:port";
fetch("http://www.mikes-marketing-tools.com/whatismyip.php", {
	"authenticate":{
		"username": "myusername",
		"password": "mypassword"
	},
  "headers": {
    "accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
    "accept-language": "en,en-US;q=0.9,cs;q=0.8,sk-SK;q=0.7,sk;q=0.6",
    "cache-control": "no-cache",
    "pragma": "no-cache",
    "sec-fetch-dest": "document",
    "sec-fetch-mode": "navigate",
    "sec-fetch-site": "same-origin",
    "sec-fetch-user": "?1",
    "sec-gpc": "1",
    "upgrade-insecure-requests": "1"
  },
  "referrer": "http://www.mikes-marketing-tools.com/whatismyip.php",
  "referrerPolicy": "strict-origin-when-cross-origin",
  "body": null,
  "method": "GET",
  "mode": "cors",
  "credentials": "include"
}).then(function (response) {
	return response.text();
}).then(function (html) {
	parser = new DOMParser();
	doc = parser.parseFromString(html, 'text/html');
	fs.appendFile('./content.html', doc.rawHTML, function(err){
		if (err) return console.log(err);
		console.log("LOADED");
	})

}).catch(function (err) {
	console.warn('Something went wrong.', err);
});

I am not sure if the authentication is handled correctly and generally the documentation and readme doesnt tell us much.

Add isomorphic-fetch as a dependency

Module not found: Can't resolve 'isomorphic-fetch' in .....node_modules\fetch-with-proxy\lib'

Please add it as a dependency.