Git Product home page Git Product logo

node-fetch-har's Introduction

node-fetch-har

npm Travis Coveralls

A Fetch API wrapper that records HAR logs for server requests made with node-fetch. You can then expose this data to get visibility into what’s happening on the server.

See also the companion project for Next.js integration, next-fetch-har.

Demo

Support

Did this project bring you joy? Want to support updates? Check out my GitHub Sponsors page.

Alternatively…

Buy Me A Coffee

Status

STABLE

Due to the wide variety seen in HTTP requests, please test thoroughly with your application and file an issue if you find any problems.

Warning

⚠️ HAR files can contain sensitive information like cookies or passwords. Since this library is for capturing what happens on the server, this is especially important because it is information that users can’t normally acces in their own browser. Be careful about sharing this data. If you provide a way to expose it, ensure it is only enabled for superusers or in secure environments.

Usage

The withHar function takes a base Fetch implementation such as node-fetch and returns a new one that records HAR entries:

import { withHar } from "node-fetch-har";
import nodeFetch from "node-fetch";

const fetch = withHar(nodeFetch);

Individual HAR entries can then accessed on the response object:

fetch("https://httpstat.us/200").then(response => {
  console.log(response.harEntry);
  return response;
});

Or by configuring withHar with an onHarEntry callback:

const fetch = withHar(nodeFetch, {
  onHarEntry: entry => console.log(entry)
});

You can also customize onHarEntry for individual requests:

const fetch = withHar(nodeFetch);

fetch("https://httpstat.us/200", {
  onHarEntry: entry => console.log(entry)
});

To disable HAR tracking for individual requests, set the har option to false:

fetch("https://httpstat.us/200", { har: false }).then(response => {
  console.log(response.harEntry); // Should be undefined.
  return response;
});

The above options will give you individual HAR entries. It’s likely that you’ll want to collect multiple requests into a single HAR log. For example, all API calls made while rendering a single page. Use the createHarLog function to generate a complete HAR object that can hold multiple entries.

You can pass the resulting object via the har option and entries will automatically be added to it:

import { withHar, createHarLog } from "node-fetch-har";
import nodeFetch from "node-fetch";

async function run() {
  const har = createHarLog();
  const fetch = withHar(nodeFetch, { har });

  await Promise.all([
    fetch("https://httpstat.us/200"),
    fetch("https://httpstat.us/200"),
    fetch("https://httpstat.us/200")
  ]);

  console.log(har);
}

You can also call createHarLog with an array of entries, if you’ve already collected them in a different way:

const har = createHarLog(entries);

…with Isomorphic Fetch

When using “universal” libraries like cross-fetch, isomorphic-fetch, or isomorphic-unfetch, make sure you only import this library and wrap the Fetch instance on the server. Not only does this library require built-in Node modules, but it’s unnecessary in the browser anyway, since you can already spy on requests (and export HAR logs) via the Network tab.

The following example assumes your bundler (e.g. webpack) is configured to strip out conditional branches based on process.browser.

import baseFetch from "isomorphic-unfetch";

let fetch = baseFetch;

if (!process.browser) {
  const { withHar } = require("node-fetch-har");
  fetch = withHar(baseFetch);
}

Redirects

Due to redirects, it is possible for a single fetch call to result in multiple HTTP requests. As you might expect, multiple HAR entries will be recorded as well.

With the Fetch API’s redirect option in follow mode (the default), calls will transparently follow redirects; that is, you get the response for the final, redirected request. Likewise, the harEntry property of the response will correspond with that final request.

To get the HAR entries for the redirects, use the har or onHarEntry options (described above). The redirects will be appended to the log and reported with onHarEntry in addition to the final entry, in the order that they were made.

Request Body

If there is no Content-Type header specified in the request, then postData will not be populated since we would not be able to populate the required mimeType field.

Additionally, params will only be populated if the Content-Type is exactly application/x-www-form-urlencoded. If it is anything else (including multipart/form-data) then text will be populated instead.

There may be limited support for exotic request body encodings.

Custom Agent

This library works by using the custom agent option supported by node-fetch. However, it should still work if you pass your own custom agent as well. The provided agent instance will have its addRequest method instrumented with the necessary HAR tracking behavior. This behavior will be skipped if the request does not originate from a Fetch instance returned by withHar.

Page Info

The second argument to createHarLog allows you to add some initial page info:

const har = createHarLog([], { title: "My Page" });

If you have additional pages within a single log, you’ll have to add them yourself:

har.log.pages.push({ title: "2nd Page" });

If not provided, a default page will be created with an ID of page_1. By default, all HAR entries will reference this page. To customize the page that entries reference, use the harPageRef option to withHar:

const fetch = withHar(nodeFetch, { har, harPageRef: "page_2" });

Or use the harPageRef option to fetch for individual requests:

await fetch(url, { harPageRef: "page_2" });

Examples

See the demo for an example of exposing an SSR HAR log from Next.js.

Run the demo like so:

$ cd demo
$ yarn
$ yarn start

TODO

  • More tests for different response types, protocols (HTTP/2), encodings, etc.

How does it work?

node-fetch supports a custom agent option. This can be used to capture very detailed information about the request all the way down to the socket level if desired. This library only uses it in a very simplistic way, to capture a few key timestamps and metadata like the HTTP version.

node-fetch-har's People

Contributors

exogen avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

node-fetch-har's Issues

Are you still interesed to maintain this repository - shall I contribute back some changes?

Hey @exogen thanks for this piece of software. I have made some significant changes and wanted to know if you would be interested if I'd contribute them back? I'm asking first , because the last commit is quite a while ago.

Changes are:

  • converted the code to Typescript
  • dropped support for all fetch implementations except node-fetch (we use only this one)
  • removed custom HarHttp and HarHttp classes (obsolete)
  • remove x-har-request-id header and instead create separate agents on each request (we don't want that the receiver is able to filter requests via this custom header)

Let me know if you are still interested to maintain this repository! I would be glad.

Support for aborted and timed out requests

Right now, if there is no response (for example, if the request is timed out or aborted), no entry will be added to the HAR log.

This means you get no visibility into these requests – you could see things long a long period of no network activity, and then a succesful retry request. But there actually was network activity, it was just a timed out request.

I'm not sure how aborted and timed out requests are represented in HAR files but will have to generate such requests in Chrome to find out.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.