Git Product home page Git Product logo

peek-readable's Introduction

Node.js CI CodeQLNPM version npm downloads Coverage Status Codacy Badge Known Vulnerabilities

peek-readable

A promise based asynchronous stream reader, which makes reading from a stream easy.

Allows to read and peek from a Readable Stream

Note that peek-readable was formally released as then-read-stream.

Usage

Installation

npm install --save peek-readable

The peek-readable contains one class: StreamReader, which reads from a stream.Readable.

Compatibility

Module: version 5 migrated from CommonJS to pure ECMAScript Module (ESM). JavaScript is compliant with ECMAScript 2019 (ES10). Requires Node.js โ‰ฅ 14.16 engine.

Examples

In the following example we read the first 16 bytes from a stream and store them in our buffer. Source code of examples can be found here.

import fs from 'node:fs';
import { StreamReader } from 'peek-readable';

(async () => {
  const readable = fs.createReadStream('JPEG_example_JPG_RIP_001.jpg');
  const streamReader = new StreamReader(readable);
  const uint8Array = new Uint8Array(16);
  const bytesRead = await streamReader.read(uint8Array, 0, 16);;
  // buffer contains 16 bytes, if the end-of-stream has not been reached
})();

End-of-stream detection:

(async () => {

  const fileReadStream = fs.createReadStream('JPEG_example_JPG_RIP_001.jpg');
  const streamReader = new StreamReader(fileReadStream);
  const buffer = Buffer.alloc(16); // or use: new Uint8Array(16);

  try {
    await streamReader.read(buffer, 0, 16);
    // buffer contains 16 bytes, if the end-of-stream has not been reached
  } catch(error) {
    if (error instanceof EndOfStreamError) {
      console.log('End-of-stream reached');
    }
  }
})();

With peek you can read ahead:

import fs from 'node:fs';
import { StreamReader } from 'peek-readable';

const fileReadStream = fs.createReadStream('JPEG_example_JPG_RIP_001.jpg');
const streamReader = new StreamReader(fileReadStream);
const buffer = Buffer.alloc(20);

(async () => {
  let bytesRead = await streamReader.peek(buffer, 0, 3);
  if (bytesRead === 3 && buffer[0] === 0xFF && buffer[1] === 0xD8 && buffer[2] === 0xFF) {
    console.log('This is a JPEG file');
  } else {
    throw Error('Expected a JPEG file');
  }

  bytesRead = await streamReader.read(buffer, 0, 20); // Read JPEG header
  if (bytesRead === 20) {
    console.log('Got the JPEG header');
  } else {
    throw Error('Failed to read JPEG header');
  }
})();

peek-readable's People

Contributors

borewit avatar dependabot-preview[bot] avatar dependabot[bot] avatar lgtm-com[bot] avatar minht11 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

peek-readable's Issues

Getting "End-Of-Stream" error when using peek-readable with FileType module

Sometimes getting an error when small data is passed to the FileType module:

FileType.fromStream(readable)
Multiple resolvers type reject, value Error: End-Of-Stream
   at PassThrough.<anonymous> (peek-readable/lib/index.js:28:46)
   at Object.onceWrapper (events.js:421:28)
   at PassThrough.emit (events.js:315:20)
   at PassThrough.EventEmitter.emit (domain.js:483:12)
   at endReadableNT (_stream_readable.js:1220:12)
   at processTicksAndRejections (internal/process/task_queues.js:84:21)

After breef look at the 'peek-readable', I would say that error occurs when 'readable' and 'end' events are raised at the same time.

The tryRead() resolves the promise, however, the reference to the promise is not set to null immediatly but in callback of the then() function.

   this.s.once('readable', () => {
                this.tryRead();
   });

Next event 'end' rejects already resolved promise:

    reject(err) {
        this.endOfStream = true;
        if (this.request) {
            this.request.deferred.reject(err);
            this.request = null;
        }
    }

Thank you in advance if somebody can take look at this.

Source maps are referenced but not included in the NPM package

Hi,

The files published on NPM contain references to JavaScript source map. //# sourceMappingURL=index.js.map at the end of the index.js file and //# sourceMappingURL=EndOfFileStream.js.map at the end of EndOfFileStream.js.

However, these files are not included in the NPM package. It's not a real issue but VSCode started to complain about it. See microsoft/vscode#102042

The references to the source maps could be removed or the files could be included in the NPM package.

Thanks.

Concurrent read operation?'

I am trying to read audio/video metadata with two distinct packages that uses your implementation.

       const fileType = require("file-type");
       const musicMetadata = require("music-metadata");
       const strtok3 = require('strtok3');
       
       const someHttpRequestStream = axios.get(url, { responseType: stream})
       
       const tokenizer = await strtok3.fromStream(someHttpRequestStream );
       
        const f =  fileType.fromTokenizer(tokenizer);
        const mm=  musicMetadata.parseFromTokenizer(tokenizer);
        const t = await Promise.all([ff, f]);

I got an error that says:

Error: Concurrent read operation?

if(this.request) throw new Error('Concurrent read operation?');

Is there anyway that I can achieve that ? I thought that maybe having a interface that allowed me to extract mimeType and extension from the library music-metadata (since it uses internally file-type if fileInfo is not provided) would be the best .

But since I could not find a way of getting duration, mime and extension in one go. I am looking for alternatives to use these packages in parallel by having an intermediate passthrough stream.

Could you help me here ?
@Borewit

this.s.once is not a function

Uncaught TypeError: this.s.once is not a function at new StreamReader (node_modules/then-read-stream/lib/index.js:28:16)

Circular dependency warning

Latest release (5.1.1) introduced a circular dependency.

File AbstractStreamReader includes EndOfStreamError from index instead of EndOfFileStream.

Ignore not implemented

ignore is mentioned in the readme but doesn't seem to be implemented:

await streamReader.ignore(16);

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.