Git Product home page Git Product logo

random-access-http's Introduction

random-access-storage

Easily make random-access-storage instances

npm install random-access-storage

A random-access-storage instance is a common interface for a storage abstraction, that provides the following core api.

  • read(offset, size) - Read a buffer at a custom offset.
  • write(offset, data) - Write a buffer to a custom offset.
  • del(offset, size) - Delete data at a custom offset.

This module exposes a base class that implements most of the plumbing and flow you'd usually want to implement when making one.

Usage

const RandomAccessStorage = require('random-access-storage')
const fs = require('fs')

const file = fileReader('index.js')

file.read(0, 10, (err, buf) => console.log('0-10: ' + buf.toString()))
file.read(40, 15, (err, buf) => console.log('40-55: ' + buf.toString()))
file.close()

function fileReader (name) {
  let fd = 0
  return new RandomAccessStorage({
    open: function (req) {
      // called once automatically before the first read call
      fs.open(name, 'r', function (err, res) {
        if (err) return req.callback(err)
        fd = res
        req.callback(null)
      })
    },
    read: function (req) {
      const buf = Buffer.allocUnsafe(req.size)
      fs.read(fd, buf, 0, buf.length, req.offset, function (err, read) {
        if (err) return req.callback(err)
        if (read < buf.length) return req.callback(new Error('Could not read'))
        req.callback(null, buf)
      })
    },
    close: function (req) {
      if (!fd) return req.callback(null)
      fs.close(fd, err => req.callback(err))
    }
  })
}

API

const storage = new RandomAccessStorage([options])

Make a new instance. Options include:

{
  createAlways: false, // always create storage on first open
  open: fn, // sets ._open
  read: fn, // sets ._read
  write: fn, // sets ._write
  del: fn, // sets ._del
  truncate: fn, // sets ._truncate
  stat: fn, // sets ._stat
  suspend: fn, // sets ._suspend
  close: fn, // sets ._close
  unlink: fn // sets ._unlink
}

storage.readable

True if the storage implements ._read.

storage.writable

True if the storage implements ._write.

storage.deletable

True if the storage implements ._del.

storage.truncatable

True if the storage implements ._truncate.

storage.statable

True if the storage implements ._stat.

storage.opened

True if the storage has been fully opened.

storage.closed

True if the storage has been fully closed.

storage.unlinked

True if the storage has been fully unlinked.

storage.writing

True if the storage is currently being written to.

storage.on('open')

Emitted when the storage is fully open.

storage.on('close')

Emitted when the storage is fully closed.

storage.on('unlink')

Emitted when the storage is fully unlinked.

storage.on('suspend')

Emitted when the storage is fully suspended

storage.on('unsuspend')

Emitted when the storage comes out of suspension.

storage.open(cb)

Explicitly open the storage. If you do not call this yourself, it will automatically called before any read/write/del/stat operation.

It is safe to call this more than once.

Triggers one call to _open if you implement that.

storage._open(req)

Implement storage open.

  • req.create is true if the storage should be created.

Call req.callback when it is fully opened.

storage.read(offset, size, callback)

Read the specified bytes from the storage at the specified byte offset. Calls the callback with a (err, buffer).

storage._read(req)

Implement storage read.

  • req.offset contains the byte offset you should read at.
  • req.size contains the amount of bytes you should read.

Call req.callback(err, buffer) when the read is completed.

Note that this is guaranteed to run after the storage has been opened and not after it has been closed.

storage.write(offset, buffer, [callback])

Write the specified buffer to the specified byte offset. Optionally pass a callback that is called with (err) when the write has completed.

storage._write(req)

Implement storage write.

  • req.offset contains the byte offset you should write at.
  • req.data contains the buffer you should write.

Call req.callback(err) when the write is completed.

Note that this is guaranteed to run after the storage has been opened and not after it has been closed.

storage.del(offset, size, [callback])

Delete the specified amount of bytes at the specified offset. Optionally pass a callback that is called with (err) when the delete has completed.

storage._del(req)

Implement storage delete.

  • req.offset contains the byte offset to delete at.
  • req.size contains the amount of bytes to delete.

Call req.callback(err) when the delete has completed.

Note that this is guaranteed to run after the storage has been opened and not after it has been closed.

storage.truncate(offset, [callback])

Truncate the storage at the specified offset. Optionally pass a callback that is called with (err) when the truncate has completed.

storage._truncate(req)

Implement storage truncate. Defaults to storage._del(req).

  • req.offset contains the byte offset to truncate at.

Call req.callback(err) when the truncate has completed.

Note that this is guaranteed to run after the storage has been opened and not after it has been closed.

storage.stat(callback)

Stat the storage. Should return an object with useful information about the underlying storage, including:

{
  size: number // how many bytes of data is stored?
}

storage._stat(req)

Implement storage stat.

Call req.callback(err, statObject) when the stat has completed.

Note that this is guaranteed to run after the storage has been opened and not after it has been closed.

storage.suspend([callback])

Suspend (temporarily close) the storage instance.

storage._suspend(req)

Implement storage suspend. Defaults to calling _close.

Optionally implement this to add a way for your storage instance to temporarily free resources.

Call req.callback(err) when the storage has been fully suspended.

storage.close([callback])

Close the storage instance.

storage._close(req)

Implement storage close.

Call req.callback(err) when the storage is fully closed.

Note this is guaranteed to run after all pending read/write/stat/del operations has finished and no methods will run after.

storage.unlink([callback])

Unlink the storage instance, removing all underlying data.

storage._unlink(req)

Implement storage unlink.

Call req.callback(err) when the storage has been fully unlinked.

Note this is guaranteed to run after .close() has been called and no methods will be run after.

License

MIT

random-access-http's People

Contributors

bcomnes avatar e-e-e avatar greenkeeper[bot] avatar greenkeeperio-bot avatar soyuka avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

random-access-http's Issues

An in-range update of pump is breaking the build 🚨

Version 1.0.3 of pump was just published.

Branch Build failing 🚨
Dependency pump
Current Version 1.0.2
Type dependency

This version is covered by your current version range and after updating it in your project the build failed.

pump is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Status Details
  • ❌ continuous-integration/travis-ci/push The Travis CI build could not complete due to an error Details

Commits

The new version differs by 2 commits.

  • 28557d8 1.0.3
  • a430c27 avoid ERR_INVALID_ARG_TYPE error on Node.js >= 9 (#23)

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

Merging http-random-access and random-access-http

I am opening this issue to discuss merging our two implementations.

I have not moved https://github.com/e-e-e/http-random-access into the random-access-storage org yet as I knew that this module already existed and it felt silly to have two versions of essentially the same thing. However they are structurally different.

I created http-random-access for use as a drop in replacement for raf and ram in dat storage. random-access-http does not work for this purpose as it requires absolute urls, and throws errors on write. http-random-access also uses axios rather than nodes native http module, so that its browser compatible.

I just updated my implementation to use the latest random-access-storage and while I was at it I made it work with absolute urls too in addition to relative urls. It now also passes your tests in addition to mine. I wanted to suggest that we simply merge my implementation into this repository and do a major version bump, and deprecate http-random-access. Although is there anything in this implementation that is not currently covered by http-random-access? Have I missed anything?

What are your thoughts @bcomnes, @mafintosh, @scriptjs, @juliangruber?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.