nodestream / nodestream Goto Github PK

Let's keep the final tarball small, the images could have several kB and are not needed for anything outside of this site.

Hey nodestream!

I just came across this project, and it looks like a really good fit for bionode-watermill. In fact, maybe their is even some (slight) overlap in our goals. (Right now we do our own checksum checking on files - and would need to do it on streams as well)

Just wanted to let you know, we have plans to integrate eventually into our project. Any help or suggestions would be greatly appreciated - see bionode/bionode-watermill#46

:)

There should be at least one "official" package which integrates nicely with Express. Other framework integrations might come later, such as Koa or Hapi. The goals of such package are described below.

• Installs itself as a body parser or hooks into existing body parser
• Watches for incoming file uploads and parses each file into a readable stream without buffering the whole file to disk
• Provides some nice mechanism (ie. a function attached to req) to configure and initiate the file upload with a previously configured Nodestream instance

Not really sure how this is actually supposed to work. The original inspiration, I guess, comes from how Skipper works. Probably it would be best to defer the actual body parsing to some existing and established library like multiparty and only extract the incoming byte streams from it.

Currently all transforms must declare their identity, which is then used at various places, ie. for exporting the transform's result data on the final object (the identity is used as object key).

This, however, limits the use of a single transform multiple times in a pipeline and forces all transforms to have unique identity across all existing transforms, whether they are core or 3rd party.

We should extend nodestream.registerTransform() to accept a string parameter as first argument, which, when supplied, will be used as the transform's identity.

I have a problem when using nodestream with stream buffers is that upload function returned promise is not resolved neither rejected inspite file is uploaded

I want to upload base64 encoded string and save it to a file so I used stream buffers for making stream instance holding base64 file and then passing it to the nodestream upload function

const Nodestream = require('nodestream')

const fs=require('fs')

const {ReadableStreamBuffer} =require('stream-buffers')

const nodestream = new Nodestream({
  adapter: 'filesystem',
  config: {
       root: [__dirname, '.storage']
  }
})

const stream=new ReadableStreamBuffer({})

stream.put(fs.readFileSync('toread.txt').toString('base64'),'base64')

//neither resolve or reject is called inspite file is called
nodestream.upload(stream).then(res=>{
   
 if(fs.existsSync(`.storage/${res.location}`))
    {
        console.log('file exist')
    }
    else
    {
        console.log('file not exist')
    }
}).catch(ex=>console.log(ex.message))

Allow developers to attach some kind of onProgress function to the upload/download request. This must not require adapter support and must be implemented here.