Git Product home page Git Product logo

connect-busboy's People

Contributors

markstos avatar mscdex avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

connect-busboy's Issues

Example not working?

Tried using the example in the Readme, had to add some boilerplate, install express etc, but I'm seeing errors when posting:

[cmorgan@homepc busboy_test]$ node testapp.js 
TypeError: Cannot call method 'on' of undefined
    at app.use.busboy.immediate (/home/cmorgan/busboy_test/testapp.js:11:14)
    at Layer.handle [as handle_request] (/home/cmorgan/busboy_test/node_modules/express/lib/router/layer.js:76:5)
    at trim_prefix (/home/cmorgan/busboy_test/node_modules/express/lib/router/index.js:263:13)
    at /home/cmorgan/busboy_test/node_modules/express/lib/router/index.js:230:9
    at Function.proto.process_params (/home/cmorgan/busboy_test/node_modules/express/lib/router/index.js:305:12)
    at /home/cmorgan/busboy_test/node_modules/express/lib/router/index.js:221:12
    at Function.match_layer (/home/cmorgan/busboy_test/node_modules/express/lib/router/index.js:288:3)
    at next (/home/cmorgan/busboy_test/node_modules/express/lib/router/index.js:182:10)
    at /home/cmorgan/busboy_test/node_modules/connect-busboy/index.js:14:14
    at Layer.handle [as handle_request] (/home/cmorgan/busboy_test/node_modules/express/lib/router/layer.js:76:5)
^C[cmorgan@homepc busboy_test]$ 

It looks like req doesn't have a .busboy element. Tried with expressjs 3.x and 4.x.

busboy 0.2.6 compatible?

Is connect-busboy compatible with busboy 0.2.6? Been hacking for almost an hour, can't get it working.

Maybe update the examples to make the usage more obvious?

can not read all chunk of the files

Screenshot from 2021-12-22 12-41-51

Hello,
after updating connect-busboy from 0.0.2 to 1.0.0, I am having an issue. The problem is that this code sometimes works but sometimes does not. seems like something gets broken while uploading a file with API and just half of the file reaches the server and the close event does not trigger. if you call the API with the same file let's say 5-6 times, this error will occur just once.

EPIPE Error when uploading large file using nodejs

I was following this article to setup a nodejs server on my local machine (which has 16 gb memory and about 170gb free disk-space) and uploaded a 20 gb file, for the first couple of times the file got uploaded successfully, but after a while i started getting EPIPE error:

error FetchError: request to http://localhost:3200/upload failed, reason: write EPIPE
    at ClientRequest.<anonymous> (/Volumes/FreeAgent GoFlex Drive/Test/multer-project/node_modules/node-fetch/lib/index.js:1455:11)
    at ClientRequest.emit (events.js:327:22)
    at Socket.socketErrorListener (_http_client.js:467:9)
    at Socket.emit (events.js:315:20)
    at emitErrorNT (internal/streams/destroy.js:100:8)
    at emitErrorCloseNT (internal/streams/destroy.js:68:3)
    at processTicksAndRejections (internal/process/task_queues.js:84:21) {
  type: 'system',
  errno: 'EPIPE',
  code: 'EPIPE'
}

When i checked, the file got uploaded partially and was about 28mb in size. I tried uploading the file from both Postman, browser and a nodejs script, but got the same EPIPE error message. I am not sure why is this happening, googling the error message didn't help. I am not sure how to overcome this. Following is my server and client code.

// server.js
const express = require("express"); // Express Web Server
const busboy = require("connect-busboy"); // Middleware to handle the file upload https://github.com/mscdex/connect-busboy
const path = require("path"); // Used for manipulation with path
const fs = require("fs-extra");  

const app = express(); // Initialize the express web server
app.use(
  busboy({
    highWaterMark: 2 * 1024 * 1024 // Set 2MiB buffer
  })
); // Insert the busboy middle-ware

const uploadPath = path.join(__dirname, "uploads/"); // Register the upload path
fs.ensureDir(uploadPath); // Make sure that he upload path exits

/**
 * Create route /upload which handles the post request
 */
app.route("/upload").post((req, res, next) => {
  req.pipe(req.busboy); // Pipe it trough busboy

  req.busboy.on("file", (fieldname, file, filename) => {
    console.log(`Upload of '${filename}' started`);

    // Create a write stream of the new file
    const fstream = fs.createWriteStream(path.join(uploadPath, filename));
    // Pipe it trough
    file.pipe(fstream);

    // On finish of the upload
    fstream.on("close", () => {
      console.log(`Upload of '${filename}' finished`);
      res.send("ok");
    });
  });
});

/**
 * Serve the basic index.html with upload form
 */
app.route("/").get((req, res) => {
  res.writeHead(200, { "Content-Type": "text/html" });
  res.write(
    '<form action="upload" method="post" enctype="multipart/form-data">'
  );
  res.write('<input type="file" name="fileToUpload"><br>');
  res.write('<input type="submit">');
  res.write("</form>");
  return res.end();
});

const server = app.listen(3200, function() {
  console.log(`Listening on port ${server.address().port}`);
});

and my client code is:

// client.js
const fs = require("fs");
const FormData = require("form-data");
const fetch = require("node-fetch");

var formdata = new FormData();
formdata.append(
  "file",
  fs.createReadStream("/Users/phantom007/My Documents/35gb.myfile")
);

var requestOptions = {
  method: "POST",
  body: formdata,
  redirect: "follow"
};

fetch("http://localhost:3200/upload", requestOptions)
  .then(response => response.text())
  .then(result => console.log(result))
  .catch(error => console.log("error", error));

"Finish" and "field" events does not work when "file" event is added.

For eg:

This works!

const registerBusboyFields = (req, res, next) => {
  req.uploadBody = {};

  req.busboy.on("field", function (name, value) {
    req.uploadBody[name] = value;
    console.log(name, value);
  });

  // Finished all busboy events
  req.busboy.on("finish", () => {
    // The finish event is triggered...
    console.log(req.uploadBody);
    next();
  });

  req.pipe(req.busboy);
};

This does not work :(

const registerBusboyFields = (req, res, next) => {
  req.uploadBody = {};
  req.busboy.on("file", function (name, file, info) {
    req.uploadBody[name] = info;
    console.log(info);
  });

  req.busboy.on("field", function (name, value) {
    req.uploadBody[name] = value;
    console.log(name, value);
  });

  // Finished all busboy events
  req.busboy.on("finish", () => {
    // The finish event is NOT triggered...
    console.log(req.uploadBody);
    next();
  });

  req.pipe(req.busboy);
};

PS: When i say "does not work", i mean, i just see the console.log happening on "file" event, but console.log does neither happen on "field" event not in "finish" event.

Am I doing something wrong?

'TypeError: Cannot call method 'on' of undefined' error with POST

Hello,

I have this piece of code:

router.route("/post")
        .get(function(req, res) {
            ...
        })
        .post(authReq, function(req, res) {
            ...
            // Get uploaded file
            var fstream
            req.pipe(req.busboy)
            req.busboy.on('file', function (fieldname, file, filename, encoding, mimetype) {
            ...
            // Start writing
            fstream = fs.createWriteStream(tmpFileURL)
            file.pipe(fstream)
            fstream.on('close', function() {
            ...
            })
        })
    })
})

This use to work perfectly. However, I don't know quite what happened, but I picked up my computer today and found this error occurring every time I attempt to post:

TypeError: Cannot call method 'on' of undefined
    at IncomingMessage.Readable.pipe (_stream_readable.js:494:8)
    at /Users/nathan/Library/Mobile Documents/com~apple~CloudDocs/Dev/LaughItUp/Code/Server/apiRoutes.js:139:8
    at Layer.handle [as handle_request] (/Users/nathan/Library/Mobile Documents/com~apple~CloudDocs/Dev/LaughItUp/Code/Server/node_modules/express/lib/router/layer.js:82:5)
    at next (/Users/nathan/Library/Mobile Documents/com~apple~CloudDocs/Dev/LaughItUp/Code/Server/node_modules/express/lib/router/route.js:100:13)
    at authReq (/Users/nathan/Library/Mobile Documents/com~apple~CloudDocs/Dev/LaughItUp/Code/Server/apiRoutes.js:17:3)
    at Layer.handle [as handle_request] (/Users/nathan/Library/Mobile Documents/com~apple~CloudDocs/Dev/LaughItUp/Code/Server/node_modules/express/lib/router/layer.js:82:5)
    at next (/Users/nathan/Library/Mobile Documents/com~apple~CloudDocs/Dev/LaughItUp/Code/Server/node_modules/express/lib/router/route.js:100:13)
    at next (/Users/nathan/Library/Mobile Documents/com~apple~CloudDocs/Dev/LaughItUp/Code/Server/node_modules/express/lib/router/route.js:94:14)
    at Route.dispatch (/Users/nathan/Library/Mobile Documents/com~apple~CloudDocs/Dev/LaughItUp/Code/Server/node_modules/express/lib/router/route.js:81:3)
    at Layer.handle [as handle_request] (/Users/nathan/Library/Mobile Documents/com~apple~CloudDocs/Dev/LaughItUp/Code/Server/node_modules/express/lib/router/layer.js:82:5)

_stream_readable.js:501
    dest.end();
         ^
TypeError: Cannot call method 'end' of undefined
    at IncomingMessage.onend (_stream_readable.js:501:10)
    at IncomingMessage.g (events.js:180:16)
    at IncomingMessage.emit (events.js:92:17)
    at _stream_readable.js:943:16
    at process._tickCallback (node.js:419:13)

In my index.js file, I have this code to use Busboy:

app.use(busboy())

Then this code to include my route:

app.use("/api", require("./apiRoutes.js")())

Thanks in advance!

SyntaxError: Unexpected token {

node_modules/busboy/lib/utils.js:465
} catch {}
^

SyntaxError: Unexpected token {

from past week I am getting this error within the library.
any one else facing it?

Node V: 10.16.0
NPM: 6.9.0
lib version: Type definitions for connect-busboy 0.0
In package.json it saying 0.0.2
also tried: 1.0.0

express 3.0?

Does this work with 3.0?

Using the raw busboy example, it works, but plugging raw busby into express and passing express headers doesn't get the files / fields. Also, using connect-busboy has the same effect.

busboy finish event

Hi,
My understanding is that. finish event will trigger after file and field event completes?
Is their any other event that will fire after field and file event completes. I need to send the response to the client after all events completes.

This is the very critical one. Please let me know

Thanks,
Vivek

How to handle undefined upload

When the upload file is undefined,event handler is registered but event 'file' never happens. It will take along time for process to stop handling this request. When I keep on doing such thing for more than five times,the node process will halt for several minutes, it will come to normal after a few minutes.

Then how can I handle this situation in NodeJS process? Don't tell me never to upload undefined, when file upload failed, I think such situation will also take place.

File get corrupt only when I upload a pdf to s3

I am directly streaming the FileStream to s3. In case of docx and xlx it is working perfectly. It upload pdf with size to s3 but the file doesn't open on download.

 req.busboy.on(
     "file",
     async function (fieldname, file, filename, encoding, mimetype) {
         
       file.on("limit", function () {
         next(
           new ErrorResponse(
             { contract: "ERR_FILE_CONTRACT_SIZE" },
             "Validation Failed"
           )
         );
         return;
       });
      
       if(file){
         const stream = new PassThrough();
         file.pipe(stream);

         // Create file object
         req.files[fieldname] = {
         stream:stream,
         name: filename,
         encoding: encoding,
         mimetype: mimetype,
         };
       }
    

     }
   );

Getting Error while uploading file "Unexpected end of form at Multipart._final"

I am using angular as a front end and Firebase cloud function (Express.js) REST API.

Busboy: 1.6.0 (also tried 1.5.0)
Node: 16

export async function uploadFile(req: Request) {
    return new Promise<any>((resolve, reject) => {
        const bb = busBoy({ headers: req.headers });
        let uploadData = { file: "", type: "" };
        const uploadId = db.collection("_").doc().id;
        let fileExt: string = "";
        let newFileName: string = "";
        const uuidToken = uuidv4();
        const bucketName = admin.storage().bucket().name;
        let fbCollTicketData: any;

        // busboy.on('file', (fieldname, file, filename, encoding, mimetype) => {
        bb.on('file', (name, file, info) => {
            console.log("Execution started, file Block");
            const { filename, encoding, mimeType } = info;
            fileExt = filename.substring(filename.lastIndexOf('.'));
            newFileName = `${uploadId}${fileExt}`;
            console.log(`File [${name}]: filename: ${filename}, encoding: ${encoding}, mimetype: ${mimeType}`);
            const filepath = path.join(os.tmpdir(), newFileName);
            uploadData = { file: filepath, type: mimeType };
            const fstream = fs.createWriteStream(filepath);
            file.pipe(fstream);
            console.log("Execution Ended, file Block");
            // file.on('data', function (data) {
            //     console.log('File [' + fieldname + '] got ' + data.length + ' bytes');
            // });
        });
        //busboy.on('field', function (fieldname, val, fieldnameTruncated, valTruncated, encoding, mimetype) {
        bb.on('field', (name, val, info) => {
            console.log("Execution started, field Block");
            //console.log('Field [' + fieldname + ']: value: ' + val);
            console.log(`Field [${name}]: value: %j`, val);
            if (name == 'data' && val) {
                fbCollTicketData = <any>JSON.parse(val);
            }
            console.log("Execution Ended, field Block");
        });
        bb.on('error', (e) => {
            console.log("Execution started, error Block");
            console.log(e);
            console.log("Execution Ended, error Block");
        });
        bb.on('close', () => {
            console.log('Done parsing form!');
            admin.storage().bucket().upload(uploadData.file, {
                destination: `${STORAGE.CLEANINGPICS}/${newFileName}`,
                resumable: false,
                contentType: uploadData.type,
                metadata: {
                    metadata: {
                        //cacheControl: 'public, max-age=31536000',
                        firebaseStorageDownloadTokens: uuidToken,
                    }
                }
            }).then(async (resp) => {
                resolve(fbCollTicketData);
                fs.unlinkSync(uploadData.file);
            }).catch(err => {
                reject(err);
            })
        });
        req.pipe(bb);
    });
};

In cloud function when i check Functions log it shows me Error "Unexpected end of form
at Multipart._final"

image

@mscdex - Please help.

file event is not triggered

Hello,

I have a simple route like this:

router.post('/locations/file', busboy(), function(req, res, next) {
   console.log('here');

    req.busboy.on('file', function (fieldname, file, filename) {
        console.log('bar');
        var filePath = path.join(os.tmpdir(), filename);
        fstream = fs.createWriteStream(filePath);
        file.pipe(fstream);

       fstream.on('close', function () {
           console.log('foo');
       });
    });

    req.pipe(req.busboy);
});

And I have a weird behaviour: The file event is not triggered everytime (with the same file and the same upload form). but the 'here' log is displayed

I'm using express 4.6.1, connect-busboy 0.0.2 with busboy 0.2.9.

Thanks

Do not parse fields.

Busboy do not parse fields. I submit the images with fields as multipart/formdata

curl http://localhost:3000/api/signup -F "[email protected]" -F "[email protected]" -F "email=bill" -F "password=hifsafafasd" -F "gender=true" -F "device_id=adsf" -F "platform=1" -v

I have the following controller code:

signup: function(req, res){
        var data = {};
        data.imagesInfo = {};
        req.busboy.on('file', function(fieldname, file, filename, encoding){
            file.on('data', function(binary) {
                data.imagesInfo[fieldname + 'Encoding'] = encoding;
                if(!data.imagesInfo[fieldname]) {
                    data.imagesInfo[fieldname]  = new Buffer(binary);
                s} else {
                    data.imagesInfo[fieldname]  = Buffer.concat([data.imagesInfo[fieldname], binary]);
                }
            });
        });

        req.busboy.on('field', function(fieldname, val){
            console.log(val);
            data[fieldname] = val;
        });

        req.pipe(req.busboy);

        req.busboy.on('finish', function() {
            console.log(data);
            req.services.users.create(data).then(function(user){
                res.json(user);
            }).catch(function(err){
                res.json({ 'error' : err });
            });
        });
    },

The values did not parsed while field name and files did. The log is:

{ imagesInfo: 
   { imageEncoding: '7bit',
     image: <Buffer ff d8 ff e0 00 10 4a 46 49 46 00 01 01 00 00 01 00 01 00 00 ff db 00 84 00 09 06 06 14 12 10 15 14 12 14 14 14 14 14 15 14 14 14 16 17 14 14 15 16 15 17 ...>,
     imageSmallEncoding: '7bit',
     imageSmall: <Buffer ff d8 ff e0 00 10 4a 46 49 46 00 01 01 00 00 01 00 01 00 00 ff db 00 84 00 09 06 07 14 13 12 15 14 13 14 15 16 14 17 14 17 18 17 18 17 17 17 17 18 16 18 ...> },
  email: '',
  password: '',
  gender: '',
  device_id: '',
  platform: '1' }

Extending this to middleware that automatically fetches all attached files and fields

I stumbled across this middleware because I am looking for a middleware that automatically grabs fields and attached files from a multipart upload but keeps the file contents available in memory (i.e. doesn't write them to disk). It looks like it is possible to extend this middleware to do the job. However, this seems like a very general problem and so I'm wondering if it makes sense to extend this middleware with this functionality. Thoughts?

async routing function stops the busboy events from firing

This works:

app.post(`/`, async (req, res) => { /* busboy code */ });

This doesn't:

app.post(`/`, async (req, res) => { await (async () => {})(); /* busboy code */ });

Seems like the act of awaiting breaks how busboy works. I can't figure out why though.

Issue upload files on iPad / iPhone

Hi,

I am using connect-busboy to upload image files on my server. It works fine with all the desktop and android devices. But it is failing for the iPad/iPhone for bigger files >=1MB. On desktop,it is working with all types of files. On iPhone/iPad,it is working with small images. Is there some limitation using that? I am using angular-file-upload on the client. I tried using the native javascript XML HTTP request as well. There also, it fails for big files.

Regards,
Manik Mittal

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.