busboy, upload few files, 'finish' event doesn't work properly

[dmitry....@wemaketraffic.com]

Hello,.

I trying to upload few files through one input[type=file] with attribue 'multiple'. All work fine but 'finish' event firing before realy all work done. Does anyone know how to fix it?

My server code:

app.post('/multiupload', function(req, res) {
    var fstream;
    var files = [];
    var busboy = new Busboy({headers: req.headers});
    busboy.on('file', function (fieldname, file, filename) {
        fstream = fs.createWriteStream(__dirname + '/../static/uploaded/' + filename);
        file.pipe(fstream);
        fstream.on('close', function(){
            console.log('file ' + filename + ' uploaded');
            files.push(filename);
        });
    });


    busboy.on('end', function(){console.log('END')});


    busboy.on('finish', function(){
        console.log('finish, files uploaded ', files);
        res.redirect('back');
    });
    req.pipe(busboy);
});

My console.log info:

file 111.gz uploaded
file 222.mp4 uploaded
file 333.jpg uploaded
finish, files uploaded  [ '111.gz', '222.mp4', '333.jpg' ]
file 444 uploaded
file 555.jpg uploaded

[Jake Wolpert]

Finish should happen before all your console log calls, but not before mine!

It takes time to read the files.

    busboy.on('file', function (fieldname, file, filename) {
        fstream = fs.createWriteStream(__dirname + '/../static/uploaded/' + filename);

        console.log('file ' + filename + ' uploading'); // mine
        file.pipe(fstream);
        fstream.on('close', function(){
            console.log('file ' + filename + ' uploaded'); /yours
            files.push(filename);
        });
    });

--
Job board: http://jobs.nodejs.org/
New group rules: https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
---
You received this message because you are subscribed to the Google Groups "nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nodejs+un...@googlegroups.com.
To post to this group, send email to nod...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/2bb9066f-e117-4ed8-98b3-407ee4c4237b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[dmitry....@wemaketraffic.com]

I need to get 'finish' event. when all files already saved. Your console may show properly, but it's not a 'finish' event :(

My trouble is 'finish' activated not in right time, as well as 'end' doesn't fire :(

Can you help me with that?

[Jake Wolpert]

Finish is happening after all the file events.

Your code does a console.log after the streams are read.

When you use my console.log at the beginning of a file event, you should see that they are all logged before finish.

It's all about the asynchronous nature of javaScript.

To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/304c4d70-b57c-4f38-aa71-1856309a638a%40googlegroups.com.

[Jake Wolpert]

You would probably need to use an async function to collect all the close events, to know exactly when all the files are saved. 

BUT since you are probably generating an html page, that will not be rendered on the browser for a while, I would just assume that the files will be saved before the page is sent to a browser, and rendered.

[dmitry....@wemaketraffic.com]

You would probably need to use an async function to collect all the close events, to know exactly when all the files are saved. 

Thanks, i will try to play with files count and stream 'close' event.

[mscdex]

On Friday, October 3, 2014 9:27:10 AM UTC-4, dmitry....@wemaketraffic.com wrote:

I trying to upload few files through one input[type=file] with attribue 'multiple'. All work fine but 'finish' event firing before realy all work done. Does anyone know how to fix it?

When the busboy instance emits `finish`, that means that all data has passed through (taking into account file stream backpressure). If it takes some time for the streams you are piping/writing to to close, busboy can't know for that so you have to track that manually. You can count the number of `close` events manually or maybe use a module like `async` to help simplify that logic.

[Jake Wolpert]

Here's a piece from one of my projects. I just added async.series.  I hope it helps.

request.pipe(

new Busboy({ headers: request.headers })

.on('file', function(fieldname, file, filename, encoding, mimetype) {

log("file",fieldname, filename, encoding, mimetype, file)

var saveTo = path.join("uploaded", path.basename(filename))

fileNames.push(saveTo)

message.push(htmlFor(saveTo,filename,mimetype,false))

var out = fs.createWriteStream(www+saveTo)

events.push(function(callback){

out.on('finish',callback)

})

file.pipe(out)

}).on('field', function(fieldname, val, valTruncated, keyTruncated) {

fieldNames[fieldname]=val // ignore arrays

// log("field",fieldname, val, valTruncated, keyTruncated)

}).on('finish', function() {

async.series(events,function(){

console.warn("UPLOADED")

//…

})

})

)

[Floby]

The problem is that the 'finish' event is not what you expect. You are looking for the 'end' event.

the 'finish' event is fired when the busboy stream writable side is done writing.