multipart problem - uploaded file ends up wrong

[Roberto Saccon]

I am trying to build an upload server, but the image I am uploading
while testing, is not an image anymore, after saving at the server.
Has anybody a working example for that ? In the testcases I also
haven't found multipart stream tests. Here is what I am doing:

function upload_file(req, res) {
req.setBodyEncoding('binary');
var stream = new multipart.Stream(req);
stream.addListener('part', function(part) {
posix.open("/tmp/test.jpg", process.O_WRONLY | process.O_TRUNC |
process.O_CREAT, 0666)
.addCallback(function (fd) {
var writtenTotal = 0;
part.addListener('body', function(chunk) {
function doWrite (_chunk) {
posix.write(fd, _chunk, null, 'binary')
.addErrback(function () {
posix.close(fd);
})
.addCallback(function (written) {
writtenTotal += written;
if (writtenTotal == stream.bytesTotal) {
sys.puts("upload complete: "); //never happens
posix.close(fd);
} else if (written < _chunk.length) {
doWrite(_chunk.slice(written));
}
});
}
doWrite(chunk);
});
});
});
stream.addListener('complete', function() {
res.sendHeader(200, {'Content-Type': 'text/plain'});
res.sendBody('uploaded');
res.finish();
});
}

And there are couple of problems with my approach. First I think there
is some encode wrong, if I look at the begin of the uploaded file
with a hex editor, it doesn't look like jpeg. Then how do I know when
to close the filedesciptor ? In the approach above, the condition
(writtenTotal == stream.bytesTotal) never becomes true, but closing
the filedesciptor in the stream-complete callback also doesn't seem to
be a good idea, because writing activity could still be ongoing when
that callback gets called (I think).

[Felix Geisendörfer]

On first scan it looks like your posix.write() calls will be executed
in random order. That's because AFAIK there is no guarantee on
ordering for the posix API.

I would highly recommend to use the file module as it will handle the
queuing of such calls for you. Here is some production (read: ugly)
code I'm using for transload.it:

https://gist.github.com/b1556e500b5edf2123c8

Let me know if that helps, otherwise I can whip up a stand-alone
example for you.

-- Felix Geisendörfer aka the_undefined

[Andrew Lunny]

I think the image is corrupted because you're calling posix.write (and
posix.close asynchronously after that) on the "body" event of the
stream, rather than the "complete" event.

Here is some sample upload code (modified from the multipart sample in
the docs) that might help:

// inside http.createServer callback
if (req.uri.path == "/upload") {

var stream = new multipart.Stream(req);

var parts = {}, names = {}, current_upload = {};

stream.addListener("part", function (part) {
var buffer = "";

part.addListener("body", function (chunk) {
buffer = buffer + chunk;
sys.puts(JSON.stringify(current_upload));
});

part.addListener("complete", function () {
parts[part.name] = buffer;
if (part.filename) {
names[part.name] = part.filename;

}
});
});

stream.addListener("complete", function() {

for (part in parts) {
uploader.create_and_write_file(names[part],parts[part]);
res.sendBody("upload complete");
res.finish();
};
});
}

var create_and_write_file = function (filename,data) {
var write_data = function (descriptor) {
var write_promise = posix.write(descriptor, data, null, "binary")

write_promise.addCallback(function (bytes) {
sys.puts(bytes + " bytes written");
posix.close(descriptor);
});

write_promise.addErrback(function () {
posix.close(descriptor);
sys.puts("an error");
})
};

posix.open(filename,process.O_CREAT |
process.O_RDWR,0755).addCallback(write_data);
};

You could also put the create_and_write_file function in the callback
for the part complete callback, if desired.

Cheers,
Andrew

> --
>
> You received this message because you are subscribed to the Google Groups "nodejs" group.
> To post to this group, send email to nod...@googlegroups.com.
> To unsubscribe from this group, send email to nodejs+un...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/nodejs?hl=en.
>
>
>



--
Andrew Lunny
Software Developer, Nitobi
604 685 9287
andrew...@nitobi.com

[Roberto Saccon]

Andrew and Felix, thanks a lot, got it working now, with the file
module instead of posix.
--
Roberto

> > For more options, visit this group athttp://groups.google.com/group/nodejs?hl=en.

>
> --
> Andrew Lunny
> Software Developer, Nitobi
> 604 685 9287

> andrew.lu...@nitobi.com