Handling binary data outside of Emscripten?

[Mateusz Wielgos]

Hi,

I used Emscripten to port libogg and libvorbis to JavaScript. In particular, I am interested in audio encoding. The encoder test example, which takes wav data from stdin and outputs it to stdout works for the most part (using node.js), but -some- characters seem to be corrupted (as viewed from a text editor).

The data in C is being output using fwrite. I think this is an encoding issue? I tried outputting the data with : process['stdout'].write(data,'binary') (in Moduel['print'] callback) but, once again, small parts of data (individual bytes here and there...) do not resemble the proper audio output, as compared to the C program audio output. The Emscripten audio file does not play while C audio file does.

I don't have any special Emscripten handling in my code (no #ifdef EMSCRIPTEN defines). 

So, in Module['print'] callback, how can I handle the binary stdout properly, using node.js?

Furthermore, I would like to go further with this and do audio encoding on the browser side (preferably using a webworker). The way that I am thinking right now, I will still use a Module['print'] callback for this, just post a message back to the main script. How do I handle the binary data at that point? In Module['print'], convert the data over to a Blob and send it back (as simple as : new Blob([mydata], { type : 'audio/ogg' }) ?).

Thanks for the help,

Mateusz

[Alon Zakai]

Module.print should just receive the bytes printed out, as well as
'null' for EOF. Perhaps null is confusing things?

Might be simpler to not use printf though, printing of binary data
isn't going to work in some platforms anyhow, and it won't be fast
since its byte by byte. Much better to just write the audio data into
a buffer in memory, then grab it with HEAPU8.subarray(start, end).

- azakai

> --
> You received this message because you are subscribed to the Google Groups
> "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to emscripten-disc...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Daniel Baulig]

Sorry for reviving an old thread but I'm running into a very similar issue right now (found this while searching for a possible solution). 

I'm having a small C program that outputs binary data to stdout using fwrite. I usually invoke it like prog >outfile. It will also output some debugging data to stderr. This works flawless when compiled using gcc, but failed when compiled with emcc and run with node. The output to stderr will occasinally be corrupted. This only happens with the next output to stderr after a fwrite call. Also, the data output to stdout is corrupted and is not the same as the data written by the gcc compiled version. I randomly checked the data in the buffer before it is written to stdout by writing some bytes to stderr using fprintf(stderr, "%x%x\t") and all the bytes i've checked in the buffer are the same for the gcc and the emcc version of the binary. So the corruption must happen after calling fwrite inside the emcc stdio implementation. I tried putting the stdout stream into binary mode with freopen(NULL, "wb", stdout); before writing binary data to it, but that didn't help either.

Anyone any suggestions?

- Daniel

[Alon Zakai]

Simplest option is to write to a file, not to stdout or stderr. For stdout and stderr, we might be missing some binary stream support in node, you might be the first person to try that...

- Alon

--

[robert.a...@gmail.com]

Hello, I'm compiling a C++ tool that outputs binary to stdout and I'm running into a similar issue but in the browser. I'm curious whether there's now a way to support this without modifying the code by any chance? Thank you!

[Alon Zakai]

I think you can define Module['stdout'], see this test:

https://github.com/emscripten-core/emscripten/blob/7e538a419c3649f3a540a57beab347aa8f6c6271/tests/test_core.py#L4895-L4926

That function should receive the character code of each byte written. So you could construct the binary data from that.

(But it would be more efficient to write to a file, and get the entire file at once as an array, usually.)

- Alon

To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/aaa95dab-0be9-439e-9b14-7b426f94d24dn%40googlegroups.com.