Newlines from C-domain are removed (or so it seems)

[Bernhard Ege]

When I compile my C-code using emscripten and in C use printf and in javascript receive the string via 'print', the newlines are removed.

This makes it quite difficult to discern the difference between these two printf's:

printf("foo");

printf("foo\n");

Visually, there is an important difference, though.

Is there some flag or some way print can be told to return the newlines?

[Shlomi Fish]

Hi!

seems to work here:

```
[shlomif@localhost shlomif-c-snippets]$ cat newline-printf.c
/*
* License is the MIT/Expat license - http://opensource.org/licenses/MIT .
* ( https://en.wikipedia.org/wiki/MIT_License ).
*/
#include <stdio.h>

int main(void)
{
printf ("Hello World!");
printf ("(emcc)\n");

return 0;
}
[shlomif@localhost shlomif-c-snippets]$ emcc newline-printf.c
[shlomif@localhost shlomif-c-snippets]$ ls -lrt *.js
-rw-r--r--. 1 shlomif shlomif 110446 Jul 15 14:36 a.out.js
[shlomif@localhost shlomif-c-snippets]$ node a.out.js
Hello World!(emcc)
[shlomif@localhost shlomif-c-snippets]$
```

Can you share a self-contained sample? See:
https://github.com/shlomif/how-to-share-code-online (read the whole thing).

Thanks in advance.

--

Shlomi Fish https://www.shlomifish.org/
https://github.com/sindresorhus/awesome - curated list of lists

At this point, I'd like to take a moment to speak to you about the Adobe PSD
format. PSD is not a good format. PSD is not even a bad format. Calling it
such would be an insult to other bad formats, such as PCX or JPEG. No, PSD is
an abysmal format.
— https://github.com/gco/xee/blob/master/XeePhotoshopLoader.m

Please reply to list if it's a mailing list post - https://shlom.in/reply .

[Alon Zakai]

One possible noticeable difference with a normal target is that we print using console.log() and other JS APIs, which can't print partial lines - they always add a newline. So we buffer, and call those APIs only when we reach a newline. As a result, you won't see partial lines printed until a newline is reached.

If that's not the issue here, that might be an unknown bug somehow, a testcase would help.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/20200715144004.58141cb4%40telaviv1.shlomifish.org.

[Bernhard Ege]

I haven't made a standalone test yet as that would require quite some time. However, I hope with the given information, that we can see what is going on and if there is anything to do about it. Problem is I am using webassembly inside webworker (works jusy fine), but it can muddy the experience and getting a clean standalone test means creating a project from scratch. I am willing to do so if we can't make a clear case of the information below:

This is how I compile my C-project (just updated to latest emscripten, problem remains):

& emcc -s WASM=1 -s ALLOW_MEMORY_GROWTH -s EXPORTED_FUNCTIONS="['_main','_malloc','_Iterate','_getImageData','_getBufferPtr']" -s EXTRA_EXPORTED_RUNTIME_METHODS='["cwrap", "getValue", "setValue"]' $(ls *.c| % {$_.FullName}) $(ls Racer/*.c | % {$_.FullName}) -I. -ferror-limit=0 -o myrekrig.js

My source is current folder and the Racer subfolder. I just want a myrekrig.js file generated (along with the webassembly).

I use the webassembly as a library, i.e. it doesn't exit, meaning I call repeatedly from javascript to wasm and have wasm repeatedly output text using printf (with and without "\n"). The printf are caught using the override in the Module setup:

var Module = {

    print: (function() { return function(text) { console.log(text); console.log(ascii_to_hexa(text); }; }

}

ascii_to_hexa simply converts text to hex and is used to verify proper operation.

I normally push the text via the message queue to the main page that appends the text to a textarea (which obeys newlines).

One example of text received that in the printf is \n terminated:

536c65657079416e742020202020302020202030202020302e30202020202030202020202030202020202030202020302e302520202020202031202020302e3025202020302e3025202020302e3025

Clearly no newline present (last 2 digits, 25, is a %-sign).

In the generated myrekrig.js, I find this section:

printChar:function(stream, curr) {

        var buffer = SYSCALLS.buffers[stream];

        assert(buffer);

        if (curr === 0 || curr === 10) {

          (stream === 1 ? out : err)(UTF8ArrayToString(buffer, 0));

          buffer.length = 0;

        } else {

          buffer.push(curr);

        }

If I remove "|| curr === 10", then the output is correct, newline or no newline. However, this also means the output is buffered for a long time (or until a buffer is filler or some other event happens), delaying the text for a long time, which is also unwanted. The hack will also be overwritten on the next compilation. I am considering simply using sprintf and push the string from wasm to js domain manually, but I hope I don't have to.

It seems the printf output is made more responsive by sending data to the JS-print method every time \n (10) or 0 (string termination) is found, which is great (if it preserves all characters, including \n, which it doesn't seem to). Could we make it operate using a smaller buffer or maybe using a forced flush (or a timeout, not sure that is possible)?

Does the above information make sense?

Thanks,

Bernhard

To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.

[Bernhard Ege]

Yes, it does that today.  But when newlines are met, they are removed before the string is handed to the js app. This seems like an error. The js app receiving the text cannot know if this particular string should be with or without newline as the framework deletes the newline. 

tor. 16. jul. 2020 02.35 skrev Alon Zakai <alon...@gmail.com>:

>  It seems the printf output is made more responsive by sending data to the JS-print method every time \n (10) or 0 (string termination) is found, which is great

It already does that, doesn't it? If not, can you share your patch that fixes it?

To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/20200715144004.58141cb4%40telaviv1.shlomifish.org.

--

You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/496ad5df-e521-41b4-bb50-a26ae13b19c6o%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/CAEX4NpRkJgD-dqAE1JP-V9w3FLESMnLtFysvNe%3Dn8QCYKhnSYg%40mail.gmail.com.

[Bernhard Ege]

I have now examined the issue a bit more, and the issue (for me) is that I don't receive any partial lines until a newline is sent.

This means lines like these output nothing:

printf("foo");

until a printf with a \n in it arrives.

This means any string I receive in javascript needs to have \n appended.

This is of course one way to solve the partial lines problem, but I was hoping to receive all bytes printf was printing, relying on fflush(stdout) to force emscripten to return all buffered text right away, including newlines.

I am using partial printed lines to show what has been started, and waiting for the result (can take minutes to arrive) to append to the line, and first then print the newline.

With the current system in effect, I am just waiting for stuff to calculate, but I don't know what until it is done.

It seems if I want to work around it, I must implement my own printf. Not ideal. Or possible figure out a way to hack the generated javascript file.

/Bernhard

To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/20200715144004.58141cb4%40telaviv1.shlomifish.org.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/496ad5df-e521-41b4-bb50-a26ae13b19c6o%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.

[Alon Zakai]

I'd be curious if you can find a better way to solve this. The current situation, as you described, is the best we've come up with so far, given the limitation that console.log always adds a newline (so we need to not send it a newline, and instead buffer until one is reached, then send the line without that).

You can experiment with that in the JS FS layer, in src/library_fs.js, and in a custom Module['stdout'] that it calls (see the code that calls Module['stdout'] in that file).

To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/20200715144004.58141cb4%40telaviv1.shlomifish.org.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/496ad5df-e521-41b4-bb50-a26ae13b19c6o%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/CAEX4NpRkJgD-dqAE1JP-V9w3FLESMnLtFysvNe%3Dn8QCYKhnSYg%40mail.gmail.com.

--

You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/1932b22b-5dfe-4f33-9043-3a04f1c03599o%40googlegroups.com.