PSA: Requiring explicit opt-in to certain JS runtime functions

[Sam Clegg]

Starting with 2.1.21, in order to make the default output size smaller some runtime functions are transitioning to library functions.  In debug builds, if you try to use these functions, you should see an actionable warning.

From the changelog:

- The `LEGACY_RUNTIME` setting is no longer enabled by default.  If you use any  
  of these legacy runtime functions (except in library code with explicit        
  dependencies) then you would need to set `LEGACY_RUNTIME` on the command line  
  or add the ones you need to `DEFAULT_LIBRARY_FUNCS_TO_INCLUDE`:                
   - addFunction                                                                
   - removeFunction                                                              
   - allocate                                                                    
   - AsciiToString                                                              
   - stringToAscii                                                              
   - UTF16ToString                                                              
   - stringToUTF16                                                              
   - lengthBytesUTF16                                                            
   - UTF32ToString                                                              
   - stringToUTF32                                                              
   - lengthBytesUTF32                                                            
   - allocateUTF8                                                                
   - allocateUTF8OnStack                                                        
   - writeStringToMemory                                                        
   - writeArrayToMemory                                                          
   - writeAsciiToMemory                                                          
   - intArrayFromString                                                          
   - intArrayToString                                                            
   - warnOnce                                                                    
   - ccall                                                                      
   - cwrap      

                                                                
  Although this is technically a breaking change for those who use these        
  functions, there are assertions in debug builds that catch such usages and        
  direct towards how to fix the issue.

I hope this change is not too disruptive and overall a move in the right direction.

cheers,

sam

[Shlomi Fish]

On Tue, 6 Sep 2022 09:06:27 -0700
"'Sam Clegg' via emscripten-discuss" <emscripte...@googlegroups.com>
wrote:

> Starting with 2.1.21, in order to make the default output size smaller some
> runtime functions are transitioning to library functions. In debug builds,
> if you try to use these functions, you should see an actionable warning.
>
> From the changelog:
>

Thanks for the warning, Sam!

--

Shlomi Fish https://www.shlomifish.org/
Chuck Norris/etc. Facts - https://www.shlomifish.org/humour/bits/facts/

If Botticelli were alive today, he’d be working for Vogue.
— https://en.wikiquote.org/wiki/Peter_Ustinov

Please reply to list if it's a mailing list post - https://shlom.in/reply .

[Floh]

I just noticed that I had two remaining calls to ccall() where I used the string argument marshalling feature (in all other cases I call functions on the C side directly). I have those cases now replaced with (essentially) "withStackSave(() => { ... allocateUTF8OnStack() ... }"

Is this "kosher"? (I'm aware of the risk that these functions may go away one day).

I don't want to impose specific compiler flags (like LEGACY_RUNTIME) on my library users, that's why I had to replace ccall() (unless there's a another way to inject ccall into the code than through compiler options?).

It would be nice to have a guaranteed set of these low level C interop functions btw (like this withStackSave, allocateUTF8OnStack stuff).

Cheers!

-Floh.

[Floh]

PS: also I just noticed that allocateUTF8OnStack is on your list of 'legacy functions', but calling this works just fine in my code. I guess those are pulled in from some library_*.js that's linked into my code?

What's the lowest level set of stack helper functions that won't go away? Is withStackSave(), stackAlloc() and stringToUTF8() guaranteed at least?

[Sam Clegg]

On Sat, Sep 10, 2022 at 10:36 AM Floh <flo...@gmail.com> wrote:

PS: also I just noticed that allocateUTF8OnStack is on your list of 'legacy functions', but calling this works just fine in my code. I guess those are pulled in from some library_*.js that's linked into my code?

What's the lowest level set of stack helper functions that won't go away? Is withStackSave(), stackAlloc() and stringToUTF8() guaranteed at least?

The hope is that we can move away from including any code that is not explicitly needed, and not include any of these functions by default.

How/where are you calling these functions?   I assume it is something like EM_JS or EM_ASM, since if you were doing it in a JS library then you could add explicitly dependencies.  

If you are indeed using EM_ASM/EM_JS and expecting to be able to call library functions then it sounds like we need to add some kind of dependency mechanism for EM_ASM/EM_JS user code.

Alternatively we could use some kind of analysis of the EM_JS/EM_ASM strings..  perhaps we could just do a dumb conservative string scanner, or use a JS parser to find any used symbols.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/39749284-d7df-4e03-aea7-83362d5ef36cn%40googlegroups.com.

[Floh]

> How/where are you calling these functions? 

Yes, it's all EM_JS in STB-style single-file C libraries (for instance: https://github.com/floooh/sokol/blob/4238be9d53f7a817f7ddf1b8d10560142052c6c7/sokol_app.h#L4519-L4528)

...if I would have to distribute a separate .js file with the header it would kinda defeat the whole idea of single-file C libraries (and I would also need to add Emscripten-specific build instructions).

>... we need to add some kind of dependency mechanism for EM_ASM/EM_JS user code

I think this would be the best solution. Maybe it could work similar like that new EM_IMPORT macro? (I just found this, don't know what it actually does.

https://github.com/emscripten-core/emscripten/blob/main/system/include/emscripten/em_macros.h

...so somewhere in my header I could have a list of all Emscripten library functions like this:

EM_JS_DEPS('withStackSave', 'allocateUTF8OnStack', 'UTF8ToString')

Cheers!

-Floh.

[Sam Clegg]

Looks promising: https://github.com/emscripten-core/emscripten/pull/17854

This depends on a small llvm linker change but otherwise seems to work fine!

To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/d825b52e-71bd-497e-acad-06e4d3aabd70n%40googlegroups.com.

[Floh]

You are my hero :)

[Floh]

The new EM_JS_DEPS() macro works nicely :)

I have just one question: does the first param "tag" have any significance? In the em_macros.h header there's this line:

EM_JS_DEPS(mylib_dep, "$allocate,$stackSave");

In my STB-style headers I just use the base name of the header file name, for instance I use withStackSave in two different headers (sokol_audio.h and sokol_app.h), and use the macro like this:

in sokol_app.h: EM_JS_DEPS(sokol_audio, "$withStackSave");

in sokol_audio.h: EM_JS_DEPS(sokol_app, "$withStackSave");

But those names (sokol_audio, sokol_app) don't have any association with any lib those headers might be compiled into.

It seems to work though, so I'm no complaining, just wondering what those tags are there for :)

[Floh]

> in sokol_app.h: EM_JS_DEPS(sokol_audio, "$withStackSave");

> in sokol_audio.h: EM_JS_DEPS(sokol_app, "$withStackSave");

...erm, obvious typos, but you know what I mean :D

[Sam Clegg]

On Sat, Oct 15, 2022 at 9:12 AM Floh <flo...@gmail.com> wrote:

The new EM_JS_DEPS() macro works nicely :)

I have just one question: does the first param "tag" have any significance?

No, the first argument is just a unique name for the set of dependencies.  It only exists because the implementation of EM_JS_DEPS relies on exporting a symbol, and exports need to have unique names.  (Perhaps I could make that more clear in the comments).

To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/6772ee9c-b7c7-4522-8218-26b5f96a09d7n%40googlegroups.com.