wasm size increase with new backend compiler

153 views
Skip to first unread message

Rohit Saini

unread,
Feb 28, 2020, 2:12:44 AM2/28/20
to emscripten-discuss
Hi All,

Previously we were using 1.38.28 emscripten version. Recently we updated to latest llvm backend 1.39.7. But with new backend there is drastic increase in wasm size. Size of my side modules become almost double and size of my main module also increased from 2.7mb to almost 4mb. Firstly we are compiling to object files then we are linking them to make wasm. Below are the flags I am passing while compiling and linking. Is this size increase intentional or do I have to change compiling or linking flags someway.

Compiling flags for main module:
   -Oz -fPIC -s DISABLE_EXCEPTION_CATCHING=0  -Wno-builtin-macro-redefined -Wno-dollar-in-identifier-extension

Compiling flags for side module:
  -std=c++14 -fPIC

Linking flags for main module:
-O3 -s EVAL_CTORS=0 --closure 0 -s ALIASING_FUNCTION_POINTERS=0 -s ELIMINATE_DUPLICATE_FUNCTIONS=1 -s ELIMINATE_DUPLICATE_FUNCTIONS_PASSES=12 --llvm-lto 3 -s FORCE_FILESYSTEM=1 -s ERROR_ON_UNDEFINED_SYMBOLS=0 -s 'EXTRA_EXPORTED_RUNTIME_METHODS=["getTempRet0","setTempRet0", "cwrap"]' -s EXPORTED_FUNCTIONS=@T4ExportedFunctions.txt -s DISABLE_EXCEPTION_CATCHING=2 -s EXCEPTION_CATCHING_WHITELIST=@ExceptionCatchingWhitelist.txt

Linking flags for side module:
-s SIDE_MODULE=1  -s WASM=1 -Oz -s EVAL_CTORS=0 --closure 0 -s ALIASING_FUNCTION_POINTERS=0 -s ELIMINATE_DUPLICATE_FUNCTIONS=1 -s ELIMINATE_DUPLICATE_FUNCTIONS_PASSES=12 --llvm-lto 3 -s ERROR_ON_UNDEFINED_SYMBOLS=0 

Gabriel Cuvillier

unread,
Feb 28, 2020, 4:07:11 AM2/28/20
to emscripte...@googlegroups.com

Hi,

Try to remove --llvm-lto 3, as it might increase binary size significantly when combined with -O2 or -O3 (and I am not sure it is very usefull without using -DWASM_OBJECT_FILES=0)

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/5f30a113-7203-4eba-9426-9f626c51ec26%40googlegroups.com.

Floh

unread,
Feb 28, 2020, 5:14:03 AM2/28/20
to emscripten-discuss
Whoops, thanks for making me aware of WASM_OBJECT_FILES being 1 by default... I've been building with LTO all the time (since the asm.js days) and was expecting that the WASM backend would behave identically there.

-Floh.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.

Gabriel Cuvillier

unread,
Feb 28, 2020, 5:32:09 AM2/28/20
to emscripte...@googlegroups.com

Glad to!  

By the way, I don't find that LTO brings any performance improvements compared to a non-LTO builds (on latests Emscripten backend). Maybe +1% at most (more probably benchmark noise to me), but sometime more than 10% worse, and at the expense of a significant link time increase and significant binary size increase!  

So I am not sure this is really an interesting option anymore...

To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/e8a9f44f-a8f5-46f2-b440-e2356f285d28%40googlegroups.com.

Floh

unread,
Feb 28, 2020, 6:16:39 AM2/28/20
to emscripten-discuss
When I last tested I generally saw quite noticeable improvements for inlined code (which LTO does across compilation units). Most of my code is straight C without inline functions in headers. I guess this scenario benefits more from LTO than typical C++ code where usually a lot of implementation code sits in inline/template functions (at the cost of increasing compile time for each compilation unit, versus increasing link-time once for LTO).

I think it depends extremely on the code base at hand whether LTO is useful or not :)

Floh

unread,
Feb 28, 2020, 6:49:37 AM2/28/20
to emscripten-discuss
...hm ok I did a quick check on my 8-bit emulators, and I have to agree. I'm seeing about 10% size agree for the generated WASM (which I'd be willing to eat if the performance would improve too), but performance isn't any noticeably better. I guess the WASM runtimes either do the inlining themselves, or can the function call overhead doesn't matter as much as it used to.

Gabriel Cuvillier

unread,
Feb 28, 2020, 6:52:12 AM2/28/20
to emscripte...@googlegroups.com

Yes, I guess you are right. It is is probably very code-dependent, and for raw C code LTO have a lot more sense than for complex C++ code with crazy template magic.

To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/a753d986-3f0a-45e1-8acd-3bb6a363fcd6%40googlegroups.com.

Gabriel Cuvillier

unread,
Feb 28, 2020, 6:57:42 AM2/28/20
to emscripte...@googlegroups.com

Well, to me LTO used to be better. But for some reason, since the switch to upstream LLVM backend, it is no more very interesting.

I have been surprised by this, and decided to disable LTO on all my projects since the switch.  But maybe there's a bug lurking somewhere: LTO is simply not correctly done anymore (?)  

@Emscripten team: any thoughts on this possibility ? or is it simply because the Wasm backend now does things extremely well ?

To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/38cfb2aa-6b9c-4dfb-a978-1c9c520d3693%40googlegroups.com.

Alon Zakai

unread,
Feb 28, 2020, 4:57:58 PM2/28/20
to emscripte...@googlegroups.com
Rohit: LTO might be the issue here as others mentioned. Specifically, with the wasm backend, object files are wasm by default. To get LTO you must build the source files with -flto, so that they contain bitcode. (wasm-ld will link both wasm and bitcode files, doing LTO where it can, but not warning if there are no actual bitcode files to LTO with.)

Note that the LLVM versions are quite different, so LTO may behave differently in the new backend. Might be worth trying --llvm-lto 1 instead of 3.

Also, might be worth doing -Os (not -O3) during link.

Gabriel: That does seem odd. Are you building source files with -flto? Worth checking if the object files are bitcode, as maybe the build system somehow didn't pass the flag through?

With that said, I haven't seen great results from LTO on the new backend either. So yeah, maybe something is wrong? I opened https://github.com/emscripten-core/emscripten/issues/10603 for more discussion.


To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/e8a9f44f-a8f5-46f2-b440-e2356f285d28%40googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/38cfb2aa-6b9c-4dfb-a978-1c9c520d3693%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/2e3a6f1d-869e-06f5-5341-a031aaa8fc09%40gmail.com.

Rohit Saini

unread,
Feb 29, 2020, 12:45:09 PM2/29/20
to emscripten-discuss
Hi Alon,

I tried removing -llvm-lto 3 and used -Os while linking, but still there is no change in size. Trying --llvm-lto 1 didn't help either.

Regards,
Rohit Saini
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.

Alon Zakai

unread,
Mar 2, 2020, 4:45:39 PM3/2/20
to emscripte...@googlegroups.com
Rohit: Can you perhaps provide the wasm files before and after, built with --profiling? (that flag ensures functions have names) perhaps by comparing them we can find what's gone wrong.

To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/8d6187d7-019e-4191-9d10-ec5e655980f3%40googlegroups.com.
Message has been deleted

Alon Zakai

unread,
Mar 5, 2020, 6:03:03 PM3/5/20
to emscripte...@googlegroups.com
Thanks Rohit!

One big difference here is the first has 130 exports while the second has 1,113.

For example, JP2KTileComponentSetEncodeOptions (C function) and _ZN10IJP2KImage16CreateCodeStreamEv (C++ function) are present in the new one, but not the old one.

Are those in the list of exported functions?

From the command above it looks like you're using SIDE_MODULE=1, and not =2, to build these. If so then it won't DCE away things not in the list of exports you provide. I'm not sure offhand why that would be different between the backends though.

On Thu, Mar 5, 2020 at 4:04 AM Rohit Saini <rohitsa...@gmail.com> wrote:
Hi Alon,

Here are the wasm files of one of my side module before (libjp2k.wasm) and after (libjp2k_new.wasm) the compiler update. Please tell if you need more info from my side.

Regards,
To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/144edb11-1b79-49ef-98e5-e220357eb713%40googlegroups.com.

Rohit Saini

unread,
Mar 6, 2020, 12:42:40 PM3/6/20
to emscripten-discuss
Hi Alon,

Thanks a lot. Passing SIDE_MODULE = 2 decreased the size of wasm a lot. But one thing I want to confirm, does something related to this changed with latest backend, because initially I was using SIDE_MODULE = 1, and size of wasm files was okk. 

Moreover I want to ask one more thing. As most of the things needs to be passed at compile time with new backend. Do I need to pass EXPORTED_FUNCTIONS=[_someFunction] at compile time too.

Regards,
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.

Alon Zakai

unread,
Mar 6, 2020, 4:37:45 PM3/6/20
to emscripte...@googlegroups.com
On Fri, Mar 6, 2020 at 9:42 AM Rohit Saini <rohitsa...@gmail.com> wrote:
Hi Alon,

Thanks a lot. Passing SIDE_MODULE = 2 decreased the size of wasm a lot. But one thing I want to confirm, does something related to this changed with latest backend, because initially I was using SIDE_MODULE = 1, and size of wasm files was okk. 

That should not change with the backend, and I verified on a small testcase now that fastcomp and upstream do the same thing.

However, if you were using fastcomp on an older version (I tested latest), then there were bugs that have been fixed. Perhaps you used an old version where the dce happened anyhow, even though it should not have. I seem to recall us fixing such a bug.


Moreover I want to ask one more thing. As most of the things needs to be passed at compile time with new backend. Do I need to pass EXPORTED_FUNCTIONS=[_someFunction] at compile time too.


EXPORTED_FUNCTIONS only matters at link time.

Regards,
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/dfb54b90-be3a-4b97-b15b-c436fd7ad6b5%40googlegroups.com.

Rohit Saini

unread,
Mar 26, 2020, 12:56:07 PM3/26/20
to emscripten-discuss
Hi Alon,

Sorry for waking up this thread after some time again.

As I mentioned, by passing SIDE_MODULE=2, size of my side modules came back to same as with previous compiler. But in my main module I was already using MAIN_MODULE=2, but still my size increased a lot with new backend compiler. You did a symbol analysis on one of my side module and told me that it has lot of export functions. Can you tell me the way, how you did it, so that I can do this analysis on my main module at my end. 

Regards,
Rohit Saini
Regards,
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.

Alon Zakai

unread,
Mar 26, 2020, 2:17:00 PM3/26/20
to emscripte...@googlegroups.com
You can use wasm-dis (part of the emsdk download, under /upstream/bin or /fastcomp/bin) to get the text form of the wasm file. Then the exports look like

(export ..)

and they are all together, near the beginning of the file.


Regards,
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/880f1ce8-c8fe-44b8-bb9d-c040f24731e6%40googlegroups.com.

Rohit Saini

unread,
Apr 5, 2020, 5:42:28 AM4/5/20
to emscripten-discuss
Hi Alon,

I used wasm-dis to disassemble my MAIN_MODULE, and issue is same as it was with my side_modules. Main_module built with compiler 1.39.7 has lot of extra imports and exports. I am using -s EXPORTED_FUNCTIONS to pass list of exported functions and also passing MAIN_MODULE = 2 flag at link time. Passing SIDE_MODULE=2 flag at linked time in my side modules solved the size problem and removed extra exports and imports. But it's not working with main module. What else could be the reasons for having these lot of export functions in main module. I am using compiler 1.39.7.

Regards,
Rohit Saini
Regards,
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.

Alon Zakai

unread,
Apr 5, 2020, 3:39:56 PM4/5/20
to emscripte...@googlegroups.com
MAIN_MODULE=2 should only be exporting what is in the EXPORTED_FUNCTIONS list (plus some dynCalls and other system things, but few of them).

If you see something else, that might be a bug we don't know about, please provide a testcase (your entire codebase would be ok if you can, or if not, please make a standalone example).

Regards,
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/2a346383-0886-4af3-a043-d899ea46c183%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages