[Dynamic linking] New and delete operators not exported or inaccessible in SIDE_MODULE

[Tapan Anand]

Hi,

I was trying to use dynamic linking using dlopen with emscripten compiler version 1.38.28 and encountered the following problem:

1. When I try to use vector in the SIDE_MODULE, it can't seem to find the new and delete operators in the Module variable.

2. The command I use to build main and side module are:

 

em++ doubler.cpp -o doubler.wasm -s SIDE_MODULE=1 -O3 -s "EXPORTED_FUNCTIONS=['_doubler']"

EMCC_FORCE_STDLIBS=1 em++ main.cpp -O3 -s WASM=1 -s MAIN_MODULE=2 -o dynlink.js -s "EXPORTED_FUNCTIONS=['_main', '_clock', '_tripler', '_printf', '_puts']"  -s "DEFAULT_LIBRARY_FUNCS_TO_INCLUDE=['clock']" -s FORCE_FILESYSTEM=1 --pre-js prejs.js

The prejs file here is there to use FS.createLazyFile to cause lazy loading of the side module.

If I change the exported function list in main_module to `['_main', '_clock', '_tripler', '__Znwm', '__ZdlPv', '_printf', '_puts']` It starts to work. Notice that I had to add the mangled symbols of new and delete operator to export list.

Also, once I did this, I didn't get any missing symbols related to the vector class itself.

So, I have two questions:

1. Do we have to export C++ runtime functions like new, delete, or even c runtime methods like printf, puts for code to work? Is that expected?

2. Since there were no errors for vector related symbols, does that mean that vector is linked with the side_module? If I also use vector in main module, will there be effectively two copies of vector? In both side and main module?

Thanks in advance!

[Alon Zakai]

This is probably because of MAIN_MODULE=2, which means you need to carefully pick what is kept alive for the other modules. Does this work with =1? If so then that's the issue. The test suite has a bunch of examples for mode 2, that might help.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/5f2ad4c5-fc8e-45a3-afab-e340f91f09f5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Tapan Anand]

Thanks for the quick reply Alon!

The C library functions puts and printf work with MAIN_MODULE=1 but the new and delete operator errors still persist if I don't manually export them.

Regarding - "which means you need to carefully pick what is kept alive for the other modules", so you mean we have to add these mangled names in the export list if we use MAIN_MODULE=2 and there is no other way to avoid this? MAIN_MODULE=1 creates a huge .js and .wasm file and hence ends up increasing the MAIN_MODULE size while the purpose of dynamic linking for me is to reduce its size.

Also, I got to know about new and delete via runtime errors, but what if I some part of code is not encountered while testing (and hence no runtime error occurs)? That may cause unknown crashes when the user uses the app for example, is that expected? Or there is a way to know unresolved symbols in advance using some technique?

On Thursday, 30 May 2019 00:00:45 UTC+5:30, Alon Zakai wrote:

This is probably because of MAIN_MODULE=2, which means you need to carefully pick what is kept alive for the other modules. Does this work with =1? If so then that's the issue. The test suite has a bunch of examples for mode 2, that might help.

On Wed, May 29, 2019 at 3:40 AM Tapan Anand <anandtap...@gmail.com> wrote:

Hi,

I was trying to use dynamic linking using dlopen with emscripten compiler version 1.38.28 and encountered the following problem:

1. When I try to use vector in the SIDE_MODULE, it can't seem to find the new and delete operators in the Module variable.

2. The command I use to build main and side module are:

 

em++ doubler.cpp -o doubler.wasm -s SIDE_MODULE=1 -O3 -s "EXPORTED_FUNCTIONS=['_doubler']"

EMCC_FORCE_STDLIBS=1 em++ main.cpp -O3 -s WASM=1 -s MAIN_MODULE=2 -o dynlink.js -s "EXPORTED_FUNCTIONS=['_main', '_clock', '_tripler', '_printf', '_puts']"  -s "DEFAULT_LIBRARY_FUNCS_TO_INCLUDE=['clock']" -s FORCE_FILESYSTEM=1 --pre-js prejs.js

The prejs file here is there to use FS.createLazyFile to cause lazy loading of the side module.

If I change the exported function list in main_module to `['_main', '_clock', '_tripler', '__Znwm', '__ZdlPv', '_printf', '_puts']` It starts to work. Notice that I had to add the mangled symbols of new and delete operator to export list.

Also, once I did this, I didn't get any missing symbols related to the vector class itself.

So, I have two questions:

1. Do we have to export C++ runtime functions like new, delete, or even c runtime methods like printf, puts for code to work? Is that expected?

2. Since there were no errors for vector related symbols, does that mean that vector is linked with the side_module? If I also use vector in main module, will there be effectively two copies of vector? In both side and main module?

Thanks in advance!

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.

[Alon Zakai]

Yes, in general the optimizer may remove anything it does not see is used, in normal compilation, and in main module/side module mode 2. It's possible some things stay alive anyhow since they are used by other things. But if nothing else uses it, you must add it to the exports yourself, or the optimizer may remove it.

To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/5f2ad4c5-fc8e-45a3-afab-e340f91f09f5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/058436c9-50d5-4a13-a499-cbca19214066%40googlegroups.com.

[Tapan Anand]

So does that mean its possible to miss some missing symbols as well since runtime errors seems to be the only way to determine them?

To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/5f2ad4c5-fc8e-45a3-afab-e340f91f09f5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.

[Alon Zakai]

Yes. Mode 2 requires careful work to provide the right symbols. The compiler can't tell what other modules that will be linked at runtime will need.

To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/5f2ad4c5-fc8e-45a3-afab-e340f91f09f5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/058436c9-50d5-4a13-a499-cbca19214066%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/f5c8beed-0aee-4a0f-8748-17321afb48dd%40googlegroups.com.

[Sam Clegg]

On Fri, May 31, 2019 at 1:25 AM Tapan Anand <anandtap...@gmail.com> wrote:
>
> So does that mean its possible to miss some missing symbols as well since runtime errors seems to be the only way to determine them?

Yes, they are currently runtime errors, because dynamic linking
happens at runtime. Are you seeing them show at at startup or after
main runs? We should at least be able to ensure these are load time
errors.

Ideally we would build a system where the MAIN_MODULE and the
SIDE_MODULE can be built in such a way that can know that list of
symbols needed by the other modules. However because we have a lot of
circular dependencies this could prove a little tricky.

>>>>> To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

>>>>> To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/5f2ad4c5-fc8e-45a3-afab-e340f91f09f5%40googlegroups.com.
>>>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.

>>> To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

>>> To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/058436c9-50d5-4a13-a499-cbca19214066%40googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>

> --
> You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/f5c8beed-0aee-4a0f-8748-17321afb48dd%40googlegroups.com.

[Tapan Anand]

Hi Sam,

   I see that error during runtime, when that branch of code is hit. Is there a way for them to atleast show up at load time so I can be sure that user wouldn't encounter any new errors in production?

>>>>> To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.

>>>>> To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/5f2ad4c5-fc8e-45a3-afab-e340f91f09f5%40googlegroups.com.
>>>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.

>>> To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.

>>> To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/058436c9-50d5-4a13-a499-cbca19214066%40googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.

[Sam Clegg]

On Sat, Jun 29, 2019 at 8:31 AM Tapan Anand <anandtap...@gmail.com> wrote:
>
> Hi Sam,
> I see that error during runtime, when that branch of code is hit. Is there a way for them to atleast show up at load time so I can be sure that user wouldn't encounter any new errors in production?
>

There is no way currently but it could be added. Do you want to open
an issue requesting this feature?

>> >>>>> To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

>> >>>>> To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/5f2ad4c5-fc8e-45a3-afab-e340f91f09f5%40googlegroups.com.
>> >>>>> For more options, visit https://groups.google.com/d/optout.
>> >>>
>> >>> --
>> >>> You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.

>> >>> To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

>> >>> To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/058436c9-50d5-4a13-a499-cbca19214066%40googlegroups.com.
>> >>> For more options, visit https://groups.google.com/d/optout.
>> >
>> > --
>> > You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.

>> > To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

>> > To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/f5c8beed-0aee-4a0f-8748-17321afb48dd%40googlegroups.com.
>> > For more options, visit https://groups.google.com/d/optout.
>

> --
> You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/e30016f3-0de3-4d3f-9c35-538b4f58e6db%40googlegroups.com.

[Tapan Anand]

Yeah I would like to. How do I go about doing that? Just create a github issue on emscripten repository? I think this is an important feature that is indispensable for using dynamic linking in production.

>> >>>>> To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.

>> >>>>> To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/5f2ad4c5-fc8e-45a3-afab-e340f91f09f5%40googlegroups.com.
>> >>>>> For more options, visit https://groups.google.com/d/optout.
>> >>>
>> >>> --
>> >>> You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.

>> >>> To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.

>> >>> To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/058436c9-50d5-4a13-a499-cbca19214066%40googlegroups.com.
>> >>> For more options, visit https://groups.google.com/d/optout.
>> >
>> > --
>> > You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.

>> > To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.

>> > To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/f5c8beed-0aee-4a0f-8748-17321afb48dd%40googlegroups.com.
>> > For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.

[Thomas Lively]

Yes, opening a GitHub issue on the Emscripten repository would be great. 

To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/02d6fbff-1de3-45d9-b119-a04231064638%40googlegroups.com.