Multiple Emscripten modules sharing one Filesystem

[Łukasz Adamczak]

I'm looking to implement a web application which combines 2 Emscripten modules.

Think an IDE with a text editor, an Assembler (module 1) and an Emulator (module 2).

https://dl.dropboxusercontent.com/u/118061/neside/neside-flow.png

The general flow I'm hoping to achieve:

1. Content of a <textarea> is written as a text file to the FS (manually written JS).

2. Emscripten module 1 ("Assembler"):

  - reads the text file from the FS

  - writes a bin file to the FS.

3. Emscripten module 2 ("Emulator"):

  - reads the bin file from the FS and acts on it

I have successfully implemented both modules in isolation.

I can also use MODULARIZE=1 and EXPORT_NAME to include 2 modules in one page.

However in this setup, I am unable to access the filesystem and - much less - share it across modules.

I could use some help with the following:

1. How do I access "FS" when I use MODULARIZE=1? FS is not public and I can't #readFile or #writeFile.

2. Can the FS be shared across separate Emscripten modules? Or will I need to write some glue code to:

  - take data out of Assembler's FS

  - put the data into Emulator's FS

Thanks!

Łukasz

[Alon Zakai]

I don't think the FS was designed for that purpose. But I'm not sure if there is a reason stopping it from working that way. You would need to somehow grab the FS object from one module and make it accessible to the other, so they share one FS (probably need some new Emscripten options for that). If you did that, things might work - at least I can't think of a reason for them not to ;)

Another option is to write glue code, as you said, to manually move around data between two FSes.

A third possible route is to use BrowserFS ( https://github.com/jvilk/BrowserFS ). It can plug into Emscripten filesystems, and I'm not sure but I think might be able to handle multiple external sources.

- Alon

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Camil Staps]

I found this discussion wanting to implement something similar. I now have something working which uses separate filesystems for each module (could not get them to use the same FS object / BrowserFS to sync them). In each I mount an IDBFS, and I make sure to call syncfs at the appropriate places so that the contents are synced. This is far from ideal (may be too slow for some use cases; probably duplicates memory), but works for me now.

Still, is there any reason why it would not be possible to pass the FS of another module in the MODULARIZEd constructor for a new module (perhaps all that is needed is `var FS=FS||...` instead of `var FS=...`?)? If it were possible, that would be a much neater solution, but perhaps I'm missing a reason why this is not feasible.

Camil

To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.

[Fernando Bitti Loureiro]

Hi Camil,

I'm another Emscripten user trying to do the same. I don't know the answer to your question but I just wanted you to know you're not the only one looking for a solution.

Fernando

[Fernando Bitti Loureiro]

Hey Camil,

I finally managed to make this work. 2 totally independent WASM side modules, compiled from different projects, can access the same filesystem without using JS to copy data to and from them.

It just uses regular Emscripten side modules. In the end, I didn't have to hack the FS or pass a module's FS to the other in order for them to share it. The way I see it, both modules just request an FS and Emscripten assures they use the same one.

This is an embryonic version of the project using MEMFS: https://web-repaq.glitch.me/

Regards,

Fernando

[Sam Clegg]

If you use SIDE_MODULEs then you are really creating just one big program..  So the module will share more than just the filesystem, they will also share memory, table, libc.  If that works for you then thats great obviously!

To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/7e1cc625-21d9-4838-850b-ef90ce518bcfo%40googlegroups.com.

[Fernando Bitti Loureiro]

Hi Sam,

This works for me now, but now that you say it, it makes me realize this solution may not be ideal for some of my future projects.

So, let me learn from you and the community: is it possible to make the modules share just the filesystem and nothing else? Would you share how or point me in the right direction?

Thanks!

On Monday, 6 July 2020 02:40:54 UTC+2, Sam Clegg wrote:

If you use SIDE_MODULEs then you are really creating just one big program..  So the module will share more than just the filesystem, they will also share memory, table, libc.  If that works for you then thats great obviously!

To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.

[Ivan Serdyuk]

Lukasz,

could you tell if StorageOS could be of help here?

Ivan

[Sam Clegg]

On Mon, Jul 6, 2020 at 12:46 AM Fernando Bitti Loureiro <fernand...@gmail.com> wrote:

Hi Sam,

This works for me now, but now that you say it, it makes me realize this solution may not be ideal for some of my future projects.

So, let me learn from you and the community: is it possible to make the modules share just the filesystem and nothing else? Would you share how or point me in the right direction?

I believe this would be a fairly large change.   Currently non if JS code generated by emscripten is designed to be shared between modules.

 

To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/7e1cc625-21d9-4838-850b-ef90ce518bcfo%40googlegroups.com.

--

You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/01f4d0b2-94f6-4cf6-bd70-93826c879cf5o%40googlegroups.com.