Inter module communication

[Eugen-Andrei Gavriloaie]

Hello everyone,

I would like to create several modules each exposing a particular protocol kind (streaming in general). I would like to keep them as modules, as the name implies. Another written-in-stone requirement that I have is to achieve 0-copy for the transferred data. Because of that, I need to be able to do inter-module sharing of data.

For example, suppose nodejs ingests some streaming data in protocol format A over an input TCP connection. Now, several consumers are connecting to the same nodejs but using a different protocol format B over n separate TCP connections.

Format A is implemented in moduleA and format B is implemented in moduleB, both loaded under the same node instance. This producer/consumer paradigm should allow write-onec-read-many (effectively, 0-copy): receive the data, store it somewhere in a common place, and have all consumers read and send from there.

How do I transfer a C/C++ pointer from moduleA to moduleB without involving passing though upper JS layer? 

In other words, is there any InterModuleCommunication framework available for native nodejs modules?

Best regards,

Andrei

[mog...@syntheticsemantics.com]

Hi Andrei,

The short answer is "You cannot".  The closest you can get is copy-in/copy-out, zero-copy sharing would make garbage collection impossible because data would be shared between heaps.

[Self-Promotion Warning]

I have written a native C addon (GitHub, npm) for Node called Extended Memory Semantics (EMS) that implements the copy-in/copy-out object sharing model, and also manages atomic access to the data.  It was designed for a persistent data model more than streaming, but may still be useful to you depending on how WORM your data is.

            -Jace