data sharing between NaCL and javascript?

[Jim]

We have some pretty big data (binary) which is loaded into NaCL and needed to be accessed frequently in javascript.
As I know, in the earlier versions of NaCL sdk, javascript code can invokes NaCL functions directly, while in later versions, only message posting can be used to communicate between NaCL and javascript.
Even though I can understand that the data sharing between different processes is nontrivial, the message mechanism is very limited and inefficient in exchanging big data.
Can anyone shed more lights on the future of NaCL development?
Thanks
Jim

[Erik Kay]

Hi Jim,

There are a few options for sharing large data between NaCl and JS.  The first is that you can store the data as a file and share a blob URL across processes.  This is great for large immutable data.

http://dev.w3.org/2006/webapi/FileAPI/#dfn-Blob

http://www.html5rocks.com/en/search?q=blob

Another option that's coming but isn't available yet is support for transferable data in postMessage.  We support it to web workers today, but not yet with NaCl.  It's on our to do list, but likely won't come until late this year.

http://updates.html5rocks.com/2011/12/Transferable-Objects-Lightning-Fast

Erik

Jim

--
You received this message because you are subscribed to the Google Groups "Native-Client-Discuss" group.
To view this discussion on the web visit https://groups.google.com/d/msg/native-client-discuss/-/xdXbAKoHRfoJ.
To post to this group, send email to native-cli...@googlegroups.com.
To unsubscribe from this group, send email to native-client-di...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/native-client-discuss?hl=en.

[David Michael]

On Mon, Aug 6, 2012 at 10:02 AM, Erik Kay <eri...@google.com> wrote:

Hi Jim,

There are a few options for sharing large data between NaCl and JS.  The first is that you can store the data as a file and share a blob URL across processes.  This is great for large immutable data.

http://dev.w3.org/2006/webapi/FileAPI/#dfn-Blob

http://www.html5rocks.com/en/search?q=blob

Another option that's coming but isn't available yet is support for transferable data in postMessage.  We support it to web workers today, but not yet with NaCl.  It's on our to do list, but likely won't come until late this year.

http://updates.html5rocks.com/2011/12/Transferable-Objects-Lightning-Fast

We provide support for ArrayBuffer in postMessage today, although the data do get copied currently. It's harder to support "transfer" for ArrayBuffer in NaCl since the transfer crosses a process boundary; we'll get there eventually. But it might be worth trying postMessage/ArrayBuffer anyway if it looks more convenient than Blob for you; the performance should be good enough for a lot of uses. You can see the "File Histogram" example for a quick example of sending binary data from JS to NaCl (the other direction works fine, too).

http://code.google.com/searchframe#OAMlx_jo-ck/src/native_client_sdk/src/examples/file_histogram/

[Jim]

Thanks Erik and David for quick response. It seems the blob url may be better since at least the javascript part do not need lots of memory to hold the big data (or keeping transferring multiple times).