Structured cloning

[Erik Bruchez]

All,

Browsers implement passing data to Web workers using "structured cloning":

http://www.w3.org/TR/html5/infrastructure.html#safe-passing-of-structured-data

So far I have serialized data between the main page and web workers
with uPickle. The result of the serialization is a String and things
just work. So that's great.

But there would be benefits to using the browser's structured cloning:

- remove the need for a Scala serialization library in some cases [1]
- possibly better performance as the cloning is implemented natively
by the browser [2]

In addition, structured cloning "preserves cycles and preserves the
identity of duplicate objects in graphs", which is not the case of
uPickle.

So I am wondering if there would be a way for at least Scala case
classes to be clonable via the browser's structured cloning algorithm,
instead of using serialization? It would be ok of course to require
that the classes be @JSExport or something else.

For the record, I tried passing a case class that way and the browser
threw a serialization error.

Would this be possible in theory?

-Erik

[1] I realize that one might be needed for Ajax calls anyway.

[2] This should be confirmed experimentally.

[Sébastien Doeraene]

Hi,

You still need a serialization-like library, but which creates JSON-like objects (structured-cloneable) instead of string/binary data. That's all.

You cannot expect to structure-clone Scala classes.

Cheers,

Sébastien

--
You received this message because you are subscribed to the Google Groups "Scala.js" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-js+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/scala-js/CAAc0PEUD2p%2Bj_TKs%2BRB%2Br_W6W1k_F8%3DXgMq6a6XDs8wFfCf-3A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Erik Bruchez]

Sébastien,

Thanks. Any quick hint as to why that is the case for Scala classes? I
guess one part is the use of the prototype chain to implement
inheritance?

I am not sure whether going through JSON-like objects will be faster,
but it might be more memory-efficient. There would be less string
concatenations, but more objects/arrays created. The Scala
serialization proper would not create a new JavaScript string for a
given Scala string.

I would hope that after that structured cloning could just pass
strings by reference (although I am not sure: the spec seems to say
they are copied, but passing immutable objects by reference might be
an acceptable optimization). In Chromium, dedicated workers are
implemented as threads in the same process, according to this
document:

https://docs.google.com/document/d/1NYMYf-_P_K2iPKlSGyv5ou_x0yL_4IRwt-DPOYqRb0s/edit#

In theory some kind of sharing by reference could be possible,
depending on whether whether two separate script environments can
share strings.

Another way to gain performance could be the use of a Transferable [1]
ArrayBuffer [2]: Scala serialization would write to that binary
buffer, which would then be transferred to the worker, and then
deserialized. This would save one data copy.

This said I do not currently have a performance issue with sending
data back and forth, but I am interested in doing things the right way
(tm).

-Erik

[1] https://html.spec.whatwg.org/multipage/infrastructure.html#transferable-objects

[2] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/ArrayBuffer

[Haoyi Li]

uPickle lets your readJs/writeJs Javascript structures the same way you read/write strings. 

To view this discussion on the web visit https://groups.google.com/d/msgid/scala-js/CAAc0PEXCrzKQ-iO1o6XyBNdVF-JNt0TmG-Je5g%2B%3Df5LTssU_Uw%40mail.gmail.com.

[Erik Bruchez]

Thanks, but unless I am mistaken, `writeJs` and `readJs` deal with
`upickle.Js.Value`, not Scala.js `js.Object`, which is what would be
needed, right?

-Erik

[Haoyi Li]

Ah, you're right. What you actually want is upickle.json.writeJs and upickle.json.readJs. Those deal with the Javascript objects (Anys)

Boopickle is supposed to write directly to bytebuffers, so that could be worth a shot too

[Erik Bruchez]

> Ah, you're right. What you actually want is upickle.json.writeJs and
> upickle.json.readJs. Those deal with the Javascript objects (Anys)

They do, but they take a Js.Value:

def writeJs(value: Js.Value): Any

What I would like I think is the same as:

def write[T: Writer](expr: T): String

but returning a js.Any (or Any which I can cast):

def write[T: Writer](expr: T): js.Any

otherwise I would need to write, I guess:

json.writeJs(writeJs(myObject))

which would create an intermediary Js.Value structure.

-Erik

[Haoyi Li]

Yeah, you're right, though it can save you the actual parsing to/from strings. The string version of these things also generates intermediary Js.Value data structures =P

[Erik Bruchez]

Got it! No big deal, at this point it's more about what the
theoretical optimum path would be anyway.

-Erik

[Sébastien Doeraene]

Hi,

Scala classes cannot be transferred directly because of the prototype chain, indeed.

Cheers,
Sébastien

[Erik Bruchez]

Thanks.

By the way if anybody has interest, I tracked down the Chromium
implementation of structured cloning:

https://chromium.googlesource.com/chromium/blink/+/master/Source/bindings/core/v8/ScriptValueSerializer.cpp

The code appears to serialize to/deserialize from a buffer. So it's
very much like any serialization/deserialization out there. If you are
not using a Transferable object, there will be a double
serialization/deserialization when exchanging data with Web workers:
one in Scala, and then a native one. At some point there might be 2
copies of the structure in Scala, 2 copies in JavaScript, and one
buffer with another copy in the middle!

-Erik

[Henry Story]

On Monday, 27 April 2015 23:05:35 UTC+2, Sébastien Doeraene wrote:

Hi,

You still need a serialization-like library, but which creates JSON-like objects (structured-cloneable) instead of string/binary data. That's all.

You cannot expect to structure-clone Scala classes.

What do you mean by a "Scala class" here? Is a scala.js class annotated with `@ScalaJSDefined` such a class? (if no, are there other annotations that indicate that a class can be cloned?) 

In my case I want to save classes to IndexDB and I can't use upickle as is because the WebCrypto library creates Opaque Objects , that cannot be transformed to strings ( or else they would not be opaque ). If I could tell if an object had as prototype the Object.prototype then we could have an extension to upickle to deal with those cases, or I could perhaps even extend scalajs-rx-idb to deal with those types of objects directly without going through upickle. It would allow one to create objects specially designed for scala-js that can be cloned easily. That would make sense in a lot of cases including transmission between web workers.

(I am really swimming in the dark here, so I may be completely off base)

[Sébastien Doeraene]

Hi,

A Scala class is any class that is not a JavaScript class. A JavaScript class is anything extending js.Any. So a Scala class is anything that does *not* extend js.Any.

If no existing pickling framework supports serializing to structurally cloneable objects, you will need your own (or a fork of an existing one).

Sébastien

To view this discussion on the web visit https://groups.google.com/d/msgid/scala-js/09301232-b8cb-4756-8d8f-979d5c631f0c%40googlegroups.com.

[Henry Story]

On Tuesday, 12 January 2016 13:12:05 UTC+1, Sébastien Doeraene wrote:

Hi,

A Scala class is any class that is not a JavaScript class. A JavaScript class is anything extending js.Any. So a Scala class is anything that does *not* extend js.Any.

If no existing pickling framework supports serializing to structurally cloneable objects, you will need your own (or a fork of an existing one).

Thanks, that's very helpful.

I tried upickle and left an bug report my understanding of the problem with the way it is written 

https://github.com/lihaoyi/upickle-pprint/issues/127

Does anyone know of a framework that is perhaps written to cover this use case? 

( Perhaps upickle will, but my feeling is that that will be a lot of work to rewrite the macros )