[racket] serialization of math/array

[Berthold Bäuml]

Hi,

will there be serialization support for math/array and math/matrix in the near future? As far as I understand in principle it should be possible at leas in a straight forward way as there are already the routines array->list and list->array.

Best,
Berthold



____________________
Racket Users list:
http://lists.racket-lang.org/users

[Neil Toronto]

On 07/16/2014 10:25 AM, Berthold Bäuml wrote:
> Hi,
>
> will there be serialization support for math/array and math/matrix in the near future? As far as I understand in principle it should be possible at leas in a straight forward way as there are already the routines array->list and list->array.

Sorry it's taken so long to reply. Part of the problem is that
`racket/serialize` isn't typed:

#lang typed/racket

> (require racket/serialize)
> serialize
Type Checker: missing type for identifier;
consider using `require/typed' to import it
identifier: serialize
from module: racket/serialize in: serialize

This and the fact that the array struct types are declared in Typed
Racket makes adding serialization tricky at best. Also, it would only
work in untyped Racket.

Generally, deserializing is hard to make type-safe, and nobody has taken
it up yet for Typed Racket. Occurrence typing should help, but would
require `deserialize` to take a predicate argument (like the second
argument to `list*->array`), which it currently doesn't do.

Neil ⊥

[Matthias Felleisen]

Lucky us. I often leave the I/O parts of my programs untyped (I write either highly imprecise unchecked signatures or I don't provide types).

[Sam Tobin-Hochstadt]

Unfortunately, I think that strategy would incur substantial overhead
for things like serialization of large arrays.

Sam

[Matthias Felleisen]

Will these costs dominate the cost of I/O here?

[Sam Tobin-Hochstadt]

That really depends what the contracts are, and if they're first-order.

Sam

On Wed, Jul 23, 2014 at 11:45 AM, Matthias Felleisen

[Matthias Felleisen]

Let's assume we serialize arrays of numbers, which is what I assume the background to the question is. In that case, the answer isn't all that obvious to me.

[Sam Tobin-Hochstadt]

De-serializing an array of numbers and then passing it to typed code
would produce a wrapper, not a first-order check, and so would be very
expensive.

What you want is something that can tell that the untyped reference is
dead after value is passed to typed code, so that a first-order check
can be used. This would require something new from Racket in the form
of a revocable reference.

Sam

On Wed, Jul 23, 2014 at 11:52 AM, Matthias Felleisen

[Matthias Felleisen]

OK. I am beginning to wonder whether we're missing something in the contract world.

[Berthold Bäuml]

> De-serializing an array of numbers and then passing it to typed code
> would produce a wrapper, not a first-order check, and so would be very
> expensive.

Even when first-order checks would be possible the cost of such a check would be significant in our application. We wan to use serialization/deserialization to send data between programs -- when running on the same computer the I/O cost (local sockets) would be almost negligible.

Would it be possible to have a typed serialization in the near future?

> What you want is something that can tell that the untyped reference is
> dead after value is passed to typed code, so that a first-order check
> can be used. This would require something new from Racket in the form
> of a revocable reference.

Does this also hold for immutable arrays? In this case a first-order check should suffice in principle.

Berthold

--
-----------------------------------------------------------------------
Berthold Bäuml -- Head of Autonomous Learning Robots Lab
DLR, Robotics and Mechatronics Center (RMC)
Münchner Str. 20, D-82234 Wessling
Phone +49 8153 282489
http://www.robotic.de/Berthold.Baeuml

[Sam Tobin-Hochstadt]

On Thu, Jul 24, 2014 at 7:21 AM, Berthold Bäuml <berthol...@dlr.de> wrote:
>> De-serializing an array of numbers and then passing it to typed code
>> would produce a wrapper, not a first-order check, and so would be very
>> expensive.
>
> Even when first-order checks would be possible the cost of such a check would be significant in our application. We wan to use serialization/deserialization to send data between programs -- when running on the same computer the I/O cost (local sockets) would be almost negligible.
>
> Would it be possible to have a typed serialization in the near future?

I think the Racket `serialize` API is likely to be higher-cost than
that, even without considering Typed Racket.

>> What you want is something that can tell that the untyped reference is
>> dead after value is passed to typed code, so that a first-order check
>> can be used. This would require something new from Racket in the form
>> of a revocable reference.
>
> Does this also hold for immutable arrays? In this case a first-order check should suffice in principle.

Yes, for immutable arrays represented as flat data (not functions) the
checks would not require wrappers.

Sam

[Berthol...@dlr.de]

So, there is currently no way make math/array serializable at all -- even when it would be ok to get an inefficient wrapped array as a result from deserialize? As far as understand, what hinders me from writing a simple typed wrapper for serialize/deserialize is that a typed struct does not support the #:property prop:serializable (simply ignored) at all and hence serialize complains about a non serializable value. Is this correct and would it be possible to add this property options with little effort?

Berthold

[Sam Tobin-Hochstadt]

Right now, there's not much support for struct properties at all in
Typed Racket. So this would probably be a non-trivial amount of work.

Sam