transit-clj: decode/encode (nested) maps and vectors?

[Jeroen van Dijk]

I've been playing with transit and looked at the decoding to see how repeated elements are being aliased etc. So far I've seen that this aliasing is limited to (longer) keywords (and maybe strings, not sure). I think it would be an improvement if we had options to mark maps and vectors as repeated elements during encoding and have these aliases automatically be decoded. Below is the use case.

I have to send data were there is a lot repetition on a higher level than keywords and strings. So e.g. I have many vectors and (nested) maps that repeat itself such as [:aaa :bbb :ccc :ddd]. Transit would compress this via something like [~1 ~2 ~3 ~4], but would not alias the entire vector. It would be very hard for transit to guess that this should happen in an efficient way as it would incur quite some overhead. I wonder though what you think about marking these data structures before encoding and have transit alias this in a way that is automatically decoded by transit on read.

Right now I have to make something at the read side to make the whole circle work. Another way would be to change the data format itself and build some kind of index, but this involves changes at the reader side as well as it then has to read and transform the index.

Please let me know your thoughts.

Thanks,

Jeroen

[Alex Miller]

I think there are a lot of consequences to remembering and checking values of arbitrary size in the cache that are problematic. As far as I know, there are no plans to add this to transit.