Intended that transients do not support metadata?

[Andy Fingerhut]

When you create a transient set, map, or vector from a persistent one with metadata, the transient has no metadata. This is true for at least 1.3.0 and the latest 1.4.0 snapshot, and probably 1.2.x as well. See the sample REPL session below.

Is this intended behavior, perhaps due to some trickiness in implementing it correctly?

Or perhaps it is just something not implemented yet?

It seems that this behavior is the underlying reason for CLJ-916:

http://dev.clojure.org/jira/browse/CLJ-916

It also seems like there are several opportunities in the Clojure code for using transients where they are not today, but that doing so would cause metadata to be lost in those cases, too, unless this behavior were changed.

Thanks,
Andy

Clojure 1.3.0
user=> (def x ^{:foo 1 :bar "x"} [1 2 3])
#'user/x
user=> (meta x)
{:foo 1, :bar "x"}
user=> x
[1 2 3]
user=> (def y (transient x))
#'user/y
user=> y
#<TransientVector clojure.lang.PersistentVector$TransientVector@4c272961>
user=> (meta y)
nil
user=> (def z (persistent! y))
#'user/z
user=> (meta z)
nil
user=> z
[1 2 3]

[Alexander Taggart]

This might be the cause of a problem I just ran into:

=> (def v (with-meta [] {:foo :bar}))

#'user/v

=> (meta v)

{:foo :bar}

=> (into v [1 2 3])

[1 2 3]

=> (meta *1)

nil

Looking at the source for into show it relies on transients.  The above happens in 1.3.0 and 1.4.0-beta1.

[Alexander Taggart]

Already in JIRA: http://dev.clojure.org/jira/browse/CLJ-916

On Monday, 5 March 2012 00:28:58 UTC-8, Andy Fingerhut wrote:

[Andy Fingerhut]

Yes, "into" losing metadata is exactly the reason that CLJ-916 was filed, and was the motivation for my message.  I even mentioned it there :-)

A quick fix for CLJ-916 is to change "into" so it doesn't use transients.  That would be correct, but lower performance.

To my mind, the more satisfying improvement is to change transients so that they preserve metadata.  Then not only can "into" remain as is, but several other core functions that do not use transients today could be sped up by using transients, and still preserve metadata.

It seems as long as the metadata attached to a collection cannot itself be a transient, i.e. the metadata must be a persistent map, then adding preservation of metadata to the following functions seems like it might be straightforward:

transient

persistent!

with-meta

vary-meta

What I don't know is whether someone else thought of a subtle reason why the straightforward implementation would be incorrect in some way, and that is why it isn't there.

If it is simply a matter of "not done yet", then I might take a crack at it.

Are there any other places that would need to be enhanced other than the 4 functions above, and the implementations of the transient collections?

Andy

--
You received this message because you are subscribed to the Google Groups "Clojure Dev" group.
To view this discussion on the web visit https://groups.google.com/d/msg/clojure-dev/-/R-VvIv0Q0QsJ.
To post to this group, send email to cloju...@googlegroups.com.
To unsubscribe from this group, send email to clojure-dev...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/clojure-dev?hl=en.

[Alan Malloy]

My assumption would be that transients don't support metadata because
it makes them (a) larger, and (b) require more constructor-arguments
to create. Transients are intended for maximum speed, and managing
metadata would impact their speed even in the very common nil-metadata
case. I'm not sure which way is right to make the trade-off, but since
putting metadata on objects which are intended to be fast and short-
lived seems like it might not be right. If I submitted a patch for
this, I'd be changing `into` to tack the metadata back on at the end,
not fixing transients to always carry metadata.

> > To view this discussion on the web visithttps://groups.google.com/d/msg/clojure-dev/-/R-VvIv0Q0QsJ.

[Andy Fingerhut]

Understood. The memory cost should be one extra 32- or 64-bit pointer to point at the metadata. In the common case that reference would only be copied once when the transient was created, and once again when the persistent object was created from the transient. Even though there are some cases where transients cannot be "bashed in place", the reason it so often happens to work is that operations like conj! assoc! etc usually return the same transient collection that was passed to them.

Andy

[Sean Devlin]

I think that the problem here is that into looses metadata, and that
transients supporting metadata is one solution. Would another route
be a wrapper layer (a decorator?) in into that preserves the metadata
of the input and attaches it to the output? This would solve the into
problem while keeping the performance of transients.

Sean

[Christophe Grand]

I think it's just an oversight. The metadata map is referenced by the root transient object which rarely change during a "transientction" so it would incur nearly no cost of having transients to remember the metadata of their source.

Christophe

--
Professional: http://cgrand.net/ (fr)
On Clojure: http://clj-me.cgrand.net/ (en)

[Paul Stadig]

Agreed. In the base case, just copy the metadata map into the persistent collection at the end.

But I'm also not seeing why it would be a performance penalty to be able to bash the metadata on a transient collection.


Paul

[David Nolen]

On Wed, Mar 14, 2012 at 10:21 AM, Paul Stadig <pa...@stadig.name> wrote:

Agreed. In the base case, just copy the metadata map into the persistent collection at the end.

But I'm also not seeing why it would be a performance penalty to be able to bash the metadata on a transient collection.


Paul

If you're updating a non-transient metadata map, I imagine that will overcome any performance benefits you'd get from transients.

David

[Paul Stadig]

Right, I was assuming the metadata map would also be transient? And in the case that there is no metadata on a transient collection is there any penalty for having the code there to deal with metadata? Or would it be pay-for-what-you-use?


Paul

[David Nolen]

It seems strange to me that the metadata would also become transient.

David