Serializing a hash/map?

703 views
Skip to first unread message

Jeff Mitchell

unread,
Jan 7, 2014, 12:37:52 PM1/7/14
to capn...@googlegroups.com
I know of a custom serialization protocol in use somewhere that is specific to C++ and has functions for serializing various types, such as std::hash and std::map.

I'd love to suggest thinking about moving to a more language-agnostic solution (such as Cap'n Proto) but a coworker was lamenting the lack of support for hashes and maps. Although my understanding is that this can be worked around by creating a struct that has both the key and value types and making a list of those structs, the rationale I was given for why direct support is necessary is that data that is being serialized often needs to be in a hash/map on both the sending and receiving sides, and that using an intermediate form for the data (such as lists of structs) rather than directly reading and writing from hashes and maps causes memory copies that would otherwise be unnecessary.

I understand that there are not native C hashes and maps, but in general they are considered standard language features. Is the lack of C support the reason that these are not supported, or some other issue?

Thanks,
Jeff

Jeff Mitchell

unread,
Jan 7, 2014, 12:45:33 PM1/7/14
to capn...@googlegroups.com
Ah hah -- more Googling found this from Kenton: "That said, it is also on my wish list to have first-class maps in Cap'n Proto, and that probably implies sets as well.  These would have an actual impact on the implementation:  you would be able to perform sub-linear lookups by key."

So I guess it's coming, which is great!

Kenton Varda

unread,
Jan 7, 2014, 5:28:20 PM1/7/14
to Jeff Mitchell, capnproto
Indeed, this is something I want to have, it just hasn't reached the top of the priority queue.  You can, of course, roll your own using a list.


--
You received this message because you are subscribed to the Google Groups "Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+...@googlegroups.com.
Visit this group at http://groups.google.com/group/capnproto.

pgau...@sevone.com

unread,
Sep 9, 2015, 3:25:09 PM9/9/15
to Cap'n Proto, jeffrey....@gmail.com
Sorry for the necro-post, but is this still on the roadmap for Cap'n Proto?


SevOne, Inc. reserves the right to monitor the transmission of this message and to take corrective action against any misuse or abuse of its e-mail system or other components of its network.

The information contained in this e-mail may be confidential and/or legally privileged. It is intended solely for the addressee.  If the reader of this message is not an intended recipient, you are hereby notified that any unauthorized review, use, disclosure, dissemination, distribution, or copying of this communication, or any of its contents, is strictly prohibited and may be unlawful. If you have received this communication in error, please reply to the sender and destroy all copies of the message.  To contact us directly, send to postm...@sevone.com

Kenton Varda

unread,
Sep 10, 2015, 1:47:38 AM9/10/15
to pgau...@sevone.com, Cap'n Proto, jeffrey....@gmail.com
Well, we've made some progress in that we now support generics. That's probably the hard part here. The other thing we need is the ability to specify that a particular type should use a custom hand-written wrapper class in place of the usual generated classes, so that you can wrap a Map(K, V) in an appropriate map-like interface. This is mostly a code generator hack, I think.

With that said, I'm pretty swamped so I still couldn't predict exactly when I'll be able to make this happen. Happy to accept patches, though! :)

-Kenton
Reply all
Reply to author
Forward
0 new messages