Cereal and pybind11

[Lee Kerley]

Hi,

Has anyone here ever used Cereal along with pybind11 to create bindings for custom objects, creating those objects in python and then using pybind11 bound functions to serialize those objects?

I've been fighting with this for a while, and have been hitting a bunch of different problems, sorts of feels like whack-a-mole.  Just wondering if anyone here has any experience to share, or thoughts on if i'm just struggling down a path thats not going to work.

My most recent problem is that when i try and serialize an object I get the error 

Trying to save an unregistered polymorphic type... which then talks about using CEREAL_REGISTER_TYPE and possibly having to use CEREAL_REGISTER_DYNAMIC_INIT.

I am already using CEREAL_REGISTER_TYPE.  All the custom classes and their serialization functions are in header files.

I am building the pybind11 module in to a shared library, that is linking against the library with my base functionality (I want to be able to use this library without the pybind11 bindings as well).

Anyone have any clues/thoughts/ideas?

Thanks

[Lee Kerley]

I made what i think is a good discovery last night.

pybind11 by default builds with -fvisibility=hidden.  When I force it to build my python module with 'default' instead all my cereal problems disappear.

Could someone here help me understand why.  I guess I need to explicity make visible some symbols for cereal to work.  

[Björn Dahlgren]

On Tuesday, 27 February 2018 19:43:42 UTC+1, Lee Kerley wrote:

I made what i think is a good discovery last night.

pybind11 by default builds with -fvisibility=hidden.  When I force it to build my python module with 'default' instead all my cereal problems disappear.

Could someone here help me understand why.  I guess I need to explicity make visible some symbols for cereal to work. 

Latest version of pybind supposedly works without setting -fvisibilty=hidden (they now set it using compiler pragmas for pybind11's headers if I've understood it correctly).
Anyhow: I've used cereal and pybind11 together for a while (tests passing so far).

[Lee Kerley]

Yes,

I've changed the visibility back to default and all is well - thank you.

Does anyone here have any information on which symbols i would need to make visible if i wanted to stay with visibility=hidden?

is there a way to get Cereal to explicitly make visible the symbols it needs?

thanks

[Lucas Watson]

Hi Lee,

Sorry to open a dead thread. Could you share how you were able to get pybind to work with cereal? I'm running into the same issues.

Thanks,

Lucas

[Lee Kerley]

I admit its not a perfect solution, but so far its working for me.  I had to restore the default visibility of the symbols.

-fvisibility=default

If i recall, pybind wants to hide them (atleast when using its cmake recipe), and Cereal needs them visible.  I'm sure there is a slightly less heavy hammer to solve this, but i haven't dug deep enough to find it yet.

hope this helps.