[Boost-users] [serialization] Polymorphic serialization with polymorphic archives problem...
Bogdan
I am unable to use the following functionality in boost 1.42 built with
Visual Studio 2008:
- serialize/deserialize a derived class via a base class pointer using
polymorphic archives.
- the class serialization is implemented in a DLL using a base polymorphic
archive.
- the actual serialization is performed in a separate executable (linked
with the DLL above); in that executable I use a polymorphic text archive.
The documentation states this scenario is legitimate; however none of the
tests shipped with boost demonstrates it (there are examples either about
polymorphic classes or polymorphic archives but no test combining both).
The example proving my point is a variation of the test using
test_dll_exported.cpp and polymorphic_derived2.cpp where I removed the
template-based archives defeating the purpose of polymorphic archives.
My example throws an exception before main(), complaining the serializers
try to register twice with the archive map singleton of the polymorphic base
class.
I described the same issue in the thread "[serialization] Can one
deserialize a derived classes via pointers to the base class using
polymorphic archives in a DLL context?" I posted in
gmane.comp.lib.boost.devel where I provided more implementation details.
Robert Ramey has graciously replied with a series of suggestions;
unfortunately my example seems to have fulfilled all the conditions he
mentioned.
So, my question is: has anyone been able to run successfully a scenario
similar to the one I described?
Thank you,
Bogdan
============================================================================
======
I am enclosing a bare-bone example containing 8 files in two project files
plus one solution file.
My environment setup contains:
_CL_=/D_HAS_ITERATOR_DEBUGGING#0 /D_SECURE_SCL#0 /wd4996 /wd4503
/DBOOST_ALL_NO_LIB
BOOST_HOME=C:\_APP_\boost_1_42_0
Boost header files are installed under %BOOST_HOME%\include while the
libraries are installed under %BOOST_HOME%\lib.
I used Boost libraries (multi-threaded DLL versions) built with no automatic
dependencies (note /DBOOST_ALL_NO_LIB above).
Their names are boost_serialization-mt.lib and boost_serialization-mt-gd.lib
for Release/Debug configurations, respectively.
The PropertyPages/ConfigurationProperties/Debugging/Environment contains:
PATH=$(BOOST_HOME)\lib;$(PATH) used while debugging.
da...@lamef.bordeaux.ensam.fr
I have the same scenario, and it's not a problem.
If i understand (my english is poor, sorry) well.
But there is differences:
- I use GCC compilers (but boost is portable).
- I use static linking.
Probably this differences should not have consequences on the result.
I didn't read your code... Had you follow this rules ?
- Register child class (a simple way is using the BOOST_CLASS_EXPORT macro).
- In the serialize method of your child class, call the
serialization of parent's class (optional). I use
BOOST_SERIALIZATION_BASE_OBJECT's macro.
If you register child class it should be ok...
Regards,
Damien.
Bogdan <bogda...@gmail.com> a écrit :
_______________________________________________
Boost-users mailing list
Boost...@lists.boost.org
http://lists.boost.org/mailman/listinfo.cgi/boost-users
Bogdan
> Hi,
> I have the same scenario, and it's not a problem.
> If i understand (my english is poor, sorry) well.
> But there is differences:
> - I use GCC compilers (but boost is portable).
> - I use static linking.
> Probably this differences should not have consequences on the result.
> I didn't read your code... Had you follow this rules ?
> - Register child class (a simple way is using the
> BOOST_CLASS_EXPORT macro).
> - In the serialize method of your child class, call the
> serialization of parent's class (optional). I use
> BOOST_SERIALIZATION_BASE_OBJECT's macro.
>
> If you register child class it should be ok...
> Regards,
> Damien.
>
>
Hi Damien,
My reply has not been posted in gmane so I try again:
Thank you for your quick reply. I believe I followed the rules:
1. polymorphic_derived1 and polymorphic_derived2 do include the
serialization of the base class via
"ar & BOOST_SERIALIZATION_BASE_OBJECT_NVP(polymorphic_base); "
2. there is no usage of BOOST_CLASS_EXPORT ( and there shouldn't be any:
according to the documentation, in a dll setup one must declare
BOOST_CLASS_EXPORT_KEY in the headers containing the classes to be
serialized and BOOST_CLASS_EXPORT_IMPLEMENT in the .cpp files). In my
example, there are exactly three declarations of BOOST_CLASS_EXPORT_KEY: one
in polymorphic_base.hpp, and two in polymorphic_derived2.hpp one for
polymorphic_derived1 and another for polymorphic_derived2. Similarly, there
are only three usages of BOOST_CLASS_EXPORT_IMPLEMENT.
3. for deeper hierarchies where every class can be potentially serialized,
one should probably register all the classes and not just the most derived
children.
I have the feeling that the type of deployment (dll vs exe) is the crux of
the problem. For that, I created four different scenarios with the same
source files and I got the following results:
1. all files in one exe with the serialization library linked statically -
WORKS AS EXPECTED.
2. all files in one exe with the serialization library linked as a dll -
WORKS AS EXPECTED.
3. serialization files in a separate dll. Both the main executable and my
dll link with the dll version of the serialization library:
main.exe
my_serialization.dll
boost_serialization.dll
my_serialization.dll
boost_serialization.dll
Builds OK but FAILS at runtime , at run time I get a C++ exception BEFORE
main() in basic_serializer_map.cpp around the line 48:
'archive_exception::multiple_code_instantiation with the message "code
instantiated in more than one module - polymorphic_base"'
4. serialization files in a separate dll. Both the main executable and my
dll link with the static version of the serialization library:
main.exe
my_serialization.dll
boost_serialization.lib
my_serialization.dll
boost_serialization.lib
Builds OK but FAILS at runtime while trying to serialize with the following
exception thrown in oserializer.hpp(line 424):struct polymorphic::save():
'unregistered void cast class polymorphic_derived1<-polymorphic_base in
function 'int __cdecl main(int,char *[])''
The source code in the scenarios above is identical with the exception of
dllexport/dllimport declaration needed for dlls. Whenever the serialization
code is relegated to a different library it seems there are problems with
boost serialization:
- if boost is linked as a static lib into a user dll, the class serializers
do not get registered at all with their associated map singleton.
- if boost is linked as a dynamic lib into a user dll, the class serializers
register twice with the map singleton of the polymorphic base archive.
I don't want to add them here but I can provide any of the projects above to
those interested.
Thank you all,
Bogdan
Robert Ramey
> 3. serialization files in a separate dll. Both the main executable
> and my dll link with the dll version of the serialization library:
>
> main.exe
> my_serialization.dll
> boost_serialization.dll
>
> my_serialization.dll
> boost_serialization.dll
>
> Builds OK but FAILS at runtime , at run time I get a C++ exception
> BEFORE main() in basic_serializer_map.cpp around the line 48:
>
> 'archive_exception::multiple_code_instantiation with the message "code
> instantiated in more than one module - polymorphic_base"'
This would indicate that the serialization for polymorphic_base is included
in BOTH the mainline and the DLL. Check that the serialization function
is NOT an inline definition. the polymorphic_base.cpp should only be needed
when compiling the DLL.
option 1
include serialization definition for polymorphic_base in
my_serialization.dll
option 2
include serialization definition for polymorphic_base in it's own dll.
> 4. serialization files in a separate dll. Both the main executable
> and my dll link with the static version of the serialization library:
>
> main.exe
> my_serialization.dll
> boost_serialization.lib
>
> my_serialization.dll
> boost_serialization.lib
>
> Builds OK but FAILS at runtime while trying to serialize with the
> following exception thrown in oserializer.hpp(line 424):struct
> polymorphic::save():
>
> 'unregistered void cast class polymorphic_derived1<-polymorphic_base
> in function 'int __cdecl main(int,char *[])''
This is probably failing due to the compiler stripping out code
not explicitly referred to. Since it's not referrred to, the linking
doesn't include it from the library.
Robert Ramey
Bogdan
====================================================
Hello Robert,
Thank you for your reply. The serialization functions are not inline: their
bodies are defined inside the DLL.
I did the following experiments:
- I skipped manually the registration exception in the debugger and it turns
out that all the serialized classes have their serializers registered twice.
- if in the main function I do not call serialize/deserialize, the program
runs ok; I agree with you, the program doesn't do anything but it proves
that the registration is triggered by the actual usage of the serialization
functionality and not by any eventual inline.
I am under the impression I followed all the directions in the documentation
and yet I keep hitting that glitch.
Thank you,
Bogdan
PS.
As for the case where boost serialization links as a static inside a user
DLL, it fails, indeed, due to the compile stripping out code. Is there a
workaround for this you would suggest?
Robert Ramey
>
> Thank you for your reply. The serialization functions are not inline:
> their bodies are defined inside the DLL.
>
> I did the following experiments:
>
> - I skipped manually the registration exception in the debugger and
> it turns out that all the serialized classes have their serializers
> registered twice. - if in the main function I do not call
> serialize/deserialize, the program runs ok; I agree with you, the
> program doesn't do anything but it proves that the registration is
> triggered by the actual usage of the serialization functionality and
> not by any eventual inline.
>
> I am under the impression I followed all the directions in the
> documentation and yet I keep hitting that glitch.
I thought I included demos and tests to implement this very scenario.
Look through the demos/tests and double check this. If there
is a demo/test that works and emulates your situation maybe
we're done. If not, perhaps you might want to modify the
demo/test so it fails and then send it to me.
Robert Ramey
>
> Thank you,
>
> Bogdan
>
> PS.
> As for the case where boost serialization links as a static inside a
> user DLL, it fails, indeed, due to the compile stripping out code. Is
> there a workaround for this you would suggest?
The serialization library uses boost/serialization/force_include.hpp to
implement this. This might be made to work but is a compiler
dependent hack. I would prefer we get to the bottom of why
the DLL situation doesn't work as I would expect.
Robert Ramey
Martin Lederhilger
I have quite the same problem (Exception multiple_code_instantiation
thrown in basic_serializer_map.cpp at line 48) as Bogdan. It can be
reproduced by changing
the pointer type from polymorphic_base to polymorphic_derived2 in the
original test_dll_exported.cpp example, which comes with the library.
void save_exported(const char *testfile)
{
test_ostream os(testfile, TEST_STREAM_FLAGS);
test_oarchive oa(os, TEST_ARCHIVE_FLAGS);
polymorphic_base *rb1 = new polymorphic_derived1;
polymorphic_derived2 *rb2 = new polymorphic_derived2;
// export will permit correct serialization
// through a pointer to a base class
oa << BOOST_SERIALIZATION_NVP(rb1);
oa << BOOST_SERIALIZATION_NVP(rb2);
delete rb1;
delete rb2;
}
I think the problem is that when oa << BOOST_SERIALIZATION_NVP(rb2); is
called, that the system registers an type in oserializer.hpp, which
leads to creation of the singletons also in the executeable.
I have this problem when serializing a shared_ptr<A> in class B in B.dll
to an object of class A in A.dll.
I tried to simply comment out this check, but it seems that I get
problems with tracking later. I have a class hierarchy like this:
C derives from B derives from A. A has a weak_ptr<A> to itself
(something like boost::enable_shared_from_this). If I serialize an
object of type C via a base pointer of type B the serialization walks
like this: C::serialize, B::serialize, A::serialize, and again
C::serialize (with a wrong this pointer - and it should not do that).
Maybe my second problem depends on the first one. I hope that my report
can be of help.
Thanks in advance for your answers,
Martin Lederhilger
Robert Ramey
> Hello Robert, hello Bogdan,
>
> I have quite the same problem (Exception multiple_code_instantiation
> thrown in basic_serializer_map.cpp at line 48) as Bogdan. It can be
> reproduced by changing
> the pointer type from polymorphic_base to polymorphic_derived2 in the
> original test_dll_exported.cpp example, which comes with the library.
>
> void save_exported(const char *testfile)
> {
> test_ostream os(testfile, TEST_STREAM_FLAGS);
> test_oarchive oa(os, TEST_ARCHIVE_FLAGS);
>
> polymorphic_base *rb1 = new polymorphic_derived1;
> polymorphic_derived2 *rb2 = new polymorphic_derived2;
>
> // export will permit correct serialization
> // through a pointer to a base class
> oa << BOOST_SERIALIZATION_NVP(rb1);
> oa << BOOST_SERIALIZATION_NVP(rb2);
>
> delete rb1;
> delete rb2;
> }
>
> I think the problem is that when oa << BOOST_SERIALIZATION_NVP(rb2);
> is called, that the system registers an type in oserializer.hpp, which
> leads to creation of the singletons also in the executeable.
Ahhhh - a very useful hint. So the rule would be that if a class
is polymorphic is to be serialized through a pointer, it should ONLY
be done through a base class pointer? That is, if ALL polymorphic
base classes are abstract - this problem will never occur?
I'll have to think about this.
> I have this problem when serializing a shared_ptr<A> in class B in
> B.dll to an object of class A in A.dll.
>
> I tried to simply comment out this check, but it seems that I get
> problems with tracking later. I have a class hierarchy like this:
> C derives from B derives from A. A has a weak_ptr<A> to itself
> (something like boost::enable_shared_from_this). If I serialize an
> object of type C via a base pointer of type B the serialization walks
> like this: C::serialize, B::serialize, A::serialize, and again
> C::serialize (with a wrong this pointer - and it should not do that).
Hmmm - why should it not do that? You may have a cycle - but
the library handles that. Seems unrelated to the other problem.
> Maybe my second problem depends on the first one. I hope that my
> report can be of help.
>
> Thanks in advance for your answers,
And thanks for your useful information.
Martin Lederhilger
in my case I want to be able to serialize an object with a pointer to
its most derived type, a pointer to it's base class,
a pointer to the base class of base class, ...
Actually this problem should happen with all
BOOST_CLASS_EXPORT_IMPLEMENTed classes
which reside in one DLL, and are serialized (+automatic register_type)
in another DLL or EXE.
Is the intention of the exception to prevent useres from using
BOOST_CLASS_EXPORT_IMPLEMENT
multiple times? If so, then the quick solution for me is to ignore the
exception (the serialisation seems to work then, but I know that a
solution for the library itself is much trickier).
Martin
Robert Ramey
> Thanks for your answer,
>
> in my case I want to be able to serialize an object with a pointer to
> its most derived type, a pointer to it's base class,
> a pointer to the base class of base class, ...
>
> Actually this problem should happen with all
> BOOST_CLASS_EXPORT_IMPLEMENTed classes
> which reside in one DLL, and are serialized (+automatic register_type)
> in another DLL or EXE.
>
> Is the intention of the exception to prevent useres from using
> BOOST_CLASS_EXPORT_IMPLEMENT
> multiple times?
My concern was that having (possibly different) implementations
of code with the same signature in different modules would lead
to bizarre behavior which would be impossible to track down. The
problem for me is that when this happens with the serialization
library, I get a message "The serialization library has a bug" which
really bugs me. Worse, it takes a huge amount of time to resolve
the issue.
> If so, then the quick solution for me is to ignore the
> exception (the serialisation seems to work then, but I know that a
> solution for the library itself is much trickier).
Of course that works.
>
> Martin
It's becoming clear to me that it's very common to have code with the
same signature duplicated across DLLS. It should be clear why this
CAN be a big problem but USUALLY won't be. It takes (a lot?) of
extra effort to avoid this situation.
I'm thinking about the best way to handle this.
Robert Ramey