Re: [Boost-users] [serialization, 1_33_0] Correct usage and implementation for serializable objects without default constructors

[Martin Proulx]

Hello,

  While using the latest release of boost, I've encountered a few questions concerning the usage of the serialization library with objects without default constructors.

  Here they are:

  1-) How can we construct an instance of a serialized object that doesn't have a default constructor?

  The only option I've figured out is to normally construct an object with bogus values, then unserialize the real values while reading from the archive. Example:

  X(0) val; // 0 being some bogus value.
  inputStream >> val; // Overwrites all values within val;

  Is this the correct way to do things?

 No.. But I think what you want can be found in the manual in section: If you specialize save/load construct data you will get what you want. Use construct in place inside of load construct data.
  Reference/Serializable/Concept/Pointers/Non-Default Constructors.

  2-) How should we split the serialization between the class' serialize member versus load/save_construct_data?

  My observations show that when un/serializing objects directly like shown above, load/save_construct_data aren't needed, so the serialize member has to handle everything. When serializing a vector<X>, load/save_construct_data does get called, so we have to un/serialize some members in those functions as well. At first glance, it seems some members will have to be un/serialized for the second time within load/save_construct_data.

  Is that correct? For a class to be correctly un/serializable for all cases, load/save_construct_data must un/serialize members that are already taken care of in the serialize member?

Nope.

  If his isn't simple, then there is something wrong. Check the code in test_non_default_ctor . If it still doesn't give you what you need, ask again.

  RObert Ramey

Hello,

Let me rediscuss everything with the example of test_non_default_ctor.cpp at hand.

In this example, we have a class A that doesn't have a default constructor.  An int is needed by the constructor, which is stored in the i member variable.  load/save_construct_data are used to un/serialize the i member, while the serialize member is used for all other members.

Looking at this test, I see a couple of behaviors that puzzle me.

-) Saving the 'a' instance in the save() function doesn't end up calling save_construct_data.  This leaves a serialized version of A which lacks the 'i' member.  This makes it impossible to fully restore the 'a' instance.  It seems problematic to me that save_construct_data isn't called in this case.  In my planned usage, I'd like to fully serialize all members, regardless if the objects are serialized directly or through pointers.  I cannot see how to do this without serializing the members needed to reconstruct the object in both serialize and save_construct_data, even if some members will end up serialized twice when an object is stored through a pointer.

What's the proper way to serialize all members of a class without a default constructor so that it gets fully serialized regardless how the objects are managed?

-) When restoring the 'a' instance within load(), in order to restore it, we first create another 'a' instance (with a different constructor argument), then restore the other members within, leaving untouched the 'i' member.  Assuming I'd like to be able to serialize and unserialize completely objects that do not have a default constructor, I'm wondering how I can fully restore an object that doesn't have a default constructor.

I currently can only see a single way, and it is the way it's done in load() for the 'a' instance; that is to construct an object with possibly irrelevant constructor arguments, and then unserialize the object from the archive.  This has the downside of fully constructing an object with temporary values that will get replaced when we unserialize the real values.

I've looked for some other mechanism relying on a copy constructor and something along the lines of load_construct_data, but I haven't found any.

Here again, what is the syntax of choice to fully restore an object that doesn't have a default constructor?

Thanks!

Martin

--
Tel: (450) 681-1681, #271

---

OKIOK

Enterprise and e-business security solutions
Solutions de sécurité d'entreprise et d'affaires électronique

Tel. : (450) 681.1681
http://www.okiok.com

This e-mail message (including attachments, if any) is intended for the use of the individual or entity to which it is addressed and may contain information that is privileged, proprietary, confidential and exempt from disclosure. If you are not the intended recipient, you are notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify the sender and erase this e-mail message immediately.
 
Le présent message électronique (y compris les pièces qui y sont annexées, le cas échéant) s'adresse au destinataire indiqué et peut contenir des renseignements de caractère privé ou confidentiel. Si vous n'êtes pas le destinataire de ce document, nous vous signalons qu'il est strictement interdit de le diffuser, de le distribuer ou de le reproduire. Si ce message vous a été transmis par erreur, veuillez en informer l'expéditeur et le supprimer immédiatement.

[Robert Ramey]

signature

In this example, we have a class A that doesn't have a default constructor.
An int is needed by the constructor, which is stored in the i member
variable. load/save_construct_data are used to un/serialize the i member,
while the serialize member is used for all other members.

Looking at this test, I see a couple of behaviors that puzzle me.

-) Saving the 'a' instance in the save() function doesn't end up calling
save_construct_data. This leaves a serialized version of A which lacks the
'i' member. This makes it impossible to fully restore the 'a' instance. It
seems problematic to me that save_construct_data isn't called in this case.

I just re-ran test_non_default_ctor. I ran under the VC 7.1 debugger and
inserted traps inside both save_construct_data and load_construct_data. II
verified that save_construct_data and load_construct_data are being invoked
exactly as they are intended. I see no problem here.

In my planned usage, I'd like to fully serialize all members, regardless if
the objects are serialized directly or through pointers. I cannot see how
to do this without serializing the members needed to reconstruct the object
in both serialize and save_construct_data, even if some members will end up
serialized twice when an object is stored through a pointer.

If the members are not pointers, the constructor is never called. so the
parameters set by the constructor are not needed. save_load_contruct_data
is only called when its necessary to create a new object.

What's the proper way to serialize all members of a class without a default
constructor so that it gets fully serialized regardless how the objects are
managed?

Exactly as described in the manual and illustrated by the example
test_non_default_ctor.

-) When restoring the 'a' instance within load(), in order to restore it, we
first create another 'a' instance (with a different constructor argument),
then restore the other members within, leaving untouched the 'i' member.
Assuming I'd like to be able to serialize and unserialize completely objects
that do not have a default constructor, I'm wondering how I can fully
restore an object that doesn't have a default constructor.

All you have to do is define save_construct_data and load_construct_data for
all types which don't have a default constructor. The library takes care of
everything else.

I've looked for some other mechanism relying on a copy constructor and
something along the lines of load_construct_data, but I haven't found any.

Somehow I'm thinking you're looking for the wrong thing. I wonder if you're
not underestimating what the library does for you. As I said, you, only
need to define save/load_construct_data for those types which don't have a
default constructor - there is nothing else required to do.

Robert Ramey



_______________________________________________
Boost-users mailing list
Boost...@lists.boost.org
http://lists.boost.org/mailman/listinfo.cgi/boost-users

[Martin Proulx]

Hello,

As this discussion is going, it seems clear that either I'm missing something somewhere or that I'm not being clear enough in exposing  my problems...  I'll directly include a sample of something similar to what I'd like to achieve that doesn't work.  Please take a look at the example, and let me know how you would modify this to make both assert succeed.  If you notice that I've missed some concept in the way the library should be used, please enlighten me!

Thanks,

Martin

---

#include <cstdlib>
#include <cassert>
#include <sstream>

#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>
#include <boost/serialization/string.hpp>

using namespace std;

class Person
{
public:
    Person(const string& name) : _name(name), _favoriteFood("none") {}
   
    const string& getName() const {return _name;}
    const string& getFavoriteFood() const {return _favoriteFood;}
    void setFavoriteFood(const string& favoriteFood) {_favoriteFood = favoriteFood;}

private:
    const string _name;
    string _favoriteFood;

    friend class boost::serialization::access;

    template<class Archive>
    void serialize(Archive & ar, const unsigned int /* file_version */){
        ar & _favoriteFood;
    }
};

namespace boost {
namespace serialization {

template<class Archive>
inline void save_construct_data(Archive & ar, const Person * person, const unsigned int /* file_version */)
{
    ar << person->getName();
}

template<class Archive>
inline void load_construct_data(Archive & ar, Person * person, const unsigned int /* file_version */)
{
    string name;
    ar >> name;
    ::new(person)Person(name);
}

} // serialization
} // namespace boost

template<class Archive>
void serializePerson(Archive& oa, const Person& someone)
{
    // save_construct_data isn't called when serializing a Person in this fashion.
    // The person's name will not appear in the archive.
    oa << someone;
}

template<class Archive>
Person UnserializePerson(Archive& ia)
{
    // I'm constructing a temporary person, with a bogus name, as it's the only workaround I've come up with.
    // There is no way that unserializing returns an actual object, like I'm needing here.
    Person someone("bogus name");

    // When unserializing in this fashion, load_construct_data isn't called either.
    ia >> someone;

    // The initial person's name is then lost.
    return someone;
}

void main(void)
{
    Person *const Joe = new Person("Joe");
    Person John("John");

    stringstream stringStream;

    boost::archive::text_oarchive oa(stringStream);
    oa << Joe;
    serializePerson(oa, John);

    boost::archive::text_iarchive ia(stringStream);
    Person* restoredPersonPointer;
    ia >> restoredPersonPointer;
    Person restoredPerson(UnserializePerson(ia));

    assert(restoredPersonPointer->getName() == Joe->getName()); // works fine.
    assert(restoredPerson.getName() == John.getName()); // fails.

[Robert Ramey]

OK I see the problem now.

 

Let me re-state it to be sure we're on the same page.

 

The class Person has a const member _name.  In C++ const members can only be set when an object is constructed.  In order to make sure no object is created without a name, There is no default constructor.  When the serialization library creates a new instance - that is when loading a pointer, the library uses the non-default constructor to initialize the const variable with the correct value.  When restoring the contents of an instance of the class Person that already exists the const member is not set.

 

Which seems totally correct to me.

 

The real question here is what is the intent in setting the member variable _name as const?  Standard C++ behavior for const members is that they

a) are only set when the instance is constructed.

b) cannot be changed thereafter.

 

Now if we do the following:

 

Person p("bob")

...

ar << p;

...

Person p1("anne");

...

ar >> p1;

 

What should p1 contain?  If you want p1 to contain "anne", then leave your program as it is.  If you want it to contain "bob" then:

 

a) remove the const from the member _name

b) implement the normal member serialization for _name.

c) at this point, loading pointers will result in _name being loaded twice. If this is a problem, you could

    i) add a default constructor and elminate the save/load construct data.  You might want to make the default constructor private and include friend boost::serialization::access to your class.  (I haven't tried this but I believe it should work.

 

An alternative to a) above would be to leave the "const" but use a const_cast inside the class serialization function.

 

At the heart of the matter here is the what we want const to mean.  Should const variables never change during the life of the class? - then we expect p1 to contain "bob".  Otherwise they are not really "const" in the traditional C++ sense and should be addressed by removing the const, or a cast or?

 

Robert Ramey

 

[Martin Proulx]

Robert Ramey wrote:

OK I see the problem now.

 

Let me re-state it to be sure we're on the same page.

 

The class Person has a const member _name.  In C++ const members can only be set when an object is constructed.  In order to make sure no object is created without a name, There is no default constructor.  When the serialization library creates a new instance - that is when loading a pointer, the library uses the non-default constructor to initialize the const variable with the correct value.  When restoring the contents of an instance of the class Person that already exists the const member is not set.

 

Which seems totally correct to me.

 

I believe there's a fondamental concept that needs to be cleared up before going any further.

What does the unserialization of an instance means?  After some thought, I now realize it can mean at least two things:

1-) It's an operation akin to the assignment operator.  Unserializing means assigning new values to an already existing object.
2-) It means reconstructing the object as it originally was, in the manner a factory would.

It seems to me as if the serialization library does a mix of both.

The first approach is what operator>> does.  You declare something, call operator>> on it, and after that, the initial object has now been assigned the serialized value.

The library also does the second interpretation as it will reconstruct an instance if needed.  load_construct_data is the factory mechanism that recreates an actual instance.  This is used to restore objects that were saved through pointers or that were contained in STL containers.

The real question here is what is the intent in setting the member variable _name as const?  Standard C++ behavior for const members is that they

a) are only set when the instance is constructed.

b) cannot be changed thereafter.

 

Now if we do the following:

 

Person p("bob")

...

ar << p;

...

Person p1("anne");

...

ar >> p1;

 

What should p1 contain?  If you want p1 to contain "anne", then leave your program as it is.  If you want it to contain "bob" then:

 

a) remove the const from the member _name

b) implement the normal member serialization for _name.

c) at this point, loading pointers will result in _name being loaded twice. If this is a problem, you could

    i) add a default constructor and elminate the save/load construct data.  You might want to make the default constructor private and include friend boost::serialization::access to your class.  (I haven't tried this but I believe it should work.

 

An alternative to a) above would be to leave the "const" but use a const_cast inside the class serialization function.

 

At the heart of the matter here is the what we want const to mean.  Should const variables never change during the life of the class? - then we expect p1 to contain "bob".  Otherwise they are not really "const" in the traditional C++ sense and should be addressed by removing the const, or a cast or?

 

All the above discussion is really a discussion about what the assignment of an object that has a const member means.  Well, in C++, it really doesn't make sense and I agree.

That's why I believe there's some tiny piece missing from the serialization library.  We should also be able to unserialize instances through a factory that returns an actual object rather than having to use operator>>.

Here's some code showing this idea that works along with the previous example.

template<class T, class Archive>
void serialize_for_factory(Archive& oa, const T& object)
{
    boost::serialization::save_construct_data(oa, &object, 0); // Do something for the file version.
    oa << object;
}

template<class T, class Archive>
T unserialize_factory(Archive& ia)
{
    unsigned char memory[sizeof(T)];
   
    T* object = reinterpret_cast<T*>(memory);

    boost::serialization::load_construct_data(ia, object, 0); // Do something for the file version.

    T unserializedObject(*object); // Uses the copy constructor;
    object->~T();

    return unserializedObject;
}

void betterWay()
{

    Person John("John");
   
    stringstream stringStream;
    boost::archive::text_oarchive oa(stringStream);

    serialize_for_factory(oa, John);

    boost::archive::text_iarchive ia(stringStream);
    Person restoredPerson(unserialize_factory<Person>(ia));

    assert(restoredPerson.getName() == John.getName()); // Works
}


Do you believe something like this fits in the serialization library?

Thanks!

Martin

[Robert Ramey]

All the above discussion is really a discussion about what the assignment of an object that has a const member means.  Well, in C++, it really doesn't make sense and I agree.

That's why I believe there's some tiny piece missing from the serialization library.  We should also be able to unserialize instances through a factory that returns an actual object rather than having to use operator>>.

Here's some code showing this idea that works along with the previous example.

 

...

 

I don't think this adds anytihing of value. 

 

Robert Ramey

[Martin Proulx]

Well, restoring objects using some sort of factory would give us a proper mechanism to restore objects with non default constructors or const members that haven't been serialized through pointers.  That would be one way to overcome the assignment like limitation of restoring through operator>>.  I believe the idea of a factory also holds well to restore objects serialized through pointers where the objects have to actually be reconstructed.

Also, fully restoring all members of an object without a default constructor currently implies writing a different serialize function depending if we are to serialize through pointers only, or directly as well: the members needed to reconstruct are either omitted or not.  This makes me believe the support for serializing through pointers is somewhat incongruous to the one for serializing directly.  It would be possible to correct this by doing something like always having save_construct_data called, and by using load_construct_data in the factory.  Something along those lines is seems to be already done when serializing and unserializing such objects from STL containers.

Thank you for your support and the serialization library; It really is a useful piece of software!

Martin

[Robert Ramey]

Well, restoring objects using some sort of factory would give us a proper mechanism to restore objects with non default constructors or const members that haven't been serialized through pointers.  That would be one way to overcome the assignment like limitation of restoring through operator>>.  I believe the idea of a factory also holds well to restore objects serialized through pointers where the objects have to actually be reconstructed.

 

what I really mean is what is the matter with

    class A {

        const X m_x

        ...

        template<class Archive>

        void serialize(Archive & ar, const unsigned int version){

            ar >> const_cast<X>(m_x);

        }

    };

Doesn't this give the same result in a simpler more transparent way?

 

 

Robert Ramey

[Martin Proulx]

It's true that this is a simple way to do the following:

A a(/* With a possible bogus and temporary value for m_x);
ia >> a;

Now if this is simpler and more transparent than doing something like this:

A a(restore_object<A>(ia));

...I'll leave for C++ language lawyers and you to decide. They just
don't mean the same thing, even if the end result might be the same.
You can achieve the same results with copy constructors or assignment
operators, but they're not the same.

As a user, what really isn't simple or transparent is that with the
above operator>> solution, for objects without a default constructor, I
have to define serialize differently depending if load/save_construct
data is called or not. Should I take care of the members needed to
reconstruct an instance or not in serialize?

I have to know enough about the library to know that serializing an "A*"
or a "vector<A>" does call them, but that serializing a "const A&"
doesn't. Now what about an array of A???

I believe making an object serializable would just be simpler if the
following would be true:

-) save_construct_data and load_construct_data are always used.
-) We have some function that constructs and returns an object instance
using load_construct_data.

This removes the possibility of doubly serialized members, the need to
const_cast, temporary objects created with bogus parameters, and the
ambiguity of what members serialize should take care of.

Martin

[Alexis Gatt]

Hi all,

I've just downloaded Boost 1.33 and built it with
VC7.0. I specified the Spirit root so that the
serialization lib would be compiled as well. I saw the
output indicating that the serialization lib was being
compiled, and I can also see the .obj files in the
bin/boost/libs/serialization/... folders, but I cannot
find the corresponding .lib or .dll anywhere?!
Did I miss something wrong, or there are some other
steps that I need to perform to obtain this .lib?

Thanks

Alex



___________________________________________________________
How much free photo storage do you get? Store your holiday
snaps for FREE with Yahoo! Photos http://uk.photos.yahoo.com

[Robert Ramey]

How did you build these? What command did you use? This is usually
addressed in the "getting started" section of the manual.

Robert Ramey

[Alexis Gatt]

--- Robdert Ramey <ra...@rrsd.com> wrote:

> How did you build these? What command did you use?
> This is usually
> addressed in the "getting started" section of the
> manual.

Thanks for the answer. I actually followed the
instructions provided in the "Getting started"
section. I first specified the root of the spirit
library by "set SPIRIT_ROOT=E:/Boost/spirit16". I then
started the compilation of Boost with the following
command "bjam "-sTOOLS=vc7" install".

THe compilation of other libraries were successful, as
the C:/Boost directory contains many .dll and .lib.
But I cant find the serialization lib.

Any ideas?

Alex



___________________________________________________________
To help you stay safe and secure online, we've developed the all new Yahoo! Security Centre. http://uk.security.yahoo.com

[Vladimir Prus]

Hi Robert,

> signatureAll the above discussion is really a discussion about what the

Is it possible for you to use proper quoting on your replies. It took me
quite a while to figure out that:

Here's some code showing this idea that works along with the previous
example.

followed by

I don't think this adds anytihing of value.

are actually words of two different persons! Besides, what that "signature"
word right at the beginning?

Thanks,
Volodya