[Boost-users] [serialization] serializing dynamic arrays
444 views
Skip to first unread message
Ivan Rachev
Jul 9, 2005, 6:56:00 PM7/9/05
to boost...@lists.boost.org
Does anyone have an idea how to serialize dynamic arrays of objects?
An example follows at the end. Its weakness shows up when an outside
pointer points to an element in the array. The problem is that the type
being serialized is T but not T*.
Thanks,
Ivan
#include <fstream>
#include <boost/archive/text_iarchive.hpp>
#include <boost/serialization/access.hpp>
#include <boost/serialization/split_member.hpp>
class A
{ friend class boost::serialization::access;
template<class Archive>
void serialize(Archive & ar, const unsigned int version) {}
};
//IVAN Element always points to the beginning of an array of
// 'Size' elements. The array is NOT null-terminated. If
// size == 0, Elements may hold an undefined value. User
// is responsible for initializing, new-ing and delete-ing.
// Assumption: if there is an outside pointer to an object
// inside this array, after loading the array
// that pointer will hold a different object.
template <typename T>
struct DynamicArray
{
int Size;
T* Element;
friend class boost::serialization::access;
template<class Archive>
void save(Archive & ar, const unsigned int version) const
{
ar & Size;
for (int i = 0; i < Size; ++i)
ar & Element[i];
}
template<class Archive>
void load(Archive & ar, const unsigned int version)
{
ar & Size;
//TODO: assert(Size >=0);
if (Size > 0)
{
Element = new T[Size];
for (int i = 0; i < Size; ++i)
ar & Element[i];
}
}
BOOST_SERIALIZATION_SPLIT_MEMBER()
};
int main()
{
DynamicArray<A> ArrayOfAs;
A* secondElement;
std::ifstream ifs("test.txt");
boost::archive::text_iarchive ia(ifs);
ia & ArrayOfAs;
ia & secondElement; // this guy will point to a copy of the
// 2nd element but NOT the 2nd element itself
delete [] ArrayOfAs.Element;
}
_______________________________________________
Boost-users mailing list
Boost...@lists.boost.org
http://lists.boost.org/mailman/listinfo.cgi/boost-users
An example follows at the end. Its weakness shows up when an outside
pointer points to an element in the array. The problem is that the type
being serialized is T but not T*.
Thanks,
Ivan
#include <fstream>
#include <boost/archive/text_iarchive.hpp>
#include <boost/serialization/access.hpp>
#include <boost/serialization/split_member.hpp>
class A
{ friend class boost::serialization::access;
template<class Archive>
void serialize(Archive & ar, const unsigned int version) {}
};
//IVAN Element always points to the beginning of an array of
// 'Size' elements. The array is NOT null-terminated. If
// size == 0, Elements may hold an undefined value. User
// is responsible for initializing, new-ing and delete-ing.
// Assumption: if there is an outside pointer to an object
// inside this array, after loading the array
// that pointer will hold a different object.
template <typename T>
struct DynamicArray
{
int Size;
T* Element;
friend class boost::serialization::access;
template<class Archive>
void save(Archive & ar, const unsigned int version) const
{
ar & Size;
for (int i = 0; i < Size; ++i)
ar & Element[i];
}
template<class Archive>
void load(Archive & ar, const unsigned int version)
{
ar & Size;
//TODO: assert(Size >=0);
if (Size > 0)
{
Element = new T[Size];
for (int i = 0; i < Size; ++i)
ar & Element[i];
}
}
BOOST_SERIALIZATION_SPLIT_MEMBER()
};
int main()
{
DynamicArray<A> ArrayOfAs;
A* secondElement;
std::ifstream ifs("test.txt");
boost::archive::text_iarchive ia(ifs);
ia & ArrayOfAs;
ia & secondElement; // this guy will point to a copy of the
// 2nd element but NOT the 2nd element itself
delete [] ArrayOfAs.Element;
}
_______________________________________________
Boost-users mailing list
Boost...@lists.boost.org
http://lists.boost.org/mailman/listinfo.cgi/boost-users
Robert Ramey
Jul 10, 2005, 12:42:05 AM7/10/05
to boost...@lists.boost.org
I believe your example should work as written. Where does it fail?
Robert Ramey
Robert Ramey
Martin Ecker
Jul 10, 2005, 4:21:58 AM7/10/05
to boost...@lists.boost.org
Hi,
Ivan Rachev wrote:
> Does anyone have an idea how to serialize dynamic arrays of objects?
>
> An example follows at the end.
Please post fully working examples. That will make it easier for people to help
you. Your example requires a file test.txt that you didn't send along in your
e-mail.
> Its weakness shows up when an outside
> pointer points to an element in the array. The problem is that the type
> being serialized is T but not T*.
This should actually work fine. I modified your sample to use a memory buffer
instead of a file and added an assert to make sure the deserialized
second-element pointer points to the correct element. The assert does
not trigger with MSVC 7.1.
#include <sstream>
#include <boost/archive/text_iarchive.hpp>
#include <boost/archive/text_oarchive.hpp>
int main(int argc, char* argv[])
{
DynamicArray<A> ArrayOfAs, ArrayOfAs2;
ArrayOfAs.Element = new A[5];
ArrayOfAs.Size = 5;
A* secondElement = &ArrayOfAs.Element[1];
A* secondElement2;
std::stringstream stream;
{
boost::archive::text_oarchive oa(stream);
oa & ArrayOfAs;
oa & secondElement; // this guy will point to a copy of the
boost::archive::text_iarchive ia(stream);
ia & ArrayOfAs2;
ia & secondElement2; // this guy will point to a copy of the
assert(secondElement2 == &ArrayOfAs2.Element[1]);
delete [] ArrayOfAs.Element;
delete [] ArrayOfAs2.Element;
}
Best Regards,
Martin
Ivan Rachev wrote:
> Does anyone have an idea how to serialize dynamic arrays of objects?
>
> An example follows at the end.
you. Your example requires a file test.txt that you didn't send along in your
e-mail.
> Its weakness shows up when an outside
> pointer points to an element in the array. The problem is that the type
> being serialized is T but not T*.
instead of a file and added an assert to make sure the deserialized
second-element pointer points to the correct element. The assert does
not trigger with MSVC 7.1.
#include <sstream>
#include <boost/archive/text_iarchive.hpp>
#include <boost/archive/text_oarchive.hpp>
{
DynamicArray<A> ArrayOfAs, ArrayOfAs2;
ArrayOfAs.Element = new A[5];
ArrayOfAs.Size = 5;
A* secondElement = &ArrayOfAs.Element[1];
A* secondElement2;
std::stringstream stream;
{
boost::archive::text_oarchive oa(stream);
oa & ArrayOfAs;
oa & secondElement; // this guy will point to a copy of the
// 2nd element but NOT the 2nd element itself
}
{
}
boost::archive::text_iarchive ia(stream);
ia & ArrayOfAs2;
ia & secondElement2; // this guy will point to a copy of the
// 2nd element but NOT the 2nd element itself
}
// does not trigger, so the above should work
}
assert(secondElement2 == &ArrayOfAs2.Element[1]);
delete [] ArrayOfAs.Element;
delete [] ArrayOfAs2.Element;
}
Best Regards,
Martin
Ivan Rachev
Jul 10, 2005, 10:36:23 AM7/10/05
to boost...@lists.boost.org
Yes, after further understanding of how boost::serialization works I
came to the conclusion that the sample works.
Thanks,
Ivan
came to the conclusion that the sample works.
Thanks,
Ivan
Ivan Rachev
Jul 10, 2005, 12:49:08 PM7/10/05
to boost...@lists.boost.org
Hi Martin,
If class A from the example is replaced by char, how should this
situation be handled? Code follows at the end.
Thanks,
Ivan
template <typename T>
struct DynamicArray
{
int Size;
T* Element;
friend class boost::serialization::access;
template<class Archive>
void save(Archive & ar, const unsigned int version) const
{
ar & Size;
for (int i = 0; i < Size; ++i)
ar & Element[i];
}
template<class Archive>
void load(Archive & ar, const unsigned int version)
{
ar & Size;
ArrayOfAs.Element = new char[5];
ArrayOfAs.Size = 5;
char* secondElement = &ArrayOfAs.Element[1];
char* secondElement2;
std::stringstream stream;
{
boost::archive::text_oarchive oa(stream);
oa & ArrayOfAs;
oa & secondElement;
}
If class A from the example is replaced by char, how should this
situation be handled? Code follows at the end.
Thanks,
Ivan
template <typename T>
struct DynamicArray
{
int Size;
T* Element;
friend class boost::serialization::access;
template<class Archive>
void save(Archive & ar, const unsigned int version) const
{
ar & Size;
for (int i = 0; i < Size; ++i)
ar & Element[i];
}
template<class Archive>
void load(Archive & ar, const unsigned int version)
{
ar & Size;
assert(Size >=0);
if (Size > 0)
{
Element = new T[Size];
for (int i = 0; i < Size; ++i)
ar & Element[i];
}
}
BOOST_SERIALIZATION_SPLIT_MEMBER()
};
int main(int argc, char* argv[])
{
DynamicArray<char> ArrayOfAs, ArrayOfAs2;
if (Size > 0)
{
Element = new T[Size];
for (int i = 0; i < Size; ++i)
ar & Element[i];
}
}
BOOST_SERIALIZATION_SPLIT_MEMBER()
};
int main(int argc, char* argv[])
{
ArrayOfAs.Element = new char[5];
ArrayOfAs.Size = 5;
char* secondElement = &ArrayOfAs.Element[1];
char* secondElement2;
std::stringstream stream;
{
boost::archive::text_oarchive oa(stream);
oa & ArrayOfAs;
oa & secondElement;
}
{
boost::archive::text_iarchive ia(stream);
ia & ArrayOfAs2;
ia & secondElement2;
}
boost::archive::text_iarchive ia(stream);
ia & ArrayOfAs2;
ia & secondElement2;
}
assert(secondElement2 == &ArrayOfAs2.Element[1]);
delete [] ArrayOfAs.Element;
delete [] ArrayOfAs2.Element;
}
delete [] ArrayOfAs.Element;
delete [] ArrayOfAs2.Element;
}
Robert Ramey
Jul 10, 2005, 4:29:00 PM7/10/05
to boost...@lists.boost.org
Addresses of primitive types are generally not tracked by default. So
replacing A with a char would not give a desired result. This is described
in the manual under the section of seriailzation traites/tracking and
serialization of pointers.
A solution would be to make a "trackable char" using STRONG_TYPE or some
other sort of wrapper to distinguish a char from a your own chartype.
Note that if you make a few small changes in your program - and are using
current CVS version (1.33 - real soon now) you will an error message to
alert you to this problem. Here is my updated version of your program.
replacing A with a char would not give a desired result. This is described
in the manual under the section of seriailzation traites/tracking and
serialization of pointers.
A solution would be to make a "trackable char" using STRONG_TYPE or some
other sort of wrapper to distinguish a char from a your own chartype.
Note that if you make a few small changes in your program - and are using
current CVS version (1.33 - real soon now) you will an error message to
alert you to this problem. Here is my updated version of your program.
Martin Ecker
Jul 10, 2005, 4:39:11 PM7/10/05
to boost...@lists.boost.org
Hi,
Ivan Rachev wrote:
> If class A from the example is replaced by char, how should this
> situation be handled? Code follows at the end.
This is not recommended. Fundamental types are not tracked by default, so it
won't work. You'd have to turn on tracking for char, but that is not a good
idea, since then _all_ values of type char that are tracked, and this can cause
quite a bit of overhead.
If you really need a pointer that directly points into an array (and you can't
redesign the code), you could wrap the array elements in a different type that
can be tracked in your serialization code. All pointers into the array would
also need to be wrapped when serialized.
Regards,
Martin
Ivan Rachev wrote:
> If class A from the example is replaced by char, how should this
> situation be handled? Code follows at the end.
won't work. You'd have to turn on tracking for char, but that is not a good
idea, since then _all_ values of type char that are tracked, and this can cause
quite a bit of overhead.
If you really need a pointer that directly points into an array (and you can't
redesign the code), you could wrap the array elements in a different type that
can be tracked in your serialization code. All pointers into the array would
also need to be wrapped when serialized.
Regards,
Martin
Robert Ramey
Jul 10, 2005, 4:32:34 PM7/10/05
to boost...@lists.boost.org
int main(int argc, char* argv[])
{
DynamicArray<char> ArrayOfAs, ArrayOfAs2;
const ArrayOfAs.Element = new char[5];
{
DynamicArray<char> ArrayOfAs, ArrayOfAs2;
ArrayOfAs.Size = 5;
char* secondElement = &ArrayOfAs.Element[1];
char* secondElement2;
std::stringstream stream;
{
boost::archive::text_oarchive oa(stream);
oa << ArrayOfAs;
char* secondElement = &ArrayOfAs.Element[1];
char* secondElement2;
std::stringstream stream;
{
boost::archive::text_oarchive oa(stream);
oa << secondElement; // error message here - serializating a
non-tracked type
// (a primitive) through a
pointer.
}
Robert Ramey
Ivan Rachev
Jul 13, 2005, 5:47:30 PM7/13/05
to boost...@lists.boost.org
Martin Ecker wrote:
> Hi,
>
> Ivan Rachev wrote:
>
>>If class A from the example is replaced by char, how should this
>>situation be handled? Code follows at the end.
>
>
> This is not recommended. Fundamental types are not tracked by default, so it
> won't work. You'd have to turn on tracking for char, but that is not a good
> idea, since then _all_ values of type char that are tracked, and this can cause
> quite a bit of overhead.
>
> If you really need a pointer that directly points into an array (and you can't
> redesign the code), you could wrap the array elements in a different type that
> can be tracked in your serialization code. All pointers into the array would
> also need to be wrapped when serialized.
>
> Regards,
> Martin
before writing my own wrappers I would like to see if the overhead of
> Hi,
>
> Ivan Rachev wrote:
>
>>If class A from the example is replaced by char, how should this
>>situation be handled? Code follows at the end.
>
>
> This is not recommended. Fundamental types are not tracked by default, so it
> won't work. You'd have to turn on tracking for char, but that is not a good
> idea, since then _all_ values of type char that are tracked, and this can cause
> quite a bit of overhead.
>
> If you really need a pointer that directly points into an array (and you can't
> redesign the code), you could wrap the array elements in a different type that
> can be tracked in your serialization code. All pointers into the array would
> also need to be wrapped when serialized.
>
> Regards,
> Martin
tracking chars is acceptable for me. I tried putting the following into
my cpp but it won't compile:
#include <boost/serialization/tracking.hpp>
BOOST_CLASS_TRACKING(char, boost::serialization::track_selectivly)
I guess I need to change the default "implementation_level" for char.
What other things do I need to change so as to make a fundamental type
trackable?
Thanks,
Ivan
Ivan Rachev
Jul 13, 2005, 10:46:32 PM7/13/05
to boost...@lists.boost.org
Martin Ecker wrote:
> Hi,
>
> Ivan Rachev wrote:
>
>>If class A from the example is replaced by char, how should this
>>situation be handled? Code follows at the end.
>
>
> This is not recommended. Fundamental types are not tracked by default, so it
> won't work. You'd have to turn on tracking for char, but that is not a good
> idea, since then _all_ values of type char that are tracked, and this can cause
> quite a bit of overhead.
>
> If you really need a pointer that directly points into an array (and you can't
> redesign the code), you could wrap the array elements in a different type that
> can be tracked in your serialization code. All pointers into the array would
> also need to be wrapped when serialized.
>
> Regards,
> Martin
> Hi,
>
> Ivan Rachev wrote:
>
>>If class A from the example is replaced by char, how should this
>>situation be handled? Code follows at the end.
>
>
> This is not recommended. Fundamental types are not tracked by default, so it
> won't work. You'd have to turn on tracking for char, but that is not a good
> idea, since then _all_ values of type char that are tracked, and this can cause
> quite a bit of overhead.
>
> If you really need a pointer that directly points into an array (and you can't
> redesign the code), you could wrap the array elements in a different type that
> can be tracked in your serialization code. All pointers into the array would
> also need to be wrapped when serialized.
>
> Regards,
> Martin
before writing my own wrappers I would like to see if the overhead of
tracking chars is acceptable for me. I tried putting the following into
my cpp but it won't compile:
#include <boost/serialization/tracking.hpp>
BOOST_CLASS_TRACKING(char, boost::serialization::track_selectivly)
I guess I need to change the default "implementation_level" for char.
What other things do I need to change so as to make a fundamental type
trackable?
Thanks,
Ivan
tracking chars is acceptable for me. I tried putting the following into
my cpp but it won't compile:
#include <boost/serialization/tracking.hpp>
BOOST_CLASS_TRACKING(char, boost::serialization::track_selectivly)
I guess I need to change the default "implementation_level" for char.
What other things do I need to change so as to make a fundamental type
trackable?
Thanks,
Ivan
Robert Ramey
Jul 14, 2005, 12:21:43 AM7/14/05
to boost...@lists.boost.org
Robert Ramey
Jul 14, 2005, 12:22:12 AM7/14/05
to boost...@lists.boost.org
How did the compile fail?
Robert Ramey
Robert Ramey
Ivan Rachev
Jul 14, 2005, 1:03:43 AM7/14/05
to boost...@lists.boost.org
Robert Ramey wrote:
> How did the compile fail?
>
> Robert Ramey
>
>
>>#include <boost/serialization/tracking.hpp>
>>BOOST_CLASS_TRACKING(char, boost::serialization::track_selectivly)
>>
>>I guess I need to change the default "implementation_level" for char.
>>What other things do I need to change so as to make a fundamental type
>>trackable?
>>
>>
>>Thanks,
>>Ivan
cpp files contains this:
> How did the compile fail?
>
> Robert Ramey
>
>
>>#include <boost/serialization/tracking.hpp>
>>BOOST_CLASS_TRACKING(char, boost::serialization::track_selectivly)
>>
>>I guess I need to change the default "implementation_level" for char.
>>What other things do I need to change so as to make a fundamental type
>>trackable?
>>
>>
>>Thanks,
>>Ivan
#include <boost/serialization/tracking.hpp>
BOOST_CLASS_TRACKING(char, boost::serialization::track_selectivly)
error C2027: use of undefined type 'boost::STATIC_ASSERTION_FAILURE<x>'
because I believe BOOST_STATIC_ASSERT inside BOOST_CLASS_TRACKING fails.
I use boost 1.32
Robert Ramey
Jul 14, 2005, 2:18:36 AM7/14/05
to boost...@lists.boost.org
Ahhh yes. Basically the STATIC_ASSERT is preventing you from setting
tracking for a primitive type. This almost HAS to be an error. Doing this
will effect serialization of all char's in your program which is almost
certainly what you don't want to do. If you want to do this, You'll have to
wrap a char in your own type so its not a primitive anymore. Then it will
pass.
I'll add a comment to where the STATIC_ASSERT traps so that one will have an
explanation when its needed.
Robert Ramey
Ivan Rachev wrote:
tracking for a primitive type. This almost HAS to be an error. Doing this
will effect serialization of all char's in your program which is almost
certainly what you don't want to do. If you want to do this, You'll have to
wrap a char in your own type so its not a primitive anymore. Then it will
pass.
I'll add a comment to where the STATIC_ASSERT traps so that one will have an
explanation when its needed.
Robert Ramey
Ivan Rachev wrote:
Ivan Rachev
Jul 15, 2005, 9:44:23 PM7/15/05
to boost...@lists.boost.org
Robert Ramey wrote:
> Ahhh yes. Basically the STATIC_ASSERT is preventing you from setting
> tracking for a primitive type. This almost HAS to be an error. Doing this
> will effect serialization of all char's in your program which is almost
> certainly what you don't want to do. If you want to do this, You'll have to
> wrap a char in your own type so its not a primitive anymore. Then it will
> pass.
>
Let's say the wrapper for char is A (from Martin's example). Then we
> Ahhh yes. Basically the STATIC_ASSERT is preventing you from setting
> tracking for a primitive type. This almost HAS to be an error. Doing this
> will effect serialization of all char's in your program which is almost
> certainly what you don't want to do. If you want to do this, You'll have to
> wrap a char in your own type so its not a primitive anymore. Then it will
> pass.
>
would have a dynamic array of As. If I try to serialize a pointer
pointing to one of these As inside the array, boost::serialization
crashes. Example and stack trace follows at the end.
Ivan
#include <sstream>
#include <boost/archive/text_iarchive.hpp>
#include <boost/archive/text_oarchive.hpp>
#include <boost/serialization/access.hpp>
#include <boost/serialization/split_member.hpp>
class A
template<class Archive>
void serialize(Archive & ar, const unsigned int version) {}
};
template <typename T>
struct DynamicArray
{
int Size;
T* Element;
friend class boost::serialization::access;
template<class Archive>
void save(Archive & ar, const unsigned int version) const
{
ar & Size;
for (int i = 0; i < Size; ++i)
ar & Element[i];
}
template<class Archive>
void load(Archive & ar, const unsigned int version)
{
ar & Size;
assert( Size >=0 );
if (Size > 0)
{
Element = new T[Size];
for (int i = 0; i < Size; ++i)
ar & Element[i];
}
}
BOOST_SERIALIZATION_SPLIT_MEMBER()
};
int main(int argc, char* argv[])
{
ArrayOfAs.Element = new A[5];
ArrayOfAs.Size = 5;
A* secondElement = &ArrayOfAs.Element[1];
ArrayOfAs.Size = 5;
A* secondElement = &ArrayOfAs.Element[1];
std::stringstream stream;
{
boost::archive::text_oarchive oa(stream);
oa & secondElement;
{
boost::archive::text_oarchive oa(stream);
oa & ArrayOfAs;
}
delete [] ArrayOfAs.Element;
}
STACK:
77e9e8bb
77e9e8bb
> _STL::__stl_debug_engine<bool>::_M_attach(_STL::__owned_list *
__l=0x004f7db4, _STL::__owned_link * __c_node=0x00000001) Line 435 C++
_STL::_Atomic_swap(volatile long * __p=0x00000001, long __q=1243908)
Line 458 C++
004f7db4
_CxxThrowException(void * pExceptionObject=0x0012fb14, const
_s__ThrowInfo * pThrowInfo=0x004b9c84) C++
boost::throw_exception<boost::archive::archive_exception>(const
boost::archive::archive_exception & e={...}) Line 40 C++
boost::archive::detail::basic_oarchive_impl::save_object(boost::archive::detail::basic_oarchive
& ar={...}, const void * t=0x004f5951, const
boost::archive::detail::basic_oserializer & bos={...}) Line 284 C++
boost::archive::detail::basic_oarchive::save_object(const void *
x=0x004f5951, const boost::archive::detail::basic_oserializer &
bos={...}) Line 395 C++
boost::archive::detail::save_non_pointer_type<boost::archive::text_oarchive,A>::save::invokex(boost::archive::text_oarchive
& ar={...}, const A & t={...}) Line 235 C++
boost::archive::detail::save_non_pointer_type<boost::archive::text_oarchive,A>::invoke(boost::archive::text_oarchive
& ar={...}, const A & t={...}) Line 309 C++
boost::archive::save<boost::archive::text_oarchive,A>(boost::archive::text_oarchive
& ar={...}, const A & t={...}) Line 551 C++
boost::archive::basic_text_oarchive<boost::archive::text_oarchive>::save_override<A>()
Line 71 C++
boost::archive::detail::interface_oarchive<boost::archive::text_oarchive>::operator<<<A>()
Line 81 C++
boost::archive::detail::interface_oarchive<boost::archive::text_oarchive>::operator&<A>()
Line 111 C++
DynamicArray<A>::save<boost::archive::text_oarchive>() Line 26 C++
boost::serialization::access::member_save<boost::archive::text_oarchive,DynamicArray<A>
>(boost::archive::text_oarchive & ar={...}, const DynamicArray<A> &
t={...}, const unsigned int file_version=0) Line 91 C++
boost::serialization::detail::member_saver<boost::archive::text_oarchive,DynamicArray<A>
>::invoke(boost::archive::text_oarchive & ar={...}, const
DynamicArray<A> & t={...}, const unsigned int file_version=0) Line 42 C++
boost::serialization::split_member<boost::archive::text_oarchive,DynamicArray<A>
>(boost::archive::text_oarchive & ar={...}, DynamicArray<A> & t={...},
const unsigned int file_version=0) Line 67 C++
DynamicArray<A>::serialize<boost::archive::text_oarchive>() Line 40 C++
boost::serialization::access::serialize<boost::archive::text_oarchive,DynamicArray<A>
>(boost::archive::text_oarchive & ar={...}, DynamicArray<A> & t={...},
const unsigned int file_version=0) Line 107 C++
boost::serialization::serialize<boost::archive::text_oarchive,DynamicArray<A>
>(boost::archive::text_oarchive & ar={...}, DynamicArray<A> & t={...},
const unsigned int file_version=0) Line 78 C++
boost::serialization::serialize_adl<boost::archive::text_oarchive,DynamicArray<A>
>(boost::archive::text_oarchive & ar={...}, DynamicArray<A> & t={...},
const unsigned int file_version=0) Line 121 C++
boost::archive::detail::oserializer<boost::archive::text_oarchive,DynamicArray<A>
>::save_object_data(boost::archive::detail::basic_oarchive & ar={...},
const void * x=0x0012ff54) Line 152 C++
boost::archive::detail::basic_oarchive_impl::save_object(boost::archive::detail::basic_oarchive
& ar={...}, const void * t=0x0012ff54, const
boost::archive::detail::basic_oserializer & bos={...}) Line 257 C++
boost::archive::detail::basic_oarchive::save_object(const void *
x=0x0012ff54, const boost::archive::detail::basic_oserializer &
bos={...}) Line 395 C++
boost::archive::detail::save_non_pointer_type<boost::archive::text_oarchive,DynamicArray<A>
>::save::invokex(boost::archive::text_oarchive & ar={...}, const
DynamicArray<A> & t={...}) Line 235 C++
boost::archive::detail::save_non_pointer_type<boost::archive::text_oarchive,DynamicArray<A>
>::invoke(boost::archive::text_oarchive & ar={...}, const
DynamicArray<A> & t={...}) Line 309 C++
boost::archive::save<boost::archive::text_oarchive,DynamicArray<A>
>(boost::archive::text_oarchive & ar={...}, const DynamicArray<A> &
t={...}) Line 551 C++
boost::archive::basic_text_oarchive<boost::archive::text_oarchive>::save_override<DynamicArray<A>
>() Line 71 C++
boost::archive::detail::interface_oarchive<boost::archive::text_oarchive>::operator<<<DynamicArray<A>
>() Line 81 C++
boost::archive::detail::interface_oarchive<boost::archive::text_oarchive>::operator&<DynamicArray<A>
>(DynamicArray<A> & t={...}) Line 111 C++
main(int argc=1, char * * argv=0x004f7478) Line 55 C++
mainCRTStartup() Line 398 C
77e9ca90
Robert Ramey
Jul 16, 2005, 12:19:32 AM7/16/05
to boost...@lists.boost.org
This is a different problem.
> DynamicArray<A> ArrayOfAs;
> ArrayOfAs.Element = new A[5];
> ArrayOfAs.Size = 5;
> A* secondElement = &ArrayOfAs.Element[1];
>
> std::stringstream stream;
> {
> boost::archive::text_oarchive oa(stream);
> oa & secondElement; // new object created here
> oa & ArrayOfAs; // attempt to reload an already loaded object
to new address - throw exception - pointer conflict
> }
the following should work:
oa & ArrayOfAs; // load objects into array
oa & secondElement; // load pointer to previouslt created object
Robert Ramey
Ivan Rachev wrote:
> DynamicArray<A> ArrayOfAs;
> ArrayOfAs.Element = new A[5];
> ArrayOfAs.Size = 5;
> A* secondElement = &ArrayOfAs.Element[1];
>
> std::stringstream stream;
> {
> boost::archive::text_oarchive oa(stream);
> oa & ArrayOfAs; // attempt to reload an already loaded object
to new address - throw exception - pointer conflict
> }
the following should work:
oa & ArrayOfAs; // load objects into array
oa & secondElement; // load pointer to previouslt created object
Robert Ramey
Ivan Rachev wrote:
A>::save::invokex(boost::archive::text_oarchive
> & ar={...}, const A & t={...}) Line 235 C++
>
>
boost::archive::detail::save_non_pointer_type<boost::archive::text_oarchive,
A>::invoke(boost::archive::text_oarchive
>
>
boost::archive::detail::save_non_pointer_type<boost::archive::text_oarchive,
Ivan Rachev
Jul 16, 2005, 8:10:48 AM7/16/05
to boost...@lists.boost.org
Robert Ramey wrote:
> This is a different problem.
>
>
>> DynamicArray<A> ArrayOfAs;
>> ArrayOfAs.Element = new A[5];
>> ArrayOfAs.Size = 5;
>> A* secondElement = &ArrayOfAs.Element[1];
>>
>> std::stringstream stream;
>> {
>> boost::archive::text_oarchive oa(stream);
>> oa & secondElement; // new object created here
>> oa & ArrayOfAs; // attempt to reload an already loaded object
>
> to new address - throw exception - pointer conflict
>
>> }
>
>
> the following should work:
>
> oa & ArrayOfAs; // load objects into array
> oa & secondElement; // load pointer to previouslt created object
>
> Robert Ramey
>
yes, it is a different problem. Could you suggest a solution to it?
> This is a different problem.
>
>
>> DynamicArray<A> ArrayOfAs;
>> ArrayOfAs.Element = new A[5];
>> ArrayOfAs.Size = 5;
>> A* secondElement = &ArrayOfAs.Element[1];
>>
>> std::stringstream stream;
>> {
>> boost::archive::text_oarchive oa(stream);
>> oa & secondElement; // new object created here
>> oa & ArrayOfAs; // attempt to reload an already loaded object
>
> to new address - throw exception - pointer conflict
>
>> }
>
>
> the following should work:
>
> oa & ArrayOfAs; // load objects into array
> oa & secondElement; // load pointer to previouslt created object
>
> Robert Ramey
>
Ivan
PS: If the array comes first the problem doesn't appear.
Ivan Rachev
Jul 16, 2005, 12:17:51 PM7/16/05
to boost...@lists.boost.org
Robert Ramey wrote:
> This is a different problem.
>
>
>> DynamicArray<A> ArrayOfAs;
>> ArrayOfAs.Element = new A[5];
>> ArrayOfAs.Size = 5;
>> A* secondElement = &ArrayOfAs.Element[1];
>>
>> std::stringstream stream;
>> {
>> boost::archive::text_oarchive oa(stream);
>> oa & secondElement; // new object created here
>> oa & ArrayOfAs; // attempt to reload an already loaded object
>
> to new address - throw exception - pointer conflict
>
>> }
I think over here we have a solution to try.
> This is a different problem.
>
>
>> DynamicArray<A> ArrayOfAs;
>> ArrayOfAs.Element = new A[5];
>> ArrayOfAs.Size = 5;
>> A* secondElement = &ArrayOfAs.Element[1];
>>
>> std::stringstream stream;
>> {
>> boost::archive::text_oarchive oa(stream);
>> oa & secondElement; // new object created here
>> oa & ArrayOfAs; // attempt to reload an already loaded object
>
> to new address - throw exception - pointer conflict
>
>> }
I believe at saving boost::serialization creates a map
(object address -> object id)
and at loading the lib creates the reverse map
(object id -> object address)
How inside the archive could I write a function like this:
ObjID GetObjectID(void* objectAddress);
Usage will be like this:
class A {
template<class Archive>
void save(Archive & ar, const unsigned int version) const
{
...
void save(Archive & ar, const unsigned int version) const
{
ObjID objId = ar.GetObjectID(this);
...
}
};
Also, after loading is done and the load map is created, how inside the
archive could I implement a function like this:
void* GetObjectAddress(ObjID objId);
Usage will be like this:
...
ObjID objId;
std::stringstream stream;
boost::archive::text_iarchive ia(stream);
ia & someObj;
ia & objId;
void* p = ar.GetObjectAddress(objId);
...
Ivan
Robert Ramey
Jul 17, 2005, 10:20:31 AM7/17/05
to boost...@lists.boost.org
The problem is is that de-serializing a pointer creates a new object in a
new address. Now one comes along an deseriailzed the same object to a fixed
address. There is no way to fix this without going back to the begining.
This situation is detected though tracking and a "pointer conflict"
exception is thrown. The solution is to be sure that if objects are
serialized directly and through pointers, the the pointer serialization is
done second.
Robert Ramey
Ivan Rachev wrote:
new address. Now one comes along an deseriailzed the same object to a fixed
address. There is no way to fix this without going back to the begining.
This situation is detected though tracking and a "pointer conflict"
exception is thrown. The solution is to be sure that if objects are
serialized directly and through pointers, the the pointer serialization is
done second.
Robert Ramey
Ivan Rachev wrote:
Ivan Rachev
Aug 8, 2005, 10:54:57 AM8/8/05
to boost...@lists.boost.org
>> The solution is to be sure that if objects are
>> serialized directly and through pointers, the
>> the pointer serialization is done second.
yes, whenever a pointer serialization is done it should be defered and serialized after the direct serialization. My believe is that the lib keeps an object-address-to-object-id mapping (BoostSaveMap) when saving and an object-id-to-object-address mapping (BoostLoadMap) when loading objects from an archive so that one and the same object does not get serialized twice. Based on this assumption there is a solition for the following problematic scenario:
>> serialized directly and through pointers, the
>> the pointer serialization is done second.
struct TChar { char ch; };
template <typename T> struct DynamicArray
{ int Size;
T* Element;
};
{ int Size;
T* Element;
typedef DynamicArray<TChar> TCharArray;
class A { TChar * ptr; } a;
class B { TCharArray array; } b;
b.array.Element = new TChar[3];
b.array.Size = 3;
a.ptr = b.array.Element[1];
ar & a; // at loading, new object created here
ar & b; // attempt to reload an already loaded object
// to new address - throw exception - pointer conflict
patch(ar); // fix to problem that needs implementation
For the solution itself, I see the following that needs to happen at saving&loading:
When saving, I see 3 steps:
1) whenever TChar* is being saved, adding the TChar* pointer to InternalPtrVector but not sending it to the archive (sending is in step 3). This vector is a mapping of internal pointer (a.ptr) to holder id (ID of a) and contains offset of the pointer (a.ptr) in the object that holds this pointer (a). So for each TChar* in the above example, this vector knows the id of the object that holds the TChar* and its offset in this object.
2) whenever DynamicArray is being saved, adding it to ArrayRangeVector and sending it to the archive. Adding is done by writing the following into the ArrayRangeVector structure: 1.array address (b.array), 2.array size (b.array.Size), 3.array id. This structure knows the range of addresses used by a particular array and the id that boost::serialization gave to the array at saving.
3) after formal serialization (in the patch func) iterate over InternalPtrVector and for each internal pointer there, check if it's within the range of any of the arrays that are stored in ArrayRangeVector. If such an array is found, write the following InternalPtrFileStruct to the archive:
Holder ID, (ID of a in the example above)
Holder Offset, (offset of a.ptr in a)
Array ID, (ID of b.array)
Array Offset ( a.ptr - b.array.Element )
The structures used when saving are:
BoostLoadMap
---------------------------------
| object ID -> object address |
---------------------------------
InternalPtrVector
-------------------------------------------------
| InternalPtr | Holder ID | Holder Offset |
-------------------------------------------------
ArrayRangeVector
-----------------------------------------------
| Array Address | Array Size | Array ID |
-----------------------------------------------
When loading, I see 1 step:
When formal loading is going on, nothing happens to TChar* members. They
are written into afterwords. Actual objects (TChar in the case above)
are loaded when loading the dynamic array. After formal serialization
(ar & a; ar & b; ) is done, TChar* members are initialized in the patch
func. What it should do is the following:
read each InternalPtrFileStruct. Look up holder object address (&a) in
BoostLoadMap by Holder ID (ID of a). Add Holder offset to this address
and the result is the address of TChar* in the case above. Now we need
to calculate the value for the TChar* member. This is done by getting
the address of the array (&b.array.Element) from BoostLoadMap using
Array ID (id of b.array). Add Array Offset to this address and the
result is the value of the TChar* memeber. So write this value into the
address of TChar*.
The structure used at loading is :
BoostLoadMap
---------------------------------
| object address -> object ID |
---------------------------------
In order for this to work there needs to be some way of accessing
BoostLoadMap and BoostSaveMap from users of boost::serialization (this
is where Robert could help). Of course, I believe the better way would
be if all this functionality is incorporated into the lib. That way or
another there needs to be serialization code similar to the following:
// user code below this line
---------------------------------------------------------------------------
template <class Archive> void serialize(Archive& ar, A& obj)
{ saveInternalPtr(obj.ptr, obj, ar);
ar & obj.other_members;
}
template <class Archive> void serialize(Archive& ar, B& obj)
{ ar & obj.array;
}
// lib code below this line
---------------------------------------------------------------------------
template <class Archive> saveInternalPtr(void* InternalPtr , void*
objectAddress, Archive& ar)
{ if (ar.is_saving) {
// get object id by objectAddress from BoostSaveMap
InternalPtrVector.insert(InternalPtr, objectID, InternalPtr -
objectAddress);
}
}
template <class Archive, typename ElemT> void serialize(Archive& ar,
DynamicArray<ElemT>& obj)
{ ar & obj.Size;
if (ar.is_saving) {
for (int i = 0; i < obj.Size; ++i) ar & obj.Element[i];
// get array id by array Address from BoostSaveMap
ArrayRangeVector.insert(&obj, obj.Size, arrayID);
} else
{
assert( obj.Size >=0 );
if (obj.Size > 0) {
obj.Element = new ElemT[obj.Size];
for (int i = 0; i < obj.Size; ++i) ar & obj.Element[i];
}
else obj.Element = null;
}
}
template <class Archive> void patch(Archive& ar)
{ ar & InternalPtrVector.size;
if (ar.is_saving)
// for each internal pointer in InternalPtrVector
// for each array in ArrayRangeVector
// if internal pointer within [ArrayRangeVector[j].arrayAddress,
ArrayRangeVector[j].arrayAddress + ArrayRangeVector[j].arraySize]
// ar & InternalPtrVector[i].holderID;
// ar & InternalPtrVector[i].holderOffset;
// ar & ArrayRangeVector[j].arrayID;
// ar & InternalPtrVector[i].InternalPtr -
ArrayRangeVector[j].arrayAddress;
else
// for (int i = 0; i < InternalPtrVector.size; ++i)
// ar & holderID;
// get holder address by holderID from BoostLoadMap
// ar & holderOffset;
// void * memberAddress = holderAddress + holderOffset; /* this is
address of TChar* in the example above */
// ar & arrayID;
// get array address by arrayID from BoostLoadMap
// ar & arrayOffset;
// int memberValue = arrayAddress + arrayOffset; /* this is value
of TChar* in the example above */
// *memberAddress = memberValue;
}
Ivan
Ivan Rachev
Aug 8, 2005, 5:04:59 PM8/8/05
to boost...@lists.boost.org
>> The solution is to be sure that if objects are
>> serialized directly and through pointers, the
>> the pointer serialization is done second.
>> serialized directly and through pointers, the
>> the pointer serialization is done second.
yes, whenever a pointer serialization is done it should be defered and serialized after the direct serialization. My believe is that the lib keeps an object-address-to-object-id mapping (BoostSaveMap) when saving and an object-id-to-object-address mapping (BoostLoadMap) when loading objects from an archive so that one and the same object does not get serialized twice. Based on this assumption there is a solition for the following problematic scenario:
struct TChar { char ch; };
struct TChar { char ch; };
template <typename T> struct DynamicArray
{ int Size;
T* Element;
{ int Size;
T* Element;
Reply all
Reply to author
Forward
0 new messages