[Boost-users] [archive] binary archive to memory stream

[Michael Powell]

Hello,

I am wrapping my head around archive, specifically binary archive.

I'd like to serialize a vector of objects in such a way, I need to
capture intermediate vector buffers (literally, to vector of uint8_t)
so that I can determine their sizes.

After which point the parent object(s) can serialize to file using
more conventional, built-in file streams and such.

It seems that binary_oarchive is helpful, but still requires there be
an ostream or streambuf in the mix. Which, as an adapter layer, while
helpful, it still leaves the question of extending streambuf to work
with ostream to vector<uint8_t> open.

I've read some docs on vectors and streambuf, but they all seem to be
geared towards a known-size vector, istream and such. I am interested
in a vector that can be streamed to, will grow in size, etc. Some
clues seem to hint at underflow, overflow type issues?

Any helpful pointers how to go about doing this?

Thanks...

Best regards,

Michael Powell
_______________________________________________
Boost-users mailing list
Boost...@lists.boost.org
http://lists.boost.org/mailman/listinfo.cgi/boost-users

[John Maddock]

> I am wrapping my head around archive, specifically binary archive.
>
> I'd like to serialize a vector of objects in such a way, I need to
> capture intermediate vector buffers (literally, to vector of uint8_t)
> so that I can determine their sizes.
>
> After which point the parent object(s) can serialize to file using
> more conventional, built-in file streams and such.
>
> It seems that binary_oarchive is helpful, but still requires there be
> an ostream or streambuf in the mix. Which, as an adapter layer, while
> helpful, it still leaves the question of extending streambuf to work
> with ostream to vector<uint8_t> open.
>
> I've read some docs on vectors and streambuf, but they all seem to be
> geared towards a known-size vector, istream and such. I am interested
> in a vector that can be streamed to, will grow in size, etc. Some
> clues seem to hint at underflow, overflow type issues?
>
> Any helpful pointers how to go about doing this?

Rather than serializing to a vector, can you not serialize to a
stringstream opened in binary mode? I believe that will still give you
all the information you require?

HTH, John.

[Michael Powell]

On Wed, Jun 18, 2014 at 10:56 AM, John Maddock <boost...@virgin.net> wrote:
>> I am wrapping my head around archive, specifically binary archive.
>>
>> I'd like to serialize a vector of objects in such a way, I need to
>> capture intermediate vector buffers (literally, to vector of uint8_t)
>> so that I can determine their sizes.
>>
>> After which point the parent object(s) can serialize to file using
>> more conventional, built-in file streams and such.
>>
>> It seems that binary_oarchive is helpful, but still requires there be
>> an ostream or streambuf in the mix. Which, as an adapter layer, while
>> helpful, it still leaves the question of extending streambuf to work
>> with ostream to vector<uint8_t> open.
>>
>> I've read some docs on vectors and streambuf, but they all seem to be
>> geared towards a known-size vector, istream and such. I am interested
>> in a vector that can be streamed to, will grow in size, etc. Some
>> clues seem to hint at underflow, overflow type issues?
>>
>> Any helpful pointers how to go about doing this?
>
>
> Rather than serializing to a vector, can you not serialize to a stringstream
> opened in binary mode? I believe that will still give you all the
> information you require?

So far so good I think. This is binary data, not "string" data. Or at
least, not to be confused with C-style-string, null-terminator
characters, per se. No chance of that being the case with
stringstream, extracting using the str() function, and so on?

Thank you...

[Michael Powell]

On Wed, Jun 18, 2014 at 5:20 PM, Michael Powell <mwpow...@gmail.com> wrote:
> On Wed, Jun 18, 2014 at 10:56 AM, John Maddock <boost...@virgin.net> wrote:
>>> I am wrapping my head around archive, specifically binary archive.
>>>
>>> I'd like to serialize a vector of objects in such a way, I need to
>>> capture intermediate vector buffers (literally, to vector of uint8_t)
>>> so that I can determine their sizes.
>>>
>>> After which point the parent object(s) can serialize to file using
>>> more conventional, built-in file streams and such.
>>>
>>> It seems that binary_oarchive is helpful, but still requires there be
>>> an ostream or streambuf in the mix. Which, as an adapter layer, while
>>> helpful, it still leaves the question of extending streambuf to work
>>> with ostream to vector<uint8_t> open.
>>>
>>> I've read some docs on vectors and streambuf, but they all seem to be
>>> geared towards a known-size vector, istream and such. I am interested
>>> in a vector that can be streamed to, will grow in size, etc. Some
>>> clues seem to hint at underflow, overflow type issues?
>>>
>>> Any helpful pointers how to go about doing this?
>>
>>
>> Rather than serializing to a vector, can you not serialize to a stringstream
>> opened in binary mode? I believe that will still give you all the
>> information you require?

It works. If I really wanted to I could throw the Boost decoration on
top of it, but so far not necessary. Thanks.

[Sebastian Messerschmidt]

Hi Michael,

I've used something like this for

writing:

typedef std::vector<char> BufferType;
typedef
boost::iostreams::stream<boost::iostreams::back_insert_device<BufferType>>
OBufferStream;
typedef boost::archive::binary_oarchive OutArchive;

Buffertype my_data_array;
OBufferStream buffer_stream(my_data_array);
OutArchive archive(buffer_stream);

archive << ... //normal boost::serialization thing here



and reading:

typedef boost::iostreams::stream<boost::iostreams::basic_array_source
<char> > IBufferStream;
typedef boost::archive::binary_iarchive InArchive;

IBufferStream i_buffer_stream(&my_data_array[0], my_data_array.size());
InArchive i_archive(i_buffer_strea,)

archive << ... //normal boost::serialization thing here

This works fine at the cost of one copy of the array for reading, and
gives you the advantage of knowing the buffer size of the serialized data.
Note, the code is more or less out of the top of my head, so I cannot
guarantee it is compiling.

cheer
Sebastian

[Robert Ramey]

stringstream is basically std::string with a layer to make it compatible with
std::streambuf. It is NOT a C string and the 00 character has no special
significance. I also believe that it is exactly what you're looking for. On
the other hand, your original explanation leaves me mystified as to what
you're actually trying to do. So I could be wrong.

Robert Ramey



--
View this message in context: http://boost.2283326.n4.nabble.com/archive-binary-archive-to-memory-stream-tp4664294p4664332.html
Sent from the Boost - Users mailing list archive at Nabble.com.