Serializing data using streams?

[Kai Mast]

Hi everybody,

Something that I miss in C++ is an easy way to write types (that aren't POD) to a buffer, and read them later.

Similar to IO and string streams, it would be cool to have a "bitstream" that does this for me.

The idea would be that folks would only need to overload the << and >> operators for a custom type.

I have written such a class for all my own projects. Is this something that could be a future proposal? Or is there maybe a better way to do serialization in C++?

The repo has code and some more examples.

https://github.com/kaimast/bitstream

Thanks,
Kai

(And sorry if this has already been suggested or there is a feature in the works that I don't know of that provides a similar feature.)

[Nicol Bolas]

On Saturday, February 3, 2018 at 7:41:48 PM UTC-5, Kai Mast wrote:

Hi everybody,

Something that I miss in C++ is an easy way to write types (that aren't POD) to a buffer, and read them later.

I'm going to pretend that by "POD" you really meant "trivially copyable".

Similar to IO and string streams, it would be cool to have a "bitstream" that does this for me.

The idea would be that folks would only need to overload the << and >> operators for a custom type.

I have written such a class for all my own projects. Is this something that could be a future proposal? Or is there maybe a better way to do serialization in C++?

The repo has code and some more examples.

https://github.com/kaimast/bitstream

Well as previously stated, your `is_pod` tests should be replaced with `is_trivially_copyable`.

So, you've provided a serialization framework that is essentially "write your own serializers". And the way to specialize the serializer for your user-created types is... what, exactly? What is the specialization point for writing your type to the stream or reading it from the stream? I'm honestly not sure how well `operator<<` overloading works in this case, since you already have a function that doesn't use SFINAE based on `is_trivially_copyable`. I don't know how the rules of overload resolution work when picking between a member function and a non-member one, but it's really not something you should leave up to esoteric rules like that.

And why is your `bitstream` not using a `vector<std::byte>` as what it writes to, or some similar standard library container? It's essentially just a stream wrapper around a container. Indeed, it would be useful to be able to have it select between `vector<std::byte>` and `span<std::byte>` as needed. And what of writing to/reading from stream buffers? If you want to write data out to a file, do you have to write it to a big memory buffer first?

Not to mention there's the lack of allocator support.

Overall, this is a pretty no-frills serialization system.

[Ricardo Fabiano de Andrade]

Nicol,

Please stop being so unwelcoming to first-timers!

As far as I know being up-to-date to the latest standard is not a requirement to post here.

Nor that's a requirement for good ideas.

There are several ways to point improvements to the OP and yours are always the worst.

If you goal is to chase away contributions to the future of C++, congrats you're getting there!

Kai,

Unofficially (since I am no ISO committee member, just someone looking into C++ improvements), my apologies for the reception to your proposal.

Even though I don't like the way `is_pod` and `std::byte` issues were pointed out they are correct indeed.

POD is no longer a thing. The concept of trivially copyable supercedes it.

And std::byte is new standard-sanctioned way to deals with binary data.

Please inform yourself about them: std::byte.

What concerns me in your proposal are two points:

1) Does it comes to invoke undefined behavior while turning objects into a stream of bytes and back?

2) Are you aware of the reflection proposals for C++? (p0194r4) If so, how do you expect your proposal to interact with it?

Other than that, I don't see the idea itself being inherently bad.

You also should look into bit_cast!

Good luck with your proposal.

--
You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to std-proposals+unsubscribe@isocpp.org.
To post to this group, send email to std-pr...@isocpp.org.
To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/e561abcd-a698-4bb1-b95e-16ef3c53533c%40isocpp.org.

[Richard Hodges]

> Please stop being so unwelcoming to first-timers!

Agreed.

To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/CA%2BfGSbP16ReuJ_cWjQz42JY-Pqr_V3tzcijrrUC0npGgHKg-JQ%40mail.gmail.com.

[Nicol Bolas]

On Sunday, February 4, 2018 at 10:40:18 AM UTC-5, Ricardo Andrade wrote:

Nicol,

Please stop being so unwelcoming to first-timers!

I didn't notice if he was a first-timer or not. I responded to the quality of the content, not to the poster of it.

As far as I know being up-to-date to the latest standard is not a requirement to post here.

You say that like that's a good thing. If someone is going to seriously suggest that something ought to be changed, is it not reasonable to expect them to be at least conversant with how it currently works?

If someone comes around and says that the definition of POD should be expanded to include more types so that they can `memcpy` them around, and their proposed POD definition mirrors TriviallyCopyable exactly, then that post is a complete waste of everyone's time.

I don't expect complete expertise. We've all been informed of esoteric rules of C++ that we didn't know beforehand. But TriviallyCopyable is not esoterica. It's been a part of C++ for six years. The top hit on Google for "C++ POD" is a StackOverflow question about what they are, and the highest voted answer explicitly states that the definition was changed in C++11, linking directly to an explanation of the StandardLayout/TriviallyCopyable distinction.

I don't expect everyone to know it. But it's hardly unreasonable to say that if you don't know it, then you probably don't know enough about the C++ standard to be able to make a serious proposal for changing it.

Nor that's a requirement for good ideas.

But we don't review ideas here; we review proposals (that's why it's called "Future Proposals" and not "Ideas"). Proposals are a lot more than just ideas.

Nobody is saying that C++ having serialization is a bad idea. But whether serialization in general is good is irrelevant next to a specific proposal for serialization. And this proposal is not good.

A good proposal would include motivation. A good proposal would show research into alternative designs and explain why the presented one is better. A good proposal would not have such obvious deficiencies in it.

Proposals exist to fix problems. But just because a problem is significant enough to be fixed does not mean that any fix for it is a good thing.

There are several ways to point improvements to the OP and yours are always the worst.

If you goal is to chase away contributions to the future of C++, congrats you're getting there!

My goal is to improve the proposal by pointing out some of its deficiencies. At no time did I direct any of my criticism at the poster in question; it was always directed at their code/proposal. If that causes "contributors to the future of C++" to be "chased away", I shudder to think what would happen at a committee meeting.

It should also be noted that serialization is well-trod ground when it comes to C++. There are many libraries out there which provide serialization features. A good, well-thought out proposal would actually talk about them, investigate their design choices, and explain why the presented design is preferred or superior.

Just look at all of the papers that get submitted to the committee. They're almost never just "here's this thing, maybe we should standardize it". They spend lots of time researching different approaches to the problem they're trying to solve.

Why should the standards of this forum be so much lower than that?

[Jake Arkinstall]

I dislike making responses like this. It isn't professional. But given that this reflects my experience, too, I will write it anyway.

On 5 Feb 2018 00:03, "Nicol Bolas" <jmck...@gmail.com> wrote:

But we don't review ideas here; we review proposals (that's why it's called "Future Proposals" and not "Ideas"). Proposals are a lot more than just ideas.

Perhaps you can spread the message to the countless people in online communities that suggest OPs post to their idea to the C++ proposals group, saying that it would be a good experience for them. Proposals ARE a lot more than just ideas, but this is called "FUTURE" proposals, and there is no "Ideas" forum. It's where proposals go to school so they can grow up in a good environment, get a good chance at a stable career and a good retirement plan, not where they get detention for wearing the wrong shoes on their first day. Maybe this is cause to create a more accessible forum below this one, or a stricter forum above this one, because there is a wide net here. And there is a wide net.

A good proposal would include motivation. A good proposal would show research into alternative designs and explain why the presented one is better. A good proposal would not have such obvious deficiencies in it.

Two points. First, it would not have harmed you to have asked those as questions clearly in the first place. Second, your attitude after still discovering that the OP is new is unchanged. Leave your ego at the keyboard for crying out loud. It isn't what you're saying, which in most places I agree with, it's how you're saying it; it doesn't make for an encouraging environment, outside of ancient Sparta.

My goal is to improve the proposal by pointing out some of its deficiencies. At no time did I direct any of my criticism at the poster in question; it was always directed at their code/proposal.

If you state that posts like this a waste of everyone's time at all reflects that attitude, you are wrong.

Why should the standards of this forum be so much lower than that?

Because this isn't the standards committee. In fact, proposals that are pretty much complete get on here and are either ignored or there are just a couple of messages saying "yep, go for it" (for example, mihailnajdenov's awesome function_view post). If you want the standards of the committee then sit on the damned thing. But if you want to help get ideas to proposal stage, even if they are rough around the edges, then this is the place to be. I agree that we need to certain standards, but we can't have people being put off from posting proposals. It's quite easy to ignore a post if it is below you, there are plenty of others who enjoy helping.

[gmis...@gmail.com]

Nicely put. It's a viewpoint I share. Thanks for taking the time to articulate this.

[schreiber...@gmail.com]

This is getting quite off-topic, but as a long time-reader of these forums, I have felt the need to intervene in the face of Nicol's attitude on a number of occasions. Now is as good a time as any...

To start with I would refrain from accusing him/her of having an oversized ego. This is not helping, and I'm not sure this is actually true. The way I see it, Nicol simply seems to have abstracted away the fact that the people who post here are not instances of the ProposalWriter class, but plain human beings. I don't think Nicol is the only one around here to make this mistake (although no one posts nearly half as often). That would be fine if the point of these forums was to present proposals for inspection and carve them out until they are perfect jewels, ready to be merged in the C++ standard. In fact, I would be reassured to know that people playing this role are as knowledgeable and keen eyed as Nicol when it comes to understanding C++ and finding flaws in said proposals. But as has been pointed out already, this is ultimately what the committee is for, and it is not clear to me that this is true of these forums as well.

That being said, the tone that you employ, Nicol, is indeed aggressive. Whether you mean it or not, that is how people perceive it (myself included). To draw one example from your first message in this thread, starting off by "pretending" something about what the OP wrote is not the nice way to go (regardless of who they are). You could have "assumed" instead, and phrased it as a question instead of a peremptory assertion. It is this type of small details that I have seen accumulating in most messages you write. And in the end they matter: rather than reading your arguments and think "oh yeah he's right", my first reaction is to be angry at you for seeming rude without provocation. You can be blunt and straight to the point, don't get me wrong, there is no need for flowers and cushions. But I think you cannot afford to be aggressive; adopting such stance is not going to get the best out of anyone, and is not going get your point through any better.

I hope you see the point, and that you still keep sharing your knowledge and your time with the people around here.

On Monday, February 5, 2018 at 10:13:30 AM UTC+1, Jake Arkinstall wrote:

I dislike making responses like this. It isn't professional. But given that this reflects my experience, too, I will write it anyway.

[Jakob Riedle]

Thanks for your courage to constructively talk about this whole topic. I mostly agree.

[ey...@sentinelone.com]

There are plenty of frameworks that do exactly that, you might want to take a look at some, for example this one

is an easy read and single header like yours: https://github.com/eyalz800/serializer