stringstream and move support

[Nicol Bolas]

On the old comp.std.c++ Usenet list, I made a suggestion about adding the ability to move `basic_string`s into and out-of `basic_stringstream`s. I'm considering making this an actual C++ proposal, so I wanted to gauge the interest in doing this.

The general idea would be fairly simple, though it would have effects on the implementation of the stringstreams. In effect, they would have to be written to either internally use `basic_string` for their array of characters, or their implementations would have to be synchronized with `basic_string`, so that memory allocated between them could be transferred appropriately.

The syntactic implication is quite simple. `basic_ostringstream` and `basic_stringstream` would have a constructor that takes a `basic_string` of the same character type. This constructor would adopt the `basic_string`'s memory, as though it had moved the `basic_string` into an internal `basic_string` instance.

There would also be a `move_str` member function which returns a `basic_string` by value (it can't be returned by `&&`), but it leaves the `basic_stringstream` or `basic_istringstream` in the infamous "valid but undefined" state.

This would be part of a larger package of general changes with respect to iostreams, such as better integration (by which I mean some integration) with `string_view` or whatever we're calling it, improved allocator support for the `stringstreams` (they don't even take allocator objects. People seemed to have completely forgotten about `stringstream`s completely when they added stateful allocators), and a couple of other things.

So is there any interest in this?

[Ville Voutilainen]

On 10 May 2013 19:14, Nicol Bolas <jmck...@gmail.com> wrote:

On the old comp.std.c++ Usenet list, I made a suggestion about adding the ability to move `basic_string`s into and out-of `basic_stringstream`s. I'm considering making this an actual C++ proposal, so I wanted to gauge the interest in doing this.

Absolutely. It would decrease the need to use strstreams, too.

[Martinho Fernandes]

On Fri, May 10, 2013 at 6:14 PM, Nicol Bolas <jmck...@gmail.com> wrote:

So is there any interest in this?

Totally! I have pined for this feature a few times before, and have at one time considered writing my own stringbuf with similar functionality, but never got to it.

[Zhihao Yuan]

I'm wondering if a `.str() &&` overload is enough for our need.

--
Zhihao Yuan

--
 
---
You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to std-proposal...@isocpp.org.
To post to this group, send email to std-pr...@isocpp.org.
Visit this group at http://groups.google.com/a/isocpp.org/group/std-proposals/?hl=en.
 
 

[Nicol Bolas]

On Friday, May 10, 2013 8:14:58 PM UTC-7, Zhihao Yuan wrote:

I'm wondering if a `.str() &&` overload is enough for our need.

How would you call that, exactly? Given some ostringstream, how do you move from it? It would be wierd to do this:

ostringstream o;
//Fill o
std::move(o).str();

It just makes more sense to do:

o.move_str();

It's also more compact, and it's clear what's going on.

[Zhihao Yuan]

Yea, it looks more length, but it's using an existing semantics, so that we don't need to explain anything about the state of the stream after calling move_str(); people already know std::move.

--
Zhihao Yuan

[Nicol Bolas]

On Friday, May 10, 2013 11:44:24 PM UTC-7, Zhihao Yuan wrote:

Yea, it looks more length, but it's using an existing semantics, so that we don't need to explain anything about the state of the stream after calling move_str(); people already know std::move.

... So the sole reason to force people to use this tortured syntax rather than a clear function call is solely to avoid having to explain what it means in the standards text? It's called `move_str`; how hard could it possibly be to understand?

It's a lot harder to explain to users why using `std::move` on the stream followed by a call to `str` results in moving the string out of the stream than it is explaining why `move_str` does that.

Furthermore, the semantics aren't correct and would generally be surprising. Consider this:

std::string s("Here is a string that contains characters.");

std::string::iterator stuff = std::move(s).begin();

What are the contents of `s` after this code? The standard is very clear what those contents are: the same contents as before. Nothing has been moved from the string yet.

Whereas `std::move(o).str()` will always leave the stream in the valid-but-unspecified state. That function will behave differently from virtually any other expression of the form `std::move(x).function_call()`.

Unless you're suggesting that `str() &&` should return a `std::basic_string &&`. In which case, you're forcing the implementation to implement `basic_stringbuf` by using an internal `basic_string`. And I don't see any reason why we need to enforce that.

[DeadMG]

Permitting easy move semantics to and from basic_stream is a good reason to enforce it.

[Marc]

Le samedi 11 mai 2013 05:14:58 UTC+2, Zhihao Yuan a écrit :

I'm wondering if a `.str() &&` overload is enough for our need.

The standard gives implementations quite a bit of freedom. Overloading .str(string&&) is probably legal without changing the standard. Overloading .str()&& (and replacing the existing .str() with .str()const&) might be allowed as well.

That doesn't mean a more explicit syntax with stronger guarantees wouldn't be welcome, but it is always good to be sure of where we are starting from.

[Ville Voutilainen]

On 11 May 2013 21:57, DeadMG <wolfei...@gmail.com> wrote:

Permitting easy move semantics to and from basic_stream is a good reason to enforce it.

move(ss).str() doesn't seem to be on the same planet with "easy move semantics", especially compared
to ss.move_str().

[DeadMG]

You're right, because when I do `f().str()`, then it works and seamlessly upgrades existing code, and move_str unnecessarily duplicates this functionality, forcing anyone who wants to use it to change all their code, for absolutely no benefit whatsoever.

[Ville Voutilainen]

On 11 May 2013 22:16, DeadMG <wolfei...@gmail.com> wrote:

You're right, because when I do `f().str()`, then it works and seamlessly upgrades existing code, and move_str unnecessarily duplicates this functionality, forcing anyone who wants to use it to change all their code, for absolutely no benefit whatsoever.

Mmkay. And then when users do x.str(), they will be utterly puzzled that they actually have to std::move(x).str() to get what
they want. The benefit is readability. The rvalue-ref overload of str() doesn't have it.

[Ville Voutilainen]

Well, to be fair, adding a stringstream(string&&) and stringstream::str(string&&) would seem to be the first

order of business, those seem uncontroversial. Getting the benefit for existing rvalues as you point out
would call for having stringstream::str() &&. Whether we additionally add stringstream::move_str() is

a nice bonus, but not something that we desperately need. I can get over the ugliness of the rvalue
str().

[Nicol Bolas]

On Saturday, May 11, 2013 12:07:31 PM UTC-7, Marc wrote:

Le samedi 11 mai 2013 05:14:58 UTC+2, Zhihao Yuan a écrit :

I'm wondering if a `.str() &&` overload is enough for our need.

The standard gives implementations quite a bit of freedom. Overloading .str(string&&) is probably legal without changing the standard. Overloading .str()&& (and replacing the existing .str() with .str()const&) might be allowed as well.

I'm not sure what you mean by "without changing the standard". Could an implementation add those overloads as an extension? Yes. But you can't rely on it "without changing the standard" to require that implementations provide them.

Also, adding `.str() &&` would change apparent behavior as well, as it would leave the stringstream in the moved-from state.

[Nicol Bolas]

On Saturday, May 11, 2013 12:16:28 PM UTC-7, DeadMG wrote:

You're right, because when I do `f().str()`, then it works and seamlessly upgrades existing code, and move_str unnecessarily duplicates this functionality, forcing anyone who wants to use it to change all their code, for absolutely no benefit whatsoever.

Even ignoring the useability issues that Ville and I have pointed out, how often do you have functions that return `stringstream`s? The most common use-case for `istringstream`s is something like this:

std::stringstream build;
build << /*stuff*/
std::string data = build.str();

Most of the time when you're working with a stringstream, you're working with an lvalue, either passed around by non-const reference (in `operator<<` overloads) or created on the stack as above.

So I would say that there is no "existing code" to "seamlessly update". Or at least, there's not nearly enough to justify doing it this way. So the vast majority of users are going to have to "change all their code" to use it. At least with the given suggestion, what they'll change their code into will make sense.

[Marc]

Le dimanche 12 mai 2013 05:26:34 UTC+2, Nicol Bolas a écrit :

On Saturday, May 11, 2013 12:07:31 PM UTC-7, Marc wrote:

Le samedi 11 mai 2013 05:14:58 UTC+2, Zhihao Yuan a écrit :

I'm wondering if a `.str() &&` overload is enough for our need.

The standard gives implementations quite a bit of freedom. Overloading .str(string&&) is probably legal without changing the standard. Overloading .str()&& (and replacing the existing .str() with .str()const&) might be allowed as well.

I'm not sure what you mean by "without changing the standard". Could an implementation add those overloads as an extension?

Not "as an extension", as a valid implementation of the standard.
 

Yes. But you can't rely on it "without changing the standard" to require that implementations provide them.

QoI issue. If your library doesn't implement it, you can file a bug with your vendor telling them that performance sucks. There are many places where the library is allowed to make needless copies, and you can only rely on the lack of copies through QoI.

Also, adding `.str() &&` would change apparent behavior as well, as it would leave the stringstream in the moved-from state.

 
Yes, that's why I was asking if the generic text about rvalues and moving in the standard library covers this case.

Again, I am not trying to prevent a more explicit syntax, which I think would be welcome, I just think this should be clarified first, and then you could in the rationale describe the difference between your new functions and those (possibly just a nicer syntax and a stronger signal to implementers, but maybe more differences).

[Nicol Bolas]

On Sunday, May 12, 2013 1:25:29 AM UTC-7, Marc wrote:

Le dimanche 12 mai 2013 05:26:34 UTC+2, Nicol Bolas a écrit :

On Saturday, May 11, 2013 12:07:31 PM UTC-7, Marc wrote:

Le samedi 11 mai 2013 05:14:58 UTC+2, Zhihao Yuan a écrit :

I'm wondering if a `.str() &&` overload is enough for our need.

The standard gives implementations quite a bit of freedom. Overloading .str(string&&) is probably legal without changing the standard. Overloading .str()&& (and replacing the existing .str() with .str()const&) might be allowed as well.

I'm not sure what you mean by "without changing the standard". Could an implementation add those overloads as an extension?

Not "as an extension", as a valid implementation of the standard.

No, you can't. `str` is not allowed to affect the state of the stream in any way. There's no viable way to move a string out of `str` that doesn't affect the stream. At least, not without making the "move" into a "copy", which defeats the entire point of the optimization.

Also, adding `.str() &&` would change apparent behavior as well, as it would leave the stringstream in the moved-from state.

 
Yes, that's why I was asking if the generic text about rvalues and moving in the standard library covers this case.

No. And there never will be.

To allow such text would create chaos. It would mean that you have no idea what the state of an object is in when you call it on something that happens to bind to the `&&` version rather than the `&` version of the function.

Movement is not something that can ever be implicit or accidental. It must be explicitly stated by every function that attempts to move.

[Olaf van der Spek]

On Friday, May 10, 2013 6:14:32 PM UTC+2, Nicol Bolas wrote:

On the old comp.std.c++ Usenet list, I made a suggestion about adding the ability to move `basic_string`s into and out-of `basic_stringstream`s. I'm considering making this an actual C++ proposal, so I wanted to gauge the interest in doing this.

Interesting!

What about variants that take the buffer by reference and input/output directly into the buffer, with buffer (views) being generic? 

Isn't an implementation allowed to move a string into a stringstream already?

[Marc]

 
Because movement is explicit (either you passed a temporary, or you passed move(obj)), binding to && is not an accident, it is done on purpose, and there is thus no way it creates chaos.

Note for instance 17.6.4.9:
"... If a program casts an lvalue to an
xvalue while passing that lvalue to a library function (e.g. by calling the function with the argument
move(x)), the program is effectively asking that function to treat that lvalue as a temporary. The
implementation is free to optimize away aliasing checks which might be needed if the argument was
an lvalue."
which in conjonction with 17.6.5.15 and 17.6.5.5 could be read to imply that we already have permission. Or not, that's not clear. But please stop with the chaos nonsense.

[Nicol Bolas]

On Sunday, May 12, 2013 6:21:23 AM UTC-7, Marc wrote:

Le dimanche 12 mai 2013 11:45:02 UTC+2, Nicol Bolas a écrit :

On Sunday, May 12, 2013 1:25:29 AM UTC-7, Marc wrote:

Le dimanche 12 mai 2013 05:26:34 UTC+2, Nicol Bolas a écrit :

On Saturday, May 11, 2013 12:07:31 PM UTC-7, Marc wrote:

Le samedi 11 mai 2013 05:14:58 UTC+2, Zhihao Yuan a écrit :

I'm wondering if a `.str() &&` overload is enough for our need.

The standard gives implementations quite a bit of freedom. Overloading .str(string&&) is probably legal without changing the standard. Overloading .str()&& (and replacing the existing .str() with .str()const&) might be allowed as well.

I'm not sure what you mean by "without changing the standard". Could an implementation add those overloads as an extension?

Not "as an extension", as a valid implementation of the standard.

No, you can't. `str` is not allowed to affect the state of the stream in any way. There's no viable way to move a string out of `str` that doesn't affect the stream. At least, not without making the "move" into a "copy", which defeats the entire point of the optimization.

Also, adding `.str() &&` would change apparent behavior as well, as it would leave the stringstream in the moved-from state.

 
Yes, that's why I was asking if the generic text about rvalues and moving in the standard library covers this case.

No. And there never will be.

To allow such text would create chaos. It would mean that you have no idea what the state of an object is in when you call it on something that happens to bind to the `&&` version rather than the `&` version of the function.

Movement is not something that can ever be implicit or accidental. It must be explicitly stated by every function that attempts to move.

 
Because movement is explicit (either you passed a temporary, or you passed move(obj)), binding to && is not an accident, it is done on purpose, and there is thus no way it creates chaos.

It creates chaos because the standard is supposed to specify the state of objects after each function. Explicitly. If a function does not say that the state of the object is affected by the function, then the function may not affect the state of that object.

Implementations cannot subvert this willy-nilly.
 

Note for instance 17.6.4.9:
"... If a program casts an lvalue to an
xvalue while passing that lvalue to a library function (e.g. by calling the function with the argument
move(x)), the program is effectively asking that function to treat that lvalue as a temporary. The
implementation is free to optimize away aliasing checks which might be needed if the argument was
an lvalue."

None of that is the actual, normative text. The normative text, the text that actually matters, is:

If a function argument binds to an rvalue reference parameter, the implementation may assume that
this parameter is a unique reference to this argument.

The terms "argument" and "parameter" are very well defined by the standard. "Argument" is the expression you use in the function call; it is what you pass in. "Parameter" is the field in the function declaration/definition which the argument is bound to.

I honestly do not know if the standard says that `x.func_name` considers `x` to be an "argument" to the member function `func_name`. But let's assume that it does (because if it doesn't, then this section clearly doesn't apply).

`std::basic_stringstream::str` does not take `this` as an "rvalue reference parameter". Therefore, any implementation that did split the function between `&` and `&&` versions and did indeed move the string out of the object would be in violation of the standard. The function as defined in the standard does not take its this parameter by rvalue reference, and therefore the function as defined by the standard cannot move from the stream.

The standard is very clear: `std::move(os).str()` shall be equivalent to `os.str()`. The state of `os` in both cases must be preserved.

Oh, and `std::basic_stringstream::str` is const (the one that returns a value). I probably should have noticed that much earlier.

[Nicol Bolas]

On Sunday, May 12, 2013 5:58:44 AM UTC-7, Olaf van der Spek wrote:

On Friday, May 10, 2013 6:14:32 PM UTC+2, Nicol Bolas wrote:

On the old comp.std.c++ Usenet list, I made a suggestion about adding the ability to move `basic_string`s into and out-of `basic_stringstream`s. I'm considering making this an actual C++ proposal, so I wanted to gauge the interest in doing this.

Interesting!

What about variants that take the buffer by reference and input/output directly into the buffer, with buffer (views) being generic?

Now you're talking about a whole new kind of `stringstream` class. It would be much easier to just expand the interface to give you control over the stream equivalent to a `basic_string` than to create a whole new stream class.

Isn't an implementation allowed to move a string into a stringstream already?

Both the constructor and `str` take the `basic_string` by `const&`. So no moving.

[Jeffrey Yasskin]

I, for one, would welcome normative text saying that if you pass an rvalue reference to any library function, the value upon return from that function is unspecified. I don't expect to have time to write the paper, but it would be great to get it for C++17.

Jeffrey

[Zhihao Yuan]

On Sun, May 12, 2013 at 10:00 AM, Nicol Bolas <jmck...@gmail.com> wrote:
> `std::basic_stringstream::str` does not take `this` as an "rvalue reference
> parameter".

But it's our job to make it work, right?

Before

std::string str() const

After

std::string str() &&
std::string str() const&

I can't see the problem...

--
Zhihao Yuan, ID lichray
The best way to predict the future is to invent it.
___________________________________________________
4BSD -- http://4bsd.biz/

[Nicol Bolas]

On Sunday, May 12, 2013 8:18:43 AM UTC-7, Zhihao Yuan wrote:

On Sun, May 12, 2013 at 10:00 AM, Nicol Bolas <jmck...@gmail.com> wrote:
> `std::basic_stringstream::str` does not take `this` as an "rvalue reference
> parameter".

But it's our job to make it work, right?

Before

  std::string str() const

After

  std::string str() &&
  std::string str() const&

I can't see the problem...

This part of the discussion was mainly about whether an implementation could back-door the functionality as an implementation detail. The answer being, "no it can't."

As for why we shouldn't do it, there are many reasons. It's not backwards compatible; it potentially breaks any code that was already of the form `std::move(os).str()`. Admittedly, that's not a huge concern, but it's still backwards incompatible needlessly. It also breaks the stringstream's ABI by creating two different functions with two different definitions when there used to be one.

Then there's the other reasons mentioned above, like not forcing the tortured `std::move(os).str()` syntax instead of the much more readable `os.move_str()` syntax?

[Nicol Bolas]

That's a terrible idea. We want movement to be something that is specified by the standard, not a random thing that might happen. Not something that some implementations could or could not do.

The actual move itself is ultimately implementation-defined. The specific details of the move are implementation-defined. But when a move is permitted (and therefore when it is not) should always be defined by the parameters a function is defined with. Not by how the user calls it.

The possibility of movement should be a function of the API, not how the API is implemented.

[Zhihao Yuan]

I see.  I had no ABI compatibility concern in mind and this problem only occurs when an explicit call to std::move was used.  Anyway, you are right.

And the next question is, does it worth to break a (almost) non-existing practice? Does it worth to add one member function semantically as  same as std::move to workaround the problem? In what way the APIs in the future can benefit from the choices people currently make?

--
Zhihao Yuan

--

[Ville Voutilainen]

On 12 May 2013 19:27, Zhihao Yuan <lic...@gmail.com> wrote:

I see.  I had no ABI compatibility concern in mind and this problem only occurs when an explicit call to std::move was used.  Anyway, you are right.

And the next question is, does it worth to break a (almost) non-existing practice? Does it worth to add one member function semantically as  same as std::move to workaround the problem? In what way the APIs in the future can benefit from the choices people currently make?

future::share is an existing example of a member function that's not strictly necessary to have, since

it would be possible to write its bare-bones counterpart, shared_future<T>(move(future)).

For the other distractions, I don't think we should rely on QoI of libraries to add the rvalue reference
overloads.

[Marc]

Le dimanche 12 mai 2013 16:00:25 UTC+2, Nicol Bolas a écrit :

On Sunday, May 12, 2013 6:21:23 AM UTC-7, Marc wrote:

Because movement is explicit (either you passed a temporary, or you passed move(obj)), binding to && is not an accident, it is done on purpose, and there is thus no way it creates chaos.

It creates chaos because the standard is supposed to specify the state of objects after each function. Explicitly. If a function does not say that the state of the object is affected by the function, then the function may not affect the state of that object.

It does. You pass a temporary, what's left of it is in an undefined (but valid) state. Which is good since it disappears immediately (or you promised not to use it anymore by using move()). Again, I insist, passing rvalues does not happen by accident, it is clear in the caller. Having functions where the standard mandates moving makes sense. But I want the library to be able to move wherever it makes sense without requiring tons of spam overloads in the standard document. For instance, in the std::integer proposal, I argued strongly that the move constructor and all the operator overloads taking rvalue references should be left out of the text, as they are implementation details (as long as we don't add guarantees like noexcept).
 
You may consider this as a chaotic subversion of the standard. I consider that your position subverts the spirit of rvalue references. And taking the standard library specification too literally doesn't help either, or we'll have to normalize an implementation.

None of that is the actual, normative text.

It explains what the people writing the normative text had in mind.
 

`std::basic_stringstream::str` does not take `this` as an "rvalue reference parameter". Therefore, any implementation that did split the function between `&` and `&&` versions and did indeed move the string out of the object would be in violation of the standard.

That's where we disagree. I think an implementation is already allowed to do it. And if it isn't, it should. That would only help.

Again, I am still in favor of a nicer more explicit syntax as well.

[Ville Voutilainen]

On 12 May 2013 21:22, Marc <marc....@gmail.com> wrote:

 

`std::basic_stringstream::str` does not take `this` as an "rvalue reference parameter". Therefore, any implementation that did split the function between `&` and `&&` versions and did indeed move the string out of the object would be in violation of the standard.

That's where we disagree. I think an implementation is already allowed to do it. And if it isn't, it should. That would only help.

17.6.5.5 Member functions [member.functions]/2:
An implementation may declare additional non-virtual member function signatures within a class:
..
-  by adding a member function signature for a member function name.
3 A call to a member function signature described in the C ++ standard library behaves as if the implementation
declares no additional member function signatures. Footnote 187
187) A valid C ++ program always calls the expected library member function, or one with equivalent behavior. An implemen-
tation may also define additional member functions that would otherwise not be called by a valid C++ program.

In other words, a library implementation is allowed to add the overloads we're talking about.

Now, let's create a proposal that actually adds those overloads explicitly, rather than rely on implementations
to do it, please.

[Ville Voutilainen]

Ah, just one more thing, says Lt. Columbo - if an rvalue reference overload actually moves, it's not equivalent
in behavior, and the behavior difference is easy to detect. I think we actually _need_ to add those overloads

to the standard and specify the semantics.

[Marc]

Le dimanche 12 mai 2013 20:46:36 UTC+2, Ville Voutilainen a écrit :

On 12 May 2013 21:39, Ville Voutilainen <ville.vo...@gmail.com> wrote:

On 12 May 2013 21:22, Marc <marc....@gmail.com> wrote:

`std::basic_stringstream::str` does not take `this` as an "rvalue reference parameter". Therefore, any implementation that did split the function between `&` and `&&` versions and did indeed move the string out of the object would be in violation of the standard.

That's where we disagree. I think an implementation is already allowed to do it. And if it isn't, it should. That would only help.

17.6.5.5 Member functions [member.functions]/2:
An implementation may declare additional non-virtual member function signatures within a class:
..
-  by adding a member function signature for a member function name.
3 A call to a member function signature described in the C ++ standard library behaves as if the implementation
declares no additional member function signatures. Footnote 187
187) A valid C ++ program always calls the expected library member function, or one with equivalent behavior. An implemen-
tation may also define additional member functions that would otherwise not be called by a valid C++ program.

In other words, a library implementation is allowed to add the overloads we're talking about.

Now, let's create a proposal that actually adds those overloads explicitly, rather than rely on implementations
to do it, please.

But let's use a better syntax for that, like Nicol's move_str() ;-)
 

Ah, just one more thing, says Lt. Columbo - if an rvalue reference overload actually moves, it's not equivalent
in behavior, and the behavior difference is easy to detect. I think we actually _need_ to add those overloads

to the standard and specify the semantics.

I think 17.6.4.9 already gives you permission to treat the rvalue reference as a temporary and thus of moving instead of copying, and the different behavior is ok. If it doesn't, I really believe it should, and that's an important point for the library.

[Nicol Bolas]

You can think that, but it doesn't. As previously stated, the definitions of "argument" and "parameter" don't allow that. It only works when the function is explicitly stated as taking an rvalue reference.