Marking more std::string_view members noexcept

[Andrey Semashev]

Hi,

I'm looking at string_view interface definition in N4606 and wondering
why so many members are not marked noexcept? For example, these
constructors:

constexpr basic_string_view(const charT* str);
constexpr basic_string_view(const charT* str, size_type len);

Assuming that the input string is not a constant expression, the first
constructor would call an equivalent of strlen(str), which in 99% is a
no-throw operation. Why not mark it
"noexcept(noexcept(traits::length(str))"? The second constructor does
not even need that and could be simply noexcept.

I'm most interested in constructors, but other members like operator[],
front(), back(), substr(), compare() also don't seem to need to ever
throw, yet they are not marked noexcept.

Is there a reason for these omissions?

Also, why there is no obvious constructor from basic_string?

Thanks.

[Daniel Krügler]

This is so, because these functions have a narrow contract. The
current guideline for LWG was described by this proposal:

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2011/n3279.pdf

> Also, why there is no obvious constructor from basic_string?

Which one in particular is in your mind?

Thanks,

- Daniel

[Nevin Liber]

The usual reason:  they are narrow contracts with preconditions.  See n3279.

 

Also, why there is no obvious constructor from basic_string?

See p0254r2.  Note:  you can still construct a basic_string_view from a basic_string via basic_string::operator basic_string_view. 

--

 Nevin ":-)" Liber  <mailto:ne...@eviloverlord.com>  +1-847-691-1404

[Andrey Semashev]

On 09/27/16 20:47, Daniel Krügler wrote:
> 2016-09-27 14:21 GMT+02:00 Andrey Semashev <andrey....@gmail.com>:
>> Hi,
>>
>> I'm looking at string_view interface definition in N4606 and wondering why
>> so many members are not marked noexcept? For example, these constructors:
>>
>> constexpr basic_string_view(const charT* str);
>> constexpr basic_string_view(const charT* str, size_type len);
>>
>> Assuming that the input string is not a constant expression, the first
>> constructor would call an equivalent of strlen(str), which in 99% is a
>> no-throw operation. Why not mark it
>> "noexcept(noexcept(traits::length(str))"? The second constructor does not
>> even need that and could be simply noexcept.
>>
>> I'm most interested in constructors, but other members like operator[],
>> front(), back(), substr(), compare() also don't seem to need to ever throw,
>> yet they are not marked noexcept.
>>
>> Is there a reason for these omissions?
>
> This is so, because these functions have a narrow contract. The
> current guideline for LWG was described by this proposal:
>
> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2011/n3279.pdf

I see. Still, it feels wrong that functions that you never expect to
throw are not marked noexcept. When you write non-throwing code you
would look at the functions your code calls and not seeing noexcept
there immediately throws a red flag.

>> Also, why there is no obvious constructor from basic_string?
>
> Which one in particular is in your mind?

I was thinking of "basic_string_view(basic_string const&)", but Nevin
already pointed me to p0254r2 and I can see there is a conversion
operator in basic_string instead. This looks like a very reasonable
solution.

[Nicol Bolas]

On Tuesday, September 27, 2016 at 2:24:48 PM UTC-4, Andrey Semashev wrote:

On 09/27/16 20:47, Daniel Krügler wrote:
> 2016-09-27 14:21 GMT+02:00 Andrey Semashev <andrey....@gmail.com>:
>> Hi,
>>
>> I'm looking at string_view interface definition in N4606 and wondering why
>> so many members are not marked noexcept? For example, these constructors:
>>
>>   constexpr basic_string_view(const charT* str);
>>   constexpr basic_string_view(const charT* str, size_type len);
>>
>> Assuming that the input string is not a constant expression, the first
>> constructor would call an equivalent of strlen(str), which in 99% is a
>> no-throw operation. Why not mark it
>> "noexcept(noexcept(traits::length(str))"? The second constructor does not
>> even need that and could be simply noexcept.
>>
>> I'm most interested in constructors, but other members like operator[],
>> front(), back(), substr(), compare() also don't seem to need to ever throw,
>> yet they are not marked noexcept.
>>
>> Is there a reason for these omissions?
>
> This is so, because these functions have a narrow contract. The
> current guideline for LWG was described by this proposal:
>
> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2011/n3279.pdf

I see. Still, it feels wrong that functions that you never expect to
throw are not marked noexcept. When you write non-throwing code you
would look at the functions your code calls and not seeing noexcept
there immediately throws a red flag.

I believe the point is that it shouldn't "throw a red flag". This is a matter of simple practicality.

C API functions cannot throw exceptions, but they equally are incapable of being marked `noexcept`. C++ libraries written before C++11 will not have non-throwing functions marked `noexcept`. And many C++ libraries even post-11 don't rigidly mark every non-throwing function as `noexcept`.

At the end of the day, C++ programmers cannot in general assume that every function not marked `noexcept` is a throwing function.

[Andrey Semashev]

On 09/27/16 23:42, Nicol Bolas wrote:
>
> On Tuesday, September 27, 2016 at 2:24:48 PM UTC-4, Andrey Semashev wrote:
>
> Still, it feels wrong that functions that you never expect to
> throw are not marked noexcept. When you write non-throwing code you
> would look at the functions your code calls and not seeing noexcept
> there immediately throws a red flag.
>

> I believe the point is that it /shouldn't/ "throw a red flag". This is a

> matter of simple practicality.
>
> C API functions cannot throw exceptions, but they equally are incapable
> of being marked `noexcept`. C++ libraries written before C++11 will not
> have non-throwing functions marked `noexcept`. And many C++ libraries
> even post-11 don't rigidly mark every non-throwing function as `noexcept`.
>
> At the end of the day, C++ programmers cannot in general assume that
> every function not marked `noexcept` is a throwing function.

That is true. But if one really wants to rely on such function in a
non-throwing code, he would have to inspect its implementation or rely
on documentation stating it won't throw. C functions (i.e. the ones that
are declared extern "C") are not a problem in this respect as they
implicitly never throw.

What I'm saying is that yes, you can use unmarked functions, but it
entails more cost on you and is more fragile (what if the function
starts to throw at some point in the future?) While by simply marking
the function you can remove that cost and make the no-throw guarantee
part of the interface.

[Edward Catmur]

It is intentional that the function might start to throw in the future, or at least in certain compilation modes. A high quality implementation might well throw from front() called on an empty string_view when built in debug mode.

noexcept is properly reserved for functions that have no preconditions, or where throwing would never be an appropriate reaction to precondition violation.

[pavel....@phystech.edu]

Hi

If there was a ctor from array reference, would it be noexcept?

template<size_t N>

constexpr basic_string_view(const charT(&str)[N]) noexcept;

Of course, you may do reinterpret_cast and pass something wrong to it, but it would be programmer's responsibility.

Thanks,

--

Pavel Kryukov

среда, 28 сентября 2016 г., 1:52:47 UTC+3 пользователь Edward Catmur написал:

[Nevin Liber]

On Wed, Oct 25, 2017 at 3:07 PM, <pavel....@phystech.edu> wrote:

Hi

If there was a ctor from array reference, would it be noexcept?

template<size_t N>

constexpr basic_string_view(const charT(&str)[N]) noexcept;

It hasn't been proposed, has it?

My guess would be no, since there is a precondition that str[N-1] == 0, giving it a narrow contract.

[Павел Игоревич Крюков]

> It hasn't been proposed, has it?

I’m not aware of that. If it is a narrow contract, I don’t see much value in such proposal. We already have a narrow contract `basic_string_view(const charT*)` ctor, and a noexcept `operator””sv`.

--

Pavel

26 окт. 2017 г., в 0:02, Nevin Liber <ne...@eviloverlord.com> написал(а):

--
You received this message because you are subscribed to a topic in the Google Groups "ISO C++ Standard - Future Proposals" group.
To unsubscribe from this topic, visit https://groups.google.com/a/isocpp.org/d/topic/std-proposals/FaBXmXBWHHI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to std-proposal...@isocpp.org.
To post to this group, send email to std-pr...@isocpp.org.
To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/CAGg_6%2BPv2BYcxMJ5aQ9uLb2qpgVZ3ThFjoWdKBNyZ4oNPm0A-A%40mail.gmail.com.

[Thiago Macieira]

On Wednesday, 25 October 2017 14:02:33 PDT Nevin Liber wrote:
> On Wed, Oct 25, 2017 at 3:07 PM, <pavel....@phystech.edu> wrote:
> > Hi
> >
> > If there was a ctor from array reference, would it be noexcept?
> >
> > template<size_t N>
> > constexpr basic_string_view(const charT(&str)[N]) noexcept;
>
> It hasn't been proposed, has it?
>
> My guess would be no, since there is a precondition that str[N-1] == 0,
> giving it a narrow contract.

Why would it have such a precondition?

--
Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org
Software Architect - Intel Open Source Technology Center

[Nevin Liber]

On Wed, Oct 25, 2017 at 6:09 PM, Thiago Macieira <thi...@macieira.org> wrote:

> > template<size_t N>
> > constexpr basic_string_view(const charT(&str)[N]) noexcept;
>
> It hasn't been proposed, has it?
>
> My guess would be no, since there is a precondition that str[N-1] == 0,
> giving it a narrow contract.

Why would it have such a precondition?

This constructor is unnecessary unless you want to differentiate between C-string literals and raw pointers (which is a breaking change BTW).

The problem is, C-string literals are not represented in the type system.

The closest, and by no means is it perfect, is a const char(&str)[N] where !str[N-1] holds.

Take the following:

string_view a("Hello");

const char s[] = { 'H', 'e', 'l', 'l', 'o' };

string_view b(s);

string_view c("Hello\0World");

Right now:

a.size() == 5;

b.size() // UB

c.size() == 5;

With this change, what should a.size(), b.size() and c.size() be?

Boost.Range has as_literal because you cannot in general deduce the intent from the type alone.

[Nevin Liber]

On Wed, Oct 25, 2017 at 7:17 PM, Nevin Liber <ne...@eviloverlord.com> wrote:

This constructor is unnecessary unless you want to differentiate between C-string literals and raw pointers (which is a breaking change BTW).

Technically, it doesn't have to be a breaking change, as long as all it does is define behavior for:

 

const char s[] = { 'H', 'e', 'l', 'l', 'o' };

string_view b(s);

Right now:

b.size() // UB

I don't see that as particularly useful.

-- 

[Thiago Macieira]

On Wednesday, 25 October 2017 18:51:43 PDT Nevin Liber wrote:
> Technically, it doesn't have to be a breaking change, as long as all it
>
> does is define behavior for:
> > const char s[] = { 'H', 'e', 'l', 'l', 'o' };
> > string_view b(s);
> >
> > Right now:
> >
> > b.size() // UB
>
> I don't see that as particularly useful.

QStringView has both a pointer and an array constructor. That means:

const char16_t str[] = u"Hello\0World";

QStringView(str).size() == 11 // constexpr
QStringView(+str).size() == 5 // not constexpr

We've also had a long-time issue with embedded nuls in arrays in other
contexts. For example:

const char data[] = "Hello World";
QByteArray ba(data);
ba[5] = '\0';
return QString::fromLatin1(ba);

Previously, QString::fromLatin1 did not have an overload for QByteArray, so it
was automatically cast to const char* and strlen applied. So this code
returned "Hello".

At some point, we added the overload, but it broke a lot of code that depended
on the null stopping the conversion. And it wasn't just a handful of cases,
plus they were quite difficult to track down.

If we had added the constructor at the moment the class was introduced, there
would have been no legacy to deal with. Now there will be, so it needs to be
done with great care.

[Nevin Liber]

On Wed, Oct 25, 2017 at 9:16 PM, Thiago Macieira <thi...@macieira.org> wrote:

On Wednesday, 25 October 2017 18:51:43 PDT Nevin Liber wrote:
> Technically, it doesn't have to be a breaking change, as long as all it
>
> does is define behavior for:
> > const char s[] = { 'H', 'e', 'l', 'l', 'o' };
> > string_view b(s);
> >
> > Right now:
> >
> > b.size() // UB
>
> I don't see that as particularly useful.

QStringView has both a pointer and an array constructor. That means:

        const char16_t str[] = u"Hello\0World";

        QStringView(str).size() == 11   // constexpr
        QStringView(+str).size() == 5   // not constexpr

I'm not sure what constexpr has to do with it.  Isn't it array vs. pointer that matters?

It also means

QStringView(b ? u"Hello\0World" : u"Howdy\0World") has subtly different behavior than QStringView(b ? u"Hello\0World" : u"Goodbye\0World"), doesn't it?

 

If we had added the constructor at the moment the class was introduced, there
would have been no legacy to deal with. Now there will be, so it needs to be
done with great care.

Again, I think matching an array is the wrong solution to the question of "is this a string literal"?

A better solution might be a magic type that, in template contexts, matches better than an array but easily decays to an array.  OTOH, it might not be (I haven't really thought about it much).  And it still wouldn't address the ternary operator problem I mentioned above.

[Thiago Macieira]

On Wednesday, 25 October 2017 19:30:21 PDT Nevin Liber wrote:
> > QStringView has both a pointer and an array constructor. That means:
> > const char16_t str[] = u"Hello\0World";
> >
> > QStringView(str).size() == 11 // constexpr
> > QStringView(+str).size() == 5 // not constexpr
>
> I'm not sure what constexpr has to do with it. Isn't it array vs. pointer
> that matters?

Just a dig at the inability to overload on constexprness.

Calculating the length of char16_t(&)[N] is easy: N - 1.

Calculating the length of char16_t* can't be done in a constexpr way.

> It also means
>
> QStringView(b ? u"Hello\0World" : u"Howdy\0World") has subtly different
> behavior than QStringView(b ? u"Hello\0World" : u"Goodbye\0World"), doesn't
> it?

Uh... I don't know. godbolting to the rescue...

Why does this call the pointer version?
template <unsigned N> void f(const char (&)[N]);
void f(const char *);

void g(bool b)
{
const char str[] = "World";
f(str);

}

> > If we had added the constructor at the moment the class was introduced,
> > there
> > would have been no legacy to deal with. Now there will be, so it needs to
> > be
> > done with great care.
>
> Again, I think matching an array is the wrong solution to the question of
> "is this a string literal"?

Indeed.

> A better solution might be a magic type that, in template contexts, matches
> better than an array but easily decays to an array. OTOH, it might not be
> (I haven't really thought about it much). And it still wouldn't address
> the ternary operator problem I mentioned above.

libstdc++ has a GCC-specific solution for string_view: __builtin_constant_p.
Using that, they can determine whether it is a string literal and they can
even do a constexpr strlen() for those. They do stop at a null, but that's by
design.

See the tricks here:

https://github.com/gcc-mirror/gcc/blob/master/libstdc%2B%2B-v3/include/bits/
char_traits.h#L220-L252

[Marc Mutz]

On 2017-10-25 22:07, pavel....@phystech.edu wrote:
> Hi
>
> If there was a ctor from array reference, would it be noexcept?
>
> template<size_t N>
> constexpr basic_string_view(const charT(&str)[N]) noexcept;

This one still has the precondition that [Char, N) is a valid range. It
might not be:

char bug[512];
auto s = string_view(bug);

but you can't check (in C++) for that.

FWIW, QStringView's Char[N] and Char* ctors are noexcept while the
(Char*, n) and (Char*, Char*) ones are not. I believe I did it like that
according to (non-/)checkability of preconditions (you can't check for
precondition violations for the first two, but you can check for some
violations for the other two). If you can't check using C++, then you
can't add an assert[1].

> Of course, you may do reinterpret_cast and pass something wrong to it,
> but it would be programmer's responsibility.

Meeting the preconditions of a function he's calling is _always_ the
programmer's responsibility.

Thanks,
Marc

[1] not entirely true: https://codereview.qt-project.org/193707

[Marc Mutz]

On 2017-10-26 05:04, Thiago Macieira wrote:
> On Wednesday, 25 October 2017 19:30:21 PDT Nevin Liber wrote:
>> > QStringView has both a pointer and an array constructor. That means:
>> > const char16_t str[] = u"Hello\0World";
>> >
>> > QStringView(str).size() == 11 // constexpr
>> > QStringView(+str).size() == 5 // not constexpr
>>
>> I'm not sure what constexpr has to do with it. Isn't it array vs.
>> pointer
>> that matters?
>
> Just a dig at the inability to overload on constexprness.

In QStringView, it's just an optimisation, because...

> Calculating the length of char16_t(&)[N] is easy: N - 1.

... this is C++11-constexpr (which we still need to care for, in Qt)
while ...

> Calculating the length of char16_t* can't be done in a constexpr way.

... is only C++14-constexpr (well, if you want to support reasonably
long strings, otherwise you could come up with a recursive
implementation).

We should probably document that the QStringView array ctor has the
precondition that it contains no embedded NULs, but it seems qdoc is too
clever for it's own good:
https://doc-snapshots.qt.io/qt5-5.10/qstringview.html doesn't have the
docs for the array ctor.

>> It also means
>>
>> QStringView(b ? u"Hello\0World" : u"Howdy\0World") has subtly
>> different
>> behavior than QStringView(b ? u"Hello\0World" : u"Goodbye\0World"),
>> doesn't
>> it?
>
> Uh... I don't know. godbolting to the rescue...
>
> Why does this call the pointer version?
> template <unsigned N> void f(const char (&)[N]);
> void f(const char *);
>
> void g(bool b)
> {
> const char str[] = "World";
> f(str);
> }

Because ordinary functions are preferred over templates. You need
constrained templates to overload on pointer vs. array:
https://www.kdab.com/qstringview-diaries-masters-overloads/ (can't seem
to link to a section, C-F for 'third').

Thanks,
Marc

[pavel....@phystech.edu]

> This one still has the precondition that [Char, N) is a valid range. It 
> might not be: 

>    char bug[512]; 
>    auto s = string_view(bug); 

Should std::string_view have such precondition, isn't it just a wrapper around pointer and size which may dangle?

You may construct a nullptr string_view using argument-less ctor (it is even noexcept) or get a valid string_view from a valid string, but then break it.

auto tmp = new std::string("Hello World");

std::string_view sv(tmp.c_str());

delete tmp;

Thanks,

--

Pavel

четверг, 26 октября 2017 г., 11:55:14 UTC+3 пользователь Marc Mutz написал:

[Marc Mutz]

On 2017-10-26 11:30, pavel....@phystech.edu wrote:
>> This one still has the precondition that [Char, N) is a valid range.
> It
>> might not be:
>
>> char bug[512];
>> auto s = string_view(bug);
>
> Should std::string_view have such precondition, isn't it just a
> wrapper around pointer and size which may dangle?

Yes, it should, IMHO. Consider;

f(std::string); // wide contract: all strings are acceptable
f(const char *s, size_t n); // narrow contract: [s,n) must be valid
range
f(std::string_view); // wide or narrow contract?

By putting the precondition on the string_view ctor, we gain wide
contracts for any string_view function. This means they can be noexcept,
and that is much more valuable than having a noexcept string_view ctor
(which is inline), because string_view-taking functions are often
out-of-line, so marking them noexcept (where no additional preconditions
prevent it) actually helps the compiler.

> You may construct a nullptr string_view using argument-less ctor (it
> is even noexcept)

Yes, and?

> or get a valid string_view from a valid string, but
> then break it.
>
> auto tmp = new std::string("Hello World");
> std::string_view sv(tmp.c_str());
> delete tmp;

Yes, this is a problem. But I'm not sure that it actually can be
formulated with the term 'precondition'. The closest I can come up with
is an external class invariant, which is a bit of an oxymoron to start
with, because classes exist to establish and maintain their invariants.

But it's not really different from std::ref, say.

Thanks,
Marc

[Thiago Macieira]

On Thursday, 26 October 2017 01:55:04 PDT Marc Mutz wrote:
> > Calculating the length of char16_t* can't be done in a constexpr way.
>
> ... is only C++14-constexpr (well, if you want to support reasonably
> long strings, otherwise you could come up with a recursive
> implementation).

No, it's not constexpr at all, since I can't run SSE or do reinterpret_cast
from pointer to integer and back in constexpr mode.

See http://code.qt.io/cgit/qt/qtbase.git/tree/src/corelib/tools/
qstring.cpp#n163

[Marc Mutz]

On 2017-10-26 18:09, Thiago Macieira wrote:
> On Thursday, 26 October 2017 01:55:04 PDT Marc Mutz wrote:
>> > Calculating the length of char16_t* can't be done in a constexpr way.
>>
>> ... is only C++14-constexpr (well, if you want to support reasonably
>> long strings, otherwise you could come up with a recursive
>> implementation).
>
> No, it's not constexpr at all, since I can't run SSE or do
> reinterpret_cast
> from pointer to integer and back in constexpr mode.
>
> See http://code.qt.io/cgit/qt/qtbase.git/tree/src/corelib/tools/
> qstring.cpp#n163

You'd think compilers would recognize an inlined strlen call and throw
their own vectoriser at it, no? If not strlen, what else could they
possibly auto-vectorise at all? :)

And, yes, you can call such functions in constexpr functions: Just loop
inline for, say, 64bytes and, if you haven't found the end by then, call
the out-of-line hand-optimized function to do the rest.

Thanks,
Marc

[Thiago Macieira]

On Thursday, 26 October 2017 10:44:04 PDT Marc Mutz wrote:
> On 2017-10-26 18:09, Thiago Macieira wrote:
> > No, it's not constexpr at all, since I can't run SSE or do
> > reinterpret_cast
> > from pointer to integer and back in constexpr mode.
> >
> > See http://code.qt.io/cgit/qt/qtbase.git/tree/src/corelib/tools/
> > qstring.cpp#n163
>
> You'd think compilers would recognize an inlined strlen call and throw
> their own vectoriser at it, no? If not strlen, what else could they
> possibly auto-vectorise at all? :)

I would. That's what GCC does:

https://code.woboq.org/gcc/libstdc++-v3/include/bits/char_traits.h.html#267

But note how that's only specialised for char_traits<char> and
char_traits<wchar_t>.

Also note how the constexpr keyword is commented out in this listing.
Unfortunately, Woboq has an older version of GCC; version 7 introduced that
__builtin_constant_p trick that I mentioned for char and wchar_t.

> And, yes, you can call such functions in constexpr functions: Just loop
> inline for, say, 64bytes and, if you haven't found the end by then, call
> the out-of-line hand-optimized function to do the rest.

That's doubly bad in my opinion:
1) it looks constexpr, but isn't for some data, leading to surprises. You
can't rely on it to create a constexpr QStringView.

2) it inserts that loop into user code that does 1 to 32 2-byte loads, instead
of the much faster 1 to 5 16-byte memory loads.