allocators and noexcept

[Lars Hagen]

Hi,

I am creating at set of vector-like structures to use as building blocks of various search structures. An important design goal is to support the strong exception guarantee as much as possible.

I was surprised to find that 1) std::allocator's construct method is never marked as noexcept, as if it might throw outside the actual constructor call and 2) this means that std::vector does not support the strong guarantee for push_back, as it does not care about the noexcept specification of the construct call, only the constructors in themselves.

Am I missing an important point here, or should calls to construct be conditionally noexcept, and containers use the noexcept value of construct when defining guarantees and choosing between copies/moves?

Lars Hagen

[David Krauss]

On 2014–03–22, at 7:10 PM, Lars Hagen <grege...@gmail.com> wrote:

> Hi,
>
> I am creating at set of vector-like structures to use as building blocks of various search structures. An important design goal is to support the strong exception guarantee as much as possible.
>
> I was surprised to find that 1) std::allocator's construct method is never marked as noexcept, as if it might throw outside the actual constructor call

It throws std::bad_alloc if it cannot allocate memory.

> and 2) this means that std::vector does not support the strong guarantee for push_back,

In general only the process of destroying things is exception-safe. Anything that allocates can fail. You can interchange between null returns and exception propagation, but failure is a possibility.

If you want to avoid exceptions from push_back, use reserve first.

If you really must avoid all exceptions ever, pre-allocate all the memory your algorithm will need before it starts. That’s the only way to avoid running out of memory.

Note that some platforms never report allocation failure to a process, but instead overcommit resources and send a signal after things go south.

[Lars Hagen]

> I was surprised to find that 1) std::allocator's construct method is never marked as noexcept, as if it might throw outside the actual constructor call

It throws std::bad_alloc if it cannot allocate memory.

I am fully aware that allocating must throw, but doesn't construct simply call placement new to construct the elements in-place. In that case, the construct method could be noexcept if T's constructor is noexcept.

 

> and 2) this means that std::vector does not support the strong guarantee for push_back,

In general only the process of destroying things is exception-safe. Anything that allocates can fail. You can interchange between null returns and exception propagation, but failure is a possibility.

If you want to avoid exceptions from push_back, use reserve first.

If you really must avoid all exceptions ever, pre-allocate all the memory your algorithm will need before it starts. That's the only way to avoid running out of memory.

Note that some platforms never report allocation failure to a process, but instead overcommit resources and send a signal after things go south.

I do not expect push_back to support the no-throw guarantee, but simply the strong guarantee that nothing is lost in the case of exceptions. The way std::vector behaves now, it starts moving elements during reallocations if T is nothrow move constructible. If the construct method fails for some unrelated reason during move construction, data is lost. I would prefer std::vector to choose copying in the case where construct(T*, T&&) may throw, as this would avoid the data loss possibility.

Lars

[Howard Hinnant]

I think you’ve identified a defect. The allocator requirements in 17.6.3.5 [allocator.requirements] should state that construct shall throw nothing except an exception propagated out of the constructor (probably using better standardeeze than that).

I think allocator_traits is ok, as it specifically says what it will do, and if it calls placement new, that is already specified with noexcept.

Whether or not we put a conditional noexcept on allocator_traits construct and std::allocator construct is a separable issue. But at the very least we need that throws constraint on construct in the allocator requirements.

Here is how to open an issue:

http://cplusplus.github.io/LWG/lwg-active.html#submit_issue

Howard

[David Krauss]

On 2014–03–22, at 11:09 PM, Howard Hinnant <howard....@gmail.com> wrote:

I think you've identified a defect.  The allocator requirements in 17.6.3.5 [allocator.requirements] should state that construct shall throw nothing except an exception propagated out of the constructor (probably using better standardeeze than that).

Huh? The default allocator spec [allocator.members] says

Remark: the storage is obtained by calling ::operator new(std::size_t)

Even if std::nothrow were used, the error would still need to be reported as an exception by vector::push_back because it does not return a status code.

[David Krauss]

Oh, never mind. The topic is std::allocator::construct. Sorry for the noise.

The requirement to call this function should perhaps be reviewed. Does it actually do anything except make work for the inliner?

[Ville Voutilainen]

I don't quite understand the question nor the way it's phrased. It's a
customization point.

[David Krauss]

Just asking if there’s a valid use case. I’ve looked and tried, but not found any.

At one point I tried using a custom allocator to adapt a container with an interposed a wrapper around the ValueType, but it later turned out to violate a requirement. IIRC, it violated the vector requirement to support array-like pointer arithmetic. So the principle might have still been valid for another container, or for code which didn’t rely on that property of vector.

However, I’ve since come to the impression that’s not what the Allocator interface designer had in mind. It seems more like the interface was designed to support object models besides the one built into C++, e.g. databases, but then all the flexibility was legislated out leaving some vestigial pass-through functions.

[David Krauss]

On 2014–03–22, at 11:39 PM, David Krauss <pot...@gmail.com> wrote:

On 2014–03–22, at 11:28 PM, Ville Voutilainen <ville.vo...@gmail.com> wrote:

I don't quite understand the question nor the way it's phrased. It's a
customization point.

Just asking if there’s a valid use case. I’ve looked and tried, but not found any.

This is apropos, not a tangent, by the way.

We need to at least identify a use case of non-trivial Allocator::construct to know whether std::allocator should guarantee noexcept, or all Allocators. If we don’t know what (besides pass-through) it might do, then how can we forbid exceptions?

Vice versa, if the customization potential has been winnowed down to nothing, it might as well be deprecated so folks don’t shoot themselves in the foot.

[Howard Hinnant]

On Mar 22, 2014, at 11:19 AM, David Krauss <pot...@gmail.com> wrote:

> The requirement to call this function should perhaps be reviewed. Does it actually do anything except make work for the inliner?
>

It *was* reviewed for C++11, and I believe the current wording surrounding it to be sound, modulo the defect Lars has identified.

I find Casey’s use of construct in his default_init_allocator here:

http://stackoverflow.com/a/21028912/576911

so motivating, that if this is the only use case ever found, it makes it worth it. But std::scoped_allocator_adaptor also customizes construct.

On Mar 22, 2014, at 11:52 AM, David Krauss <pot...@gmail.com> wrote:

> We need to at least identify a use case of non-trivial Allocator::construct to know whether std::allocator should guarantee noexcept, or all Allocators. If we don’t know what (besides pass-through) it might do, then how can we forbid exceptions?

Just for the record, no one is suggesting that allocator::construct be noexcept. Just that it only propagate exceptions coming out of the object being constructed. If allocator::construct throws an exception after it successfully constructs an object, then we actually loose basic exception safety (i.e. we start leaking memory).

Howard

[Alisdair Meredith]

This is the customization point that allows scoped_allocator_adapter to pass appropriate allocators to container elements.  It is similarly used by the polymorphic memory resources in the Library Fundamentals TS.

Sent from my iPad

--

---
You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to std-discussio...@isocpp.org.
To post to this group, send email to std-dis...@isocpp.org.
Visit this group at http://groups.google.com/a/isocpp.org/group/std-discussion/.

[Alisdair Meredith]

In order to preserve basic exception safety, we need to be sure to destroy any constructed object if an exception is thrown by something other than the new object's constructor.

My simple example of a potentially throwing allocator is one that logs every operation. If the attempt to write to the log fails, that operation may throw.

I am not suggesting that this is a compelling example, but it does seem like a reasonably simple case that could cause exactly these problems. Do we want to outlaw such allocators?

Sent from my iPad

[Lars Hagen]

I agree about the possible usefulness of the customization points, but I feel that the correct solution is for all allocators to be responsible for propagating noexcept-information from the actual constructors/destructors, unless they may throw themselves.

The containers could then specify behaviour and exception guarantees with respect to whether construct/destruct is noexcept, not  whether the constructors/destructor is noexcept.

Lars

[Howard Hinnant]

On Mar 22, 2014, at 12:02 PM, Alisdair Meredith <alis...@me.com> wrote:

> My simple example of a potentially throwing allocator is one that logs every operation. If the attempt to write to the log fails, that operation may throw.
>
> I am not suggesting that this is a compelling example, but it does seem like a reasonably simple case that could cause exactly these problems. Do we want to outlaw such allocators?

If the logging operation happens before the construction of the element (and throws), there is no problem. If the logging operations throws after the construction of the element, then the client (the container):

1. Can see an exception coming out of allocator_traits::construct, but does not know whether or not the value_type has been constructed.

2. Therefore the container does not know whether or not to catch the exception, destruct the value_type, and then rethrow. Or perhaps, catch the exception, record that a value_type has been constructed (or not), say by incrementing the end() pointer, and then rethrow.

I.e., for vector::push_back, the time from the end of the value_type being constructed, until the time the vector registers the construction with ++end(), the lifetime of that value_type is not owned.

To make it work, somehow the allocator::construct would have to communicate whether or not it actually constructed a value_type. Or perhaps allocator::construct would have to be responsible for destructing the value_type if it threw an exception after constructing it.

Howard

[David Krauss]

On 2014–03–22, at 11:57 PM, Alisdair Meredith <alis...@me.com> wrote:

> This is the customization point that allows scoped_allocator_adapter to pass appropriate allocators to container elements. It is similarly used by the polymorphic memory resources in the Library Fundamentals TS.

Ah, thanks! I didn’t realize how scoped allocators worked. The polymorphic allocators also look pretty cool.

N3916 is marked “adopted” in the list, does this mean adopted into the TS (not the main standard)?

[David Krauss]

On 2014–03–23, at 12:21 AM, Howard Hinnant <howard....@gmail.com> wrote:

> To make it work, somehow the allocator::construct would have to communicate whether or not it actually constructed a value_type. Or perhaps allocator::construct would have to be responsible for destructing the value_type if it threw an exception after constructing it.

It would work by adapting the model I mentioned earlier, where the allocator adds a shim and the allocated object is actually a subobject. The log operation would be done in the constructor of the shim, and an exception would cause destruction of already-constructed subobjects.

[Howard Hinnant]

Perhaps the correct resolution is something along the lines of:

Add a paragraph to [allocator.requirements]:

If a.construct(c, args) throws an exception, then there shall not be an object constructed at c.

Howard

[Felipe Magno de Almeida]

On Sat, Mar 22, 2014 at 12:57 PM, Howard Hinnant
<howard....@gmail.com> wrote:
> On Mar 22, 2014, at 11:19 AM, David Krauss <pot...@gmail.com> wrote:

[snip]

> > On Mar 22, 2014, at 11:52 AM, David Krauss <pot...@gmail.com> wrote:
> >
> > We need to at least identify a use case of non-trivial Allocator::construct to know whether std::allocator should guarantee noexcept, or all Allocators. If we don't know what (besides pass-through) it might do, then how can we forbid exceptions?
>
> Just for the record, no one is suggesting that allocator::construct be noexcept. Just that it only propagate exceptions coming out of the object being constructed. If allocator::construct throws an exception after it successfully constructs an object, then we actually loose basic exception safety (i.e. we start leaking memory).

Isn't basic expcetion safety already implied in the standard library?
If a function fails to give the guarantee basic guarantee, then that
code is buggy. We don't need to say a function can't have bugs.

If the construct throws an exception and has already constructed the
object, it must destroy it. It seems obvious and implied.

> Howard

Regards,
--
Felipe Magno de Almeida

[thedownsideo...@gmail.com]

I think the issue doesn't only affect push_back, but more generally, the insert functions; and the usage of move_if_noexcept. Some time ago, I asked about it, but that question unfortunately didn't get much attention.
I agree that a conditional noexcept is required for my_allocator::construct, since it's not required to use a certain constructor of the value type as far as I know (see the linked question). Calling my_allocator::construct with a non-const rvalue can invoke the copy ctor of the value type even though the value type has an accessible noexcept move ctor.

A more realistic example is that the custom allocator provides an additional argument to the ctor of the value type. Instead of a move ctor, a ctor taking a value_type&& and that additional argument could be called. The check whether the move ctor of the value type is noexcept then isn't appropriate (for move_if_noexcept), since that ctor is not called.

I'm happy you've brought up this issue again.