Compile time conditionally make operator new() noexcept

[Matthew Fioravante]

Right now, operator new() will throw if memory allocation fails.

The idea is that in theory your application can catch these exceptions and properly clean up. In practice, almost nobody actually handles memory allocation exceptions. Most of the time there is no way to handle it properly. Aside from very small carefully controlled situations, its impossible to unit test what happens when each and every allocation fails. If your application runs of memory you're better off just crashing.

The consequence of this design decision, is that everything that might do a memory allocation needs to be exception safe. We need to write exception safe code and the compiler must also assume any allocation can throw and then setup the unwinding structures.

A good example of this, would be something like vector::push_back(). If push_back needs to reallocate, it will move construct each element. If move can throw, it will need to fallback to copy. If copy can throw, there needs to be cleanup code in place that will destroy all of the previously copied elements and deallocate the new buffer. Finally, for the programmer calling push_back. I need to be sure to order the other stuff before and after it correctly, so my state doesn't get messed up in the off chance that it throws.

For the case of allocations, all of this exception cleanup code is in practice dead code. It increases our binary size which also negatively impacts the Icache.

Most compilers provide some variant of -fno-exceptions which entirely disables exceptions for a performance gain. But what if we want to use exceptions carefully in a few places but not suffer the cost of them globally from allocations?

So the idea is to have some way to make operator new() noexcept at compile time. Probably via a #define. This differs from the nothrow version of new because it actually calls std::terminate() if the allocation fails, instead of returning nullptr. No only can the compiler assume new will never throw, it could also assume it never returns a nullptr.

Also this approach is better than using an allocator. You just switch it on and everything works. ABI compatibility should not be a problem as separately compiled code can be compiled with or without the feature. For any opaque function call that isn't noexcept, we still need to assume it can throw anything. This optimization would only be possible in inline context.

This noexcept new mode could also improve application correctness. Almost nobody ever writes careful code to handle exceptions thrown by new. 

People can write catch blocks for std::exception anticipating other errors. This will catch allocation errors as well. Since its impossible to unit test every possible allocation site, its unlikely to be handled properly. It would be better to just crash at the allocation site, with a ready stack trace for debugging.

At a minimum, most people write try {} catch {} blocks around the entire application in main. This completely obscures the source of an exception, especially an out of memory error which could happen almost anywhere.

If I accidentally try to allocate INT_MAX somewhere, I'd rather get a core dump at the site of the bug than a clean exit from main in a catch{} block.

I don't have hard data on any of this, but I suspect the following things are likely true:

1. Most C++ applications don't need to handle out of memory conditions and just calling std::terminate() immediately at the allocation site is the best possible solution.

2. The largest source of "possibly can throw" in most applications come from things that *might* allocate memory. Mostly from std:: containers.

3. This mode could give us most of the performance benefits of -fno-exceptions while still allowing us to use exceptions in certain parts of the system.

4. This mode can help us make our systems much simpler and more correct, as we don't have to worry about the effectively infinite explosion of possible application states from throwing allocations.

5. After removing allocations from concern, tools that analyse how exceptions are used in programs might actually be meaningful and possible.

Would you use noexcept new in any of your applications?

[Thiago Macieira]

On quinta-feira, 23 de março de 2017 12:05:46 PDT Matthew Fioravante wrote:
> The idea is that in theory your application can catch these exceptions and
> properly clean up. In practice, almost nobody actually handles memory
> allocation exceptions. Most of the time there is no way to handle it
> properly. Aside from very small carefully controlled situations, its
> impossible to unit test what happens when each and every allocation fails.

Actually, that's not very difficult.

Override your operator new with:

pid_t pid = fork();
if (!pid) {
// child process
throw std::bad_alloc();
return malloc(size);

To make the output more readable, you probably want to introduce a cross-
process synchronisation before malloc(). It would also help with avoiding lots
of COW of the process pages.

--
Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org
Software Architect - Intel Open Source Technology Center

[Greg Marr]

On Thursday, March 23, 2017 at 3:05:46 PM UTC-4, Matthew Fioravante wrote:

Right now, operator new() will throw if memory allocation fails.

The idea is that in theory your application can catch these exceptions and properly clean up. In practice, almost nobody actually handles memory allocation exceptions. Most of the time there is no way to handle it properly. Aside from very small carefully controlled situations, its impossible to unit test what happens when each and every allocation fails. If your application runs of memory you're better off just crashing.

So the idea is to have some way to make operator new() noexcept at compile time. Probably via a #define. This differs from the nothrow version of new because it actually calls std::terminate() if the allocation fails, instead of returning nullptr. No only can the compiler assume new will never throw, it could also assume it never returns a nullptr.

You have been able to do this since at least C++98.

http://en.cppreference.com/w/cpp/memory/new/set_new_handler

std::new_handler set_new_handler( std::new_handler new_p );

Makes new_p the new global new-handler function and returns the previously installed new-handler.

The new-handler function is the function called by allocation functions whenever a memory allocation attempt fails. Its intended purpose is one of three things:

1) make more memory available

2) terminate the program (e.g. by calling std::terminate)

3) throw exception of type std::bad_alloc or derived from std::bad_alloc.

The default implementation throws std::bad_alloc. The user can install his own new-handler, which may offer behavior different than the default one.

[Matthew Fioravante]

On Thursday, March 23, 2017 at 9:20:18 PM UTC-5, Greg Marr wrote:

You have been able to do this since at least C++98.

http://en.cppreference.com/w/cpp/memory/new/set_new_handler

std::new_handler set_new_handler( std::new_handler new_p );

Makes new_p the new global new-handler function and returns the previously installed new-handler.

The new-handler function is the function called by allocation functions whenever a memory allocation attempt fails. Its intended purpose is one of three things:

1) make more memory available

2) terminate the program (e.g. by calling std::terminate)

3) throw exception of type std::bad_alloc or derived from std::bad_alloc.

The default implementation throws std::bad_alloc. The user can install his own new-handler, which may offer behavior different than the default one.

set_new_handler() is a runtime operation. You could get the safety benefits but the compiler will be unable to optimize out exception handling as the new handler function pointer type is not noexcept.

[Nicol Bolas]

On Thursday, March 23, 2017 at 3:05:46 PM UTC-4, Matthew Fioravante wrote:

Right now, operator new() will throw if memory allocation fails.

The idea is that in theory your application can catch these exceptions and properly clean up. In practice, almost nobody actually handles memory allocation exceptions. Most of the time there is no way to handle it properly. Aside from very small carefully controlled situations, its impossible to unit test what happens when each and every allocation fails. If your application runs of memory you're better off just crashing.

The consequence of this design decision, is that everything that might do a memory allocation needs to be exception safe. We need to write exception safe code and the compiler must also assume any allocation can throw and then setup the unwinding structures.

A good example of this, would be something like vector::push_back(). If push_back needs to reallocate, it will move construct each element. If move can throw, it will need to fallback to copy. If copy can throw, there needs to be cleanup code in place that will destroy all of the previously copied elements and deallocate the new buffer. Finally, for the programmer calling push_back. I need to be sure to order the other stuff before and after it correctly, so my state doesn't get messed up in the off chance that it throws.

For the case of allocations, all of this exception cleanup code is in practice dead code. It increases our binary size which also negatively impacts the Icache.

Most compilers provide some variant of -fno-exceptions which entirely disables exceptions for a performance gain. But what if we want to use exceptions carefully in a few places but not suffer the cost of them globally from allocations?

So the idea is to have some way to make operator new() noexcept at compile time. Probably via a #define. This differs from the nothrow version of new because it actually calls std::terminate() if the allocation fails, instead of returning nullptr. No only can the compiler assume new will never throw, it could also assume it never returns a nullptr.

Also this approach is better than using an allocator. You just switch it on and everything works. ABI compatibility should not be a problem as separately compiled code can be compiled with or without the feature. For any opaque function call that isn't noexcept, we still need to assume it can throw anything. This optimization would only be possible in inline context.

This noexcept new mode could also improve application correctness. Almost nobody ever writes careful code to handle exceptions thrown by new. 

People can write catch blocks for std::exception anticipating other errors. This will catch allocation errors as well. Since its impossible to unit test every possible allocation site, its unlikely to be handled properly. It would be better to just crash at the allocation site, with a ready stack trace for debugging.

At a minimum, most people write try {} catch {} blocks around the entire application in main.

... they do? I don't believe there's a rash of people doing that in `main`. For obvious reasons.

This completely obscures the source of an exception, especially an out of memory error which could happen almost anywhere.

If I accidentally try to allocate INT_MAX somewhere, I'd rather get a core dump at the site of the bug than a clean exit from main in a catch{} block.

I don't have hard data on any of this, but I suspect the following things are likely true:

1. Most C++ applications don't need to handle out of memory conditions and just calling std::terminate() immediately at the allocation site is the best possible solution.

2. The largest source of "possibly can throw" in most applications come from things that *might* allocate memory. Mostly from std:: containers.

That's a pretty bold statement. You're basically ignoring every other thing in the standard library that throws something else. Which is quite a lot (std::filesystem is a big one, just off the top of my head).

Not to mention all of the non-standard library code which might also throw.

3. This mode could give us most of the performance benefits of -fno-exceptions while still allowing us to use exceptions in certain parts of the system.

How, exactly?

It is my understanding that the costs of exception handling are measured in two ways: executable code size increase, due to the need to provide dynamic runtime stack unwinding infrastructure; and runtime performance, which in most implementations is primarily assessed at the time the exception is thrown/caught. The amount that isn't due to actually throwing/catching exceptions happens when a `try` block is encountered.

So long as your application can throw exceptions anywhere, the compiler must still provide stack unwinding infrastructure and whatever runtime performance costs happen on `try` and `throw`s.

The performance problem with exception handling is primarily about the cost of actually throwing/catching such errors. Which, due in part to the need to minimize the non-exception case, is quite expensive. That is, people in the high-performance/real-time world don't want to spend tens of thousands of cycles just to catch an error condition more conveniently. That causes unpleasant stalls in systems that really shouldn't have unpleasant stalls.

Of course, there's something else that causes "unpleasant stalls": memory allocation. Which is why that too is avoided in performance-critical code. So it seems to me that memory allocation already is being avoided in places where you would turn this feature on.

4. This mode can help us make our systems much simpler and more correct, as we don't have to worry about the effectively infinite explosion of possible application states from throwing allocations.

If you premise #1 is correct, then most systems don't care about the "infinite explosion of possible application states from throwing allocations" since they never bother to catch such allocations to begin with.

5. After removing allocations from concern, tools that analyse how exceptions are used in programs might actually be meaningful and possible.

In what way are they not "meaningful" or "possible" presently?

Would you use noexcept new in any of your applications?

No.

[Sean Middleditch]

On Thursday, March 23, 2017 at 12:05:46 PM UTC-7, Matthew Fioravante wrote:

Right now, operator new() will throw if memory allocation fails.

This topic has been discussed by SG14 as one of the possible approaches to solving the "many projects in games/embedded/realtime/etc. use -fno-exceptions dialect of C++" problem.

The immediate proto-feedback from the committee is that no option that bifurcates the standard language will be considered for standardization. ... which to me just means that they enthusiastically support bifurcating the language because -fno-exceptions already exists and is widely used and the whole point is to make it unnecessary, but whatever.

There may very well be a case to be made for non-throwing new as a new implementation-specific option, e.g. something like -fnoexcept-new, though I'm unsure who the target audience for that would be.

Most compilers provide some variant of -fno-exceptions which entirely disables exceptions for a performance gain. But what if we want to use exceptions carefully in a few places but not suffer the cost of them globally from allocations?

An idea I've seen floated before was something like -fnoexcept-true-by-default . Again, that would never be a part of standard C++, and any code relying on it would not be compatible with standard C++ (even more so than -fno-exceptions). The difference between the two being that one would let you opt in to using exceptions by using noexcept(false) on a case-by-case basis. Aside from being non-standard and even more of a compatibly problem than just disabling them completely, I also don't think it actually solves enough of the problems with exceptions to be worth the bother.
 

Also this approach is better than using an allocator. You just switch it on and everything works. ABI compatibility should not be a problem as separately compiled code can be compiled with or without the feature. For any

Separately compiled code will definitely still be an ABI problem here. If you compile foo.cpp with noexcept new (and so it compiles its push_back to not support throwing new) but then link it with a module that has throwing new, you'll have ODR violations of push_back.

Adding optional noexcept new is an ABI breakage. Granted, so is inconsistent use of -fno-exceptions, so it's not a *ahem* new ABI breakage.

 

opaque function call that isn't noexcept, we still need to assume it can throw anything. This optimization would only be possible in inline context.

This assumes the leaves are inlined. I can compile an inlined push_back that calls out to extern allocator function implemented in a TU with throwing new. :)

 

3. This mode could give us most of the performance benefits of -fno-exceptions while still allowing us to use exceptions in certain parts of the system.

Don't worry too much about the performance side of this. Modern compilers do a fairly good job here in terms of runtime performance in non-exceptional cases. There's certainly other problems, but I'm not getting involved with debating those for the 576th time. :)

Would you use noexcept new in any of your applications?

No, I already have -fno-exceptions. Any change here is going to need to add a significant value over the status quo and it's going to have to do it without trying to split the language into standardized dialects. Good luck with that. :) Though if you have ideas or want to bounce anything past folks with lots of experience wrestling with exceptions or operating without them, you might have luck asking on the SG14 reflector.

[Thiago Macieira]

On sexta-feira, 24 de março de 2017 14:29:44 PDT Sean Middleditch wrote:
> There may very well be a case to be made for non-throwing new as a new
> implementation-specific option, e.g. something like -fnoexcept-new, though
> I'm unsure who the target audience for that would be.

And this being a compiler-specific extension, you can already achieve this by
simply adding this forward declaration somewhere in a header:

void *operator new(std::size_t) noexcept;

GCC and ICC grok it just fine and remove exceptional codepaths, whereas Clang
"sees" a previous declaration of operator new that conflicts with that. Play
with the compiler selection at
https://godbolt.org/g/sBokXT

MSVC didn't change its code generation (you need to change the arguments to
the compiler to -O2 -EHsc).

Just make sure you don't #include <new>. Or convince your compiler vendor to
implement this.

[Marc]

Yes, I would, we seem to write the same type of applications ;-) . I already regularly add an inline definition of operator new (that's illegal but for no good reason AFAICS) that just calls malloc (I don't even check for NULL and terminate), but that does not change the noexcept status of types so any code-level optimization like move_if_noexcept is missed. Note that to make a noexcept new really useful, many more functions should be conditionally noexcept, which is really painful to get right without noexcept(auto). By the way, I cannot use -fno-exceptions, we do use exceptions for something else (that happens in a normal run, not to report errors).
I don't really expect anything to happen at the committee level, providing this kind of option seems more like something you should ask your compiler vendor.

[ol...@join.cc]

Op donderdag 23 maart 2017 20:05:46 UTC+1 schreef Matthew Fioravante:

Would you use noexcept new in any of your applications?

 I'd love to use it in all my apps.

I've never seen any app handle out-of-memory better then crashing.

[Sergey Zubkov]

I've never seen any app handle out-of-memory better then crashing.

Ever tried opening a file too large for Photoshop or Notepad++?

At one point last year I did a little survey of just "catch bad_alloc" (which misses many kinds of OOM handling) in Debian source code and there were a few hundred opensource packages with perfectly sensible bad_alloc handling (including Notepad++). 

[Nicol Bolas]

On Monday, April 3, 2017 at 10:35:31 PM UTC-4, Sergey Zubkov wrote:

I've never seen any app handle out-of-memory better then crashing.

Ever tried opening a file too large for Photoshop or Notepad++?

I assumed they just checked the file size before they loaded it, and changed how they worked based on that.

[olafv...@gmail.com]

Op dinsdag 4 april 2017 04:35:31 UTC+2 schreef Sergey Zubkov:

I've never seen any app handle out-of-memory better then crashing.

Ever tried opening a file too large for Photoshop or Notepad++?

Nope

 

At one point last year I did a little survey of just "catch bad_alloc" (which misses many kinds of OOM handling) in Debian source code and there were a few hundred opensource packages with perfectly sensible bad_alloc handling (including Notepad++). 

I find bad_alloc 6x in Notepad++'s source. First one is:

} catch (std::bad_alloc&) {

errorStatus = SC_STATUS_BADALLOC;

} catch (...) {

errorStatus = SC_STATUS_FAILURE;

}

 

AFAIK the catch all isn't recommended practice..

Second one is a comment: wl[i] = new WordList(); // (works or THROWS bad_alloc EXCEPTION)

in a function with raw new and delete that's obviously not exception safe.

Third, fifth and sixth one are like the first one.

Fourth one:

int SCI_METHOD Document::AddData(char *data, int length) {

try {

int position = Length();

InsertString(position, data, length);

} catch (std::bad_alloc &) {

return SC_STATUS_BADALLOC;

} catch (...) {

return SC_STATUS_FAILURE;

}

return 0;

}

Is this what we call perfectly sensible?

And are we sure all state is in a good state after an exception is thrown and caught?

[Sergey Zubkov]

Is this what we call perfectly sensible?

Yes, undo of all data structures built so far and a pop-up informing the user that operation they requested could not be perfomed (file load, print preview, image resize, etc) is one of the most common types of OOM handlers in the wild, though I like seeing non-interactive handling like audacity's buffer shrinking or scylladb's transaction rollback or xorp's corrupt packet dropping, or armadillo's STL-like space/time tradeoff etc.

In my professional career, C++ was chosen exactly because of the resilience and guarantees offered by exception handling and it's a little scary to see how popular the meme "If your application runs of memory you're better off just crashing" has become.

[olafv...@gmail.com]

Op dinsdag 4 april 2017 13:46:01 UTC+2 schreef Sergey Zubkov:

Is this what we call perfectly sensible?

Yes, undo of all data structures built so far

Are we sure this is done correctly? Is it tested properly? I'm not a fan of silent data corruption.

[Sergey Zubkov]

Yes, undo of all data structures built so far

Are we sure this is done correctly? Is it tested properly? I'm not a fan of silent data corruption.

I am sure stack unwinding is done correctly and, in acceptable C++ code, invariants are guaranteed by construction. Testing of error handling is a worthwhile discussion, but that would take the focus away from the topic of this thread - I'm just here to point out that many (though not a majority) C++ applications handle OOM conditions by means other than termination, and for some it is a critical part of operation.

[Olaf van der Spek]

Fair enough. But for apps that don't catch bad_alloc having it be
noexcept would be a benefit, wouldn't it?

--
Olaf

[Sergey Zubkov]

Fair enough. But for apps that don't catch bad_alloc having it be
noexcept would be a benefit, wouldn't it?

Well.. yes, there are benefits to being able to put "noexcept" on many functions that might transitively end up allocating something.

This is viral. In fact, there is precedent: g_malloc from the ubiquitous glib terminates on OOM. They have a "try" variant, but numerous libraries that use glib just call g_malloc, and that infects a large part of the Linux ecosystem. For example, Mozilla does some heroics to survive many OOMs, except when the last straw allocation happens to be a g_malloc deep in the call stack across several third-party libraries. 

[Thiago Macieira]

Em quarta-feira, 5 de abril de 2017, às 06:07:05 PDT, Sergey Zubkov escreveu:
> Well.. yes, there are benefits to being able to put "noexcept" on many
> functions that might transitively end up allocating something.
> This is viral. In fact, there is precedent: g_malloc from the ubiquitous
> glib terminates on OOM. They have a "try" variant, but numerous libraries
> that use glib just call g_malloc, and that infects a large part of the
> Linux ecosystem. For example, Mozilla does some heroics to survive many
> OOMs, except when the last straw allocation happens to be a g_malloc deep
> in the call stack across several third-party libraries.

The rule of thumb is that you only check for success in large allocations
(multiple megabytes). Anything small is not worth the test and is best handled
by terminating the application. After a failed large allocation, the
application can reasonably continue working, assuming that it may allocate
small amounts as it's required to react to other user events. It may even show
an OOM message to the user.

The problem with large allocations is that they can be served by an
overcommitted memory map, so they can succeed even if there's no memory left
to back it up. Usually, by the time this happens, the system has been swapping
hard for some time or at least the kernel is aggressively dropping clean pages
and having to swap them back in, so it's either way in a state of such
slowness that the user is unlikely to have allowed it to last.

When small allocations start failing, it's highly unlikely that the
application can continue working. If the system has been allowed to reach such
a state, your application will be competing with others, so some allocations
may succeed and the majority will not. It's also unlikely you will reach the
point where you can actually save any data.

And then there's the OOM killer...

[Ville Voutilainen]

Even small allocations can sometimes be gracefully recovered from;
they may be caused by
a resource limit set by ulimit. In such cases, the small allocations
do not fail because of
necessarily competing with other applications.

[Olaf van der Spek]

2017-04-05 15:07 GMT+02:00 Sergey Zubkov <cubb...@gmail.com>:
>> Fair enough. But for apps that don't catch bad_alloc having it be
>> noexcept would be a benefit, wouldn't it?
>>
> Well.. yes, there are benefits to being able to put "noexcept" on many
> functions that might transitively end up allocating something.

I'm not sure 'manual' noexcept is the right a recipe here. It'd be
nice if the compiler could take care of this.

> This is viral. In fact, there is precedent: g_malloc from the ubiquitous
> glib terminates on OOM. They have a "try" variant, but numerous libraries
> that use glib just call g_malloc, and that infects a large part of the Linux
> ecosystem. For example, Mozilla does some heroics to survive many OOMs,
> except when the last straw allocation happens to be a g_malloc deep in the
> call stack across several third-party libraries.

--
Olaf

[Thiago Macieira]

Em quarta-feira, 5 de abril de 2017, às 08:19:50 PDT, Ville Voutilainen
escreveu:

> Even small allocations can sometimes be gracefully recovered from;
> they may be caused by
> a resource limit set by ulimit. In such cases, the small allocations
> do not fail because of
> necessarily competing with other applications.

True, but not always. This is a hard to solve problem in a modern multitasking
system with virtual memory. Even if a single process is limited due to a
ulimit and not by overall OOM condition, if you're reaching the point where
small allocations fail, you're already running the risk of being killed by
SIGBUS because a page that was overcommitted cannot be provided without
exceeding the limit.