[llvm-dev] Catching exceptions while unwinding through -fno-exceptions code

[Everett Maus via llvm-dev]

Hey all:

I wanted to bring up something that was discussed a few years ago around the behavior of exceptions when interacting with code compiled with -fno-exceptions. (In https://lists.llvm.org/pipermail/llvm-dev/2017-February/109992.html and https://lists.llvm.org/pipermail/llvm-dev/2017-February/109995.html)

It's possible to compile (and link/etc.) code with -fexceptions for some compilation units and -fno-exceptions for others.  Unlike the behavior of noexcept (which requires termination), this doesn't have a specified behavior in the C++ standard as far as I can tell.  However, it can lead to memory leaks & other issues (e.x. with TSAN, it messes up the tracking of the current stack frame).

I'd be interested in looking into potentially doing the work to add an option to clang/etc. to terminate when an exception traverses code compiled with -fno-exceptions, instead of simply allowing the unwinder to walk through the stack frame & leak memory/etc. (possibly behind a flag?).  This particular issue bit a team I work closely with, and I'd imagine it could be causing subtle issues for other clang users.

I'm mostly concerned with solving this on Linux/x86_64, although if there's a way to solve it more generally I'm open to looking into doing that instead.

I /think/ the right place to change this (from the discussions I linked) would be in the LLVM -> assembly layer, adding an appropriate .gcc_except_table for functions that are determined to be unable to throw exceptions (either due to noexcept or due to -fno-exceptions). Then the unwinder would find .eh_frame but no entry in the .gcc_except_table and should terminate (via  __gxx_personality_v0).

Am I understanding that correctly?  What's the best way to propose this sort of change to clang? (document/just try to look at putting together a PR/other?)

Alternatively--one other thing that occurred to me is that it could be reasonably cheap to simply add try/catch blocks that report an UBSAN error in all methods that shouldn't be able to throw an exception.  This obviously doesn't fix the code-generation problem and would lead to larger binary sizes, but that seems less bad for an UBSAN build in particular.  That would likely meet my needs around wanting a way to automatically detect this behavior/problem, but might not address the more generic issue.

Thanks,

--

--EJM

[Modi Mo via llvm-dev]

If you don’t need to capture more information and can just terminate, you can directly register std::terminate as the personality routine as opposed to __gxx_personality_v0 or __CxxFrameHandler3/4 (Windows) which lets you omit other metadata and work cross-platform.

 

Modi

[Everett Maus via llvm-dev]

So yes--that would work.

I think for that route, though, it would be probably useful to try to output some more information than just std::terminate does. (i.e. that would work as a proof of concept, but getting more debug information the way you do with noexcept functions or functions without a handler would be more user friendly).

The other reason I was hesitant to go that route is that I was worried about the impact of that on binary size (as that's... I think 4+ bytes per function added for the .gcc_except_table sections, as opposed to just 4 bytes for the empty section).  That seems fine for a small binary, but seems likely to be expensive-to-prohibitive for large binaries/libraries.

--EJM

--

--EJM

[Hubert Tong via llvm-dev]

On Tue, Dec 8, 2020 at 6:52 AM Everett Maus via llvm-dev <llvm...@lists.llvm.org> wrote:

So yes--that would work.

I think for that route, though, it would be probably useful to try to output some more information than just std::terminate does. (i.e. that would work as a proof of concept, but getting more debug information the way you do with noexcept functions or functions without a handler would be more user friendly).

The user can control what std::terminate does.

 

The other reason I was hesitant to go that route is that I was worried about the impact of that on binary size (as that's... I think 4+ bytes per function added for the .gcc_except_table sections, as opposed to just 4 bytes for the empty section).  That seems fine for a small binary, but seems likely to be expensive-to-prohibitive for large binaries/libraries.

Not sure how much binary size balances with other concerns, but it sounds to me that the methods proposed are ones that would result in false positives where unwinding through the frame would have resulted in no action even when compiled with exceptions fully on.

Perhaps leaving functions that would otherwise be "transparent" to exception handling alone is already implied?

If the baseline for the new mode is instead the fully exception-aware one, then I think one form of the desired functionality (with high granularity in terms of avoiding false positives) is to:

error on compiling try-blocks,

make cleanup landing pads terminate,

leave noexcept(true) alone,

and, for dynamic exception specifications, call terminate instead of unexpected.

[Reid Kleckner via llvm-dev]

I would suggest using a custom personality function for this. It will optimize better and be much smaller than using a standard personality function. It saves the LSDA tables.

LLVM supports custom personality functions, so only clang changes are required. You could either do something like add a flag to override the EH personality with a custom one, or come up with a new dedicated fno-exceptions termination personality and add it to compiler-rt.

_______________________________________________
LLVM Developers mailing list
llvm...@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

[Everett Maus via llvm-dev]

That makes sense.  Really appreciate the feedback, all.

I think the approach I'll look at implementing is probably to:

- Implement a dedicated 'termination' personality function (in compiler-rt?) that does appropriate logging + exit.

- Add a flag (-fterminate-exceptions?). This is because this is a very clear behavior change, so at least providing an opt-in/opt-out mechanism seems important. Possible option: make this an enum, have 'strict' = all exceptions crash, 'normal' = exceptions passing through methods that would require a cleanup /etc. terminate, none (default) = current behavior where things just leak/etc.

- During code generation, when -fno-exceptions is turned on, if -fterminate-exceptions was passed, it changes the personality function from being not-present to being the dedicated -fno-exceptions termination personality function.

Not sure how much binary size balances with other concerns, but it sounds to me that the methods proposed are ones that would result in false positives where unwinding through the frame would have resulted in no action even when compiled with exceptions fully on.

Perhaps leaving functions that would otherwise be "transparent" to exception handling alone is already implied?

So I think this is actually not ideal behavior, at least for the use case I have in mind.

I think I'd prefer (and the team I partner with would prefer) /any/ exception passing from code compiled with -fexceptions to code compiled with -fno-exceptions to halt & return an error, even if the method wouldn't have participated in exception handling (e.x. no classes that need to have destructors called, etc.)  I think the most desirable behavior is "the program halts if exceptions pass through code compiled without exceptions enabled".

There's a few reasons for this:

First, because you can imagine that you could wind up with a situation where a "happy path" will usually work (but then you get an unexpected halt on a less well tested path).

Second, because you can imagine a situation where that winds up putting code in a very weird position where adding a local variable with a destructor that must be called changes how a particular method participates in exception handling from "it just passes exceptions through"  to "it crashes". This could leave code in a weird state where it's hard to reason about the impact of a change or it goes from a perceived "working fine" state to a crashing state.

Third, it makes using *SAN code harder and less predictable. The reason I became aware of this issue at all is that various sanitizers will insert landing pads to keep track of stack unwinding (but don't do that with -fno-exceptions code or if they believe an exception cannot pass through the method).  That then leads to very, very weird behavior with those sanitizers (memory leaks, very weird/hard to unravel stacks as new frames get consistently added, etc.).  Sure--you could make the *SAN build terminate in that case instead of just behaving weirdly, but it'd be nice to get that behavior for normal code (since it's unrelated to the *SAN behavior).

If having an intermediate scenario makes sense (only halting when a method would have taken part in exception handling), then making the flag have 3 states (strict/normal/none) seems like the right choice to me.

Thoughts/feedback on this approach?

I have verified that changing the personality function to std::terminate during code generation (via a boolean flag/etc.) does exactly what I'm looking for (but also has an impact on intermediate sizes of about what I expected).

Thanks,

--EJM

 

--

--EJM

[Everett Maus via llvm-dev]

One additional question, actually:

From looking at the current location of exception-handling code, would it make more sense to have the personality function in libcxxabi (where __gxx_personality_v0 and friends live) or in compiler-rt (which doesn't seem to have any exception handling code at the moment)?

Thanks,

Everett Maus

--

--EJM

[James Y Knight via llvm-dev]

Why is adding a new personality function useful? Can't you share a single LSDA table for every noexcept function in a TU (both those implicitly noexcept due to -fno-exceptions and for those marked "noexcept")?

[Modi Mo via llvm-dev]

> - Add a flag (-fterminate-exceptions?). This is because this is a very clear behavior change, so at least providing an opt-in/opt-out mechanism seems important. Possible option: make this an enum, have 'strict' = all exceptions crash, 'normal' = exceptions passing through methods that would require a cleanup /etc. terminate, none (default) = current  behavior where things just leak/etc.

 

+1. This definitely needs to be an opt-in feature over the current default behavior.

 

> I think I'd prefer (and the team I partner with would prefer) /any/ exception passing from code compiled with -fexceptions to code compiled with -fno-exceptions to halt & return an error, even if the method wouldn't have participated in exception handling (e.x. no classes that need to have destructors called, etc.)  I think the most desirable behavior is "the program halts if exceptions pass through code compiled without exceptions enabled".

 

Fully agreed. Mixing -fno-exceptions code with -fexceptions code is already dangerous and an anti-pattern. Propagating an exception successfully past code which is -fno-exceptions is asking for problems when LLVM has been explicitly instructed to build with the knowledge that this scenario will not occur. To that end, a “non-strict” mode means the compiler has to reason about exceptions with -fno-exceptions passed which makes this seem more of a sub-flag for -fexceptions funnily enough.

 

> I have verified that changing the personality function to std::terminate during code generation (via a boolean flag/etc.) does exactly what I'm looking for (but also has an impact on intermediate sizes of about what I expected).

 

Nice! I’m curious about the impact of sizes. The relevant CIE + FDE with exceptions look like the following:

 

From llvm-dwarfdump:

 

Has handler:

000000a8 0000001c ffffffff CIE

  Version:               1

  Augmentation:          "zPLR"

  Code alignment factor: 1

  Data alignment factor: -8

  Return address column: 16

  Personality Address: 00201a50 // gxx_personality_v0

  Augmentation data:     03 50 1A 20 00 03 1B

 

  DW_CFA_def_cfa: reg7 +8

  DW_CFA_offset: reg16 -8

  DW_CFA_nop:

  DW_CFA_nop:

 

000000c8 00000024 00000024 FDE cie=00000024 pc=00001120...0000117f

  LSDA Address: 002006b0

 

No Handler:

00000000 00000014 ffffffff CIE

  Version:               1

  Augmentation:          "zR"

  Code alignment factor: 1

  Data alignment factor: -8

  Return address column: 16

  Augmentation data:     1B

 

  DW_CFA_def_cfa: reg7 +8

  DW_CFA_offset: reg16 -8

  DW_CFA_nop:

  DW_CFA_nop:

 

00000018 00000010 0000001c FDE cie=0000001c pc=fffffc00...fffffc2f

  DW_CFA_advance_loc: 4

  DW_CFA_undefined: reg16

 

 

Registering terminate/custom handler as a handler allows you to bypass the “L” Augmentation (since you don’t need an LSDA in such cases) so the total cost should be:

1. “P” augmentation string, 1 byte
2. Encoding in Augmentation data of Personality Routine, 1 byte
3. Relocation to Personality Routine, 4 bytes

 

So it should add 6 bytes to every function. You can’t get away from adding an address to each function’s .eh_frame entry with how CIE/FDE works. I think the extreme that is possible is encoding this scenario in a new character in the Augmentation String but that will severely limit what you can do outside of directly calling abort() along with changing the ABI 😃.

 

> From looking at the current location of exception-handling code, would it make more sense to have the personality function in libcxxabi (where __gxx_personality_v0 and friends live) or in compiler-rt (which doesn't seem to have any exception handling code at the moment)?

 

It depends on the amount of functionality here and how language specific this is. As it stands I don’t think this is C++ specific and could extend to other llvm language targets which makes it a better candidate with compiler-rt. It would be good to flesh out what behavior you want out of the personality function then go from there.

 

Modi

[Modi Mo via llvm-dev]

Actually a single CIE can host multiple FDEs corresponding to multiple functions so it should be less than 6 bytes per function.

[Reid Kleckner via llvm-dev]

Using existing personality functions requires emitting an LLVM IR cleanup around every function when building in this -fterminate-exceptions mode. Then all functions would have IR like this:

invoke void @foo(...) unwind to %terminate

invoke void @bar(...) unwind to %terminate

...

landingpad cleanup ...

  call void @myterminate()

[Reid Kleckner via llvm-dev]

Bleh, early send. We'd also have to overcome the issue of functions being nounwind with -fno-exceptions, which means LLVM would optimize those invokes to calls immediately.

[Everett Maus via llvm-dev]

It depends on the amount of functionality here and how language specific this is. As it stands I don’t think this is C++ specific and could extend to other llvm language targets which makes it a better candidate with compiler-rt. It would be good to flesh out what behavior you want out of the personality function then go from there.

That makes sense.

I think there's one "must have" behavior and a few "very nice-to-have but not required" behaviors.

The "must have" for the handler would be "it terminates the program when an exception passes through code compiled with -fno-exceptions, hopefully in a way that can be easily identified as different from a normal/healthy program exit".

The nice-to-have list is probably longer:

1. Printing the exception type that was thrown

2. Recovering the place the exception was thrown /from/ and printing the function name & offset.

3. Printing the function name that the unwinder reached that led to it exiting.

4. (Extra fancy/pie in the sky "what I'd want if I were debugging this situation"): printing the full demangled stack to the location the exception was thrown from or the method that led to the exit.

I expect that (1) requires some language-specific handling (RTTI in C++/etc.), so that'd imply that it might be good to have a libcxxabi implementation even if there's also a compiler-rt version (it looks like this is the case with __gxx_personality_v0 which is defined both in compiler-rt here: https://github.com/llvm/llvm-project/blob/main/compiler-rt/lib/builtins/gcc_personality_v0.c#L172 and in libcxxabi here: https://github.com/llvm/llvm-project/blob/main/libcxxabi/src/cxa_personality.cpp#L953

I also think that possibly (2)/(3)/(4) might require some non-compiler-rt components, but I'm not certain if they would (I haven't dug into what's in compiler-rt in depth quite yet).  That may also be reasonable to defer or to have a sanitizer handle.

--EJM

--

--EJM

[Modi Mo via llvm-dev]

The gcc_personality_v0 in compiler-rt is actually kind of an oddball. It maps to a custom GCC extension `__attribute__((cleanup(xx)))` (https://gcc.gnu.org/onlinedocs/gcc/Common-Variable-Attributes.html) which is “destructors but for C” which has to work if you have:

C++ code calling C code calling C++ code

To ensure that this attribute is correctly maintained when throwing from C++ to C++ code with C code sandwiched in the middle.

 

For (1) if the exception type desired is the C++ exception type, then that’ll definitely move this into libcxxabi. There’s a lower level code (https://github.com/llvm-project/llvm-project-dev/blob/master/libcxxabi/src/cxa_exception.hpp#L23) that exists to identify if an exception is from Clang/GCC/MSVC which would still be suitable for compiler-rt.

 

I think the rest of nice-to-have also move it towards libcxxabi. Stack unwinding and function context for C++ have ABI specifications that lock it into a C++ runtime library.

 

My suggestion would be to get a std::terminate implementation with acceptable size bloat tested and working. That by itself can be reviewed and is self-contained such that a more robust solution can easily take its place later. As Hubert points out, you can also override std::terminate with https://en.cppreference.com/w/cpp/error/set_terminate so there’s quite some flexibility there.

 

Adding runtime code is (in my experience) pretty difficult because of backwards-compatibility and the larger surface area.

 

Modi

[Fangrui Song via llvm-dev]

On 2020-12-09, Everett Maus via llvm-dev wrote:
>>
>> It depends on the amount of functionality here and how language specific
>> this is. As it stands I don’t think this is C++ specific and could extend
>> to other llvm language targets which makes it a better candidate with
>> compiler-rt. It would be good to flesh out what behavior you want out of
>> the personality function then go from there.
>>
>
>That makes sense.
>
>I think there's one "must have" behavior and a few "very nice-to-have but
>not required" behaviors.
>
>The "must have" for the handler would be "it terminates the program when an
>exception passes through code compiled with -fno-exceptions, hopefully in a
>way that can be easily identified as different from a normal/healthy
>program exit".

If you don't need .eh_frame and .eh_frame_hdr otherwise (no need to
appease debuggers/profilers/crash reporters/other stack unwinders),
you can simply add -fno-asynchronous-unwind-tables.

Clang behavior:

* `-fno-exceptions` && `-fno-asynchronous-unwind-tables` => no `.eh_frame`
* `-fno-exceptions` => no `.gcc_except_table`
* noexcept && -fexceptions => call __clang_call_terminate

libgcc_s/libunwind libsupc++/libc++abi behavior:

* no .eh_frame => __cxa_throw calls std::terminate since _Unwind_RaiseException returns
* .eh_frame + empty .gcc_except_table => __gxx_personality_v0 calls std::terminate since no call site code range matches
* .eh_frame without .gcc_except_table => pass-through

In short, if you just require the feature to work like a poor man's sanitizer,
-fno-exceptions -fno-asynchronous-unwind-tables already work.

>The nice-to-have list is probably longer:
>1. Printing the exception type that was thrown

I believe this requires a new personality as Reid mentioned.

>2. Recovering the place the exception was thrown /from/ and printing the
>function name & offset.

If there is a custom personality, it can call _Unwind_GetIP (Itanium C++ ABI)
or _Unwind_GetIPInfo (GCC extension, implemented in llvm-project libunwind)
to retrieve the instruction pointer, then you'll need a symbolizer.

>3. Printing the function name that the unwinder reached that led to it
>exiting.

This is more complex because the personality will have to actually unwind the stack
(pass control to Level 1 libunwind, i.e. it cannot simply return _URC_FATAL_PHASE1_ERROR).

>4. (Extra fancy/pie in the sky "what I'd want if I were debugging this
>situation"): printing the full demangled stack to the location the
>exception was thrown from or the method that led to the exit.

_Unwind_Backtrace, which exists as a nongnu libunwind/GCC extension.

>I expect that (1) requires some language-specific handling (RTTI in
>C++/etc.), so that'd imply that it might be good to have a libcxxabi
>implementation even if there's also a compiler-rt version (it looks like
>this is the case with __gxx_personality_v0 which is defined both in
>compiler-rt here:
>https://github.com/llvm/llvm-project/blob/main/compiler-rt/lib/builtins/gcc_personality_v0.c#L172
>and in libcxxabi here:
>https://github.com/llvm/llvm-project/blob/main/libcxxabi/src/cxa_personality.cpp#L953
>
>I also think that possibly (2)/(3)/(4) might require some non-compiler-rt
>components, but I'm not certain if they would (I haven't dug into what's in
>compiler-rt in depth quite yet). That may also be reasonable to defer or
>to have a sanitizer handle.

There should be no compiler-rt side change. __gcc_personality_v0 is in compiler-rt/libgcc_s
simply because it is used by C :( and it cannot use libsupc++/libc++abi :)

>>>>>> *From: *llvm-dev <llvm-dev...@lists.llvm.org> on behalf of

>>>>>> Everett Maus via llvm-dev <llvm...@lists.llvm.org>

>>>>>> *Reply-To: *Everett Maus <evm...@google.com>
>>>>>> *Date: *Monday, December 7, 2020 at 12:47 PM
>>>>>> *To: *"llvm...@lists.llvm.org" <llvm...@lists.llvm.org>
>>>>>> *Subject: *[llvm-dev] Catching exceptions while unwinding through

[James Y Knight via llvm-dev]

On Tue, Dec 8, 2020 at 2:26 PM Everett Maus via llvm-dev <llvm...@lists.llvm.org> wrote:

That makes sense.  Really appreciate the feedback, all.

I think the approach I'll look at implementing is probably to:

- Implement a dedicated 'termination' personality function (in compiler-rt?) that does appropriate logging + exit.

I still think we don't actually need to introduce a new personality function -- just reusing the standard C++ one, instead.

- Add a flag (-fterminate-exceptions?). This is because this is a very clear behavior change, so at least providing an opt-in/opt-out mechanism seems important. Possible option: make this an enum, have 'strict' = all exceptions crash, 'normal' = exceptions passing through methods that would require a cleanup /etc. terminate, none (default) = current behavior where things just leak/etc.

I agree we probably do need a flag for this change in behavior, but I think this flag is going to be very difficult to name and explain, because there are currently different behaviors depending on the target and other flags you pass. This existing behavior is very inconsistent and confusing...

1. With -fno-asynchronous-unwind-tables -fno-exceptions, no unwind info is emitted, and therefore, attempting to throw an exception through such a function already aborts. This is the historically-common behavior.

2. With -fasynchronous-unwind-tables -fno-exceptions, unwind info is emitted for functions, and that unwind info currently specifies that exceptions can be thrown through such functions transparently.

To make matters worse, the default of -fasynchronous-unwind-tables is different per-platform. In Clang, it's enabled for x86-64-linux, and off for i386-linux and ppc64-linux. And, in GCC, the default value even depends on other flags, and is customized in some distro's builds of the compiler. Whether it's on or off for i386-linux depends on whether you specify -fomit-frame-pointer or not...except that Debian (and maybe other distros?) have patched the spec file to enable async unwind tables, regardless.

- During code generation, when -fno-exceptions is turned on, if -fterminate-exceptions was passed, it changes the personality function from being not-present to being the dedicated -fno-exceptions termination personality function.

We'll also probably want to adjust the inliner, because it otherwise prohibits inlining between functions with different personality routines (which will be relevant with cross-TU inlining with LTO).

Anyways -- what I'd really like to see done here is to BOTH fix the handling of C++ "noexcept" functions to have minimal overhead, and then simply use the same mechanism to implement -fno-exceptions. This means we do in general need to be able to implement this behavior with the __gxx_personality_v0 personality, because noexcept functions can have try/catch blocks inside.

I believe that the current design of noexcept -- which requires explicit invoke and landingpads saying that they terminate -- should be revisited. It adds a lot of unnecessary overhead.

What we need is a way to explicitly declare that we want the personality-routine's default action to be taken on unwind -- implemented via leaving a gap in the LSDA table. I think we might want to do this both as syntax on invoke [e.g. `invoke void @xyz() to label %l unwind default`], as well as a function attribute declaring that any call in the function should be treated as having the default action of the personality function.

"nothrow" would emit this new function attribute in combination with nounwind. Inlining a function having this attribute into a function not having the attribute would need to convert `call ...` into `invoke ... unwind default` -- but since the inliner already has such code, that's not a big deal.

Not sure how much binary size balances with other concerns, but it sounds to me that the methods proposed are ones that would result in false positives where unwinding through the frame would have resulted in no action even when compiled with exceptions fully on.

Perhaps leaving functions that would otherwise be "transparent" to exception handling alone is already implied?

So I think this is actually not ideal behavior, at least for the use case I have in mind.

I think I'd prefer (and the team I partner with would prefer) /any/ exception passing from code compiled with -fexceptions to code compiled with -fno-exceptions to halt & return an error, even if the method wouldn't have participated in exception handling (e.x. no classes that need to have destructors called, etc.)  I think the most desirable behavior is "the program halts if exceptions pass through code compiled without exceptions enabled".

I agree that is a desirable behavior.

There's the additional complication of C code.  Historically, before -fasynchronous-unwind-tables became a common default, throwing through C functions would also abort, by default. That's why Glibc builds C functions that might call user C++ code with -fexceptions (example: qsort) -- that way it can ensure that unwinding a C++ exception does actually work properly.

Unfortunately, unlike in C++, we don't have a ready-made personality function we can use for C functions which has "terminate" behavior. The _gcc_personality_v0 doesn't terminate upon failing to find a LSDA table entry.

[Everett Maus via llvm-dev]

In short, if you just require the feature to work like a poor man's sanitizer,
-fno-exceptions -fno-asynchronous-unwind-tables already work.

& 

I agree we probably do need a flag for this change in behavior, but I think this flag is going to be very difficult to name and explain, because there are currently different behaviors depending on the target and other flags you pass.

I guess my take is that the way it should behave is "If exceptions are disabled on a TU via -fno-exceptions, and this flag is passed to that TU, then we should guarantee that any method in it that an exception handler passes through is guaranteed to exit".

I think the complexity of the state of -fno-asynchronous-unwind-tables vs. -fasynchronous-unwind-tables that you highlighted makes an additional case for something that ideally won't be affected by those flags but does work well with -fno-exceptions.

That absolutely seems, at least to me, to imply that we need something other than just -fno-asynchronous-unwind-tables (consider that:

-fno-asynchronous-unwind-tables + -fno-exceptions does exit, but -fno-asynchronous-unwind-tables + -funwind-tables + -fno-exceptions does not).  It'd be nice to have something that guaranteed this behavior for all translation units compiled with -fno-exceptions (and perhaps implied that unwind tables are set up in the correct way; becoming incompatible with -fno-asynchronous-unwind-tables + -fno-unwind-tables).

The other concern I have is the current error message for this behavior from the existing personality function; it'd be good to surface not just "an uncaught exception caused the code to terminate" but that the exception handler had to traverse through code that was not compiled with exceptions in mind (or with Jame's proposal, where we also change noexcept to use this, where we have -fterminate-exceptions + fno-exceptions --> all methods are implied to be noexcept; which might be neater, to be honest.).

There should be no compiler-rt side change. __gcc_personality_v0 is in compiler-rt/libgcc_s
simply because it is used by C :( and it cannot use libsupc++/libc++abi :)

& 

There's the additional complication of C code.  Historically, before -fasynchronous-unwind-tables became a common default, throwing through C functions would also abort, by default. That's why Glibc builds C functions that might call user C++ code with -fexceptions (example: qsort) -- that way it can ensure that unwinding a C++ exception does actually work properly. 

Unfortunately, unlike in C++, we don't have a ready-made personality function we can use for C functions which has "terminate" behavior. The _gcc_personality_v0 doesn't terminate upon failing to find a LSDA table entry.

This is unfortunate, actually, because compatibility with code being compiled as C and linked with C++ that an exception may pass through is definitely something it'd be good to detect.  The prototype implementation I have using custom personality functions that I have does enable this at the moment by just adding a separate compiler-rt personality function.  I suppose we could also modify __gcc_personality_v0 to handle this case in some way.

After doing some prototyping, at least initially I think the following is a reasonable proposal for using  that should meet the initial requirements I'd want for this feature:

1) Adding a new personality routine to libcxxabi (tentatively named "__cxx_noexceptions_terminate_personality_v0") which will exit with a message containing the detail that an exception was passed through code compiled without exception handling, along with exception type (if libcxxabi is compiled without LIBCXXABI_SILENT_TERMINATE; otherwise it would simply invoke std::terminate & exit).

2) Adding a new personality routine to compiler-rt (tentatively named "__noexceptions_terminate_personality_v0") which will exit with a less verbose message, but provide the same 'exit the program with some sort of message' functionality for C code compiled with -fno-exceptions if an exception is passed through it. (E.x. if something is compiled with all methods that are in an extern "C" { ... } block or similar, this would insert the C handler for those methods.)

3) Adding a new flag, -fterminate-exceptions (=strict or =none (the default)), to clang, which when set to 'strict' will add these new personality functions to all methods when -fno-exceptions is also set (and would have no effect otherwise).  I'm opting for an enumeration instead of a boolean flag here as one could imagine wanting to add an intermediate state in the future where it only halts if it would result in cleanups not occurring (but I propose not doing that work in the initial version).  This flag shouldn't change the behavior of code compiled with -fexceptions, and it should default to not affecting the behavior of code compiled with -fno-exceptions.

Other notes:

- This shouldn't result in changes to functions marked with 'noexcept(true)' unless they are also compiled with -fno-exceptions (in which case they would get the new personality function like all other methods).  The current "add a landing pad to the caller that invokes __clang_call_terminate" behavior would still be present.

- I expect that the new personality routine would wind up wanting to call a handler instead of terminating itself (to support the optional demanging).  That would imply also adding a new member "__cxa_noexcept_terminate_handler"  or similar that has its behavior defined behind a compilation flag (like the other handlers in src/cxa_default_handlers.cpp).

- I think it's reasonable to defer printing stack information or information about where the exception was thrown/where it encountered a function compiled with -fno-exceptions and this flag until later. (That would potentially be an output message change, though--I don't know what the requirements are around text-output behavior changes. Potentially, I could see just gating that behind further restrictions or a separate compiler flag/etc. since calling out to the symbolizer seems like it might be expensive.)

- This seems to have an adverse effect on binary size, but it's less than 5% without optimizations, which is probably OK.

- My take is that this sort of feature is likely one that you'd want during test/debug/fuzzing builds, but probably would drop during an optimized build. From that perspective, I'm not as concerned about size bloat or the fact that inlining/optimizations will work less well.

Option 2:

I'm also going to look at prototyping something like what James is suggesting tomorrow, and I'll write up that as an alternate proposal.

It does seem like it'd be a win to clean up the current issue with noexcept functions needing to add landing pads everywhere.  However, one thing I dislike with the current proposal of just leaving the LSDA empty, aside from the fact that it'll be hard to make it work with the C personality function, is that I don't think there's a good way (via just leaving the LSDA missing) to flag he difference between "it's missing from the table because it wasn't compiled in with -fasynchronous-unwind-table or -funwind-table" versus "it was explicitly removed because we want this to be required to halt." I feel like for debugging/etc. purposes, it would probably be good to flag the difference (as the exception would 'seem' like it should be caught to the programmer).

Thanks,

Everett Maus

--

--EJM

[Everett Maus via llvm-dev]

So, I've been digging into this a bit more over the past week or so.

I think both options are viable.

A custom personality function is definitely easier (and does seem to be necessary for C code); so it may actually be ideal to do a mixed version of the proposal (custom personality for C code, normal personality + different tables for C++ code and noexcept functions).

The work to re-do how exception tables are emitted for noexcept functions is much larger, but does seem like it would work pretty well & give the desired behavior.

The general idea of the second approach would be:  

- We add a new attribute (noexcept/haltonunwind/name still in process) to LLVM IR.

- In Clang: by default, noexcept functions receive this attribute.  Also, so do all functions in a translation unit compiled with -fno-exceptions + -fterminate-exceptions. (So we would have a number of functions with "nounwind noexcept" attributes on them or similar.)  This would affect the code generation for noexcept functions as well (removing the need for the explicit terminate handler block).

- In LLVM, when emitting unwind tables, if a function that has this particular attribute, we only emit regions for explicit try/catch blocks, and intentionally leave other areas out of the exception table (but we do still emit the default personality function/CFI/LSDA/etc. for the method). This leads to there being a "gap" in the table (and for functions compiled with -fno-exceptions, this means basically emitting a table that has no call sites/ranges in it at all). That then causes __gxx_personality_v0 to halt.

I'm still trying to figure out if/how this would work for Windows/other exception models like SJLG or WASM, though, which adds some complexity to this proposal that I'd need to figure out.

Open question about process--now that I've done enough prototyping to potentially put together a bit more of a proposal, is it best to just start a new thread for discussion of the two options?

Thanks,

Everett Maus

--

--EJM

[Hubert Tong via llvm-dev]

On Tue, Dec 22, 2020 at 7:57 PM Everett Maus via llvm-dev <llvm...@lists.llvm.org> wrote:

So, I've been digging into this a bit more over the past week or so.

I think both options are viable.

A custom personality function is definitely easier (and does seem to be necessary for C code); so it may actually be ideal to do a mixed version of the proposal (custom personality for C code, normal personality + different tables for C++ code and noexcept functions).

The work to re-do how exception tables are emitted for noexcept functions is much larger, but does seem like it would work pretty well & give the desired behavior.

The general idea of the second approach would be:  

- We add a new attribute (noexcept/haltonunwind/name still in process) to LLVM IR.

- In Clang: by default, noexcept functions receive this attribute.  Also, so do all functions in a translation unit compiled with -fno-exceptions + -fterminate-exceptions. (So we would have a number of functions with "nounwind noexcept" attributes on them or similar.)  This would affect the code generation for noexcept functions as well (removing the need for the explicit terminate handler block).

- In LLVM, when emitting unwind tables, if a function that has this particular attribute, we only emit regions for explicit try/catch blocks, and intentionally leave other areas out of the exception table (but we do still emit the default personality function/CFI/LSDA/etc. for the method). This leads to there being a "gap" in the table (and for functions compiled with -fno-exceptions, this means basically emitting a table that has no call sites/ranges in it at all). That then causes __gxx_personality_v0 to halt.

What is the nature of this "halting"? If it's not calling a terminate_handler, then the behaviour is not correct to use by default for noexcept functions.

 

[Fāng-ruì Sòng via llvm-dev]

On Tue, Dec 22, 2020 at 9:08 PM Hubert Tong via llvm-dev <llvm...@lists.llvm.org> wrote:
>
> On Tue, Dec 22, 2020 at 7:57 PM Everett Maus via llvm-dev <llvm...@lists.llvm.org> wrote:
>>
>> So, I've been digging into this a bit more over the past week or so.
>>
>> I think both options are viable.
>>
>> A custom personality function is definitely easier (and does seem to be necessary for C code); so it may actually be ideal to do a mixed version of the proposal (custom personality for C code, normal personality + different tables for C++ code and noexcept functions).
>>
>> The work to re-do how exception tables are emitted for noexcept functions is much larger, but does seem like it would work pretty well & give the desired behavior.
>>
>> The general idea of the second approach would be:  
>> - We add a new attribute (noexcept/haltonunwind/name still in process) to LLVM IR.
>> - In Clang: by default, noexcept functions receive this attribute.  Also, so do all functions in a translation unit compiled with -fno-exceptions + -fterminate-exceptions. (So we would have a number of functions with "nounwind noexcept" attributes on them or similar.)  This would affect the code generation for noexcept functions as well (removing the need for the explicit terminate handler block).
>> - In LLVM, when emitting unwind tables, if a function that has this particular attribute, we only emit regions for explicit try/catch blocks, and intentionally leave other areas out of the exception table (but we do still emit the default personality function/CFI/LSDA/etc. for the method). This leads to there being a "gap" in the table (and for functions compiled with -fno-exceptions, this means basically emitting a table that has no call sites/ranges in it at all). That then causes __gxx_personality_v0 to halt.
>
> What is the nature of this "halting"? If it's not calling a terminate_handler, then the behaviour is not correct to use by default for noexcept functions.

The default action of the personality (i.e. when LSDA exists but PC belongs to a gap in the LSDA). For __gxx_personality_v0, it is __cxa_begin_catch + std::terminate (for a foreign exception the std::get_terminate() handler).

A function attribute is definitely useful - which can give us the GCC eh_must_not_throw size optimization benefits. I haven't thought a lot about whether `invoke ... unwind default` (default action for invoke) is needed.

I may be a bit conservative on whether a new personality should be introduced.

--
宋方睿

[Everett Maus via llvm-dev]

The nature of "halting" is in cxa_personality.ccp, which is slightly more complex than __clang_call_terminate but not 

, and then either call the terminate handler for the exception directly or call std::terminate if it is a foreign exception.  So that behavior largely matches the noexcept behavior as far as I can tell (which calls __clang_call_terminate, which sets exception context via __cxa_begin_catch and then calls std::terminate, which checks the global exception context and if an exception handler is present, calls it, and if one is not simply calls the default termination handler--so basically they have the same outcome).

At least in the absence of optimizations, I don't think we actually /need/ to add the invoke--as long as we change the behavior for when the personality function and the exception table is emitted to force them to be emitted for a noexcept function.  A call to a function marked "noexcept" + "nounwind" is still guaranteed to not pass an exception out (the semantics of "call", at least as I understand them?).

At the moment, that's what my very raw prototype does--instead of changing the logic around when a call vs. invoke is emitted, I simply changed the code generation logic in LLVM for the exception tables for itanium style exceptions, so that they emit appropriately empty (or gapped/incomplete/etc.) tables appropriately in the presence of exceptions.  However I'm not completely sure if that'll be valid in the face of all optimizations--I'm still working out some details there.

--EJM

--

--EJM

[Everett Maus via llvm-dev]

Whoops!  First sentence should read "slightly more complex than __clang_call_terminate but looks like it has the same external behavior".

--

--EJM