throw std::exception with stack trace (portable)

[ste.ri...@gmail.com]

To find the root cause of bugs easier, it is very helpful to have stack trace from the location where the exception is thrown.

Surely I can throw an exception class derived from std:exception and use backtrace on Linux or StackWalk64 on Windows, with the appropriate symbols (=> not portable), 

or have a try catch "everywhere" and add file and line information (=> too much code to add), 

or not to catch the exception at all and let the Operation System write the dump file(=>not portable, some exceptions can be handled inside the program).

I thought there should be an easy portable way to add stack information to a std::exception. Does anybody have some ideas?

best regards,

Stefan

[Andrew Marlow]

I have been wanting this for years but there seems to be little interest unfortunately. We all know the challenges of doing it portably including demangling challenges but these are well known problems that have been solved before.

--

---
You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to std-discussio...@isocpp.org.
To post to this group, send email to std-dis...@isocpp.org.
Visit this group at https://groups.google.com/a/isocpp.org/group/std-discussion/.

--
Regards,

Andrew Marlow
http://www.andrewpetermarlow.co.uk

[Richard Hodges]

You can implement this easily in a portable way by adding a function try block to each function. In the exception handler use std::throw_with_nested to rethrow the exception chain. Pass the function name (__func__) as the argument to the runtime_error you throw.

There is an example of unwrapping the nested exceptions on cppreference.com  

[Nathan Ernst]

You can implement this easily in a portable way by adding a function try block to each function. In the exception handler use std::throw_with_nested to rethrow the exception chain. Pass the function name (__func__) as the argument to the runtime_error you throw.

There is an example of unwrapping the nested exceptions on cppreference.com

I wouldn't necessarily call doing this manually easy - although, not difficult, either.  It would require every call to be "annotated" , which is a very easy mistake to make.

I've implemented similar backtraces under linux, but I'd posit a reason it's not standard: I would think that in order to properly implement this, there may be several dynamic memory allocations (i.e. maintain the stack of functions that were involved), which could, in turn, cause other failures and just mask the original problem. Additionally, how would this play with inlined functions?

Additionally, I think this could fly in the face of the underlying C++ philosophy of only pay for that which you use.

I would generally agree that such a facility would be useful, however.

Regards

[Dilip Ranganathan]

This is not a trivial undertaking if its need to be made portable. My employer has open sourced

this effort here:

<https://github.com/bloomberg/bde/tree/master/groups/bal/balst>

You might need to poke around a bit to figure how to compose a rich Exception object filled with

backtrace information, but I'd start with:
<https://github.com/bloomberg/bde/blob/master/groups/bal/balst/balst_stacktraceutil.h>

It does require certain amount of buy-in into other parts of the library (dependencies and such) but

I haven't seen this problem addressed anywhere else in quite the way we have.

[Richard Hodges]

what could be more portable that the use of standard libraries? We use this paradigm in the two (large!) projects that I run. It works perfectly well on all platforms, is extremely efficient (stack trace is only constructed during an exception) and does not pollute program logic in any way.

It’s so effective at tracing bugs that we no longer need to produce megabytes of logs on the off-chance that there may be a bug to track down.

void foo()

try

{

   // do something

}

catch(…)

{

  std::throw_with_nested(std::runtime_error(__func__));

}

in fact, we use a variadic macro to build the exception so that arguments can be captured during the rollback.

e.g.

void foo(int arg1, std::string& const arg2)

try

{

  // do something

}

UTILS_CATCH_ARGS(__func__, arg1, arg2)

This is better than a stack trace since it allows us to capture extra information (such as a digest of the contents of each *this in the exception chain) and it all gets pushed into a log file.

Almost all bugs are fixed without even resorting to a debugger.

[Matthew Woehlke]

On 2016-04-15 02:02, ste.ri...@gmail.com wrote:
> To find the root cause of bugs easier, it is very helpful to have stack
> trace from the location where the exception is thrown.
>

> I thought there should be an easy portable way to add stack information to
> a std::exception. Does anybody have some ideas?

It sounds like you are asking for an opt-in (i.e. at the point where the
exception is created) mechanism for adding a stack trace. Is that correct?

If yes, I would drop everything about exceptions and just ask for a
portable way of obtaining a stack trace. I thought about writing a paper
for this once, but haven't done so. I think it would be a good addition
to the standard. If phrased such that only the API must be available,
but it isn't required to do anything, I think this would be useful to
stave off possible objections such as out of memory situations or
platforms that can't implement it reasonably. Then, if anyone is slow to
adopt it (i.e. platforms that don't implement it even though they
could), you can just file QoI bugs against those implementations.

--
Matthew

[Viacheslav Usov]

On Fri, Apr 15, 2016 at 4:19 PM, Matthew Woehlke <mwoehlk...@gmail.com> wrote:

> If yes, I would drop everything about exceptions and just ask for a portable way of obtaining a stack trace.

I would say that supporting a stack trace without an exception is more of an effort and probably a pessimisation than supporting a stack trace when an exception is thrown. When an exception is thrown, the implementation has to do stack unwinding. So it will have some supporting code for that, which can only be optimized away if the the throw is optimized away, and it will normally have some statically allocated data that could be translated into a stack trace, not necessarily at runtime. None of that needs to be present elsewhere.

I would say having a mechanism coupled with std::exception that collects those "pointers", which are not yet human readable, is a good thing, because certain people and certain companies may have policies that would prohibit exposing too much symbolic information about their code to third parties; "pointers" can be made lightweight, too. Another mechanism could translate those pointers into human readable stack traces, if appropriate symbolic info is available.

Cheers,

V.

[Tony V E]

I think a portable stack trace library would be a great addition to boost. 

I think making it standardized is an order of magnitude harder.

Sent from my BlackBerry portable Babbage Device

From: ste.ri...@gmail.com Sent: Friday, April 15, 2016 2:02 AM To: ISO C++ Standard - Discussion Reply To: std-dis...@isocpp.org Subject: [std-discussion] throw std::exception with stack trace (portable)

--

[Tony V E]

Why are we conflating bugs with exceptions? (sure the venn diagram might overlap but there is also a big gap)

Exceptions are for *coping* with the non-happy path‎ of execution.  And coping means that pre-written code is going to execute to cope. And then probably the program can continue along its way. 

We cope with bugs via future code, not pre-written code. 

If I ‎write some bug-detection code (ie bad state detection*), I might throw an exception or I might terminate, or I might "panic save"*, or tell the test framework, or log and continue or...

A stack trace can be useful in many cases.

---

*bad state detection is why exceptions and bugs are often conflated. Both deal with 'non happy' state. 

*panic save is where you save the user's document (to a different filename, don't overwrite the original) ‎and then (attempt to) tell the user "sorry, hope you can recover something from here" and then exit

Sent from my BlackBerry portable Babbage Device

From: Viacheslav Usov Sent: Friday, April 15, 2016 1:13 PM To: std-dis...@isocpp.org Reply To: std-dis...@isocpp.org Subject: Re: [std-discussion] Re: throw std::exception with stack trace (portable)

[Matthew Woehlke]

On 2016-04-15 13:13, Viacheslav Usov wrote:

> On Fri, Apr 15, 2016 at 4:19 PM, Matthew Woehlke wrote:
>> If yes, I would drop everything about exceptions and just ask for a
>> portable way of obtaining a stack trace.
>
> I would say that supporting a stack trace without an exception is more of
> an effort

I might be willing to believe this if obtaining stack traces at
arbitrary points of execution was not *already supported*, at least by
Windows and Linux.

> and probably a pessimisation than supporting a stack trace when
> an exception is thrown. When an exception is thrown, the implementation has
> to do stack unwinding. So it will have some supporting code for that, which
> can only be optimized away if the the throw is optimized away, and it will
> normally have some statically allocated data that could be translated into
> a stack trace, not necessarily at runtime. None of that needs to be present
> elsewhere.

That sounds like you want the operation to involve compiler magic, which
would make it a *language* feature, rather than a library feature.

Maybe an implementation could do that anyway as a QoI thing, but
*requiring* it to work that way (i.e. making it a language feature)
seems... ambitious. I think a library approach would have a better
chance at being accepted by the committee.

Plus, see Tony's point. I can think of many situations where I might
want a stack trace but I'm *not* throwing an exception.

I also have to wonder if your exception-based stack collection doesn't
quit working as soon as it hits a `catch`... which would be
unacceptable, of course...

> I would say having a mechanism coupled with std::exception that collects
> those "pointers", which are not yet human readable, is a good thing,
> because certain people and certain companies may have policies that would
> prohibit exposing too much symbolic information about their code to third
> parties

That's both orthogonal (see below) and also moot. If I have pointers,
either a) I can resolve those to symbols, regardless if the code did it
for me automatically, or b) I can't, period. Code that has been
symbol-stripped is just not going to be able to produce names from
pointers. Any proposal will need to take this into consideration.

> Another mechanism could translate those pointers into human readable
> stack traces, if appropriate symbolic info is available.

I do agree with this, however. In fact, I would make any such proposal
necessarily operate in 2-3 phases:

1. (Optional) Obtain count of available stack pointers¹.
2. Obtain up to N stack pointers.
3. Translate arbitrary stack pointers to symbol names.

Having (2) and (3) [available²] as separate operations is orthogonal to
whether or not exception unwinding is used to help with (2).

(2) should accept an input buffer which is filled with pointers.
(Possibly an overload that accepts a std::vector should also be
provided, but one taking a previously allocated memory block is mandatory.)

(3) should be usable on any pointer obtained in any fashion, not just by
(2). (No guarantees that it will *work*, of course...)

(1) isn't required, but it helps with (2), and I can imagine uses for
(1) that don't use (2) or (3), e.g. an algorithm that detects if it has
gone into an infinite recursion state and terminates itself gracefully
*before* crashing due to stack exhaustion.

(¹ In case of platforms where `void*` is not sufficient, let's assume
that when I say "[stack] pointer" I'm really talking about something
that the standard would specify as an "opaque" type.)

(² We might want to *additionally* provide a convenience function that
combines (2) and (3).)

--
Matthew

[Dilip Ranganathan]

On Fri, Apr 15, 2016 at 1:57 PM, Matthew Woehlke <mwoehlk...@gmail.com> wrote:

Plus, see Tony's point. I can think of many situations where I might
want a stack trace but I'm *not* throwing an exception.

This is possible with the library I posted else thread.

 

> I would say having a mechanism coupled with std::exception that collects
> those "pointers", which are not yet human readable, is a good thing,
> because certain people and certain companies may have policies that would
> prohibit exposing too much symbolic information about their code to third
> parties

That's both orthogonal (see below) and also moot. If I have pointers,
either a) I can resolve those to symbols, regardless if the code did it
for me automatically, or b) I can't, period. Code that has been
symbol-stripped is just not going to be able to produce names from
pointers. Any proposal will need to take this into consideration.

> Another mechanism could translate those pointers into human readable
> stack traces, if appropriate symbolic info is available.

I do agree with this, however. In fact, I would make any such proposal
necessarily operate in 2-3 phases:

1. (Optional) Obtain count of available stack pointers¹.
2. Obtain up to N stack pointers.
3. Translate arbitrary stack pointers to symbol names.

All 3 of the above are also possible with the library posted else thread.

It could be an useful starting point for standardization. 

[Viacheslav Usov]

On Fri, Apr 15, 2016 at 7:42 PM, Tony V E <tvan...@gmail.com> wrote:

Why are we conflating bugs with exceptions? (sure the venn diagram might overlap but there is also a big gap)

Exceptions are for *coping* with the non-happy path‎ of execution.  And coping means that pre-written code is going to execute to cope. And then probably the program can continue along its way. 

We cope with bugs via future code, not pre-written code. 

If I ‎write some bug-detection code (ie bad state detection*), I might throw an exception or I might terminate, or I might "panic save"*, or tell the test framework, or log and continue or...

A stack trace can be useful in many cases.

I did not debate that (nor did I conflate). My remarks was based on the premise that stack unwinding is solely specified in an exception context. That is something that every (conformant) implementation must have.

It is quite possible that certain implementation can do more.

Cheers,

V.

[Viacheslav Usov]

On Fri, Apr 15, 2016 at 7:57 PM, Matthew Woehlke <mwoehlk...@gmail.com> wrote:

On 2016-04-15 13:13, Viacheslav Usov wrote:

> I might be willing to believe this if obtaining stack traces at arbitrary points of execution was not *already supported*, at least by Windows and Linux.

That is more like "arcanely supported by some implementations". Stack unwinding, however, is a language feature.

> That sounds like you want the operation to involve compiler magic, which would make it a *language* feature, rather than a library feature.

Well, std::exception would be able to provide a list of stack pointers, That is technically a library feature, although compiler magic is required.

> That's both orthogonal (see below) and also moot. If I have pointers, either a) I can resolve those to symbols, regardless if the code did it for me automatically, or b) I can't, period. Code that has been symbol-stripped is just not going to be able to produce names from pointers. Any proposal will need to take this into consideration.

I am not seeing why that was called moot; I think we are in a violent agreement here. The code that throws may be symbol-stripped, but it can still collect those pointers and save/forward them to some other code that has the symbolic info.

Cheers,

V.

[Matthew Woehlke]

On 2016-04-15 14:47, Viacheslav Usov wrote:

> On Fri, Apr 15, 2016 at 7:57 PM, Matthew Woehlke wrote:
>> I might be willing to believe this if obtaining stack traces at arbitrary
>> points of execution was not *already supported*, at least by Windows and
>> Linux.
>
> That is more like "arcanely supported by some implementations".

Sounds like a good reason to standardize it :-).

> Stack unwinding, however, is a language feature.

...which I can *turn off*, and many users do. For this and other
reasons, I don't want to limit obtaining traces to exception contexts.

Encouraging implementations to leverage unwinding to produce better
traces *when available* is fine, and even — pardon the repetition —
encouraged :-).

>> That sounds like you want the operation to involve compiler magic, which
>> would make it a *language* feature, rather than a library feature.
>
> Well, std::exception would be able to provide a list of stack pointers,
> That is technically a library feature, although compiler magic is required.

If you're going to hide the language feature ("compiler magic") behind a
library facade, is there a benefit to restricting it to an exception
context?

>> On 2016-04-15 13:13, Viacheslav Usov wrote:

>>> I would say having a mechanism coupled with std::exception that
>>> collects those "pointers", which are not yet human readable, is a
>>> good thing, because certain people and certain companies may have
>>> policies that would prohibit exposing too much symbolic
>>> information about their code to third parties
>>

>> That's both orthogonal (see below) and also moot. If I have
>> pointers, either a) I can resolve those to symbols, regardless if
>> the code did it for me automatically, or b) I can't, period. Code
>> that has been symbol-stripped is just not going to be able to
>> produce names from pointers. Any proposal will need to take this
>> into consideration.
>
> I am not seeing why that was called moot; I think we are in a violent
> agreement here.

It may also be that I misunderstood your original point. As I read it,
you implied that separating obtaining pointers and resolving names is
useful for "proprietary" code in order to *prevent* leakage of
proprietary information. But separation isn't needed for that reason,
because if the symbols exist, I can resolve them "by hand" anyway, and
if they don't, the automatic mechanism will anyway fail to resolve them.
So separation for that reason is pointless ("moot").

But maybe you meant that separation is useful *because* resolution will
not be possible. I didn't read your comment this way because I would
expect any reasonable implementation to still report the pointer in that
case; even with a non-separated implementation, the pointers would still
be available (if in a less machine-readable format) for later analysis.
(That's not to disagree with your point, with which I actually agree;
just to explain why I read your comment as I did.)

> The code that throws may be symbol-stripped, but it can still collect
> those pointers and save/forward them to some other code that has the
> symbolic info.

Okay, so really you want separation in order to collect a trace now, but
delay resolution. Most of the reasons I can think for separation are the
same in essence, though e.g. to delay memory allocation in case the
reason I need a trace has to do with memory problems (low, corrupt,
etc.), or because I might be collecting a trace that may or may not be
thrown out (e.g. hand-written code to do something like what
valgrind-memcheck does) and don't want to pay (time, memory) for resolution.

All good reasons :-).

--
Matthew

[Nicol Bolas]

On Friday, April 15, 2016 at 2:47:26 PM UTC-4, Viacheslav Usov wrote:

On Fri, Apr 15, 2016 at 7:57 PM, Matthew Woehlke <mwoehlk...@gmail.com> wrote:

On 2016-04-15 13:13, Viacheslav Usov wrote:

> I might be willing to believe this if obtaining stack traces at arbitrary points of execution was not *already supported*, at least by Windows and Linux.

That is more like "arcanely supported by some implementations". Stack unwinding, however, is a language feature.

Stack unwinding is a language feature. But the nature of the stack itself is not.

If a function gets inlined, it probably disappears from the executed output and the stack trace. But by the rules of the standard, the effect of stack unwinding through an inlined call must still function. Which is fine.
 

> That sounds like you want the operation to involve compiler magic, which would make it a *language* feature, rather than a library feature.

Well, std::exception would be able to provide a list of stack pointers,

... why? That all rather depends on how stack unwinding is implemented. I'm not sure I see a need for an implementation to have "stack pointers".

[Tony V E]

Sorry, I should mention that my 'conflate' remarks were more for the whole thread, not just a reply to you. I was originally typing a response to the OP, but then also wanted to respond to your post... sorry.

Sent from my BlackBerry portable Babbage Device

From: Viacheslav Usov Sent: Friday, April 15, 2016 2:35 PM

To: std-dis...@isocpp.org Reply To: std-dis...@isocpp.org Subject: Re: [std-discussion] Re: throw std::exception with stack trace (portable)

--

[Richard Hodges]

The c++ standard makes no mention of a stack. The memory model makes no assumption that there is one.

In fact, in optimised code, there is often no stack use at all because of extensive inlining.

The only time you'll ever need a stack trace is when you don't have a debugger handy - I.e. Production code. In this case it's almost useless because of inlining.

In the light of this, I don't see that it is feasible , desirable (or even possible) to impose a stack trace on implementors.

C++ is not like other languages. It expresses intent. The compiler transforms that intent into 'as if' code. 

It is wiser to focus efforts on guaranteeing that the correct intent is specified. This is (in the main) achieved through copious use of standard library constructs, idioms and patterns, static asserts, early parameter checking and good, informative exceptions.

On Friday, 15 April 2016, Viacheslav Usov <via....@gmail.com> wrote:e 

--

[Philipp Stephani]

Richard Hodges <hodg...@gmail.com> schrieb am Sa., 16. Apr. 2016 um 11:58 Uhr:

The c++ standard makes no mention of a stack. The memory model makes no assumption that there is one.

The C++ standard does mention the stack. It is the thing that gets unwound during stack unwinding.

 

The only time you'll ever need a stack trace is when you don't have a debugger handy - I.e. Production code. In this case it's almost useless because of inlining.

In practice only a few simple functions are inlined, and most functions are not (and often cannot be) inlined. Stack traces tend to be very useful and valuable in practice.

 

In the light of this, I don't see that it is feasible , desirable (or even possible) to impose a stack trace on implementors.

Is is both feasible, desirable and possible. In fact, many implementations provide stack traces today. Implementations should be allowed to omit frames for inlined functions, just like in other programming languages, e.g. Java.

 

C++ is not like other languages. It expresses intent. The compiler transforms that intent into 'as if' code. 

All other programming languages are exactly alike in this respect. Otherwise optimizations wouldn't be possible at all.

 

It is wiser to focus efforts on guaranteeing that the correct intent is specified. This is (in the main) achieved through copious use of standard library constructs, idioms and patterns, static asserts, early parameter checking and good, informative exceptions.

 That typically (e.g. in the case of Boost.Exception) requires cooperation by all parties involved and significant boilerplate and/or macros.

[Edward Catmur]

Adding to this: stack traces are useful precisely when deciding whether to open a crash in a debugger - assuming you haven't suppressed core generation, in which case they can still be used to determine whether a crash is known and whether to enable core generation.

Because stack traces are lightweight and do not require manual intervention or any special machinery to generate, they can be used in the first line of bug triage and prioritization. It doesn't matter that they're less than perfect.

[Andrew Marlow]

On 16 April 2016 at 11:30, Philipp Stephani <p.ste...@gmail.com> wrote:

Richard Hodges <hodg...@gmail.com> schrieb am Sa., 16. Apr. 2016 um 11:58 Uhr:

The c++ standard makes no mention of a stack. The memory model makes no assumption that there is one.

The C++ standard does mention the stack. It is the thing that gets unwound during stack unwinding.

The only time you'll ever need a stack trace is when you don't have a debugger handy - I.e. Production code. In this case it's almost useless because of inlining.

In practice only a few simple functions are inlined, and most functions are not (and often cannot be) inlined. Stack traces tend to be very useful and valuable in practice.

Hear, hear!
 

In the light of this, I don't see that it is feasible , desirable (or even possible) to impose a stack trace on implementors.

Is is both feasible, desirable and possible. In fact, many implementations provide stack traces today. Implementations should be allowed to omit frames for inlined functions, just like in other programming languages, e.g. Java.

Indeed. I think it would be perfectly ok for the stack trace produced to omit functions that had been inlined.
 

[Bjorn Reese]

On 04/16/2016 01:34 PM, Edward Catmur wrote:

> Because stack traces are lightweight and do not require manual intervention or any special machinery to generate

How are you going to generate a stack trace on MIPS without "any
special machinery"?

[Edward Catmur]

Using the unwind tables, which are required for exception support (but can be provided even if you are compiling without exceptions). It's the same on x86 with frame pointer omission.

In any case, this would be a conditionally supported feature; whether that argues against standardization is another matter.

[FrankHB1989]

在 2016年4月16日星期六 UTC+8下午6:30:33，Philipp Stephani写道：

Richard Hodges <hodg...@gmail.com> schrieb am Sa., 16. Apr. 2016 um 11:58 Uhr:

The c++ standard makes no mention of a stack. The memory model makes no assumption that there is one.

 

The C++ standard does mention the stack. It is the thing that gets unwound during stack unwinding.

 

If so, show the reference.
 

I believe not. The stack has LIFO semantics but the frames of activation record do not. I don't find any assumption of the implementation in the standard. So it only specifies the stack unwinding (which does need LIFO) but not the stack itself.

BTW I think it a potential defect to leave the specification of activation record out. It confuses users about the scope of the language, makes them wrongly to believe the activation record should be implemented as a stack and makes some wording harder to compose.
 

The only time you'll ever need a stack trace is when you don't have a debugger handy - I.e. Production code. In this case it's almost useless because of inlining.

In practice only a few simple functions are inlined, and most functions are not (and often cannot be) inlined. Stack traces tend to be very useful and valuable in practice.

 

In the light of this, I don't see that it is feasible , desirable (or even possible) to impose a stack trace on implementors.

Is is both feasible, desirable and possible. In fact, many implementations provide stack traces today. Implementations should be allowed to omit frames for inlined functions, just like in other programming languages, e.g. Java.

 

Please note it is not so easy to make it mandatory by a few assertions like here. Java is a very different monster in this aspect: firstly it has nothing to support freestanding implementations, second it does have at one canonical related specification about the intermediate representation (i.e. the JVM spec) to help make the task here being "portable" easily enough.
 

C++ is not like other languages. It expresses intent. The compiler transforms that intent into 'as if' code. 

All other programming languages are exactly alike in this respect. Otherwise optimizations wouldn't be possible at all.

 

No. C does like this, though it has only the equivalence of "as-if" rules, without some other more subtle rules aiming to gain such intent (e.g. copy elision). Many other languages are simpler: they just fully document the intended (including explicitly unspecified) behavior of the implementations, in the language specification, in the target intermediate representation specification. in both, or being more simple - leave them out when there presents a reference implementation. There also exist a few languages strictly specify the formal semantics of the language rules without mention the intent, but seems to be similar to the "as-if" rules on implementation when the behavior is not explicitly specified.
 

[FrankHB1989]

在 2016年4月16日星期六 UTC+8下午7:34:36，Edward Catmur写道：

Adding to this: stack traces are useful precisely when deciding whether to open a crash in a debugger - assuming you haven't suppressed core generation, in which case they can still be used to determine whether a crash is known and whether to enable core generation.

Because stack traces are lightweight and do not require manual intervention or any special machinery to generate, they can be used in the first line of bug triage and prioritization. It doesn't matter that they're less than perfect.

I am not against to the feature but I don't think it should be mandatory in every cases which violates the zero cost abstract principle.

To make it usable in a debugger should be a correct direction. (In fact I do need such standardized interface to reduce some daily work.)

To make this possible in the level of the specification, it can be conditionally-supported, or only mandated for hosted implementations when appropriate (really implementable in a lightweight enough way). Otherwise, it can be equipped with a runtime with some replacement mechanism like "operator new" and let users decide when to use. Both choices are not easy to be reasonably portable without the aid of the standard.



.

[Asiri Rathnayake]

On Mon, Apr 18, 2016 at 3:36 AM, FrankHB1989 <frank...@gmail.com> wrote:

在 2016年4月16日星期六 UTC+8下午6:30:33，Philipp Stephani写道：

Richard Hodges <hodg...@gmail.com> schrieb am Sa., 16. Apr. 2016 um 11:58 Uhr:

The c++ standard makes no mention of a stack. The memory model makes no assumption that there is one.

 

The C++ standard does mention the stack. It is the thing that gets unwound during stack unwinding.

 

If so, show the reference.
 

I believe not. The stack has LIFO semantics but the frames of activation record do not. I don't find any assumption of the implementation in the standard. So it only specifies the stack unwinding (which does need LIFO) but not the stack itself.

How else would you implement "frames of activation records" if not a stack? You need a stack, this is basic CS.

 

BTW I think it a potential defect to leave the specification of activation record out.

You are confusing platform ABI (e.g. [1]) with the language standard.

[1] https://mentorembedded.github.io/cxx-abi/abi.html

[Viacheslav Usov]

On Fri, Apr 15, 2016 at 9:23 PM, Matthew Woehlke <mwoehlk...@gmail.com> wrote:

> Okay, so really you want separation in order to collect a trace now, but delay resolution.

Yes. I think this is more powerful and more flexible than just spitting out (or not) a text blob.

Cheers,

V.

[Viacheslav Usov]

On Sat, Apr 16, 2016 at 1:56 AM, Nicol Bolas <jmck...@gmail.com> wrote:

On Friday, April 15, 2016 at 2:47:26 PM UTC-4, Viacheslav Usov wrote:

On Fri, Apr 15, 2016 at 7:57 PM, Matthew Woehlke <mwoehlk...@gmail.com> wrote:

On 2016-04-15 13:13, Viacheslav Usov wrote:

> I might be willing to believe this if obtaining stack traces at arbitrary points of execution was not *already supported*, at least by Windows and Linux.

That is more like "arcanely supported by some implementations". Stack unwinding, however, is a language feature.

Stack unwinding is a language feature. But the nature of the stack itself is not.

The stack is not a language feature, I am fully aware of that and that was a primary motivation for looking at that from the stack unwinding angle, because that is something that the standard has.

Another primary motivation was that, as mentioned by others, with frame pointer omission, one cannot back-trace the stack unless relying on the exception machinery.

So stack unwinding is closest to stack tracing both standard-wise and implementation-wise.

... why? That all rather depends on how stack unwinding is implemented. I'm not sure I see a need for an implementation to have "stack pointers".

I did not specify what "stack pointers" were. Nor should the standard, this can be implementation-defined. It is sufficient that std::exception (or something available in a catch block) has "something" that gives the caller implementation-defined stack pointers. And there should be another "something" that can, under implementation-defined conditions, convert them them to strings of an implementation-defined format. An implementation not supporting that can trivially return an empty collection of stack pointers/converted strings.

Cheers,

V.

[FrankHB1989]

在 2016年4月18日星期一 UTC+8下午2:42:48，Asiri Rathnayake写道：

On Mon, Apr 18, 2016 at 3:36 AM, FrankHB1989 <frank...@gmail.com> wrote:

在 2016年4月16日星期六 UTC+8下午6:30:33，Philipp Stephani写道：

Richard Hodges <hodg...@gmail.com> schrieb am Sa., 16. Apr. 2016 um 11:58 Uhr:

The c++ standard makes no mention of a stack. The memory model makes no assumption that there is one.

 

The C++ standard does mention the stack. It is the thing that gets unwound during stack unwinding.

 

If so, show the reference.
 

I believe not. The stack has LIFO semantics but the frames of activation record do not. I don't find any assumption of the implementation in the standard. So it only specifies the stack unwinding (which does need LIFO) but not the stack itself.

How else would you implement "frames of activation records" if not a stack? You need a stack, this is basic CS.

 

But I don't necessarily need it just only as a stack. It can be a vector or even a hash table. On the other hand, "the stack" might be too easily confused with the stack provided by the ISA-level ABI (which should not be in the language rules).

BTW I think it a potential defect to leave the specification of activation record out.

You are confusing platform ABI (e.g. [1]) with the language standard.

No, I don't. It is an issue of the lack of terminology in the language standard to distinguish something from the corresponding entity in ABI specifications.

[1] https://mentorembedded.github.io/cxx-abi/abi.html

[Matthew Woehlke]

On 2016-04-16 06:30, Philipp Stephani wrote:

> Richard Hodges schrieb am Sa., 16. Apr. 2016 um 11:58 Uhr:
>> In the light of this, I don't see that it is feasible , desirable (or even
>> possible) to impose a stack trace on implementors.
>
> Is is both feasible, desirable and possible. In fact, many implementations
> provide stack traces today. Implementations should be allowed to omit
> frames for inlined functions, just like in other programming languages,
> e.g. Java.

First off, I want to agree with this. Definitely the standard should not
make any assertions as to the "quality" of the trace, i.e. we expect
that inlined functions will either a) not show up, or b) "hide" the
function that called them.

That said, something just occurred to me... it seems like we could do
better here. If a function is inlined, as I understand, a call to that
function is replaced with a *copy* of that function. My experience with
stack traces is that when this happens, the resolved stack pointer gives
the name and source line of the inlined function being executed.
However, since that function is a *copy*, shouldn't we also know what
function and source line *called* the inline?

This would imply that there might exist a function that, given a "frame
pointer", can resolve a second pointer that called the first, in the
case that the first is an inlined call. (Maybe current debugging
information does not give us a way to resolve this information?) This
might potentially be an API we would want to include.

--
Matthew

[Matthew Woehlke]

On 2016-04-16 05:58, Richard Hodges wrote:
> In fact, in optimised code, there is often no stack use at all because of
> extensive inlining.

I'm going to have to concur with Philipp in disagreeing. I see stack
traces from "optimized" code often enough to know that this isn't true.
I would guess that the number of real calls that are "hidden" by
inlining is typically 15%-50%. In particular, note that inlining is not
likely¹ across a shared library boundary. Also note that inlining is a
speed / size tradeoff; it's unlikely cache concerns will ever go away to
the extent that we can expand every possible code path fully inline.
After all, the performance improvement is something like O(N) vs. a size
cost of something more like O(N^2) or O(2^N). (I don't know the actual
numbers, I just know that the size order is higher than the speed order.)

(¹ I'm not all that familiar with "prelinking" to guess if it could
dynamically inline across library boundaries in previously compiled
code. You'd at least need to keep a copy of the original code in such
case to redo or scrap the optimization when the library changes. Also,
you're still in the low-order-speed vs. higher-order-size tradeoff
territory.)

p.s. The KDE crash reported collects stack traces (using gdb) for
inclusion in bug reports. (This feature wouldn't help there, since the
application is already crashing; I'm just mentioning it as a
counter-argument to the claim that stack traces are useless.)

> In the light of this, I don't see that it is feasible , desirable (or even
> possible) to impose a stack trace on implementors.

(...and also...)

On 2016-04-17 11:47, Edward Catmur wrote:
> In any case, this would be a conditionally supported feature; whether that
> argues against standardization is another matter.

I recommended that *only the API* would be mandated. That is, an
implementation which always fails / returns no data / etc. is
conforming. Providing stubs is not much of an imposition. Always
providing the API, so that portable code doesn't need to employ
conditional compilation, is IMHO preferable. Also, it may be that
whether or not a result can be achieved is dependent on compile flags or
even run-time variables that make it difficult or maybe even impossible
to determine if the feature should be available at compile time.

Vendors that can reasonably provide a "real" implementation will likely
do so (and can be pressured by customers to do so as a matter of QoI).
Platforms that can't reasonably provide an implementation would *not* be
required to go to heroic efforts; they could just leave the
implementation as doing nothing.

And of course, as has been mentioned to death by now, those platforms
likely to provide a "real" implementation are those platforms that
*already have* an implementation. All that's really being recommended
here is to provide a standard, C++ interface.

--
Matthew

[Nicol Bolas]

On Monday, April 18, 2016 at 5:15:42 AM UTC-4, Viacheslav Usov wrote:

On Sat, Apr 16, 2016 at 1:56 AM, Nicol Bolas <jmck...@gmail.com> wrote:

On Friday, April 15, 2016 at 2:47:26 PM UTC-4, Viacheslav Usov wrote:

On Fri, Apr 15, 2016 at 7:57 PM, Matthew Woehlke <mwoehlk...@gmail.com> wrote:

On 2016-04-15 13:13, Viacheslav Usov wrote:

> I might be willing to believe this if obtaining stack traces at arbitrary points of execution was not *already supported*, at least by Windows and Linux.

That is more like "arcanely supported by some implementations". Stack unwinding, however, is a language feature.

Stack unwinding is a language feature. But the nature of the stack itself is not.

The stack is not a language feature, I am fully aware of that and that was a primary motivation for looking at that from the stack unwinding angle, because that is something that the standard has.

Another primary motivation was that, as mentioned by others, with frame pointer omission, one cannot back-trace the stack unless relying on the exception machinery.

Your assumption is that "exception machinery" would be capable of rebuilding stack frame data. You have yet to justify this assumption.

So stack unwinding is closest to stack tracing both standard-wise and implementation-wise.

... why? That all rather depends on how stack unwinding is implemented. I'm not sure I see a need for an implementation to have "stack pointers".

I did not specify what "stack pointers" were. Nor should the standard, this can be implementation-defined. It is sufficient that std::exception (or something available in a catch block) has "something" that gives the caller implementation-defined stack pointers. And there should be another "something" that can, under implementation-defined conditions, convert them them to strings of an implementation-defined format. An implementation not supporting that can trivially return an empty collection of stack pointers/converted strings.

That doesn't make sense.

You introduce the concept of "stack pointers", without actually saying what they are, mean, or do. You then require every implementation to have them, even though implementations may not necessarily need them for anything. Then you say that these "stack pointers" can be converted into implementation-defined strings. But we have no idea what these "stack pointers" actually represent.

In which case, you're left with a feature that defines nothing. You may as well say that you can call a function that returns an implementation-defined string. In fact, it'd be far better to do it that way.

After all, if exception machinery is truly capable of rebuilding the stack even when it has been optimized out, then surely it must be possible for the implementation to use this exception machinery to reconstruct the stack without actually throwing an exception. The machinery is there, whether you throw the exception or not, after all. It simply inaccessible. So why not just provide a function that can access it and assemble it into a coherent stack trace?

I see no reason why the feature of getting a stack trace needs to be associated with exception handling, even if it uses exception machinery to create that trace.

[Nicol Bolas]

On Monday, April 18, 2016 at 10:45:06 AM UTC-4, Matthew Woehlke wrote:

On 2016-04-17 11:47, Edward Catmur wrote:
> In any case, this would be a conditionally supported feature; whether that
> argues against standardization is another matter.

I recommended that *only the API* would be mandated. That is, an
implementation which always fails / returns no data / etc. is
conforming. Providing stubs is not much of an imposition. Always
providing the API, so that portable code doesn't need to employ
conditional compilation, is IMHO preferable. Also, it may be that
whether or not a result can be achieved is dependent on compile flags or
even run-time variables that make it difficult or maybe even impossible
to determine if the feature should be available at compile time.

Vendors that can reasonably provide a "real" implementation will likely
do so (and can be pressured by customers to do so as a matter of QoI).
Platforms that can't reasonably provide an implementation would *not* be
required to go to heroic efforts; they could just leave the
implementation as doing nothing.

And of course, as has been mentioned to death by now, those platforms
likely to provide a "real" implementation are those platforms that
*already have* an implementation. All that's really being recommended
here is to provide a standard, C++ interface.

Hmmm.

I disagree that a null return value should be allowed as a valid implementation. However, I also do not believe that implementations should be required to go to extensive effort to create a quality stack trace.

Getting some form of trace should be guaranteed. Maybe not an especially useful form, but some form. Even if all of the names and locations are just memory addresses, something still needs to be there. We obviously can't guarantee that any particular function call will add one more level to the trace, and so forth. But returning an empty string shouldn't be valid.

Implementations should at least make an effort to come up with something.

I also think that having a consistent format for the stack trace is important. There are certain things we ought to be able to do in a platform-independent way:

1: Separate individual levels in the stack trace.

2: Be able to distinguish "function name", "line number" and "filename" in each level of the trace. Note that none of these need to actually be what they say they are. That is, "function name" could be a memory address or a post-mangled name. It may or may not have parameter types listed. "Line number" could be a byte offset or nothing at all. "Filename" could be an empty string. The point is, you need to be able to parse all of these elements from the string in a platform-neutral way.

[Thiago Macieira]

Em segunda-feira, 18 de abril de 2016, às 10:21:18 PDT, Nicol Bolas escreveu:
> 1: Separate individual levels in the stack trace.
>
> 2: Be able to distinguish "function name", "line number" and "filename" in
> each level of the trace. Note that none of these need to actually be what
> they say they are. That is, "function name" could be a memory address or a
> post-mangled name. It may or may not have parameter types listed. "Line
> number" could be a byte offset or nothing at all. "Filename" could be an
> empty string. The point is, you need to be able to parse all of these
> elements from the string in a platform-neutral way.

Existing practice is <execinfo.h> header from glibc, which has two functions
of interest:

/* Store up to SIZE return address of the current program state in
ARRAY and return the exact number of values stored. */
extern int backtrace (void **__array, int __size) __nonnull ((1));


/* Return names of functions from the backtrace list in ARRAY in a newly
malloc()ed memory block. */
extern char **backtrace_symbols (void *const *__array, int __size)
__THROW __nonnull ((1));

If you couple that with the demangler from cxxabi.h:

char*
__cxa_demangle(const char* __mangled_name, char* __output_buffer,
size_t* __length, int* __status);

you can get pretty decent function names in a stack trace, at least for
exported functions. We're using that in Qt and you can ask each line of a
debugging output to include the stack trace by using %[backtrace] in your
QT_MESSAGE_PATTERN environment variable.

File names and line numbers aren't stored in the unwind stack. They're only
present in the debugging information, which isn't loaded into memory. Asking
for that is too much.

Finally, when I really want a stack trace, I want to see the values to the
parameters in each frame. We're not going to get that from the library. That's
a debugger's job.

--
Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org
Software Architect - Intel Open Source Technology Center

[Viacheslav Usov]

On Mon, Apr 18, 2016 at 7:10 PM, Nicol Bolas <jmck...@gmail.com> wrote:

> Your assumption is that "exception machinery" would be capable of rebuilding stack frame data. You have yet to justify this assumption.

It is not an assumption, it is a fact in certain implementations, with at least one particular implementation already mentioned in this thread.

> In which case, you're left with a feature that defines nothing.

Since when is "implementation-defined" equivalent to "nothing"?

> The machinery is there, whether you throw the exception or not, after all

Justify that. Note that I said in my very first message in this thread: "it will have some supporting code for that, which can only be optimized away if the the throw is optimized away".

Cheers,

V.

[Nicol Bolas]

On Tuesday, April 19, 2016 at 3:57:15 AM UTC-4, Viacheslav Usov wrote:

On Mon, Apr 18, 2016 at 7:10 PM, Nicol Bolas <jmck...@gmail.com> wrote:

> Your assumption is that "exception machinery" would be capable of rebuilding stack frame data. You have yet to justify this assumption.

It is not an assumption, it is a fact in certain implementations, with at least one particular implementation already mentioned in this thread.

vtable pointers are "a fact in certain implementations." But we don't standardize them. The standard does not enforce or even assume that virtual objects are implemented using vtable pointers. Oh, the standard has language that makes it possible to use vtable pointers. But the standard does nothing to favor or disfavor that implementation.

Nor should it do so for exception handling.

> In which case, you're left with a feature that defines nothing.

Since when is "implementation-defined" equivalent to "nothing"?

A function which returns a string who's nature, format, and contents are implementation defined is pretty meaningless.

Do you see a lot of code making use of `type_info::name`?

> The machinery is there, whether you throw the exception or not, after all

Justify that.

Because the compiler cannot tell, in most cases, if a function will certainly not participate in stack unwinding. C++ isn't Java; we aren't required to annotate each function call with the list of exceptions that can propagate through it. As such, most functions will have to have exception machinery built for them, because the compiler will be unable to tell if stack unwinding will happen to them. So that code needs to be there, just in case.

Note that I'm referring to the compile-time machinery, not the runtime machinery that might be built when execution enters a `try` block.

[Viacheslav Usov]

On Tue, Apr 19, 2016 at 5:59 PM, Nicol Bolas <jmck...@gmail.com> wrote:

> vtable pointers are "a fact in certain implementations." But we don't standardize them.

Two messages earlier I said: "stack unwinding is closest to stack tracing both standard-wise and implementation-wise." You singled out the latter part and generalized that into absurdity. Yes, and your point is?

> A function which returns a string who's nature, format, and contents are implementation defined is pretty meaningless.

Any language linkage that is not "C" or "C++" is like that. Pretty meaningless you say? Why would the standard have such a thing?

> Because the compiler cannot tell, in most cases, if a function will certainly not participate in stack unwinding.

Oh, you are referring to "a fact in certain implementation"? But, you know, "we don't standardize them".

And here is another fact for you: https://msdn.microsoft.com/en-us/library/1deeycx5.aspx

(begin quote)

If you use /EHa, the image may be larger and might perform less well because the compiler does not optimize a try block as aggressively. It also leaves in exception filters that automatically call the destructors of all local objects even if the compiler does not see any code that can throw a C++ exception. This enables safe stack unwinding for asynchronous exceptions as well as for C++ exceptions. When you use /EHs, the compiler assumes that exceptions can only occur at a throw statement or at a function call. This allows the compiler to eliminate code for tracking the lifetime of many unwindable objects, and this can significantly reduce code size.

(end quote)

Do you see that your imaginary fact cannot be reconciled with a real fact? You should be more careful with your generalizations, Nicol.

Cheers,

V.

[Nicol Bolas]

On Tuesday, April 19, 2016 at 1:00:02 PM UTC-4, Viacheslav Usov wrote:

On Tue, Apr 19, 2016 at 5:59 PM, Nicol Bolas <jmck...@gmail.com> wrote:

> vtable pointers are "a fact in certain implementations." But we don't standardize them.

Two messages earlier I said: "stack unwinding is closest to stack tracing both standard-wise and implementation-wise." You singled out the latter part and generalized that into absurdity. Yes, and your point is?

My point is the same as it was before: we should not add language to the standard that makes assumptions about how stack unwinding is implemented. Your "stack pointer" nonsense in particular. That's why I brought up vtable pointers. They're both implementation details that the standard does not, and should not require implementations to use.
 

> A function which returns a string who's nature, format, and contents are implementation defined is pretty meaningless.

Any language linkage that is not "C" or "C++" is like that. Pretty meaningless you say? Why would the standard have such a thing?

So that you can actually use it and rely on it. So that you can do something more than stick it in a file or throw it on the screen.

> Because the compiler cannot tell, in most cases, if a function will certainly not participate in stack unwinding.

Oh, you are referring to "a fact in certain implementation"?

No, I am referring to reality.

Here's a function:

using FuncPtr = void(*)();

void SomeFunc(FuncPtr ptr)
{
  ptr();
}

Can the compiler tell, in all possible cases, that `SomeFunc` will participate in stack unwinding? No. With aggressive, global optimizations, it may be able to tell in specific cases. But not generally.

So long as the flow of execution through a function is not static, the compiler cannot know for certain if a function will need stack unwinding machinery.

Oh and here's the thing. If we add a standard library function to generate a stack trace, and the implementer wants to make it use stack unwinding machinery to generate that trace (remember: that's not a requirement of stack traces in general, only in your personal idea)... that implementation would need nothing more than to choose to treat calls to that standard library function as equivalent to a `throw` that never gets caught.
 

And here is another fact for you: https://msdn.microsoft.com/en-us/library/1deeycx5.aspx

(begin quote)

If you use /EHa, the image may be larger and might perform less well because the compiler does not optimize a try block as aggressively. It also leaves in exception filters that automatically call the destructors of all local objects even if the compiler does not see any code that can throw a C++ exception. This enables safe stack unwinding for asynchronous exceptions as well as for C++ exceptions. When you use /EHs, the compiler assumes that exceptions can only occur at a throw statement or at a function call. This allows the compiler to eliminate code for tracking the lifetime of many unwindable objects, and this can significantly reduce code size.

(end quote)

Do you see that your imaginary fact cannot be reconciled with a real fact? You should be more careful with your generalizations, Nicol.

What "real fact" are we talking about? According to that document, exceptions in the most optimal case are assumed to happen when `throw` is encountered or when you call a function (presumably ones without `noexcept` or similar constructs).

I very clearly said "compiler cannot tell, in most cases". Do you write a lot of functions that only call explicitly qualified `noexcept` functions, or don't call functions at all? My point still stands: the majority of code will still have to have exception machinery, because the majority of code does not limit itself to `noexcept` functions.

It should also be noted that the primary purpose of /EHa is not about C++ exceptions (which by the standard, cannot possibly happen outside of an explicit `throw` statement or a non-`noexcept` function call). /EHa is, as the text you posted shows, primarily about dealing with Microsoft exceptions which can be generated asynchronously at any time.

So the behavior that /EHs provides is really just standard C++.

[Viacheslav Usov]

Top posting because I cannot see how I could write this response inline.

In another message I said that "stack pointers" could be as either part of std::exception or something that can be used in a catch block. Thinking more on the latter, I like that "something that can be used in a catch block" better. But then, taking into account that as far as I can tell this entire mechanism needs to be implementation-defined anyway, what would happen if that something is used outside a catch block? Probably something implementation-defined again. Therefore, we can just say that "something" can be called anywhere to obtain implementation-defined "stack pointers".

So I think we could propose either a class or a function, say std::stacktrace, that can be instantiated/called anywhere. To avoid discussing the nature of "stack pointers", we can specify that the class/function then can make available to its users a pointer into an array of chars, whose length is also provided by the class/function. The length and content of that array is implementation-defined.

We also need another mechanism to translate the byte buffer returned by std:;stacktrace into human readable text.

Cheers,

V.

On Fri, Apr 15, 2016 at 7:57 PM, Matthew Woehlke <mwoehlk...@gmail.com> wrote:

On 2016-04-15 13:13, Viacheslav Usov wrote:

> On Fri, Apr 15, 2016 at 4:19 PM, Matthew Woehlke wrote:
>> If yes, I would drop everything about exceptions and just ask for a
>> portable way of obtaining a stack trace.
>
> I would say that supporting a stack trace without an exception is more of
> an effort

I might be willing to believe this if obtaining stack traces at
arbitrary points of execution was not *already supported*, at least by
Windows and Linux.

> and probably a pessimisation than supporting a stack trace when
> an exception is thrown. When an exception is thrown, the implementation has
> to do stack unwinding. So it will have some supporting code for that, which
> can only be optimized away if the the throw is optimized away, and it will
> normally have some statically allocated data that could be translated into
> a stack trace, not necessarily at runtime. None of that needs to be present
> elsewhere.

That sounds like you want the operation to involve compiler magic, which
would make it a *language* feature, rather than a library feature.

Maybe an implementation could do that anyway as a QoI thing, but
*requiring* it to work that way (i.e. making it a language feature)
seems... ambitious. I think a library approach would have a better
chance at being accepted by the committee.

Plus, see Tony's point. I can think of many situations where I might
want a stack trace but I'm *not* throwing an exception.

I also have to wonder if your exception-based stack collection doesn't
quit working as soon as it hits a `catch`... which would be
unacceptable, of course...

> I would say having a mechanism coupled with std::exception that collects
> those "pointers", which are not yet human readable, is a good thing,
> because certain people and certain companies may have policies that would
> prohibit exposing too much symbolic information about their code to third
> parties

That's both orthogonal (see below) and also moot. If I have pointers,
either a) I can resolve those to symbols, regardless if the code did it
for me automatically, or b) I can't, period. Code that has been
symbol-stripped is just not going to be able to produce names from
pointers. Any proposal will need to take this into consideration.

> Another mechanism could translate those pointers into human readable
> stack traces, if appropriate symbolic info is available.

I do agree with this, however. In fact, I would make any such proposal
necessarily operate in 2-3 phases:

1. (Optional) Obtain count of available stack pointers¹.
2. Obtain up to N stack pointers.
3. Translate arbitrary stack pointers to symbol names.

Having (2) and (3) [available²] as separate operations is orthogonal to
whether or not exception unwinding is used to help with (2).

(2) should accept an input buffer which is filled with pointers.
(Possibly an overload that accepts a std::vector should also be
provided, but one taking a previously allocated memory block is mandatory.)

(3) should be usable on any pointer obtained in any fashion, not just by
(2). (No guarantees that it will *work*, of course...)

(1) isn't required, but it helps with (2), and I can imagine uses for
(1) that don't use (2) or (3), e.g. an algorithm that detects if it has
gone into an infinite recursion state and terminates itself gracefully
*before* crashing due to stack exhaustion.

(¹ In case of platforms where `void*` is not sufficient, let's assume
that when I say "[stack] pointer" I'm really talking about something
that the standard would specify as an "opaque" type.)

(² We might want to *additionally* provide a convenience function that
combines (2) and (3).)

--
Matthew

[Viacheslav Usov]

On Tue, Apr 19, 2016 at 7:59 PM, Nicol Bolas <jmck...@gmail.com> wrote:

> My point is the same as it was before: we should not add language to the standard that makes assumptions about how stack unwinding is implemented. Your "stack pointer" nonsense in particular.

The only nonsense here is yours. Nowhere did I suggest adding any such language to the standard.

> So that you can actually use it and rely on it. So that you can do something more than stick it in a file or throw it on the screen.

Oh, so something wholly implementation-defined is not nothing and may even be useful? Why are you contradicting yourself?

> Can the compiler tell, in all possible cases, that `SomeFunc` will participate in stack unwinding? No. With aggressive, global optimizations, it may be able to tell in specific cases. But not generally.

The compiler does not need to deal with all possible cases at any given time. Compiling a program is not equivalent to proving a theorem in CS. Not being able to prove something when compiling a particular piece of code does not nullify that ability in a million other places. Optimizing away function calls and all that comes with them is the bread and butter of contemporary C++ optimizers. Your generalization is irrelevant.

> Oh and here's the thing. If we add a standard library function to generate a stack trace, and the implementer wants to make it use stack unwinding machinery to generate that trace (remember: that's not a requirement of stack traces in general, only in your personal idea)... that implementation would need nothing more than to choose to treat calls to that standard library function as equivalent to a `throw` that never gets caught.

"treat calls to that standard library function as equivalent to a `throw` that never gets caught" means really just that: call to std::terminate.

> According to that document, exceptions in the most optimal case are assumed to happen when `throw` is encountered or when you call a function (presumably ones without `noexcept` or similar constructs).

The parenthetical conditional of yours is not what the document said. It did not specify the class of functions that the compiler assume will throw. So, you are wrong.

> I very clearly said "compiler cannot tell, in most cases". Do you write a lot of functions that only call explicitly qualified `noexcept` functions, or don't call functions at all? My point still stands: the majority of code will still have to have exception machinery, because the majority of code does not limit itself to `noexcept` functions.

That is making implication from wrong premises.

> So the behavior that /EHs provides is really just standard C++.

Yes. And the text I posted implies very trivially that in that mode ("standard C++") the compiler does analyse exception-generating code and eliminates exception-handling machinery very aggressively. Contrary to your claims.

Even though I find your participation in this discussion nothing but militantly illogical nitpicking, it has had a positive effect just by keeping me thinking about it longer. So I thank you for that. As I described in another message, I no longer think it necessary to tie stack tracing with exceptions.

Cheers,

V.

[Nicol Bolas]

On Wednesday, April 20, 2016 at 4:58:35 AM UTC-4, Viacheslav Usov wrote:

Top posting because I cannot see how I could write this response inline.

In another message I said that "stack pointers" could be as either part of std::exception or something that can be used in a catch block. Thinking more on the latter, I like that "something that can be used in a catch block" better. But then, taking into account that as far as I can tell this entire mechanism needs to be implementation-defined anyway, what would happen if that something is used outside a catch block? Probably something implementation-defined again. Therefore, we can just say that "something" can be called anywhere to obtain implementation-defined "stack pointers".

So I think we could propose either a class or a function, say std::stacktrace, that can be instantiated/called anywhere. To avoid discussing the nature of "stack pointers", we can specify that the class/function then can make available to its users a pointer into an array of chars, whose length is also provided by the class/function. The length and content of that array is implementation-defined.

We also need another mechanism to translate the byte buffer returned by std:;stacktrace into human readable text.

What's the point of making these separate actions? Why bother with the intermediate step of implementation-defined byte arrays? What are you going to do, stick them in a binary log-file? It's not like they have any inherent meaning, nor are they required to be useful between executions of the program.

Just give us the human readable stack trace.

[Edward Catmur]

But they are useful between executions if you don't apply aslr, or if you do you can de-relocate them, or ship the relocation table along with the binary stack trace. The advantage of a binary stack trace is that it is compact, takes a predictable amount of space and can reliably be hashed for triage and analysis.

Admittedly, it could be more complicated if you're dynamically loading modules.

> Just give us the human readable stack trace.

Human readable tends to imply less useful for machines.

[Viacheslav Usov]

On Wed, Apr 20, 2016 at 3:19 PM, Nicol Bolas <jmck...@gmail.com> wrote:

> What's the point of making these separate actions?

Separating those actions makes it unnecessary to embed symbolic information information into the compiled and linked executable, as has been mentioned more than once in his thread.

> nor are they required to be useful between executions of the program.

Indeed they are not, being implementation-defined.

Cheers,

V.

[Nicol Bolas]

So, you want to have this two-step process for cases where you're not using DLLs/SOs and you have either turned off or otherwise subverted ASLR. Oh, and the format of such binary data is well-documented. Is this a particularly common case?

Also, how exactly can you predict how much space a stack trace takes? It is, after all, highly dependent on a runtime concept: where you were when it was generated.

In any case, I believe that this can be resolved without a two-stage process: use formats.

That is, when you get a stack trace, you specify a format like `printf` that defines how the contents of each level of the trace are formatted. Formatting characters would include:

%F: Function name of that level of the trace, as an implementation-defined string. If not available, then nothing.
%L: Line number for that level of the trace, as a string. If not available, then nothing.
%E: Filename for that level of the trace, as a string. If not available, then nothing.
%B: Binary representation of that level of the trace, as an implementation-defined sequence of hexidecimal characters.
%b: Binary representation of that level of the trace, as an implementation-defined sequence of bytes.
%n: Number of bytes in the binary representation of that level of the trace, as an `unsigned int` integer type, as though the implementation performed `memcpy(output, &size_var, sizeof(unsigned int));`

So if you wanted a textual trace, you could do:

trace("%F(0x%B)[%L]\n");

If you want a binary trace, you could do:

trace("%n%b");

Note that the trace is generated as a single sequence of bytes, based entirely on the format. So the implementations will not add characters between the levels of the trace. In the first case, "\n" is used to section the trace into levels, but you could use whatever you like. In the second case, you have to parse it by reading the implementation-defined byte array back.

Admittedly, reading back the binary trace is a bit inelegant. But `unsigned int` followed by that number of bytes is sufficient for processing the whole thing into whatever format you desire.

So there's no clumsy two-step process, and everyone can still get the data they desire.

[Nicol Bolas]

On Wednesday, April 20, 2016 at 5:50:34 AM UTC-4, Viacheslav Usov wrote:

On Tue, Apr 19, 2016 at 7:59 PM, Nicol Bolas <jmck...@gmail.com> wrote:

> My point is the same as it was before: we should not add language to the standard that makes assumptions about how stack unwinding is implemented. Your "stack pointer" nonsense in particular.

The only nonsense here is yours. Nowhere did I suggest adding any such language to the standard.

And I quote:

> Well, std::exception would be able to provide a list of stack pointers, That is technically a library feature, although compiler magic is required.

That would be an addition to the standard.
 

> So that you can actually use it and rely on it. So that you can do something more than stick it in a file or throw it on the screen.

Oh, so something wholly implementation-defined is not nothing and may even be useful? Why are you contradicting yourself?

> Can the compiler tell, in all possible cases, that `SomeFunc` will participate in stack unwinding? No. With aggressive, global optimizations, it may be able to tell in specific cases. But not generally.

The compiler does not need to deal with all possible cases at any given time. Compiling a program is not equivalent to proving a theorem in CS. Not being able to prove something when compiling a particular piece of code does not nullify that ability in a million other places. Optimizing away function calls and all that comes with them is the bread and butter of contemporary C++ optimizers. Your generalization is irrelevant.

> Oh and here's the thing. If we add a standard library function to generate a stack trace, and the implementer wants to make it use stack unwinding machinery to generate that trace (remember: that's not a requirement of stack traces in general, only in your personal idea)... that implementation would need nothing more than to choose to treat calls to that standard library function as equivalent to a `throw` that never gets caught.

"treat calls to that standard library function as equivalent to a `throw` that never gets caught" means really just that: call to std::terminate.

Since you wish to be pedantic about it, what I meant was to force the generation of stack unwinding machinery for every function between that and `main` in the call graph. If the compiler is able to search through the call graph in order to remove unwinding machinery from functions that call functions that don't throw, then the compiler would be able to add unwinding machinery to functions that call functions that call this library function (or throw exceptions).
 

> According to that document, exceptions in the most optimal case are assumed to happen when `throw` is encountered or when you call a function (presumably ones without `noexcept` or similar constructs).

The parenthetical conditional of yours is not what the document said. It did not specify the class of functions that the compiler assume will throw. So, you are wrong.

... what? The parenthetical is essentially irrelevant to the overall point.

Also, any C++11 compiler worth its salt wouldn't bother looking through a function marked `noexcept` to see if it throws exceptions. I would only be wrong if VS were a stupid compiler.

> I very clearly said "compiler cannot tell, in most cases". Do you write a lot of functions that only call explicitly qualified `noexcept` functions, or don't call functions at all? My point still stands: the majority of code will still have to have exception machinery, because the majority of code does not limit itself to `noexcept` functions.

That is making implication from wrong premises.

> So the behavior that /EHs provides is really just standard C++.

Yes. And the text I posted implies very trivially that in that mode ("standard C++") the compiler does analyse exception-generating code and eliminates exception-handling machinery very aggressively. Contrary to your claims.

That's not what the text says or implies. Here is the key part of the text:

> the compiler assumes that exceptions can only occur at a throw statement or at a function call. This allows the compiler to eliminate code for tracking the lifetime of many unwindable objects

Now, explain to me this. If you're implementing stack unwinding logic, why would you need to have "code for tracking the lifetime of many unwindable objects"? Those objects' lifetimes are static properties, not runtime ones. Exceptions can only be thrown at a function call or at an explicit `throw`, after all. So for every point of an exception, the lifetimes of the objects is a static property. So why would you need code to track those lifetimes?

You would only need to track lifetimes if exceptions could be thrown at other times. Which is exactly what /EHa is all about permitting. That's a language extension; /EHs is simply forcing the standard-conforming behavior, thus eliminating all of the stuff that /EHa would have added.

/EHs is more optimized, but only relative to /EHa. It does not promise that it will remove unwinding logic from most things; it only promises that it will remove the complex logic that /EHa would have added to most things.

General unwinding logic is different from what this paragraph is talking about.

[Matthew Woehlke]

On 2016-04-20 09:19, Nicol Bolas wrote:
> On Wednesday, April 20, 2016 at 4:58:35 AM UTC-4, Viacheslav Usov wrote:
>> In another message I said that "stack pointers" could be as either part of
>> std::exception or something that can be used in a catch block. Thinking
>> more on the latter, I like that "something that can be used in a catch
>> block" better. But then, taking into account that as far as I can tell this
>> entire mechanism needs to be implementation-defined anyway, what would
>> happen if that something is used outside a catch block? Probably something
>> implementation-defined again. Therefore, we can just say that "something"
>> can be called anywhere to obtain implementation-defined "stack pointers".
>>
>> So I think we could propose either a class or a function, say
>> std::stacktrace, that can be instantiated/called anywhere. To avoid
>> discussing the nature of "stack pointers", we can specify that the
>> class/function then can make available to its users a pointer into an array
>> of chars, whose length is also provided by the class/function. The length
>> and content of that array is implementation-defined.
>>
>> We also need another mechanism to translate the byte buffer returned by
>> std:;stacktrace into human readable text.
>
> What's the point of making these separate actions?

- Symbol resolution is expensive (both in time, and in necessary use of
dynamically allocated memory, as opposed to "stack pointers" which are
almost certainly of fixed size); we may be collecting a trace that may
never be used, so paying this cost up front is stupid. Even if I am
definitely going to print the trace, I may not want to do resolution
immediately.

- Symbol resolution may not be possible at run time due to missing debug
information, but may be possible "offline", so it can be useful to dump
just the "stack pointers" without resolving them.

- We might be collecting a trace for some other reason entirely for
which the resolved symbols aren't even interesting. (See for example my
other message suggesting reasons why having a separate call to *only get
the stack depth* might be interesting.)

- It's lower level. I'm not aware of any case where the implementation
would not be a two-step process anyway, so not providing direct access
to this API is unnecessarily limiting.

- I might obtain a "stack pointer" by some other means and want to
resolve it.

> Why bother with the intermediate step of implementation-defined byte
> arrays?

...because a "stack pointer" may not be isomorphic with `void*` on every
platform. (That said, I don't like the byte array, either; I'd propose
an opaque type.)

> Just give us the human readable stack trace.

Existing implementations don't work this way, for a reason.

That said, I'd encourage providing a convenience function to combine the
operations; it just shouldn't be the *only available* API.

On 2016-04-20 10:16, Nicol Bolas wrote:
> Also, how exactly can you predict how much space a stack trace takes?
> It is, after all, highly dependent on a runtime concept: where you
> were when it was generated.

Typically, when requesting a trace, you pass a limit on the number of
entries to be returned. Accordingly, it is trivial to compute an upper
bound on the required memory. (Especially if I am doing this for e.g. a
panic handler, I might want to reserve this memory in advance.)

--
Matthew

[Viacheslav Usov]

On Wed, Apr 20, 2016 at 4:16 PM, Nicol Bolas <jmck...@gmail.com> wrote:

> So, you want to have this two-step process for cases where you're not using DLLs/SOs and you have either turned off or otherwise subverted ASLR.

Why does that exclude DLLs/SOs and ASLR? It is very trivial to produce a list of modules, including that of the main program, complete with the base addresses of whatever sections/segments they are transformed into memory. You can do so without having any symbolic information. It is very trivial to include that info into a binary stack trace. That is essentially what the minidump feature in Windows does, working just fine just about always.

Cheers,

V.

[Viacheslav Usov]

On Wed, Apr 20, 2016 at 4:20 PM, Nicol Bolas <jmck...@gmail.com> wrote:

> And I quote:

I do not believe this branch of the discussion is of interest for third parties. If you would like to continue it privately, let me know.

Cheers,

V.

[Matthew Woehlke]

Some of the discussion in this thread seems to be caused various
perceptions of what a portable API to obtain stack traces might look
like. To that end, I would like to submit the following for consideration.

This isn't a proposal yet, just some notes I wrote down on what the
actual API might look like. Some noteworthy features incorporated include:

- Ability to resolve the (static) call site of an inlined call.
- Ability to turn dynamically offset pointers into absolute pointers.

This is not yet complete. For example, there is no resolution function
at all. Some mechanism to format a `frame` (i.e. without resolution) is
probably needed. Methods to apply and remove offsets may be needed?
Also, there will likely be at least one "push button" convenience
function to combine operations. (As noted, however, I'm opposed to
making this the *only* available API.)

p.s. If anyone would like to co-author a proposal, please let me know.

--
Matthew

[Viacheslav Usov]

On Wed, Apr 20, 2016 at 5:01 PM, Matthew Woehlke <mwoehlk...@gmail.com> wrote:

> ...because a "stack pointer" may not be isomorphic with `void*` on every platform. (That said, I don't like the byte array, either; I'd propose an opaque type.)

The reason I switched from "stack pointers" to a byte buffer is that the former seem to imply that the implementation may be able to resolve a binary chunk that the stack is into a collection of entities right there and then. That need not be the case I believe.

Then again, insisting on a single byte buffer may also be both restrictive and wasteful, requiring memory allocation to pack possibly disjoint data. Therefore, I think the interface should be something like a begin/end iterator pair, with the iterator pointing to a byte array.

Cheers,

V.

[Andrew Marlow]

What not model the interface on what Java has done? There is the means to get the trace as a string as well as a function that logs the trace directly where it gets the string internally. In both cases we do not need to get pointers to stack frame objects or anything like that. Remember, the whole point of asking for this is as aid in debugging and error reporting.

--

---
You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to std-discussio...@isocpp.org.
To post to this group, send email to std-dis...@isocpp.org.
Visit this group at https://groups.google.com/a/isocpp.org/group/std-discussion/.

--
Regards,

Andrew Marlow
http://www.andrewpetermarlow.co.uk

[Viacheslav Usov]

On Wed, Apr 20, 2016 at 6:46 PM, Andrew Marlow <marlow...@gmail.com> wrote:

> What not model the interface on what Java has done?

Because in this particular case the difference between C++ and Java is significant.

> There is the means to get the trace as a string as well as a function that logs the trace directly where it gets the string internally.

Java can always obtain symbolic names through its reflection. C++ has nothing like that, and may or may not, or may only partially be able to do that right there and then. Being able to obtain some binary information is just about the only thing that is truly portable in this case. Translation in situ is not.

Cheers,

V.

[Miro Knejp]

Here's an idea: why not stop arguing about whether a certain word is
compatible with standard terminology and instead focus your energy on
the actual feature? This whole bikeshedding nonsense is completely
counterproductive. Design the feature first, analyse if it's compatible
with implementations, and *then* worry about how to integrate it with
standard terminology. Until then it's a waste of time. Just use terms
everybody knows and understands.

So here's my contribution:

1. We certainly don't want the runtime to insert any additional
instructions into functions to track the current activation frame, so
whatever machinery is used to determine the stack trace is going to have
to do some magic based on the current instruction pointer. This usually
involves some kind of pointer chasing in tables or the stack itself.
2. It may or may not be able to detect inlined functions. For me not
having them is a sacrifice I'm willing to make if I can get traces for
everything else. I consider this a QoI issue.
3. A developer might chose not to include human-readable strings for
symbols in order to reduce binary size or enforce code obfuscation.
4. I'd like a simple abstraction that doesn't require me to deal with
pointers or arrays or stuff like that.

From these points I suggest a range-based approach:

auto stacktrace = std::capture_stack()
cout << stacktrace.depth();
cout << stacktrace.has_source_names(); // Are human-readable file/source
names present?
cout << stacktrace.has_function_names(); // Are human-readable function
names present?
cout << stacktrace.has_line_numbers(); // Are line numbers for source
files available?

for(const auto& frame : stracktrace) {
cout << frame.source_id(); // An id that can be used with a
compiler-generated file/tool to get the source file name
cout << frame.line_number(); // Line number into source file if
has_line_numbers == true otherwise 0

cout << frame.source_name(); // "/usr/project/file.cpp" if
has_source_names == true otherwise ""

cout << frame.base(); // A void(*)() containing the function's address
cout << frame.offset(); // Offset in bytes/chars/addressable units
from the beginning of the function
cout << frame.function_id(); // An id that can be used with a
compiler-generated file/tool to get the function name
cout << frame.function_name(); // "project::foo(int, int)" if
has_function_names == true otherwise ""
}

Implementations that do not support certain features (or were disabled
by the developer) are acounted for and you don't need to change the code
to handle it.

Making this a range to iterate over gives the implementation the freedom
to traverse the stack frame list only on-demand if it makes sense for
that particular implementation to do so. It also allows the
implementation to not allocate any temporary memory for storing the
stack frames for this particular trace. If the code is compiled with
compiler flags that disable features like function or source names the
corresponding functions can be made nop in the library.

It also gives the developer the most freedom to do with the stack frames
whatevery they need.

[Matthew Woehlke]

On 2016-04-20 15:24, Miro Knejp wrote:
> 1. We certainly don't want the runtime to insert any additional
> instructions into functions to track the current activation frame,

Definitely agreed.

> 2. It may or may not be able to detect inlined functions. For me not
> having them is a sacrifice I'm willing to make if I can get traces for
> everything else. I consider this a QoI issue.

Agreed. From my experience, it *might* be possible to detect if an entry
is an inline call, and if so, determine the parent call point (or at
least something close). This may be expensive, and may require loading
debugging symbols.

> 4. I'd like a simple abstraction that doesn't require me to deal with
> pointers or arrays or stuff like that.

So... you want only iterator based access? Why? Maybe I don't understand
why dealing with arrays is bad. Please see also the API I proposed
elsewhere in the thread which (optionally) uses a std::vector.

I think an API that takes a previously allocated block of data is an
important feature. As a debugging API, it may be that the program needs
to generate a trace in a situation where memory allocation could be
problematic.

> Making this a range to iterate over gives the implementation the freedom
> to traverse the stack frame list only on-demand if it makes sense for
> that particular implementation to do so.

This suggests that the trace cannot be copied and/or is invalidated if
not used immediately. IMHO there are enough problems with such a design
that it's not even worth discussing.

Note that you probably can't implement `depth()` without walking the
stack, so even in your API as presented there is very little actual
benefit to such an approach.

> It also allows the implementation to not allocate any temporary
> memory for storing the stack frames for this particular trace.

Or... you could pre-allocate the memory, as in my API.

I don't see how your implementation would even work *without* allocating
memory; at least the trace object itself would need to hold a pointer to
where in the stack is currently being traversed (two, if it isn't
read-once, which I would consider another non-starter). This is on top
of whatever scratch space is needed for operation, which won't be
reusable if stack walking is interspersed with other operations e.g.
symbol name resolution.

> If the code is compiled with compiler flags that disable features
> like function or source names the corresponding functions can be made
> nop in the library.

This won't work. I can have any combination of libraries loaded
dynamically at runtime that do and don't have symbols, and can request
traces from either. There is no way (without forbidding runtime dynamic
library loading, at least) to know for sure if symbols will be present
or not at compile time. I don't see such a limitation being worthwhile.
It's also somewhat moot if you separate tracing and symbol resolution.

> It also gives the developer the most freedom to do with the stack frames
> whatevery they need.

Like... save them for later use? Sorry, the only benefit I see to this
design is having resolution return a struct (or even a class). I'd
already suggested doing so in my API.

--
Matthew

[Andrew Marlow]

On 20 April 2016 at 20:24, Miro Knejp <miro....@gmail.com> wrote:

1. We certainly don't want the runtime to insert any additional instructions into functions to track the current activation frame, so whatever machinery is used to determine the stack trace is going to have to do some magic based on the current instruction pointer. This usually involves some kind of pointer chasing in tables or the stack itself.
2. It may or may not be able to detect inlined functions. For me not having them is a sacrifice I'm willing to make if I can get traces for everything else. I consider this a QoI issue.

Indeed. Debuggers can already provide stack traces,which includes demangling symbols. The code necessary to do this depends on the compiler but so what? It means the implementation must be compiler aware. But this is true for loads of other aspects of library implementation.

 

3. A developer might chose not to include human-readable strings for symbols in order to reduce binary size or enforce code obfuscation.

I think there is no need to worry about this. I come back to what debuggers are capable of doing. When the compilation is in debug mode the debugger stack trace often contains source level information such as file and line number. But without this info it might just give addresses. Again, so what? These limitations are perfectly acceptable and understood in the context of the debugger behaviour. I would like a stack trace facility in C++ that simply provides the same.

[Miro Knejp]

Am 20.04.2016 um 22:06 schrieb Matthew Woehlke:
> On 2016-04-20 15:24, Miro Knejp wrote:
>> 1. We certainly don't want the runtime to insert any additional
>> instructions into functions to track the current activation frame,
> Definitely agreed.
>
>> 2. It may or may not be able to detect inlined functions. For me not
>> having them is a sacrifice I'm willing to make if I can get traces for
>> everything else. I consider this a QoI issue.
> Agreed. From my experience, it *might* be possible to detect if an entry
> is an inline call, and if so, determine the parent call point (or at
> least something close). This may be expensive, and may require loading
> debugging symbols.
>
>> 4. I'd like a simple abstraction that doesn't require me to deal with
>> pointers or arrays or stuff like that.
> So... you want only iterator based access? Why? Maybe I don't understand
> why dealing with arrays is bad. Please see also the API I proposed
> elsewhere in the thread which (optionally) uses a std::vector.

If the API codifies a vector or array then the implementation and user
have no other choice than to use a vector or array, even if that
pessimizes what the implementation or user already have available. With
a range you can still copy everything to a preallocated array, or a
vector, or any other sequence container you may have. The range
interface doesn't limit the implementation's capabilities: if it has to
create a block of memory for internal use, it can do so and store it in
the range. If it can traverse the data structures freely, let it do so.
My question is: why arbitrarily restrict your options?

copy(stacktrace(), back_inserter(my_vector));
copy(stacktrace() | transformed([] (const auto& frame) { return
frame.function_name(); }), ostream_iterator<string>(cout, "\n"));

Do you see where this is going? If you want to deal with an array, you
can chose to do so. If you want something else, you're free to do that, too.

>
> I think an API that takes a previously allocated block of data is an
> important feature. As a debugging API, it may be that the program needs
> to generate a trace in a situation where memory allocation could be
> problematic.

Then with a range based API it has the freedom to have a preallocated
block available for the trace. But it isn't *forced* to. If you're on a
system with limited memory you preallocate an array to hold just what
you need and then copy as much as you can. If memory is not your concern
you have the option to store the information however and where ever you
wish. You can even chose a different solution depending on whether
you're dealing with a std::bad_alloc or something else. I don't see how
limiting your number of options is a good thing.

>
>> Making this a range to iterate over gives the implementation the freedom
>> to traverse the stack frame list only on-demand if it makes sense for
>> that particular implementation to do so.
> This suggests that the trace cannot be copied and/or is invalidated if
> not used immediately. IMHO there are enough problems with such a design
> that it's not even worth discussing.
>
> Note that you probably can't implement `depth()` without walking the
> stack, so even in your API as presented there is very little actual
> benefit to such an approach.

Then let it return 0 if it doesn't have the information available just
as in your API. After iterating the range you know how many elements
there are.

>
>> It also allows the implementation to not allocate any temporary
>> memory for storing the stack frames for this particular trace.
> Or... you could pre-allocate the memory, as in my API.

Yes, my interface gives you *and* the runtime the freedom to do either,
depending on what is most efficient for you *and* the runtime.

>
> I don't see how your implementation would even work *without* allocating
> memory; at least the trace object itself would need to hold a pointer to
> where in the stack is currently being traversed (two, if it isn't
> read-once, which I would consider another non-starter). This is on top
> of whatever scratch space is needed for operation, which won't be
> reusable if stack walking is interspersed with other operations e.g.
> symbol name resolution.

On x86 the most trivial form of stack tracing is using the IP to find
the current function's base address for name lookup and then iteratively
following the return address stored in the function prolog until it
reaches a known point like main(). No allocation required, only side
tables with static information. All the strings are static as well so a
string_view does the job, again no allocation by the runtime required.
It's an iterative algorithm that is a perfect fit for a range interface.
If you have an IP -> name mapping available it can give you very usable
results. This obviously skips over inlined functions, but that's where
more sophisticated solutions can be applied using additional debug
information and effort by the runtime. Making the stacktrace object
non-copyable prevents problems with reading a stack that no longer
exists, and since the stacktrace object only represents the algorithm to
retrieve the information, not the actual trace data itself, that's not
an issue in reality.

>
>> If the code is compiled with compiler flags that disable features
>> like function or source names the corresponding functions can be made
>> nop in the library.
> This won't work. I can have any combination of libraries loaded
> dynamically at runtime that do and don't have symbols, and can request
> traces from either. There is no way (without forbidding runtime dynamic
> library loading, at least) to know for sure if symbols will be present
> or not at compile time. I don't see such a limitation being worthwhile.
> It's also somewhat moot if you separate tracing and symbol resolution.

Right, scratch that then. It was a QoI point anyway.

>
>> It also gives the developer the most freedom to do with the stack frames
>> whatevery they need.
> Like... save them for later use? Sorry, the only benefit I see to this
> design is having resolution return a struct (or even a class). I'd
> already suggested doing so in my API.

You also acknowledge the need for an iterator based approach with the
next() function. In your API the only *actually* required operations for
traversing the stack are "trace(&frame, 1)" and "frame = next(frame)"
and "offset = base(frame)". Everything else can be trivially implemented
in terms of those with simple algorithms. So why not go the full mile
and give it a proper interface that doesn't attempt to hide what it
actually is?

The point is the runtime can't possibly know what anyone might want to
do with the information, neither do you or I. Not seeing another benefit
and thus calling it a day is an argumentum ad ignorantium. My approach
presents a separation of concerns:
1. The runtime gives you the algorithm to traverse the call stack
2. You use the algorithm to get the data and do with it what you want

[Edward Catmur]

On Wed, Apr 20, 2016 at 11:29 PM, Miro Knejp <miro....@gmail.com> wrote:

Then with a range based API it has the freedom to have a preallocated block available for the trace. But it isn't *forced* to. If you're on a system with limited memory you preallocate an array to hold just what you need and then copy as much as you can. If memory is not your concern you have the option to store the information however and where ever you wish. You can even chose a different solution depending on whether you're dealing with a std::bad_alloc or something else. I don't see how limiting your number of options is a good thing.

Or on a truly resource-constrained system, you can even push minimal information out over a wire to somewhere that has the space to process it properly.

 

On x86 the most trivial form of stack tracing is using the IP to find the current function's base address for name lookup and then iteratively following the return address stored in the function prolog until it reaches a known point like main(). No allocation required, only side tables with static information. All the strings are static as well so a string_view does the job, again no allocation by the runtime required. It's an iterative algorithm that is a perfect fit for a range interface. If you have an IP -> name mapping available it can give you very usable results. This obviously skips over inlined functions, but that's where more sophisticated solutions can be applied using additional debug information and effort by the runtime. Making the stacktrace object non-copyable prevents problems with reading a stack that no longer exists, and since the stacktrace object only represents the algorithm to retrieve the information, not the actual trace data itself, that's not an issue in reality.

I think the issue here with inlined functions points to a more general problem, which is that the physical stack trace (pcs of return locations) do not necessarily correspond 1-1 with logical stack frames (as in experimental/source_location). In addition to inline functions (1-n) there are trampolines/thunks that should not appear in the logical stack trace; meanwhile function/COMDAT folding implies that a single physical frame could correspond to one of multiple logical frames that can (at least in principle) be distinguished by the address of their caller. To my thinking this points to tracing as a range generator and symbol resolution as a range transformer.

[Viacheslav Usov]

On Wed, Apr 20, 2016 at 9:24 PM, Miro Knejp <miro....@gmail.com> wrote:

auto stacktrace = std::capture_stack()
cout << stacktrace.depth();
cout << stacktrace.has_source_names(); // Are human-readable file/source names present?
cout << stacktrace.has_function_names(); // Are human-readable function names present?
cout << stacktrace.has_line_numbers(); // Are line numbers for source files available?

for(const auto& frame : stracktrace) {
  cout << frame.source_id(); // An id that can be used with a compiler-generated file/tool to get the source file name
  cout << frame.line_number(); // Line number into source file if has_line_numbers == true otherwise 0

  cout << frame.source_name(); // "/usr/project/file.cpp" if has_source_names == true otherwise ""

  cout << frame.base(); // A void(*)() containing the function's address
  cout << frame.offset(); // Offset in bytes/chars/addressable units from the beginning of the function
  cout << frame.function_id(); // An id that can be used with a compiler-generated file/tool to get the function name
  cout << frame.function_name(); // "project::foo(int, int)" if has_function_names == true otherwise ""
}

I do not think it was explicitly mentioned in your points 1 - 4, but your code does not have a clear separation between "capture" and "parse". Is this something that you really meant to be desirable?

My impression is that you wanted to combine those two phases within one stacktrace object, and extract information conditionally based on its has_X methods. That is, in principle, portable, but is unnecessarily baroque because truly portable code can only use the subset that works everywhere always, and that is binary stack capture, as has already been explained in this thread. The success of those has_X calls will depend on the capabilities and the configuration of the toolchain; I do not think we really want a close tie between the way we use the (theoretically) portable API and those settings.

Your code also seems to assume that it can always iterate over stack frames; that is generally not granted without access to debugging information.

I believe that a truly portable API need not have anything but a binary capture mode, which, as I mentioned earlier, can be iterator based.

Thinking more about the parse phase, I am not even sure we need a C++ API for that. Personally, having an implementation-provided tool (e.g., its standard debugger) that can parse that capture and show me the call stack would be sufficient.

Cheers,

V.

[Viacheslav Usov]

On Thu, Apr 21, 2016 at 12:04 PM, Viacheslav Usov <via....@gmail.com> wrote:

> iterate over stack frames; that is generally not granted without access to debugging information.

Another consideration: when stack is being captured, it can have been corrupted. In this case iteration over stack frames will be at least unreliable. A binary capture could still be amenable to offline analysis. And again, because the code calling the stack trace API cannot know whether the stack is good or bad, it should better only use what is more likely to work. I realize that this consideration is very deep in the domain of unspecified behaviour, but, practically, I can easily imagine that dumping the stack is the only thing available for post mortem debugging, and stack corruption is one of the most common bugs in C++ programs..

Cheers,

V.

[Andrew Marlow]

On Thursday, 21 April 2016, Viacheslav Usov <via....@gmail.com> wrote:

On Thu, Apr 21, 2016 at 12:04 PM, Viacheslav Usov <via....@gmail.com> wrote:

> iterate over stack frames; that is generally not granted without access to debugging information.

Another consideration: when stack is being captured, it can have been corrupted. In this case iteration over stack frames will be at least unreliable. 

Agreed. Which is why I am in favour of a simple string being returned from the getStack function or whatever it winds up being called.  My aim in being able to portably get at the stack trace is just for debugging and logging. It is not to do anything more complicated than that. I don't expect to be able to perform program introspection and or manipulate stack frames, I think all that's beyond portable c++for all the reasons people have already given. 

[Edward Catmur]

Returning a "simple string" requires memory allocation, so is unsuitable for a worst-case fault handler.

[Viacheslav Usov]

On Thu, Apr 21, 2016 at 2:20 PM, Andrew Marlow <marlow...@gmail.com> wrote:

> Agreed. Which is why I am in favour of a simple string being returned from the getStack function or whatever it winds up being called.  My aim in being able to portably get at the stack trace is just for debugging and logging.

Oh. I think I did not get that initially. Just to make sure: do you mean the string can contain anything, say just a bunch of non-printable bytes? Basically a glorified binary buffer?

Cheers,

V.

[Miro Knejp]

Am 21.04.2016 um 12:04 schrieb Viacheslav Usov:

On Wed, Apr 20, 2016 at 9:24 PM, Miro Knejp <miro....@gmail.com> wrote:

auto stacktrace = std::capture_stack()
cout << stacktrace.depth();
cout << stacktrace.has_source_names(); // Are human-readable file/source names present?
cout << stacktrace.has_function_names(); // Are human-readable function names present?
cout << stacktrace.has_line_numbers(); // Are line numbers for source files available?

for(const auto& frame : stracktrace) {
  cout << frame.source_id(); // An id that can be used with a compiler-generated file/tool to get the source file name
  cout << frame.line_number(); // Line number into source file if has_line_numbers == true otherwise 0

  cout << frame.source_name(); // "/usr/project/file.cpp" if has_source_names == true otherwise ""

  cout << frame.base(); // A void(*)() containing the function's address
  cout << frame.offset(); // Offset in bytes/chars/addressable units from the beginning of the function
  cout << frame.function_id(); // An id that can be used with a compiler-generated file/tool to get the function name
  cout << frame.function_name(); // "project::foo(int, int)" if has_function_names == true otherwise ""
}

I do not think it was explicitly mentioned in your points 1 - 4, but your code does not have a clear separation between "capture" and "parse". Is this something that you really meant to be desirable?

Not exactly sure what you mean by "parse". As mentioned in a previous message, the stacktrace object hides the algorithm to traverse the call stack and you use it to collect the information you need. A range/iterator interface has the big benefit that it can work lazily if the runtime supports it and avoid additional resources costs that would otherwise be necessary for a temporary array/vector/etc.

My impression is that you wanted to combine those two phases within one stacktrace object, and extract information conditionally based on its has_X methods. That is, in principle, portable, but is unnecessarily baroque because truly portable code can only use the subset that works everywhere always, and that is binary stack capture, as has already been explained in this thread. The success of those has_X calls will depend on the capabilities and the configuration of the toolchain; I do not think we really want a close tie between the way we use the (theoretically) portable API and those settings.

I'm not attached to the has_X methods, and the more I think about it they are probably unreliable in the presence of dynamically loaded libraries that were built with different levels of debug information. However if a get_name() method is codified in the interface then it also works *everywhere always*, it simply may return an empty string and you have to be prepared to deal with it, but the code is still perfectly portable giving you base() and offset() always.

Your code also seems to assume that it can always iterate over stack frames; that is generally not granted without access to debugging information.

If it cannot iterate over stack frames then how is it supposed to provide a stack trace for arbitrary points in your program? Remember it's a range with a begin and end iterator. If the implementation can only find 2 stack frames and then gets lost then the loop simply exits after 2 iterations and you're done. If it cannot provide any information at all the loop body never runs. Any argument against the iterability of the stack walking algorithm is an argument against the very feature regardless of the interface, since the implementation *must* do *some* work at runtime unless you have a program where every function has only one statically known unique call stack. In cases where the implementation cannot walk the stack interspersed with user calls it can chose to do it all at once in the range's constructor and stash all the state it needs in the range object. The point is to let the implementation decide what the most efficient/safe way of walking the stack is, and let you decide what the most efficient way is for you to make use of the information said algorithm delivers.

I believe that a truly portable API need not have anything but a binary capture mode, which, as I mentioned earlier, can be iterator based.

Now you're contradicting yourself. So is stack walking *always* possible to be iterator based or not? My code assumes exactly that and you didn't seem to accept that. If it ends up to be accepted that an iterator-based approach is desireable then not making it a range would be a big mistake IMHO.

Thinking more about the parse phase, I am not even sure we need a C++ API for that. Personally, having an implementation-provided tool (e.g., its standard debugger) that can parse that capture and show me the call stack would be sufficient.

If the runtime has the information at hand why not make it available to the user?

[Matthew Woehlke]

On 2016-04-20 18:29, Miro Knejp wrote:
> If the API codifies a vector or array then the implementation and user
> have no other choice than to use a vector or array, even if that
> pessimizes what the implementation or user already have available. With
> a range you can still copy everything to a preallocated array, or a
> vector, or any other sequence container you may have.

...and if the implementation needs to work with an array anyway, then
you've penalized *those* implementations by forcing anyone that needs to
keep the trace around to immediately make a copy that would have been
unnecessary.

Can you give an example of an implementation for which an array would
not be efficient *and* does not have the property that the result is
invalidated if I make a copy of it for later? Because I am not a fan of
an object that needs to be transformed in order to safely retain it for
later use. That sort of subtle "gotcha" seems like a great way to
encourage bugs.

> copy(stacktrace(), back_inserter(my_vector));

How would you do this when the target is a `frame*` of fixed size? I
still think that's an important use case that needs to be handled.

> Am 20.04.2016 um 22:06 schrieb Matthew Woehlke:
>> Note that you probably can't implement `depth()` without walking the
>> stack, so even in your API as presented there is very little actual
>> benefit to such an approach.
>
> Then let it return 0 if it doesn't have the information available just
> as in your API. After iterating the range you know how many elements
> there are.

Unacceptable. `depth()` may return 0 iff a stack trace cannot be
produced. The criteria for returning 0 are "not supported" (read: 'not
possible') and "cannot be completed", *not* "inefficient". Artificially
not supporting the operation, or having its value change for an already
collected trace, are not acceptable.

>> On 2016-04-20 15:24, Miro Knejp wrote:
>>> It also allows the implementation to not allocate any temporary
>>> memory for storing the stack frames for this particular trace.
>>

>> I don't see how your implementation would even work *without* allocating
>> memory; at least the trace object itself would need to hold a pointer to
>> where in the stack is currently being traversed (two, if it isn't
>> read-once, which I would consider another non-starter). This is on top
>> of whatever scratch space is needed for operation, which won't be
>> reusable if stack walking is interspersed with other operations e.g.
>> symbol name resolution.
>
> On x86 the most trivial form of stack tracing is using the IP to find
> the current function's base address for name lookup and then iteratively
> following the return address stored in the function prolog until it
> reaches a known point like main(). No allocation required, only side
> tables with static information.

You need two items of state: the original IP when the trace was
collected, and the IP of the current iteration. You need the latter so
that when you increment the iterator, it knows where it was and where it
was going. You need the former so that your iterator isn't read-once,
which would be horrible.

Yes, it's less memory than storing the whole trace (unless your trace
only has two entries; then it would be the same), but it's memory you
still have to keep around until you're doing with the trace, above and
beyond whatever the trace functions need for scratch memory... which
also has to be reallocated every time you step through the stack. (Note:
I'm assuming this is stack memory, but still...)

> Making the stacktrace object non-copyable prevents problems with
> reading a stack that no longer exists, and since the stacktrace
> object only represents the algorithm to retrieve the information, not
> the actual trace data itself, that's not an issue in reality.

Any API that does not allow me to copy a lightweight trace is a
non-starter. "Lightweight" on at least x86 means `sizeof(void*)*k`,
where `k` is the number of trace entries. Any effort above and beyond
collecting the IP's is also strongly undesirable.

>>> It also gives the developer the most freedom to do with the stack frames
>>> whatevery they need.
>>
>> Like... save them for later use? Sorry, the only benefit I see to this
>> design is having resolution return a struct (or even a class). I'd
>> already suggested doing so in my API.
>
> You also acknowledge the need for an iterator based approach with the
> next() function.

No, I don't. Please *read* the documentation of that function (or look
carefully at the API, for that matter; it is a *stateless* function).
The `next` I present is a *static* operation that attempts to determine
the caller of inlined code. It does not require an active program stack.
It *may* be very expensive (e.g. on at least some platforms, likely
requires loading debug symbols). It *will* return a null `frame` for any
input `frame` that is not part of an inlined function. It *may* just
return a null `frame`, period.

("Next" is probably not the best name, but I couldn't come up with
anything better when I wrote it. Suggestions?)

> In your API the only *actually* required operations for
> traversing the stack are "trace(&frame, 1)" and "frame = next(frame)"
> and "offset = base(frame)".

No; you need `trace(&frames, n)`, and *maybe*, depending on how symbol
resolution is implemented (and if you e.g. want to save a trace to use
in a different process instance) `offset`. You most certainly *do not*
need `next`, and some platforms may well have an implementation of
`next` that does nothing. If you only collect the top of stack, *at
best* you can resolve its static callers if it is an inlined function.

The only functions in my API that use non-global state are `size` and
`trace`. Also, unlike your API, nothing gives you back any data that is
in any way dependent on mutable program state (aside from unloading
libraries).

--
Matthew

[Matthew Woehlke]

On 2016-04-21 05:23, Edward Catmur wrote:
> I think the issue here with inlined functions points to a more general
> problem, which is that the physical stack trace (pcs of return locations)
> do not necessarily correspond 1-1 with logical stack frames (as in
> experimental/source_location). In addition to inline functions (1-n) there
> are trampolines/thunks that should not appear in the logical stack trace;
> meanwhile function/COMDAT folding implies that a single physical frame
> could correspond to one of multiple logical frames that can (at least in
> principle) be distinguished by the address of their caller. To my thinking
> this points to tracing as a range generator and symbol resolution as a
> range transformer.

I have to *strongly* disagree with this, at least with the notion that
"tracing" (i.e. the initial process used to collect a trace) involves
resolving these issues.

For some use cases, it is *imperative* that collecting the trace be as
efficient as possible. This means doing a walk of the assembly-level
call stack *and nothing else*, and deferring any translation between
this "physical stack trace" and a logical stack trace until later. (This
is specifically why I decided to keep `next` as a separate function in
my notional API, rather than declaring its function to be built into
`trace`.) Symbol resolution is separated for similar reasons.

Specifically, I may need to collect (and retain copies of!) a *lot* of
stack traces, but only very rarely if ever actually do anything with
them. (Example: a library collects a stack trace every time an object¹
is allocated in order to dump those traces if it detects any objects not
freed at end of program execution. *Many* objects will be allocated, so
collecting the trace must be cheap, but a well behaved program will leak
few or none, so only few or no traces will be displayed and accordingly
will ever need to be "cleaned up".)

(¹ i.e. some base class from which most or all classes of the library
derive.)

--
Matthew

[Edward Catmur]

On Thu, Apr 21, 2016 at 4:20 PM, Matthew Woehlke <mwoehlk...@gmail.com> wrote:

On 2016-04-20 18:29, Miro Knejp wrote:
> If the API codifies a vector or array then the implementation and user
> have no other choice than to use a vector or array, even if that
> pessimizes what the implementation or user already have available. With
> a range you can still copy everything to a preallocated array, or a
> vector, or any other sequence container you may have.

...and if the implementation needs to work with an array anyway, then
you've penalized *those* implementations by forcing anyone that needs to
keep the trace around to immediately make a copy that would have been
unnecessary.

Taking a backtrace is essentially the same as walking a linked list; there's not much about that that suggests that an array is a natural interface. Sure, existing platform-specific implementations are specified in terms of arrays, but that's because that's they're written to C, which doesn't have the expressive power of C++.

 

Can you give an example of an implementation for which an array would
not be efficient *and* does not have the property that the result is
invalidated if I make a copy of it for later? Because I am not a fan of
an object that needs to be transformed in order to safely retain it for
later use. That sort of subtle "gotcha" seems like a great way to
encourage bugs.

A stacktrace range generator is the same idea as istream_iterator or filesystem::directory_iterator; it's a concept that programmers should be reasonably well acquainted with.

 

> copy(stacktrace(), back_inserter(my_vector));

How would you do this when the target is a `frame*` of fixed size? I
still think that's an important use case that needs to be handled.

copy(stacktrace() | sliced(0, n), target);

 

> Making the stacktrace object non-copyable prevents problems with
> reading a stack that no longer exists, and since the stacktrace
> object only represents the algorithm to retrieve the information, not
> the actual trace data itself, that's not an issue in reality.

Any API that does not allow me to copy a lightweight trace is a
non-starter. "Lightweight" on at least x86 means `sizeof(void*)*k`,
where `k` is the number of trace entries. Any effort above and beyond
collecting the IP's is also strongly undesirable.

Copying the stacktrace into a container shouldn't be the responsibility of the API, since you can do it using external facilities, e.g.: boost::copy_range<vector<stack_entry>>(stacktrace())

I think we're in violent agreement here; I'm advocating separating the collection stage ("tracing") from the translation stage - I called the latter "symbol resolution", but that was me being lazy as it involves other forms of translation e.g. inline call expansion, thunk elimination, fold expansion. I'm not sure I see a need to separate inline call expansion from other forms of translation - it seems to all be very implementation-specific - but I could be persuaded otherwise.

The power of presenting collection as a range generator and translation as a range transformer is that it's entirely up to the end user how to compose them, thereby encompassing the majority of use cases.

[Viacheslav Usov]

On Thu, Apr 21, 2016 at 3:39 PM, Miro Knejp <miro....@gmail.com> wrote:

> Not exactly sure what you mean by "parse". As mentioned in a previous message, the stacktrace object hides the algorithm to traverse the call stack and you use it to collect the information you need.

Traverse and collect could mean a few different things. Let's just use x86 as a hopefully universally understandably example. "Traversing and collecting" could be as trivial as just copying the entire byte range from the stack's bottom to its current top. And can be as complicated as singling out call frames and return addresses, and translating all that into the symbolic names and source file locations. On x86, there is a conventional frame pointer register that makes frame iteration quite straightforward without symbolic info - when a certain protocol is followed by the program, which is not mandatory, and when the stack has not been corrupted, which is unknown.

Roughly speaking, anything beyond "just copying" can be considered parsing without a guaranteed success, but I would be willing to grant the implementation the right to determine what it thinks it can capture efficiently with a reasonable hope for success.

> If it cannot iterate over stack frames then how is it supposed to provide a stack trace for arbitrary points in your program?

Earlier in this thread I indicated that "arbitrary points" may be harder than "exception points". More generally, it may be impossible to iterate when the stack is being captured, but may be possible later in a different environment. In the example above, if the program is built with "frame pointer omission", iterating over stack frames is only possible with debugging information. If the latter is stripped from the executable, that is not possible. But it may be possible if the stack is captured in a binary form and later analysed in an environment that has the debugging info.

> Now you're contradicting yourself. So is stack walking *always* possible to be iterator based or not?

"Iterator-based" does not necessarily mean "stack-walking". The implementation may need to capture a few different regions of memory to make stack walking possible (with debugging info present at a later stage), hence the need to iterate over them. But they do not need to correspond directly to stack frames.

> If the runtime has the information at hand why not make it available to the user?

To make all the information available, normally some configuration will be required in the general case. In some cases all that is required is embedded into the program itself and any modules it uses. But, in my opinion, we need to regard that as an exception and API must be configurable - and portable. And I have certain doubts as to whether it can really be portable, not just superficially. I am talking about using portable API in this way for platform A and in that way for platform B, or, worse, in this way for platform X and toolchain settings Y & Z.

Cheers,

V.

[Tony V E]

Don't forget coroutines. The stack might not be contiguous (IIUC)

Sent from my BlackBerry portable Babbage Device

From: Viacheslav Usov Sent: Thursday, April 21, 2016 12:51 PM To: std-dis...@isocpp.org Reply To: std-dis...@isocpp.org Subject: Re: [std-discussion] Re: throw std::exception with stack trace (portable)

--

[Thiago Macieira]

On quinta-feira, 21 de abril de 2016 13:14:26 PDT Tony V E wrote:
> Don't forget coroutines. The stack might not be contiguous (IIUC)

Just like threads: you have one stack per thread. In an environment with
coroutines, I'd like to know the state of all coroutines, running or not.
That's very discontiguous.

--
Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org
Software Architect - Intel Open Source Technology Center

[Patrice Roy]

Yeah, stack traces with coroutines might look like stack traces in callback-laden systems such as those based on Node.js. That doesn't discredit the idea of stack traces, though (it might just be the wrong use-case).

On a technical note: I hope any stack trace proposal will make it program-readable (something we can translate into a human-readable format on demand, but program-readable first and foremost).

[Myriachan]

Not about coroutines:

Since sometimes stack traces aren't directly possible, perhaps a rule could be that the implementation need only provide a stack trace containing functions that either a) have objects that need to be destructed during an unwind, or b) have an active try {} containing the unwind point (return address), with the exclusion that inlined functions may appear as their containing function instead.  Of course, additional functions could be provided in the trace if the implementation is aware of them as a QoI issue.

In x86-32 Windows, it is not possible in a fully-generic way to make a stack trace without extra information that not all executables today have (even when they don't play tricks).  However, it is possible to walk the SEH exception chain, which covers the a) and b) cases I mentioned.  Functions that meet neither a) nor b) are usually not in the SEH chain, which is why I suggest it that way.

The x86-32 Windows design can be applied generically, because all implementations must be capable of unwinding to those functions.  I guess that my meaning is, "if you can unwind to it, you can trace the stack to it".

Note that x86-64 Windows and ARM Windows are quite different in this regard: on x86-64 and ARM Windows, the assembly language ABI is strict and there is additional metadata that can used to make a "perfect" stack trace (aside from the effects of inlining and tail-call optimizations).

Melissa

[Myriachan]

I just thought of a change to what I said:

a) have objects that need to be destructed during an unwind

This should probably should say:

a) have objects that need to call a destructor that has potential side effects that may affect the observable behavior of the program

The reason for the change is that compilers would still be able to omit unwind code if the only destructor(s) to be called would optimize to nothing.

[Christopher Jefferson]

On 26 April 2016 at 02:33, Myriachan <myri...@gmail.com> wrote:

Not about coroutines:

Since sometimes stack traces aren't directly possible, perhaps a rule could be that the implementation need only provide a stack trace containing functions that either a) have objects that need to be destructed during an unwind, or b) have an active try {} containing the unwind point (return address), with the exclusion that inlined functions may appear as their containing function instead.  Of course, additional functions could be provided in the trace if the implementation is aware of them as a QoI issue. 

....

 

The x86-32 Windows design can be applied generically, because all implementations must be capable of unwinding to those functions.  I guess that my meaning is, "if you can unwind to it, you can trace the stack to it".

I think a simpler rule would just be to say "some functions can be missing". This has the advantage that it provides an easy, valid, implementation for compilers which don't know how to do any stack traces (just miss everything), and makes life easier when optimisation levels increase and stack trace functions get lost (as they often are when debugging real code with high optimisation levels).

Of course, as a QoI issue, I would expect with optimisations off/low, most compilers would produce good quality backtraces.

Chris

[tap...@yahoo.com]

Exception with stack trace sounds nice - but would it be possible as very first step standardize stack trace itself (without any exception handling).

I can provide proposal for API - from this project:

https://sourceforge.net/projects/diagnostic/

See ResolveStack.h / ResolveStack.cpp / ResolveStackM.h / ResolveStackM.cpp
https://sourceforge.net/p/diagnostic/svn/HEAD/tree/src/ 

Design: https://sourceforge.net/p/diagnostic/svn/HEAD/tree/ - see chapter 7.

It has some cross links to stack overflow forum / message / posts for it.