Allocation failure of automatic objects

206 views
Skip to first unread message

FrankHB1989

unread,
Dec 1, 2017, 10:41:20 AM12/1/17
to ISO C++ Standard - Discussion
Does the standard have normative rules to guarantee allocation failure of automatic objects always consistent and predictable?

And what about requirements/limitations on nested function calls, with or without such guarantee?


Nicol Bolas

unread,
Dec 1, 2017, 11:31:14 AM12/1/17
to ISO C++ Standard - Discussion
On Friday, December 1, 2017 at 10:41:20 AM UTC-5, FrankHB1989 wrote:
Does the standard have normative rules to guarantee allocation failure of automatic objects always consistent and predictable?

Consider proper tail recursion. This is an optimization that a compiler can make to recursive functions that essentially removes local variables from the stack. But whether it can do so depends on a myriad of factors: the target ABI, the particular function arguments, possibly even aspects of the CPU itself.

If tail recursion is so implementation and build-target dependent, how could the standard spell out requirements for when it can happen? It can't. Instead, by not detailing specifically how "consistent and predictable" automatic object allocation is, it permits tail recursion as an optimization.

Similarly, if all of a function's automatic variables fit into registers, and that function doesn't call any other functions... why take up precious stack space for them? That optimization would again require automatic object allocation to not be "consistent and predictable", since such allocation patterns would change based on adding a (non-inlined) function call.
 
And what about requirements/limitations on nested function calls, with or without such guarantee?

There are no requirements or limitations. Or rather, there are limitations, but the standard doesn't require that they be spelled out. There's a recognition that implementations have finite resources and that these resources can be exhausted, leading to program termination.

Nothing more specific than that is said with regard to function recursion or automatic object "allocation" failure.
 

FrankHB1989

unread,
Dec 1, 2017, 12:32:08 PM12/1/17
to ISO C++ Standard - Discussion


在 2017年12月2日星期六 UTC+8上午12:31:14,Nicol Bolas写道:
On Friday, December 1, 2017 at 10:41:20 AM UTC-5, FrankHB1989 wrote:
Does the standard have normative rules to guarantee allocation failure of automatic objects always consistent and predictable?

Consider proper tail recursion. This is an optimization that a compiler can make to recursive functions that essentially removes local variables from the stack. But whether it can do so depends on a myriad of factors: the target ABI, the particular function arguments, possibly even aspects of the CPU itself.

Well, that's exactly what I expect. However, IIRC it is only mandated in language like Scheme. Without such guarantee, I can't even write an interpreter targetting an object language which has proper tail recursion guarantee in portable C++ without some sophisticated magic.

If tail recursion is so implementation and build-target dependent, how could the standard spell out requirements for when it can happen? It can't. Instead, by not detailing specifically how "consistent and predictable" automatic object allocation is, it permits tail recursion as an optimization.

So, is it essentially intentionally unspecified? In reality, stack overflow may likely cause the state of program unrecoverable, which is pretty like some cases of undefined behavior... That also sounds strange, comparing to the definition of "unspecified behavior", which concerns with "well-formed program construct and correct data". I can hardly imagine the case where stack overflow is not a bug.

BTW, as per ISO C 4p2, it seems to be undefined in C.

Similarly, if all of a function's automatic variables fit into registers, and that function doesn't call any other functions... why take up precious stack space for them? That optimization would again require automatic object allocation to not be "consistent and predictable", since such allocation patterns would change based on adding a (non-inlined) function call.
 
And what about requirements/limitations on nested function calls, with or without such guarantee?

There are no requirements or limitations. Or rather, there are limitations, but the standard doesn't require that they be spelled out. There's a recognition that implementations have finite resources and that these resources can be exhausted, leading to program termination.

Interestingly, [implimits] suggests there are limitations which can't determine compliance. Would such limitation (if any) be a candidate of this list?

Nicol Bolas

unread,
Dec 1, 2017, 1:33:44 PM12/1/17
to ISO C++ Standard - Discussion
On Friday, December 1, 2017 at 12:32:08 PM UTC-5, FrankHB1989 wrote:
在 2017年12月2日星期六 UTC+8上午12:31:14,Nicol Bolas写道:
On Friday, December 1, 2017 at 10:41:20 AM UTC-5, FrankHB1989 wrote:
Does the standard have normative rules to guarantee allocation failure of automatic objects always consistent and predictable?

Consider proper tail recursion. This is an optimization that a compiler can make to recursive functions that essentially removes local variables from the stack. But whether it can do so depends on a myriad of factors: the target ABI, the particular function arguments, possibly even aspects of the CPU itself.

Well, that's exactly what I expect. However, IIRC it is only mandated in language like Scheme. Without such guarantee, I can't even write an interpreter targetting an object language which has proper tail recursion guarantee in portable C++ without some sophisticated magic.

Nonsense. If you're writing an interpreter, then your interpretation loop is simply to interpret each instruction as it appears. A function call instruction does not mean you do a function call in the interpreter. It simply changes where you get your next instruction from. The return address is stored on the interpretation stack, along with whatever values are needed for the function call.

Go look at the Lua interpreter, which does have guaranteed proper tail calls (not just recursion; any call of the form `return some_function(...)` does not increase the stack). It doesn't need to do "sophisticated magic" to do that.

If tail recursion is so implementation and build-target dependent, how could the standard spell out requirements for when it can happen? It can't. Instead, by not detailing specifically how "consistent and predictable" automatic object allocation is, it permits tail recursion as an optimization.

So, is it essentially intentionally unspecified? In reality, stack overflow may likely cause the state of program unrecoverable, which is pretty like some cases of undefined behavior... That also sounds strange, comparing to the definition of "unspecified behavior", which concerns with "well-formed program construct and correct data". I can hardly imagine the case where stack overflow is not a bug.

BTW, as per ISO C 4p2, it seems to be undefined in C.

Similarly, if all of a function's automatic variables fit into registers, and that function doesn't call any other functions... why take up precious stack space for them? That optimization would again require automatic object allocation to not be "consistent and predictable", since such allocation patterns would change based on adding a (non-inlined) function call.
 
And what about requirements/limitations on nested function calls, with or without such guarantee?

There are no requirements or limitations. Or rather, there are limitations, but the standard doesn't require that they be spelled out. There's a recognition that implementations have finite resources and that these resources can be exhausted, leading to program termination.

Interestingly, [implimits] suggests there are limitations which can't determine compliance. Would such limitation (if any) be a candidate of this list?

No. No implementation could reasonably quantify these limitations, so they would all say "fixed limits exist, but are unknown". Which tells you nothing you didn't already know.

Michael Kilburn

unread,
Dec 1, 2017, 2:04:16 PM12/1/17
to std-dis...@isocpp.org
Does the standard have normative rules to guarantee allocation failure of automatic objects always consistent and predictable?

Afaik, yes -- it assumes local storage is large enough to run your program (i.e. automatic objects allocation failure never happens). The idea is that (knowing details of your implementation) you've analyzed your program and figured out maximum local storage usage and ensured it is available prior to running your program. But implementation is allowed to use "run-time discovery" instead -- i.e. it will run your program (in hope that whatever resources are available is enough) until it hits a limit at which point it is up to implementation -- normally it terminates your program simply because it discovered that environment were not capable of providing required guarantees to C++ runtime (and therefore can't expect program to behave). Implementation can also suspend your program (probably indefinitely) with aim to resume execution when resources become available.

In any case from C++ standpoint -- local storage allocation never fails. If underlying environment can't provide this guarantee -- program cannot execute, no diagnostic (from C++) is required.


On Fri, Dec 1, 2017 at 9:41 AM, FrankHB1989 <frank...@gmail.com> wrote:
Does the standard have normative rules to guarantee allocation failure of automatic objects always consistent and predictable?

And what about requirements/limitations on nested function calls, with or without such guarantee?


--

---
You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to std-discussion+unsubscribe@isocpp.org.
To post to this group, send email to std-dis...@isocpp.org.
Visit this group at https://groups.google.com/a/isocpp.org/group/std-discussion/.



--
Sincerely yours,
Michael.

Ville Voutilainen

unread,
Dec 1, 2017, 2:14:14 PM12/1/17
to std-dis...@isocpp.org
On 1 December 2017 at 21:04, Michael Kilburn <crusad...@gmail.com> wrote:
>> Does the standard have normative rules to guarantee allocation failure of
>> automatic objects always consistent and predictable?
>
> Afaik, yes

No, not at all. See
http://eel.is/c++draft/intro.compliance#2.1

If the mentioned resource limits are exceeded, you get undefined
behavior. There's no hint of "always consistent and predictable"
anywhere near any of it.

Michael Kilburn

unread,
Dec 1, 2017, 2:22:42 PM12/1/17
to std-dis...@isocpp.org
That was somewhat "tongue-in-cheek" answer. It can never happen (from C++ program POV) -- therefore it is "always consistent and predictable". :-)

FrankHB1989

unread,
Dec 1, 2017, 9:21:46 PM12/1/17
to ISO C++ Standard - Discussion


在 2017年12月2日星期六 UTC+8上午2:33:44,Nicol Bolas写道:
On Friday, December 1, 2017 at 12:32:08 PM UTC-5, FrankHB1989 wrote:
在 2017年12月2日星期六 UTC+8上午12:31:14,Nicol Bolas写道:
On Friday, December 1, 2017 at 10:41:20 AM UTC-5, FrankHB1989 wrote:
Does the standard have normative rules to guarantee allocation failure of automatic objects always consistent and predictable?

Consider proper tail recursion. This is an optimization that a compiler can make to recursive functions that essentially removes local variables from the stack. But whether it can do so depends on a myriad of factors: the target ABI, the particular function arguments, possibly even aspects of the CPU itself.

Well, that's exactly what I expect. However, IIRC it is only mandated in language like Scheme. Without such guarantee, I can't even write an interpreter targetting an object language which has proper tail recursion guarantee in portable C++ without some sophisticated magic.

Nonsense. If you're writing an interpreter, then your interpretation loop is simply to interpret each instruction as it appears. A function call instruction does not mean you do a function call in the interpreter. It simply changes where you get your next instruction from. The return address is stored on the interpretation stack, along with whatever values are needed for the function call.

True. However, to simulate the low-level architecture of a concrete machine (either virtual or not) does seem the magic, comparing to simple REPL. With C++, I have to manually reify the activation records explicitly. I know the transition has to occur in somewhere of the language implementation stack (typically consisted of one or more kinds of ISA level machine code, portable IRs and high-level languages), but now it occurs at too high-level and that is not I want, because it requires too many details which should have been buried in underlying implementations. Has C++ to be bare-metal in the case?

Go look at the Lua interpreter, which does have guaranteed proper tail calls (not just recursion; any call of the form `return some_function(...)` does not increase the stack). It doesn't need to do "sophisticated magic" to do that.

Well, "recursion" and "calls" are synonymous here. The recursion means unbounded calls. (Though I actually use the latter in my documents, even before I've heard similar wording is also used in several other non-Scheme based languages, to avoid misinterpretation from readers.)

Lua 5+ does good job here if all I want is an interpreter (besides the language design), by providing low-level (but still portable) low-level abstraction in a form of VM. But it is a pain to work more with it (like partial specialization on the interpreter). PyPy is something sounds better here, but the object languages support no better proper tail calls than CPython.

BTW, http://lua-users.org/wiki/ProperTailRecursion: "Complain if your favourite programming language does not. " :)

And more importantly, as the step of evolution, their design does not qualified enough to address my need in the object language:
  1. No mandatory of globally separation of phases (to be friendly with program specification, supercompilation, etc)
  2. No mandatory of GC (to be friendly with deterministic & bounded resource management)
  3. First-class functions (to be friendly with ... modern programmers)
  4. First-class delimited continuations (to be friendly with both programmers and underlying implementations)
  5. Good interoperation with C++ and extensible to others with reasonably little effort
  6. Allowing to be JIT/AOT compiled to multiple targets (in future)
Or in short (but less precisely), I want a general-purposed language implementation like klisp + Racket + Chez Scheme/nanopass with opt-out GC, opt-in interops and portable (as possible) infrastructure.

The least magical I've known is trampolined style code (as klisp does).

FrankHB1989

unread,
Dec 1, 2017, 9:25:44 PM12/1/17
to ISO C++ Standard - Discussion


在 2017年12月2日星期六 UTC+8上午3:14:14,Ville Voutilainen写道:
Ah, great. That is what I have missing. Thank you.


FrankHB1989

unread,
Dec 1, 2017, 9:29:44 PM12/1/17
to ISO C++ Standard - Discussion


在 2017年12月2日星期六 UTC+8上午3:22:42,Michael Kilburn写道:
It does not occur in pure abstraction machine semantics. The problem is, in reality (even with an conforming implementation), it does occur. I have even no way to detect it reliably with aid of the language standard (unless to implement the language by myself).


Michael Kilburn

unread,
Dec 1, 2017, 9:50:20 PM12/1/17
to std-dis...@isocpp.org
It does not occur in pure abstraction machine semantics. The problem is, in reality (even with an conforming implementation), it does occur.

I explained why it "does occur" -- because implementation is allowed "to discover" it's limitation (that would otherwise will prevent it from running your program) at run-time.


I have even no way to detect it reliably with aid of the language standard (unless to implement the language by myself).

this is because on language level this situation (failure to allocate from local storage) simply doesn't happen. To deal with it you have to use implementation-specific tools (if it provides them) that are outside of language.

Let me provide an analogy:
- lets say you run a program on some hardware and your program is written on assumption that if value is written at certain address in memory -- all subsequent reads from that location will return the same value
- unfortunately such hardware can't be built thanks to entropy of universe and cosmic rays -- with time, sooner or later, this assumption will break
- instead of giving up we relax our requirements for underlying layer (i.e. hardware in our case) -- it is no longer required to give us this guarantee upfront, but we still need it to provide correct program execution
- ... so we design hardware in such way that it checks on every access whether guarantee still holds (e.g. via checksum or checking against a copy) -- and if it is discovered that smth went wrong, from your program's perspective time stops
- hardware may or may not recover from the issue, if it does -- your program won't notice it; and if it doesn't -- it won't notice it either, because it's execution will cease to happen

same for local storage allocations in C++...


--

---
You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to std-discussion+unsubscribe@isocpp.org.
To post to this group, send email to std-dis...@isocpp.org.
Visit this group at https://groups.google.com/a/isocpp.org/group/std-discussion/.



--
Sincerely yours,
Michael.

FrankHB1989

unread,
Dec 1, 2017, 11:37:06 PM12/1/17
to ISO C++ Standard - Discussion


在 2017年12月2日星期六 UTC+8上午10:50:20,Michael Kilburn写道:
It does not occur in pure abstraction machine semantics. The problem is, in reality (even with an conforming implementation), it does occur.

I explained why it "does occur" -- because implementation is allowed "to discover" it's limitation (that would otherwise will prevent it from running your program) at run-time.


I have even no way to detect it reliably with aid of the language standard (unless to implement the language by myself).

this is because on language level this situation (failure to allocate from local storage) simply doesn't happen. To deal with it you have to use implementation-specific tools (if it provides them) that are outside of language.

Let me provide an analogy:
- lets say you run a program on some hardware and your program is written on assumption that if value is written at certain address in memory -- all subsequent reads from that location will return the same value
- unfortunately such hardware can't be built thanks to entropy of universe and cosmic rays -- with time, sooner or later, this assumption will break
- instead of giving up we relax our requirements for underlying layer (i.e. hardware in our case) -- it is no longer required to give us this guarantee upfront, but we still need it to provide correct program execution
- ... so we design hardware in such way that it checks on every access whether guarantee still holds (e.g. via checksum or checking against a copy) -- and if it is discovered that smth went wrong, from your program's perspective time stops
- hardware may or may not recover from the issue, if it does -- your program won't notice it; and if it doesn't -- it won't notice it either, because it's execution will cease to happen

same for local storage allocations in C++...

I think I know the situation... Just too bad for the cases those are quite easily observable by end users, so I have to fill the gap of abstraction and expectation by myself, relying on platform-dependent ways or even case-by-case analysis...


On Fri, Dec 1, 2017 at 8:29 PM, FrankHB1989 <frank...@gmail.com> wrote:


在 2017年12月2日星期六 UTC+8上午3:22:42,Michael Kilburn写道:
On Fri, Dec 1, 2017 at 1:14 PM, Ville Voutilainen <ville.vo...@gmail.com> wrote:
On 1 December 2017 at 21:04, Michael Kilburn <crusad...@gmail.com> wrote:
>> Does the standard have normative rules to guarantee allocation failure of
>> automatic objects always consistent and predictable?
>
> Afaik, yes

No, not at all. See
http://eel.is/c++draft/intro.compliance#2.1

If the mentioned resource limits are exceeded, you get undefined
behavior. There's no hint of "always consistent and predictable"
anywhere near any of it.

That was somewhat "tongue-in-cheek" answer. It can never happen (from C++ program POV) -- therefore it is "always consistent and predictable". :-)

It does not occur in pure abstraction machine semantics. The problem is, in reality (even with an conforming implementation), it does occur. I have even no way to detect it reliably with aid of the language standard (unless to implement the language by myself).


--

---
You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to std-discussio...@isocpp.org.

To post to this group, send email to std-dis...@isocpp.org.
Visit this group at https://groups.google.com/a/isocpp.org/group/std-discussion/.



--
Sincerely yours,
Michael.

Michael Kilburn

unread,
Dec 2, 2017, 3:25:29 AM12/2/17
to std-dis...@isocpp.org
On Fri, Dec 1, 2017 at 10:37 PM, FrankHB1989 <frank...@gmail.com> wrote:
在 2017年12月2日星期六 UTC+8上午10:50:20,Michael Kilburn写道:
It does not occur in pure abstraction machine semantics. The problem is, in reality (even with an conforming implementation), it does occur.

I explained why it "does occur" -- because implementation is allowed "to discover" it's limitation (that would otherwise will prevent it from running your program) at run-time.


I have even no way to detect it reliably with aid of the language standard (unless to implement the language by myself).

this is because on language level this situation (failure to allocate from local storage) simply doesn't happen. To deal with it you have to use implementation-specific tools (if it provides them) that are outside of language.

Let me provide an analogy:
- lets say you run a program on some hardware and your program is written on assumption that if value is written at certain address in memory -- all subsequent reads from that location will return the same value
- unfortunately such hardware can't be built thanks to entropy of universe and cosmic rays -- with time, sooner or later, this assumption will break
- instead of giving up we relax our requirements for underlying layer (i.e. hardware in our case) -- it is no longer required to give us this guarantee upfront, but we still need it to provide correct program execution
- ... so we design hardware in such way that it checks on every access whether guarantee still holds (e.g. via checksum or checking against a copy) -- and if it is discovered that smth went wrong, from your program's perspective time stops
- hardware may or may not recover from the issue, if it does -- your program won't notice it; and if it doesn't -- it won't notice it either, because it's execution will cease to happen

same for local storage allocations in C++...

I think I know the situation...

Another reason for this is the fact that often your call tree depth depends on external data provided at run-time. I.e. it is simply impossible to pre-calculate your local storage requirements. Even when it is possible -- some implementations (MSVC) use stack for exceptions, this makes it much harder to figure out max possible stack usage (because while exception is in-flight -- stack "below" exception is not used for function calls made during unwinding).

 
Just too bad for the cases those are quite easily observable by end users, so I have to fill the gap of abstraction and expectation by myself, relying on platform-dependent ways or even case-by-case analysis...

Language certainly can recognize this situation and provide some model for handling mechanism. But which one? It certainly can't be via exceptions or return values. That leaves signals (inherited from C) or you have to add smth new.

Not sure about how C++ standard treats signals and if they are supported on every platform. I know they behave differently even from one Linux version to another.

Even if you come up with something -- suppose you ran out of local storage and that mechanism called another part of your code (presumably using another pre-allocated stack)... What it is going to do? There is no way to "add more stack to that thread" on typical implementation. Nor there is a good way to interrupt execution of that thread -- noexcept functions are way too important/convenient to make same mistake early versions of Java did.

If all you want to do is to print something to console (using another thread/stack) and exit (or freeze that particular thread) -- then yes, it is probably possible to change language to provide for this. But then you'll have questions about practical usefulness of that.

FrankHB1989

unread,
Dec 2, 2017, 7:45:26 AM12/2/17
to ISO C++ Standard - Discussion


在 2017年12月2日星期六 UTC+8下午4:25:29,Michael Kilburn写道:
On Fri, Dec 1, 2017 at 10:37 PM, FrankHB1989 <frank...@gmail.com> wrote:
在 2017年12月2日星期六 UTC+8上午10:50:20,Michael Kilburn写道:
It does not occur in pure abstraction machine semantics. The problem is, in reality (even with an conforming implementation), it does occur.

I explained why it "does occur" -- because implementation is allowed "to discover" it's limitation (that would otherwise will prevent it from running your program) at run-time.


I have even no way to detect it reliably with aid of the language standard (unless to implement the language by myself).

this is because on language level this situation (failure to allocate from local storage) simply doesn't happen. To deal with it you have to use implementation-specific tools (if it provides them) that are outside of language.

Let me provide an analogy:
- lets say you run a program on some hardware and your program is written on assumption that if value is written at certain address in memory -- all subsequent reads from that location will return the same value
- unfortunately such hardware can't be built thanks to entropy of universe and cosmic rays -- with time, sooner or later, this assumption will break
- instead of giving up we relax our requirements for underlying layer (i.e. hardware in our case) -- it is no longer required to give us this guarantee upfront, but we still need it to provide correct program execution
- ... so we design hardware in such way that it checks on every access whether guarantee still holds (e.g. via checksum or checking against a copy) -- and if it is discovered that smth went wrong, from your program's perspective time stops
- hardware may or may not recover from the issue, if it does -- your program won't notice it; and if it doesn't -- it won't notice it either, because it's execution will cease to happen

same for local storage allocations in C++...

I think I know the situation...

Another reason for this is the fact that often your call tree depth depends on external data provided at run-time. I.e. it is simply impossible to pre-calculate your local storage requirements. Even when it is possible -- some implementations (MSVC) use stack for exceptions, this makes it much harder to figure out max possible stack usage (because while exception is in-flight -- stack "below" exception is not used for function calls made during unwinding).

 
Just too bad for the cases those are quite easily observable by end users, so I have to fill the gap of abstraction and expectation by myself, relying on platform-dependent ways or even case-by-case analysis...

Language certainly can recognize this situation and provide some model for handling mechanism. But which one? It certainly can't be via exceptions or return values. That leaves signals (inherited from C) or you have to add smth new.
Is throwing std::bad_alloc object allowed by current C++ standard?

Not sure about how C++ standard treats signals and if they are supported on every platform. I know they behave differently even from one Linux version to another.

It seems that they are quite limited. C++ does not extend much compared to C, and C mandates only a few cases.

Even if you come up with something -- suppose you ran out of local storage and that mechanism called another part of your code (presumably using another pre-allocated stack)... What it is going to do? There is no way to "add more stack to that thread" on typical implementation. Nor there is a good way to interrupt execution of that thread -- noexcept functions are way too important/convenient to make same mistake early versions of Java did.

At least the program would have opportunity to report error... as usual std::bad_alloc cases.

Allowing reallocation of the whole stack may be also feasible sometimes, though I'd admit it may be error-prone in general. .

If all you want to do is to print something to console (using another thread/stack) and exit (or freeze that particular thread) -- then yes, it is probably possible to change language to provide for this. But then you'll have questions about practical usefulness of that.

Well, despite that I actually need things radically different now, I believe this is more or less useful. It is like the cases of std::terminated to be called - why not just leave them UB and allow crash?

Thiago Macieira

unread,
Dec 2, 2017, 2:03:04 PM12/2/17
to std-dis...@isocpp.org
On Saturday, 2 December 2017 04:45:26 PST FrankHB1989 wrote:
> > Language certainly can recognize this situation and provide some model for
> > handling mechanism. But which one? It certainly can't be via exceptions or
> > return values. That leaves signals (inherited from C) or you have to add
> > smth new.
>
> Is throwing std::bad_alloc object allowed by current C++ standard?

From where?

Paraphrasing Monty Python, when a thread reaches stack overflow, it is dead,
it's gone from this to a better one, it has gone to meet its maker. It's an
ex-thread.

No generic recovery is possible, no generic reporting mechanism other than an
out-of-stack (including other threads) handler. Not unless you add overhead to
every single stack allocation.

> > and that mechanism called another part of your code (presumably using
> > another pre-allocated stack)... What it is going to do? There is no way to
> > "add more stack to that thread" on typical implementation. Nor there is a
> > good way to interrupt execution of that thread -- *noexcept *functions
> > are way too important/convenient to make same mistake early versions of
> > Java did.
> >
> At least the program would have opportunity to report error... as usual
> std::bad_alloc cases.

It's too late if the stack has already exhausted. That's the same as heap
allocation succeeding, but the page faulting in failing.

You need to detect that the exhaustion would happen before it actually
happens. That's what std::bad_alloc does.

> Allowing reallocation of the whole stack may be also feasible sometimes,
> though I'd admit it may be error-prone in general. .

Stack cannot be reallocated, except in very restricted, specially constructed
cases.

It can be grown, but that's what it's already doing on any modern OS. If you
exhausted the stack, it's because the growth reached its limit.

--
Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org
Software Architect - Intel Open Source Technology Center

Michael Kilburn

unread,
Dec 2, 2017, 4:31:45 PM12/2/17
to std-dis...@isocpp.org
On Sat, Dec 2, 2017 at 6:45 AM, FrankHB1989 <frank...@gmail.com> wrote:
在 2017年12月2日星期六 UTC+8下午4:25:29,Michael Kilburn写道:
Just too bad for the cases those are quite easily observable by end users, so I have to fill the gap of abstraction and expectation by myself, relying on platform-dependent ways or even case-by-case analysis...

Language certainly can recognize this situation and provide some model for handling mechanism. But which one? It certainly can't be via exceptions or return values. That leaves signals (inherited from C) or you have to add smth new.
Is throwing std::bad_alloc object allowed by current C++ standard?

Certainly not from "noexcept" functions. Besides, "throwing" in all current implementations means "call another function(s)" and you can't do it -- no stack space left. Doubt anyone will be thrilled if you try to enforce "throw" to not use current stack.

Even if you come up with something -- suppose you ran out of local storage and that mechanism called another part of your code (presumably using another pre-allocated stack)... What it is going to do? There is no way to "add more stack to that thread" on typical implementation. Nor there is a good way to interrupt execution of that thread -- noexcept functions are way too important/convenient to make same mistake early versions of Java did.

At least the program would have opportunity to report error... as usual std::bad_alloc cases.

Well, depends on what "reporting" means. You'll have to define it's meaning, figure out a way how to do it without using current stack and then argue with language lawyers about it practical necessity and effect on other C++ implementations that may not even use stack to model local storage.

 
Allowing reallocation of the whole stack may be also feasible sometimes, though I'd admit it may be error-prone in general. .

If all you want to do is to print something to console (using another thread/stack) and exit (or freeze that particular thread) -- then yes, it is probably possible to change language to provide for this. But then you'll have questions about practical usefulness of that.

Well, despite that I actually need things radically different now, I believe this is more or less useful. It is like the cases of std::terminated to be called - why not just leave them UB and allow crash?

Well, first of all UB doesn't mean crash -- it can be anything. Afaik, language doesn't define "crash" at all. Second, in your case -- how would you call std::terminate()? it needs stack space too...

I don't think it is impossible to devise such mechanism and put it into language (but I also argued that throwing from destructors should be allowed ;) ). Questions are:
- details of such mechanism behavior
- it's practical benefits
- it's implementation costs
- details of typical implementation (e.g. in GCC on x86)

--
Sincerely yours,
Michael.

FrankHB1989

unread,
Dec 2, 2017, 9:55:13 PM12/2/17
to ISO C++ Standard - Discussion


在 2017年12月3日星期日 UTC+8上午3:03:04,Thiago Macieira写道:
On Saturday, 2 December 2017 04:45:26 PST FrankHB1989 wrote:
> > Language certainly can recognize this situation and provide some model for
> > handling mechanism. But which one? It certainly can't be via exceptions or
> > return values. That leaves signals (inherited from C) or you have to add
> > smth new.
>
> Is throwing std::bad_alloc object allowed by current C++ standard?

From where?
... Another emergency area?


Paraphrasing Monty Python, when a thread reaches stack overflow, it is dead,
it's gone from this to a better one, it has gone to meet its maker. It's an
ex-thread.

No generic recovery is possible, no generic reporting mechanism other than an
out-of-stack (including other threads) handler. Not unless you add overhead to
every single stack allocation.

OK. The case should be that the thread had been exited with a pending allocation failure status (bad_alloc, or whatever else).

> > and that mechanism called another part of your code (presumably using
> > another pre-allocated stack)... What it is going to do? There is no way to
> > "add more stack to that thread" on typical implementation. Nor there is a
> > good way to interrupt execution of that thread -- *noexcept *functions
> > are way too important/convenient to make same mistake early versions of
> > Java did.
> >
> At least the program would have opportunity to report error... as usual
> std::bad_alloc cases.

It's too late if the stack has already exhausted. That's the same as heap
allocation succeeding, but the page faulting in failing.

You need to detect that the exhaustion would happen before it actually
happens. That's what std::bad_alloc does.

I see. So comes extra overhead and complexity, sigh...

I'd hope that page faults have never be invented in the style today so everyone has to meet and solve the real problems, without half-baked solutions (and other mess like overcommit)...MMU actually does a lot but only a few can be used by programmers, sigh.


> Allowing reallocation of the whole stack may be also feasible sometimes,
> though I'd admit it may be error-prone in general. .

Stack cannot be reallocated, except in very restricted, specially constructed
cases.

It can be grown, but that's what it's already doing on any modern OS. If you
exhausted the stack, it's because the growth reached its limit.

The actual work should be done by the processors. OS only configures it.

Thiago Macieira

unread,
Dec 2, 2017, 10:19:43 PM12/2/17
to std-dis...@isocpp.org
On Saturday, 2 December 2017 18:55:13 PST FrankHB1989 wrote:
> > Paraphrasing Monty Python, when a thread reaches stack overflow, it is
> > dead,
> > it's gone from this to a better one, it has gone to meet its maker. It's
> > an
> > ex-thread.
> >
> > No generic recovery is possible, no generic reporting mechanism other than
> > an
> > out-of-stack (including other threads) handler. Not unless you add
> > overhead to
> > every single stack allocation.
>
> OK. The case should be that the thread had been exited with a pending
> allocation failure status (bad_alloc, or whatever else).

You're mixing concepts here. An allocation failure does not exit the thread.
It's a recoverable condition.

Failing to serve a page during a page fault is an unrecoverable condition.
Failing to serve a page that is part of the stack implies the stack state is
now corrupt, which in turn means the corner stone of throwing known as "stack
unwinding" is not possible either. A stack overflow cannot be easily predicted
before it happens (and even then it would incur overhead), and after it has
happened, the thread can't save anything or do anything. It's over.

Besides, there is one thing that can react to a segmentation fault: an alt-
stack signal handler. If you want that in your application, you already have
the tools.

> I'd hope that page faults have never be invented in the style today so
> everyone has to meet and solve the real problems, without half-baked
> solutions (and other mess like overcommit)...MMU actually does a lot but
> only a few can be used by programmers, sigh.

The MMU is not the problem. It's your OS's virtual memory manager. It's
perfectly possible to use paging with static memory maps that are always
backed by RAM. That's how functionally-safe operating systems and hypervisors
do it. Most real-time operating systems will do that too.

You can choose to program only for them and leave the mainstream OSes behind.

FrankHB1989

unread,
Dec 2, 2017, 10:29:32 PM12/2/17
to ISO C++ Standard - Discussion


在 2017年12月3日星期日 UTC+8上午5:31:45,Michael Kilburn写道:
On Sat, Dec 2, 2017 at 6:45 AM, FrankHB1989 <frank...@gmail.com> wrote:
在 2017年12月2日星期六 UTC+8下午4:25:29,Michael Kilburn写道:
Just too bad for the cases those are quite easily observable by end users, so I have to fill the gap of abstraction and expectation by myself, relying on platform-dependent ways or even case-by-case analysis...

Language certainly can recognize this situation and provide some model for handling mechanism. But which one? It certainly can't be via exceptions or return values. That leaves signals (inherited from C) or you have to add smth new.
Is throwing std::bad_alloc object allowed by current C++ standard?

Certainly not from "noexcept" functions. Besides, "throwing" in all current implementations means "call another function(s)" and you can't do it -- no stack space left. Doubt anyone will be thrilled if you try to enforce "throw" to not use current stack.
Hmm, it can be elided in some cases but just too difficult in general. Certainly I should admit the stack can't be reused normally, so there needs to be some other buffer if space is required. But... if the thread is expected always to die, why not just jump instead of call?

Well, this may have ABI issues... I'm merely talking about possibilities.

Even if you come up with something -- suppose you ran out of local storage and that mechanism called another part of your code (presumably using another pre-allocated stack)... What it is going to do? There is no way to "add more stack to that thread" on typical implementation. Nor there is a good way to interrupt execution of that thread -- noexcept functions are way too important/convenient to make same mistake early versions of Java did.

At least the program would have opportunity to report error... as usual std::bad_alloc cases.

Well, depends on what "reporting" means. You'll have to define it's meaning, figure out a way how to do it without using current stack and then argue with language lawyers about it practical necessity and effect on other C++ implementations that may not even use stack to model local storage.

This is like handling std::bad_alloc. It is eventually up to user, with certain limitations (more strictly than usual cases). They can even be conditionally supported, however, UB is still not involved and other threads can remain alive.
 
Allowing reallocation of the whole stack may be also feasible sometimes, though I'd admit it may be error-prone in general. .

If all you want to do is to print something to console (using another thread/stack) and exit (or freeze that particular thread) -- then yes, it is probably possible to change language to provide for this. But then you'll have questions about practical usefulness of that.

Well, despite that I actually need things radically different now, I believe this is more or less useful. It is like the cases of std::terminated to be called - why not just leave them UB and allow crash?

Well, first of all UB doesn't mean crash -- it can be anything. Afaik, language doesn't define "crash" at all. Second, in your case -- how would you call std::terminate()? it needs stack space too...

Yes, it is "allowed". So they are similar. The handler of "std::terminated" is called by the implementation, not by user. It has freedom to pre-allocate the needed buffer, at least in hosted implementations. (Note now exception handling is even required in freestanding ones...)

I don't think it is impossible to devise such mechanism and put it into language (but I also argued that throwing from destructors should be allowed ;) ). Questions are:
- details of such mechanism behavior
- it's practical benefits
- it's implementation costs
- details of typical implementation (e.g. in GCC on x86)

Well, that may need a detailed proposal. The points here are allowance and conformance. Since resource exhaustion causes UB currently, the former is resolved; all I want (except ones I believed no sane way to be resolved like deploying spaghetti stack as requirements) remained is why it has to be UB Is it contradict with the current language design directly? AFAIK, no.

There still be some insights which are not off-topic too much (I think). They are mostly about "practical benefits".

Portability is at least one answer to it, and avoiding UB is one of the necessary condition. Detailed mechanism would bring more persuasiveness, but let it to be in std-proposals. First let's try proving it false about feasibility here, if you think it necessary.

Second, UB effect the whole program. To keep other thread work as expected is the practical need. In reality, unless there is only one thread, it is not likely that all threads have to died in same manner.

For costs... at least they are not easy to be zero. It is probably not useful for freestanding implementations as in hosted implementations, so it may be excluded there. Otherwise, I hope it can be afford by mainstream implementations.

--
Sincerely yours,
Michael.

FrankHB1989

unread,
Dec 2, 2017, 10:44:22 PM12/2/17
to ISO C++ Standard - Discussion


在 2017年12月3日星期日 UTC+8上午11:19:43,Thiago Macieira写道:
On Saturday, 2 December 2017 18:55:13 PST FrankHB1989 wrote:
> > Paraphrasing Monty Python, when a thread reaches stack overflow, it is
> > dead,
> > it's gone from this to a better one, it has gone to meet its maker. It's
> > an
> > ex-thread.
> >
> > No generic recovery is possible, no generic reporting mechanism other than
> > an
> > out-of-stack (including other threads) handler. Not unless you add
> > overhead to
> > every single stack allocation.
>
> OK. The case should be that the thread had been exited with a pending
> allocation failure status (bad_alloc, or whatever else).

You're mixing concepts here. An allocation failure does not exit the thread.
It's a recoverable condition.

Yes, they are separated concepts. I put them together only to weaken the requirements to make more implementable in the sense of standard conformance.


Failing to serve a page during a page fault is an unrecoverable condition.
Failing to serve a page that is part of the stack implies the stack state is
now corrupt, which in turn means the corner stone of throwing known as "stack
unwinding" is not possible either. A stack overflow cannot be easily predicted
before it happens (and even then it would incur overhead), and after it has
happened, the thread can't save anything or do anything. It's over.

Besides, there is one thing that can react to a segmentation fault: an alt-
stack signal handler. If you want that in your application, you already have
the tools.
That would fall back to platform-dependent solutions. And I actually does not need so powerful tool currently. (Especially when I have to work around for platforms without an MMU... I even don't have segmentation faults... though it is more or less freestanding cases.)

> I'd hope that page faults have never be invented in the style today so
> everyone has to meet and solve the real problems, without half-baked
> solutions (and other mess like overcommit)...MMU actually does a lot but
> only a few can be used by programmers, sigh.

The MMU is not the problem. It's your OS's virtual memory manager. It's
perfectly possible to use paging with static memory maps that are always
backed by RAM. That's how functionally-safe operating systems and hypervisors
do it. Most real-time operating systems will do that too.

So why mainstream OSes does not work like them? If the problem is too much overhead is incurred so manufacturers abandon the mechanism, it is the case what I said.

You can choose to program only for them and leave the mainstream OSes behind.

If it can run on my machine for daily work, I'd like to...

Nicol Bolas

unread,
Dec 2, 2017, 10:47:26 PM12/2/17
to ISO C++ Standard - Discussion
On Saturday, December 2, 2017 at 10:29:32 PM UTC-5, FrankHB1989 wrote:
在 2017年12月3日星期日 UTC+8上午5:31:45,Michael Kilburn写道:
On Sat, Dec 2, 2017 at 6:45 AM, FrankHB1989 <frank...@gmail.com> wrote:
在 2017年12月2日星期六 UTC+8下午4:25:29,Michael Kilburn写道:
Even if you come up with something -- suppose you ran out of local storage and that mechanism called another part of your code (presumably using another pre-allocated stack)... What it is going to do? There is no way to "add more stack to that thread" on typical implementation. Nor there is a good way to interrupt execution of that thread -- noexcept functions are way too important/convenient to make same mistake early versions of Java did.

At least the program would have opportunity to report error... as usual std::bad_alloc cases.

Well, depends on what "reporting" means. You'll have to define it's meaning, figure out a way how to do it without using current stack and then argue with language lawyers about it practical necessity and effect on other C++ implementations that may not even use stack to model local storage.

This is like handling std::bad_alloc.

It is nothing like handling `std::bad_alloc`. That is an exception, and there are rules for when those get thrown, where they get caught, and what happens when they do get caught.

You cannot unwind the stack if you have a stack overflow; since that risks overflowing again. And if you cannot unwind the stack, then you cannot destroy any of the objects on that stack. The absolute best you could do is say that the thread-stack never terminates in this situation. But that leaves the program broken; who knows if that stack is holding a mutex that will never be released?

Stack exhaustion is not a survivable condition. The entire C and C++ execution model relies on the stack existing. It would be exceedingly difficult to write a multithreaded program of significance where a thread can stack overflow, terminate in some meaningful way, and the program still proceed reasonably.

The most you could do is give people a standard function to test how close you are to exhaustion, so that the caller can choose to terminate in some way.

It is eventually up to user, with certain limitations (more strictly than usual cases). They can even be conditionally supported, however, UB is still not involved and other threads can remain alive.

I don't think it is impossible to devise such mechanism and put it into language (but I also argued that throwing from destructors should be allowed ;) ). Questions are:
- details of such mechanism behavior
- it's practical benefits
- it's implementation costs
- details of typical implementation (e.g. in GCC on x86)

Well, that may need a detailed proposal. The points here are allowance and conformance. Since resource exhaustion causes UB currently, the former is resolved; all I want (except ones I believed no sane way to be resolved like deploying spaghetti stack as requirements) remained is why it has to be UB Is it contradict with the current language design directly? AFAIK, no.

There still be some insights which are not off-topic too much (I think). They are mostly about "practical benefits".

Portability is at least one answer to it, and avoiding UB is one of the necessary condition.

C and C++ have existed for a combined total of over half a century. And yet, programmers have managed to write portable programs just fine despite this being UB the entire time.

Why is this a problem for portability in the real world?

This sounds very much like people who want `int` to be required to be exactly 32-bits, 2's complement, and claim that if it isn't, then they cannot write portable programs. Not everything has to be well-defined in order for a program to be portable.

Detailed mechanism would bring more persuasiveness, but let it to be in std-proposals. First let's try proving it false about feasibility here, if you think it necessary.

Second, UB effect the whole program. To keep other thread work as expected is the practical need. In reality, unless there is only one thread, it is not likely that all threads have to died in same manner.

For costs... at least they are not easy to be zero. It is probably not useful for freestanding implementations as in hosted implementations, so it may be excluded there. Otherwise, I hope it can be afford by mainstream implementations.

To what end? What would be accomplished by this which cannot otherwise be done?


FrankHB1989

unread,
Dec 2, 2017, 11:26:41 PM12/2/17
to ISO C++ Standard - Discussion


在 2017年12月3日星期日 UTC+8上午11:47:26,Nicol Bolas写道:
On Saturday, December 2, 2017 at 10:29:32 PM UTC-5, FrankHB1989 wrote:
在 2017年12月3日星期日 UTC+8上午5:31:45,Michael Kilburn写道:
On Sat, Dec 2, 2017 at 6:45 AM, FrankHB1989 <frank...@gmail.com> wrote:
在 2017年12月2日星期六 UTC+8下午4:25:29,Michael Kilburn写道:
Even if you come up with something -- suppose you ran out of local storage and that mechanism called another part of your code (presumably using another pre-allocated stack)... What it is going to do? There is no way to "add more stack to that thread" on typical implementation. Nor there is a good way to interrupt execution of that thread -- noexcept functions are way too important/convenient to make same mistake early versions of Java did.

At least the program would have opportunity to report error... as usual std::bad_alloc cases.

Well, depends on what "reporting" means. You'll have to define it's meaning, figure out a way how to do it without using current stack and then argue with language lawyers about it practical necessity and effect on other C++ implementations that may not even use stack to model local storage.

This is like handling std::bad_alloc.

It is nothing like handling `std::bad_alloc`. That is an exception, and there are rules for when those get thrown, where they get caught, and what happens when they do get caught.

You cannot unwind the stack if you have a stack overflow; since that risks overflowing again. And if you cannot unwind the stack, then you cannot destroy any of the objects on that stack. The absolute best you could do is say that the thread-stack never terminates in this situation. But that leaves the program broken; who knows if that stack is holding a mutex that will never be released?

Is I have said, if it works, I don't expect it would directly reuse the current stack.
But it is a good question about how to handling resources shared with other threads. In such case, is it possible that the stack can be unwound remotely/asynchronously?

Stack exhaustion is not a survivable condition. The entire C and C++ execution model relies on the stack existing. It would be exceedingly difficult to write a multithreaded program of significance where a thread can stack overflow, terminate in some meaningful way, and the program still proceed reasonably.

Not every case. Consider a strictly conforming `int main(){}` :)

I agree with the points of difficulties. What I want is (ideally) that it can be guaranteed never overflow until the free store is exhausted (which may be detected like catching std::bad_alloc). It seems far more difficult to be supported even by any hosted implementation. (Otherwise I have always to work it around for my task by trampolines, etc.)

The most you could do is give people a standard function to test how close you are to exhaustion, so that the caller can choose to terminate in some way.

It is eventually up to user, with certain limitations (more strictly than usual cases). They can even be conditionally supported, however, UB is still not involved and other threads can remain alive.

I don't think it is impossible to devise such mechanism and put it into language (but I also argued that throwing from destructors should be allowed ;) ). Questions are:
- details of such mechanism behavior
- it's practical benefits
- it's implementation costs
- details of typical implementation (e.g. in GCC on x86)

Well, that may need a detailed proposal. The points here are allowance and conformance. Since resource exhaustion causes UB currently, the former is resolved; all I want (except ones I believed no sane way to be resolved like deploying spaghetti stack as requirements) remained is why it has to be UB Is it contradict with the current language design directly? AFAIK, no.

There still be some insights which are not off-topic too much (I think). They are mostly about "practical benefits".

Portability is at least one answer to it, and avoiding UB is one of the necessary condition.

C and C++ have existed for a combined total of over half a century. And yet, programmers have managed to write portable programs just fine despite this being UB the entire time.

That depends on how much portability you have to meet.

Why is this a problem for portability in the real world?

In reality, the level of portability on high-level languages (like C++) and platform-dependent solutions has difference of a factor N where N is how many radically different categories of platforms have to support. If you need only portability across several platforms look alike each other enough, and there is sufficient documentation to elide the minor differences, then N ≈ 1, and you won't have too much extra work to do besides some common assumptions from these platforms, so the process is still "as-if" they are portable after the extra work and the result would kept reasonably reusable.

Sorry, this is really not my cases in reality... and I found nowhere to put the combined description of portable behavior other than the language standards.

This sounds very much like people who want `int` to be required to be exactly 32-bits, 2's complement, and claim that if it isn't, then they cannot write portable programs. Not everything has to be well-defined in order for a program to be portable.
It isn't. If there is no exact 32-bits 2's complement integer, use of other integer (correctly) does not incur extra UB directly. The extra work to remain portability is quite limited and trivial: to test all options of integer representations allowed by the standard, using conditional inclusion, etc. Note one reason of adding `std::int32_t` to the standard is to prevent reinvented wheels in vain; that is also different to my case where there is almost no reliable wheels unless the standard has required it.

(BTW, I prefer `std::ptrdiff_t`, `std::intptr_t` and `std::int_leastN_t` over `int` or `intN_t` in the sense of portability as possible, for significance of semantic properties.)


Detailed mechanism would bring more persuasiveness, but let it to be in std-proposals. First let's try proving it false about feasibility here, if you think it necessary.

Second, UB effect the whole program. To keep other thread work as expected is the practical need. In reality, unless there is only one thread, it is not likely that all threads have to died in same manner.

For costs... at least they are not easy to be zero. It is probably not useful for freestanding implementations as in hosted implementations, so it may be excluded there. Otherwise, I hope it can be afford by mainstream implementations.

To what end? What would be accomplished by this which cannot otherwise be done?


Conformance, if you insist other difference is not important in reality.


Thiago Macieira

unread,
Dec 3, 2017, 12:25:46 AM12/3/17
to std-dis...@isocpp.org
On Saturday, 2 December 2017 19:44:22 PST FrankHB1989 wrote:
> So why mainstream OSes does not work like them? If the problem is too much
> overhead is incurred so manufacturers abandon the mechanism, it is the case
> what I said.

Going off-topic here, but the long story short is that here are decades of
experience with VMMs that say it should be that way. A lot of applications
allocate memory they don't use and a great deal of memory they do use, it's
only for a short time so it can be swapped out. Then there's clean memory,
which can be simply dropped as it can be reloaded from disk.

There's not much overhead incurred. It will not get dropped.

You need to study the subject a little more before you can make suggestions
for improvements.

> > You can choose to program only for them and leave the mainstream OSes
> > behind.
>
> If it can run on my machine for daily work, I'd like to...

Obviously not. Modern VMM works just fine for "daily work" type of
applications. And that's not even talking about applications that use (gasp!)
garbage collection...

In any case, how is a modern VMM hindering your daily work? Are you
programming functionally-safe applications?

Thiago Macieira

unread,
Dec 3, 2017, 12:34:33 AM12/3/17
to std-dis...@isocpp.org
On Saturday, 2 December 2017 19:29:32 PST FrankHB1989 wrote:
> Second, UB effect the whole program.

Correct. All UB affects the whole program, all threads.

If you want to survive stack overflow, then we start by defining "stack
overflow". That's not part of the standard. Therefore, your solution is not
provided by the standard. Please read your platform's documentation.

If you want to survive a null pointer dereference, then read what your
platform does in that case. If you want to survive a division by zero, find
out how your platform delivers those.

Thiago Macieira

unread,
Dec 3, 2017, 12:45:46 AM12/3/17
to std-dis...@isocpp.org
On Saturday, 2 December 2017 20:26:41 PST FrankHB1989 wrote:
> Is I have said, if it works, I don't expect it would directly reuse the
> current stack.
> But it is a good question about how to handling resources shared with other
> threads. In such case, is it possible that the stack can be unwound
> remotely/asynchronously?

Again, please read up on your platform's architecture about stacks and the
concept of "context switching" before you make suggestions like this. There
are many things that are unique to a thread, like the thread ID, the platform-
specific thread pointer and the *stack*.

Unwinding the stack means running code *with* that stack. If the stack is
exhausted, the unwinding won't work. Even completely noexcept destructors may
call functions and create new automatic-storage variables. That requires
stack. If you could add more memory to the stack, you wouldn't have exhausted
it in the first place.

Not to mention that if the stack exhausted, then it's corrupt. Trying to
unwind a corrupt stack is GIGO - garbage in, garbage out.

> Sorry, this is really not my cases in reality... and I found nowhere to put
> the combined description of portable behavior other than the language
> standards.

Ever heard of POSIX? Single UNIX Specification and XPG?

Nicol Bolas

unread,
Dec 3, 2017, 10:42:56 AM12/3/17
to ISO C++ Standard - Discussion
On Saturday, December 2, 2017 at 11:26:41 PM UTC-5, FrankHB1989 wrote:
在 2017年12月3日星期日 UTC+8上午11:47:26,Nicol Bolas写道:
On Saturday, December 2, 2017 at 10:29:32 PM UTC-5, FrankHB1989 wrote:
在 2017年12月3日星期日 UTC+8上午5:31:45,Michael Kilburn写道:
On Sat, Dec 2, 2017 at 6:45 AM, FrankHB1989 <frank...@gmail.com> wrote:
在 2017年12月2日星期六 UTC+8下午4:25:29,Michael Kilburn写道:

Why is this a problem for portability in the real world?

In reality, the level of portability on high-level languages (like C++) and platform-dependent solutions has difference of a factor N where N is how many radically different categories of platforms have to support. If you need only portability across several platforms look alike each other enough, and there is sufficient documentation to elide the minor differences, then N ≈ 1, and you won't have too much extra work to do besides some common assumptions from these platforms, so the process is still "as-if" they are portable after the extra work and the result would kept reasonably reusable.

Sorry, this is really not my cases in reality... and I found nowhere to put the combined description of portable behavior other than the language standards.

OK, consider a program that takes a number as input and will perform recursion based on that number; larger numbers mean more recursion. Is that program portable?

Right now, the answer as far as the standard is concerned is "yes". Sure, for each platform, there is some maximum number that you can provide, beyond which you'll exhaust the available resources. But the program itself is portable.

You're effectively saying that you want the program to be considered non-portable, simply because it could exhaust resources. That is, a program is not "portable" to you unless it cannot overflow the stack.

This is not a reasonable definition of portability. It is certainly not a useful one...

This sounds very much like people who want `int` to be required to be exactly 32-bits, 2's complement, and claim that if it isn't, then they cannot write portable programs. Not everything has to be well-defined in order for a program to be portable.
It isn't. If there is no exact 32-bits 2's complement integer, use of other integer (correctly) does not incur extra UB directly. The extra work to remain portability is quite limited and trivial: to test all options of integer representations allowed by the standard, using conditional inclusion, etc. Note one reason of adding `std::int32_t` to the standard is to prevent reinvented wheels in vain; that is also different to my case where there is almost no reliable wheels unless the standard has required it.

(BTW, I prefer `std::ptrdiff_t`, `std::intptr_t` and `std::int_leastN_t` over `int` or `intN_t` in the sense of portability as possible, for significance of semantic properties.)

Detailed mechanism would bring more persuasiveness, but let it to be in std-proposals. First let's try proving it false about feasibility here, if you think it necessary.

Second, UB effect the whole program. To keep other thread work as expected is the practical need. In reality, unless there is only one thread, it is not likely that all threads have to died in same manner.

For costs... at least they are not easy to be zero. It is probably not useful for freestanding implementations as in hosted implementations, so it may be excluded there. Otherwise, I hope it can be afford by mainstream implementations.

To what end? What would be accomplished by this which cannot otherwise be done?


Conformance, if you insist other difference is not important in reality.

"Conformance" to what?


FrankHB1989

unread,
Dec 3, 2017, 12:05:33 PM12/3/17
to ISO C++ Standard - Discussion


在 2017年12月3日星期日 UTC+8下午1:25:46,Thiago Macieira写道:
On Saturday, 2 December 2017 19:44:22 PST FrankHB1989 wrote:
> So why mainstream OSes does not work like them? If the problem is too much
> overhead is incurred so manufacturers abandon the mechanism, it is the case
> what I said.

Going off-topic here, but the long story short is that here are decades of
experience with VMMs that say it should be that way. A lot of applications
allocate memory they don't use and a great deal of memory they do use, it's
only for a short time so it can be swapped out. Then there's clean memory,
which can be simply dropped as it can be reloaded from disk.

There's not much overhead incurred. It will not get dropped.

You need to study the subject a little more before you can make suggestions
for improvements.

I think I've misread your previous words. As of paging, your points are most valid. But pages (backed by RAM or not) still do not solve the problem magically. Paging is not the right tool to solve the problem in nature because it has too coarse granularity in allocation. The original point here is that MMU involves in address translation, not necessarily concrete management mechanism like paging. Segmentation can actually do something further for this problem, but it do incur some other unnecessary complexity and inflexibility. I'd expect some other internal state exported by MMU available differently from current ISA to aid runtime check of frames of activation records (not necessarily the stack) in some hardware-accelerated way.

> > You can choose to program only for them and leave the mainstream OSes
> > behind.
>
> If it can run on my machine for daily work, I'd like to...

Obviously not. Modern VMM works just fine for "daily work" type of
applications. And that's not even talking about applications that use (gasp!)
garbage collection...

In any case, how is a modern VMM hindering your daily work? Are you
programming functionally-safe applications?

OK. My daily work relies on mainstream OSes because most applications only available on them. Modern VMM does not always hindering my work, but it does not work well in every case especially when I am work on the machine where is no enough RAM space. As an end-user, I have to always carefully terminate some programs periodically to avoid too many GBs are ate in vein, or make my HDD quiet. Garbage collected programs do work worse for these cases in general. But anyway, this is another topic.

And it sounds strange if I have to write programs only work with hypervisor...

FrankHB1989

unread,
Dec 3, 2017, 12:12:32 PM12/3/17
to ISO C++ Standard - Discussion


在 2017年12月3日星期日 UTC+8下午1:34:33,Thiago Macieira写道:
On Saturday, 2 December 2017 19:29:32 PST FrankHB1989 wrote:
> Second, UB effect the whole program.

Correct. All UB affects the whole program, all threads.

If you want to survive stack overflow, then we start by defining "stack
overflow". That's not part of the standard. Therefore, your solution is not
provided by the standard. Please read your platform's documentation.

Well, this is normally not enough. Platform's documentation would not provide enough details in implementation of the language in general. Some assumptions can be provided by ISA manual, the remained things relying on ... guessing. It can even still as unpredictable as UB if not deep dug.

If you want to survive a null pointer dereference, then read what your
platform does in that case. If you want to survive a division by zero, find
out how your platform delivers those.

Perhaps also some DR in some cases...

Thiago Macieira

unread,
Dec 3, 2017, 12:18:24 PM12/3/17
to std-dis...@isocpp.org
On Sunday, 3 December 2017 09:05:32 PST FrankHB1989 wrote:
> I think I've misread your previous words. As of paging, your points are
> most valid. But pages (backed by RAM or not) still do not solve the problem
> magically. Paging is not the right tool to solve the problem in nature
> because it has too coarse granularity in allocation. The original point
> here is that MMU involves in address translation, not necessarily concrete
> management mechanism like paging. Segmentation can actually do something
> further for this problem, but it do incur some other unnecessary complexity
> and inflexibility. I'd expect some other internal state exported by MMU
> available differently from current ISA to aid runtime check of frames of
> activation records (*not necessarily the stack*) in some
> hardware-accelerated way.

Explain a little more. What would you like to see done that can't be done
right now? What purpose would this serve? What could applications do that they
can't right now?

> OK. My daily work relies on mainstream OSes because most applications only
> available on them. Modern VMM does not always hindering my work, but it
> does not work well in every case especially when I am work on the machine
> where is no enough RAM space. As an end-user, I have to always carefully
> terminate some programs periodically to avoid too many GBs are ate in vein,
> or make my HDD quiet. Garbage collected programs do work worse for these
> cases in general. But anyway, this is another topic.

More than likely, you can only run those programs in the first place *because*
of modern VMM. If paging and overcommitting were not availbale, you'd only run
half of those programs at a time and some of them would crash unexpectedly
before you realised you should close them to free up memory.

> And it sounds strange if I have to write programs only work with
> hypervisor...

It's not strange. If you *need* the guarantee that memory you asked for was
delivered and was not taken away, like in a functionally-safe application, you
need an OS that gives you that. So choose your OS carefully. Don't use a
mainstream, multi-purpose OS.

FrankHB1989

unread,
Dec 3, 2017, 12:32:02 PM12/3/17
to ISO C++ Standard - Discussion


在 2017年12月3日星期日 UTC+8下午1:45:46,Thiago Macieira写道:
On Saturday, 2 December 2017 20:26:41 PST FrankHB1989 wrote:
> Is I have said, if it works, I don't expect it would directly reuse the
> current stack.
> But it is a good question about how to handling resources shared with other
> threads. In such case, is it possible that the stack can be unwound
> remotely/asynchronously?

Again, please read up on your platform's architecture about stacks and the
concept of "context switching" before you make suggestions like this. There
are many things that are unique to a thread, like the thread ID, the platform-
specific thread pointer and the *stack*.
This time I'd like to say no. The stack is not in the standard. So is context switching. So is the threading model.

Even native threads provided by OS are expected in most implementations, they are not mandated. Even there is the stack, it is not necessarily the native one. Even the threads are one-to-one mapped to the OS thread, they are not necessary native to processors (ISA-level; SMT on microarchitectural is another story). There is too much room...


Unwinding the stack means running code *with* that stack. If the stack is
exhausted, the unwinding won't work. Even completely noexcept destructors may
call functions and create new automatic-storage variables. That requires
stack. If you could add more memory to the stack, you wouldn't have exhausted
it in the first place.

No. Stack unwinding is mandated, but not the stack. It only means the code might be run as-if on that stack. I don't see why it can't be simulated where the runtime has the whole control, besides interops/compatibility issues.

Not to mention that if the stack exhausted, then it's corrupt. Trying to
unwind a corrupt stack is GIGO - garbage in, garbage out.

This is the status quo, a true problem to be resolved.

> Sorry, this is really not my cases in reality... and I found nowhere to put
> the combined description of portable behavior other than the language
> standards.

Ever heard of POSIX? Single UNIX Specification and XPG?

Seriously, do you really rely on the portability across platforms in reality? Not to mention freestanding implementations, they don't even applicable well for Windows...

(POSIX support is always not first-class citizen supported in Windows. WSL is still in beta and is still suffering from several severe syscall bugs in current version so it is not ready to replace the host environment.)

Another question... are there working drafts of them available conveniently as WG21 documents?

FrankHB1989

unread,
Dec 3, 2017, 12:47:09 PM12/3/17
to ISO C++ Standard - Discussion


在 2017年12月3日星期日 UTC+8下午11:42:56,Nicol Bolas写道:
On Saturday, December 2, 2017 at 11:26:41 PM UTC-5, FrankHB1989 wrote:
在 2017年12月3日星期日 UTC+8上午11:47:26,Nicol Bolas写道:
On Saturday, December 2, 2017 at 10:29:32 PM UTC-5, FrankHB1989 wrote:
在 2017年12月3日星期日 UTC+8上午5:31:45,Michael Kilburn写道:
On Sat, Dec 2, 2017 at 6:45 AM, FrankHB1989 <frank...@gmail.com> wrote:
在 2017年12月2日星期六 UTC+8下午4:25:29,Michael Kilburn写道:

Why is this a problem for portability in the real world?

In reality, the level of portability on high-level languages (like C++) and platform-dependent solutions has difference of a factor N where N is how many radically different categories of platforms have to support. If you need only portability across several platforms look alike each other enough, and there is sufficient documentation to elide the minor differences, then N ≈ 1, and you won't have too much extra work to do besides some common assumptions from these platforms, so the process is still "as-if" they are portable after the extra work and the result would kept reasonably reusable.

Sorry, this is really not my cases in reality... and I found nowhere to put the combined description of portable behavior other than the language standards.

OK, consider a program that takes a number as input and will perform recursion based on that number; larger numbers mean more recursion. Is that program portable?

Right now, the answer as far as the standard is concerned is "yes". Sure, for each platform, there is some maximum number that you can provide, beyond which you'll exhaust the available resources. But the program itself is portable.

You're effectively saying that you want the program to be considered non-portable, simply because it could exhaust resources. That is, a program is not "portable" to you unless it cannot overflow the stack.

This is not a reasonable definition of portability. It is certainly not a useful one...


The usefulness depends.

It is easy to define. It can be clearly precise. The term strictly conforming program in ISO C captures the intention exactly.

Besides the property of the definition itself, instances also count. A strictly conforming C program is not so useful mostly because ISO C requires too few so almost any real program does not fall in the category. The similar ones in C++ are hopefully not so limited.

This sounds very much like people who want `int` to be required to be exactly 32-bits, 2's complement, and claim that if it isn't, then they cannot write portable programs. Not everything has to be well-defined in order for a program to be portable.
It isn't. If there is no exact 32-bits 2's complement integer, use of other integer (correctly) does not incur extra UB directly. The extra work to remain portability is quite limited and trivial: to test all options of integer representations allowed by the standard, using conditional inclusion, etc. Note one reason of adding `std::int32_t` to the standard is to prevent reinvented wheels in vain; that is also different to my case where there is almost no reliable wheels unless the standard has required it.

(BTW, I prefer `std::ptrdiff_t`, `std::intptr_t` and `std::int_leastN_t` over `int` or `intN_t` in the sense of portability as possible, for significance of semantic properties.)

Detailed mechanism would bring more persuasiveness, but let it to be in std-proposals. First let's try proving it false about feasibility here, if you think it necessary.

Second, UB effect the whole program. To keep other thread work as expected is the practical need. In reality, unless there is only one thread, it is not likely that all threads have to died in same manner.

For costs... at least they are not easy to be zero. It is probably not useful for freestanding implementations as in hosted implementations, so it may be excluded there. Otherwise, I hope it can be afford by mainstream implementations.

To what end? What would be accomplished by this which cannot otherwise be done?


Conformance, if you insist other difference is not important in reality.

"Conformance" to what?

The language specification.

In the worst case, if the portability is guaranteed in such manner but it still not meet my requirements, as the vendor, I can at most fork the language specification to derive a dialect, not the language + POSIX + ISA documents + other messes...

(Yes, this is about the usability of the spec itself.)



FrankHB1989

unread,
Dec 3, 2017, 12:53:36 PM12/3/17
to ISO C++ Standard - Discussion


在 2017年12月4日星期一 UTC+8上午1:18:24,Thiago Macieira写道:
On Sunday, 3 December 2017 09:05:32 PST FrankHB1989 wrote:
> I think I've misread your previous words. As of paging, your points are
> most valid. But pages (backed by RAM or not) still do not solve the problem
> magically. Paging is not the right tool to solve the problem in nature
> because it has too coarse granularity in allocation. The original point
> here is that MMU involves in address translation, not necessarily concrete
> management mechanism like paging. Segmentation can actually do something
> further for this problem, but it do incur some other unnecessary complexity
> and inflexibility. I'd expect some other internal state exported by MMU
> available differently from current ISA to aid runtime check of frames of
> activation records (*not necessarily the stack*) in some
> hardware-accelerated way.

Explain a little more. What would you like to see done that can't be done
right now? What purpose would this serve? What could applications do that they
can't right now?
As said previously, to write implementation of a language (at least) with first-class continuations in direct (non-trampolined, non-CPS transformed) style without construction of software stack. Unless proper tail call is required, it is not implementable in a portable way. Note manual transformation usually involve *huge* amount of work.

I've thought this was too difficult to achieve for about conformance requirements, so I weakened the requirements later.

Thiago Macieira

unread,
Dec 3, 2017, 12:54:42 PM12/3/17
to std-dis...@isocpp.org
On Sunday, 3 December 2017 09:32:02 PST FrankHB1989 wrote:
> > Unwinding the stack means running code *with* that stack. If the stack is
> > exhausted, the unwinding won't work. Even completely noexcept destructors
> > may
> > call functions and create new automatic-storage variables. That requires
> > stack. If you could add more memory to the stack, you wouldn't have
> > exhausted
> > it in the first place.
> >
> No. Stack unwinding is mandated, but not the stack. It only means the code
> might be run *as-if* on that stack. I don't see why it can't be simulated
> where the runtime has the whole control, besides interops/compatibility
> issues.

The code is not on the stack. The code is in the code pages.

The stack is what stack pointer register points to. If you point to the actual
stack, you're on that stack. If you point elsewhere, you're not on the actual
stack, but then you can't unwind it either.

Are you suggesting copying some pages off the top of the stack (lowest
addresses) to some altstack place where the destructors could run?

> > Not to mention that if the stack exhausted, then it's corrupt. Trying to
> > unwind a corrupt stack is GIGO - garbage in, garbage out.
> >
> This is the status quo, a true problem to be resolved.

We're saying it's not. Stack overflows do not usually happen in normal
applications. When they happen, they're usually due to bugs in the code in the
first place (unbounded recursion). So if it's due to bugs, there's no
guarantee that providing a recovery mechanism will help and there's also no
need for it as non-buggy code doesn't have the problem in the first place.

> > Ever heard of POSIX? Single UNIX Specification and XPG?
>
> Seriously, do you really rely on the portability *across *platforms in
> reality? Not to mention freestanding implementations, they don't even
> applicable well for Windows...

Yes, I do. Oh, sure, there's a lot of bugs everywhere that I need to work
around. But the vast majority of the the Unix code I write is just plainly
cross-platform.

Freestanding doesn't apply to me.

> Another question... are there working drafts of them available conveniently
> as WG21 documents?

Working drafts of what?

Thiago Macieira

unread,
Dec 3, 2017, 12:59:06 PM12/3/17
to std-dis...@isocpp.org
On Sunday, 3 December 2017 09:53:36 PST FrankHB1989 wrote:
> > Explain a little more. What would you like to see done that can't be done
> > right now? What purpose would this serve? What could applications do that
> > they
> > can't right now?
>
> As said previously, to write implementation of a language (at least) with
> first-class continuations in direct (non-trampolined, non-CPS transformed)
> style without construction of software stack. Unless proper tail call is
> required, it is not implementable in a portable way. Note manual
> transformation usually involve *huge* amount of work.

Sorry, this went over my head. Can you elaborate on what functionality you
need from the language, OS or processor/ISA to achieve that implementation
that you don't have today?

FrankHB1989

unread,
Dec 3, 2017, 1:14:12 PM12/3/17
to ISO C++ Standard - Discussion


在 2017年12月4日星期一 UTC+8上午1:54:42,Thiago Macieira写道:
On Sunday, 3 December 2017 09:32:02 PST FrankHB1989 wrote:
> > Unwinding the stack means running code *with* that stack. If the stack is
> > exhausted, the unwinding won't work. Even completely noexcept destructors
> > may
> > call functions and create new automatic-storage variables. That requires
> > stack. If you could add more memory to the stack, you wouldn't have
> > exhausted
> > it in the first place.
> >
> No. Stack unwinding is mandated, but not the stack. It only means the code
> might be run *as-if* on that stack. I don't see why it can't be simulated
> where the runtime has the whole control, besides interops/compatibility
> issues.

The code is not on the stack. The code is in the code pages.

The stack is what stack pointer register points to. If you point to the actual
stack, you're on that stack. If you point elsewhere, you're not on the actual
stack, but then you can't unwind it either.

Are you suggesting copying some pages off the top of the stack (lowest
addresses) to some altstack place where the destructors could run?

OK, *with* the stack is right. Sorry for my typo.

I did not suggest the concrete way to implement. I did suggest it can be out of current stack, so this can be a choice. Another way is to transform the destructor calls to stackless style.

(As I said previously, if I merely want to make other threads alive, no destructor needs to be called. It is certainly dangerous for shared resources, though.)

FrankHB1989

unread,
Dec 3, 2017, 1:19:43 PM12/3/17
to ISO C++ Standard - Discussion


在 2017年12月4日星期一 UTC+8上午1:59:06,Thiago Macieira写道:
On Sunday, 3 December 2017 09:53:36 PST FrankHB1989 wrote:
> > Explain a little more. What would you like to see done that can't be done
> > right now? What purpose would this serve? What could applications do that
> > they
> > can't right now?
>
> As said previously, to write implementation of a language (at least) with
> first-class continuations in direct (non-trampolined, non-CPS transformed)
> style without construction of software stack. Unless proper tail call is
> required, it is not implementable in a portable way. Note manual
> transformation usually involve *huge* amount of work.

Sorry, this went over my head. Can you elaborate on what functionality you
need from the language, OS or processor/ISA to achieve that implementation
that you don't have today?

If guarantee of proper tail calls is not the functionality... for example, a call/cc variant meets zero overhead principle?

FrankHB1989

unread,
Dec 3, 2017, 1:31:17 PM12/3/17
to ISO C++ Standard - Discussion


在 2017年12月4日星期一 UTC+8上午1:54:42,Thiago Macieira写道:
On Sunday, 3 December 2017 09:32:02 PST FrankHB1989 wrote:
It is considered not usual in imperative style code (though it still seldom occurs). It is actually quite normal in so-called functional code. And when I am implement a language deliberately with such style, I can't enforce the limitation on users' code as same as C++; that would kill most of usability. The workarounds are either obviously not portable or with obvious overhead. This is better fixed in the language implementing it.

As of my weakened requirements... true, it is not so significant merely to workaround bugs.

> > Ever heard of POSIX? Single UNIX Specification and XPG?
>
> Seriously, do you really rely on the portability *across *platforms in
> reality? Not to mention freestanding implementations, they don't even
> applicable well for Windows...

Yes, I do. Oh, sure, there's a lot of bugs everywhere that I need to work
around. But the vast majority of the the Unix code I write is just plainly
cross-platform.

Freestanding doesn't apply to me.
 
Sorry, that is not my case.

> Another question... are there working drafts of them available conveniently
> as WG21 documents?

Working drafts of what?

POSIX/SUS/XPG...

FrankHB1989

unread,
Dec 3, 2017, 1:36:46 PM12/3/17
to ISO C++ Standard - Discussion


在 2017年12月4日星期一 UTC+8上午2:31:17,FrankHB1989写道:
I should be talking about the code leads to stack overflow, not the stack overflow itself. It the stack overflow in the end, the implementation has bugs.

Thiago Macieira

unread,
Dec 3, 2017, 1:37:57 PM12/3/17
to std-dis...@isocpp.org
On Sunday, 3 December 2017 10:19:42 PST FrankHB1989 wrote:
> > Sorry, this went over my head. Can you elaborate on what functionality you
> > need from the language, OS or processor/ISA to achieve that implementation
> > that you don't have today?
> >
> If guarantee of proper tail calls is not the functionality... for example,
> a call/cc variant meets zero overhead principle?

What's overhead for you in a function call? Obviously a function call needs to
store somewhere the return address. In some architectures, there's also some
other state that needs to be stored. That's not overhead, that's the normal
part.

Thiago Macieira

unread,
Dec 3, 2017, 1:38:58 PM12/3/17
to std-dis...@isocpp.org
On Sunday, 3 December 2017 10:14:12 PST FrankHB1989 wrote:
> (As I said previously, if I merely want to make other threads alive, no
> destructor needs to be called. It is certainly dangerous for shared
> resources, though.)

Then just suspend this thread forever if it overflows its stack.

Any locked mutexes and other contexts will stay locked forever too.

Thiago Macieira

unread,
Dec 3, 2017, 1:41:46 PM12/3/17
to std-dis...@isocpp.org
On Sunday, 3 December 2017 10:31:16 PST FrankHB1989 wrote:
> It is considered not usual in imperative style code (though it still
> seldom occurs). It is actually quite normal in so-called functional code.
> And when I am implement a language deliberately with such style, I can't
> enforce the limitation on users' code as same as C++; that would kill most
> of usability. The workarounds are either obviously not portable or with
> obvious overhead. This is better fixed in the language implementing it.

Sorry, any code that recurses on anything besides O(1) (if I'm feeling
generous, O(log n)) on the number of elements is buggy. Redesign it.

Nicol Bolas

unread,
Dec 3, 2017, 2:00:02 PM12/3/17
to ISO C++ Standard - Discussion


On Sunday, December 3, 2017 at 12:53:36 PM UTC-5, FrankHB1989 wrote:


在 2017年12月4日星期一 UTC+8上午1:18:24,Thiago Macieira写道:
On Sunday, 3 December 2017 09:05:32 PST FrankHB1989 wrote:
> I think I've misread your previous words. As of paging, your points are
> most valid. But pages (backed by RAM or not) still do not solve the problem
> magically. Paging is not the right tool to solve the problem in nature
> because it has too coarse granularity in allocation. The original point
> here is that MMU involves in address translation, not necessarily concrete
> management mechanism like paging. Segmentation can actually do something
> further for this problem, but it do incur some other unnecessary complexity
> and inflexibility. I'd expect some other internal state exported by MMU
> available differently from current ISA to aid runtime check of frames of
> activation records (*not necessarily the stack*) in some
> hardware-accelerated way.

Explain a little more. What would you like to see done that can't be done
right now? What purpose would this serve? What could applications do that they
can't right now?
As said previously, to write implementation of a language (at least) with first-class continuations in direct (non-trampolined, non-CPS transformed) style without construction of software stack.

And why do you need to do that? This sounds very much like defining yourself into a problem. What's wrong with a software stack? Interpreters have been using them for years without problems. What's wrong with trampolined or CPS-transformed style?

Why do you need a way to do this in a non-platform-specific way? After all, the only reason I can see for wanting to do any of this is if you're trying to use JIT or something similar. And JIT has to be platform-specific, by its very nature.

A good motivation for an idea comes from practical problems; your problem here seems like an artificially created one, used to justify this particular solution.

Nicol Bolas

unread,
Dec 3, 2017, 2:22:17 PM12/3/17
to ISO C++ Standard - Discussion
On Sunday, December 3, 2017 at 12:47:09 PM UTC-5, FrankHB1989 wrote:
在 2017年12月3日星期日 UTC+8下午11:42:56,Nicol Bolas写道:
On Saturday, December 2, 2017 at 11:26:41 PM UTC-5, FrankHB1989 wrote:
在 2017年12月3日星期日 UTC+8上午11:47:26,Nicol Bolas写道:
On Saturday, December 2, 2017 at 10:29:32 PM UTC-5, FrankHB1989 wrote:
在 2017年12月3日星期日 UTC+8上午5:31:45,Michael Kilburn写道:
On Sat, Dec 2, 2017 at 6:45 AM, FrankHB1989 <frank...@gmail.com> wrote:
在 2017年12月2日星期六 UTC+8下午4:25:29,Michael Kilburn写道:

Why is this a problem for portability in the real world?

In reality, the level of portability on high-level languages (like C++) and platform-dependent solutions has difference of a factor N where N is how many radically different categories of platforms have to support. If you need only portability across several platforms look alike each other enough, and there is sufficient documentation to elide the minor differences, then N ≈ 1, and you won't have too much extra work to do besides some common assumptions from these platforms, so the process is still "as-if" they are portable after the extra work and the result would kept reasonably reusable.

Sorry, this is really not my cases in reality... and I found nowhere to put the combined description of portable behavior other than the language standards.

OK, consider a program that takes a number as input and will perform recursion based on that number; larger numbers mean more recursion. Is that program portable?

Right now, the answer as far as the standard is concerned is "yes". Sure, for each platform, there is some maximum number that you can provide, beyond which you'll exhaust the available resources. But the program itself is portable.

You're effectively saying that you want the program to be considered non-portable, simply because it could exhaust resources. That is, a program is not "portable" to you unless it cannot overflow the stack.

This is not a reasonable definition of portability. It is certainly not a useful one...


The usefulness depends.

It is easy to define.

What is "easy to define"? The limits of the stack? How could you possibly define that?

The behavior of a stack overflow? There's no effective way to define that; if there was, then `std::thread` would have a way to terminate the given thread. It doesn't have such a thing, because the idea that a thread can be terminated by outside causes is not something the C++ object model can handle.

And what of forward progress guarantees under such a system? If a thread can just stop, for whatever reason, C++ just doesn't work anymore.

Nothing of what you're talking about can be deemed "easy to define".

It can be clearly precise. The term strictly conforming program in ISO C captures the intention exactly.

Besides the property of the definition itself, instances also count. A strictly conforming C program is not so useful mostly because ISO C requires too few so almost any real program does not fall in the category.

How do you figure?

The similar ones in C++ are hopefully not so limited.

This sounds very much like people who want `int` to be required to be exactly 32-bits, 2's complement, and claim that if it isn't, then they cannot write portable programs. Not everything has to be well-defined in order for a program to be portable.
It isn't. If there is no exact 32-bits 2's complement integer, use of other integer (correctly) does not incur extra UB directly. The extra work to remain portability is quite limited and trivial: to test all options of integer representations allowed by the standard, using conditional inclusion, etc. Note one reason of adding `std::int32_t` to the standard is to prevent reinvented wheels in vain; that is also different to my case where there is almost no reliable wheels unless the standard has required it.

(BTW, I prefer `std::ptrdiff_t`, `std::intptr_t` and `std::int_leastN_t` over `int` or `intN_t` in the sense of portability as possible, for significance of semantic properties.)

Detailed mechanism would bring more persuasiveness, but let it to be in std-proposals. First let's try proving it false about feasibility here, if you think it necessary.

Second, UB effect the whole program. To keep other thread work as expected is the practical need. In reality, unless there is only one thread, it is not likely that all threads have to died in same manner.

For costs... at least they are not easy to be zero. It is probably not useful for freestanding implementations as in hosted implementations, so it may be excluded there. Otherwise, I hope it can be afford by mainstream implementations.

To what end? What would be accomplished by this which cannot otherwise be done?


Conformance, if you insist other difference is not important in reality.

"Conformance" to what?

The language specification.

My question was "What would be accomplished by this which cannot otherwise be done?" Your answer was "Conformance [to] the language specification".

That's not an answer.

Compilers already conform to the language specification. Programs already conform to the language specification. You're talking about changing the language specification to have different conformance requirements. Such changes will not increase "conformance".

So again, what is it that you're trying to gain from this?

In the worst case, if the portability is guaranteed in such manner but it still not meet my requirements, as the vendor, I can at most fork the language specification to derive a dialect, not the language + POSIX + ISA documents + other messes...

Nothing is stopping them from creating such a fork right now. If an IHV wanted to, they could fork the spec and specify all kinds of behavior that was implementation-defined or even undefined. The possibility of stack overflows and their behavior could easily be part of such additions.

So again, defining this will not improve things in this way.

FrankHB1989

unread,
Dec 3, 2017, 8:53:12 PM12/3/17
to ISO C++ Standard - Discussion


在 2017年12月4日星期一 UTC+8上午2:37:57,Thiago Macieira写道:
On Sunday, 3 December 2017 10:19:42 PST FrankHB1989 wrote:
> > Sorry, this went over my head. Can you elaborate on what functionality you
> > need from the language, OS or processor/ISA to achieve that implementation
> > that you don't have today?
> >
> If guarantee of proper tail calls is not the functionality... for example,
> a call/cc variant meets zero overhead principle?

What's overhead for you in a function call? Obviously a function call needs to
store somewhere the return address. In some architectures, there's also some
other state that needs to be stored. That's not overhead, that's the normal
part.
You need to study the subject a little more... Traditional call/cc would need to copy the whole stack without something more magical. (And strictly speaking, the arguments to be "called" are not usual functions.) In general, I don't need most of the frames it would capture, but I have no way to specify in the interface. So this directly violate the principle a lot, as well as semantic problems like how to interact with RAII. (P0534R0 excludes full continuations for the very same reason.) The overhead seems to be theoretically unavoidable so I would only consider the variant which allows me to specify a portion of top frames I needed. But even this one is not easy to implement natively (with platform-dependent code).

BTW, a limited case of the feature involved is called "exception handling" (which has limited the direction of continuation propagation)... I'm glad to see you treat it as a function call to keep the core language clean :)

FrankHB1989

unread,
Dec 3, 2017, 8:58:19 PM12/3/17
to ISO C++ Standard - Discussion


在 2017年12月4日星期一 UTC+8上午2:38:58,Thiago Macieira写道:
On Sunday, 3 December 2017 10:14:12 PST FrankHB1989 wrote:
> (As I said previously, if I merely want to make other threads alive, no
> destructor needs to be called. It is certainly dangerous for shared
> resources, though.)

Then just suspend this thread forever if it overflows its stack.

Any locked mutexes and other contexts will stay locked forever too.

This does not sound sane. It leaks resources not shared, and it is problematic on shared resources. The locks it holds would probably block others so this can be another can of worms...

FrankHB1989

unread,
Dec 3, 2017, 9:02:04 PM12/3/17
to ISO C++ Standard - Discussion


在 2017年12月4日星期一 UTC+8上午2:41:46,Thiago Macieira写道:
On Sunday, 3 December 2017 10:31:16 PST FrankHB1989 wrote:
> It is considered not usual in imperative style code (though it still
> seldom occurs). It is actually quite normal in so-called functional code.
> And when I am implement a language deliberately with such style, I can't
> enforce the limitation on users' code as same as C++; that would kill most
> of usability. The workarounds are either obviously not portable or with
> obvious overhead. This is better fixed in the language implementing it.

Sorry, any code that recurses on anything besides O(1) (if I'm feeling
generous, O(log n)) on the number of elements is buggy. Redesign it.

For example, tree traversing (even with known limited depth)?

FrankHB1989

unread,
Dec 3, 2017, 9:24:10 PM12/3/17
to ISO C++ Standard - Discussion


在 2017年12月4日星期一 UTC+8上午3:00:02,Nicol Bolas写道:


On Sunday, December 3, 2017 at 12:53:36 PM UTC-5, FrankHB1989 wrote:


在 2017年12月4日星期一 UTC+8上午1:18:24,Thiago Macieira写道:
On Sunday, 3 December 2017 09:05:32 PST FrankHB1989 wrote:
> I think I've misread your previous words. As of paging, your points are
> most valid. But pages (backed by RAM or not) still do not solve the problem
> magically. Paging is not the right tool to solve the problem in nature
> because it has too coarse granularity in allocation. The original point
> here is that MMU involves in address translation, not necessarily concrete
> management mechanism like paging. Segmentation can actually do something
> further for this problem, but it do incur some other unnecessary complexity
> and inflexibility. I'd expect some other internal state exported by MMU
> available differently from current ISA to aid runtime check of frames of
> activation records (*not necessarily the stack*) in some
> hardware-accelerated way.

Explain a little more. What would you like to see done that can't be done
right now? What purpose would this serve? What could applications do that they
can't right now?
As said previously, to write implementation of a language (at least) with first-class continuations in direct (non-trampolined, non-CPS transformed) style without construction of software stack.

And why do you need to do that? This sounds very much like defining yourself into a problem. What's wrong with a software stack? Interpreters have been using them for years without problems. What's wrong with trampolined or CPS-transformed style?
This is nothing wrong to be merely implementable. But this is of (relatively) bad quality, about both performance and maintenance of the code, with a quite significant factor. And it's a pity to work hard to just bypass the stack provided by underling implementations.


Why do you need a way to do this in a non-platform-specific way? After all, the only reason I can see for wanting to do any of this is if you're trying to use JIT or something similar. And JIT has to be platform-specific, by its very nature.

I don't argue that it should be totally non-platform-specific. If I need to target native ISA directly, certainly it should be platform-specific. It has to be platform-specific to utilize more power of the platform. But that is only one case. I suggest it should be more portable than the status quo for other cases, provided a common abstraction fit to several well-known problems, to save the manpower of duplication of error-prone works like manual CPS transformation. This is also one reason we need coroutines or resumable functions in the language.

A good motivation for an idea comes from practical problems; your problem here seems like an artificially created one, used to justify this particular solution.
Both the need to saving the amount of work and the need of abstraction are practical.

And consider alternatives. I can roll one more my own IR to be compiled merely to overcome the problem, which is also more or less the status quo in methodology used by other projects. But it is not specific only to my case. I can also reuse IRs from other projects, but I still have to port them by myself once there is a backend target and/or a feature I need is not well-supported directly, which may involve even more work compared to a fresh IR design by myself. Why it should be implemented separatedly instead of built in a high-level general-purposed language like C++?




Thiago Macieira

unread,
Dec 3, 2017, 9:36:56 PM12/3/17
to std-dis...@isocpp.org
On Sunday, 3 December 2017 18:02:03 PST FrankHB1989 wrote:
> > Sorry, any code that recurses on anything besides O(1) (if I'm feeling
> > generous, O(log n)) on the number of elements is buggy. Redesign it.
> >
> For example, tree traversing (even with known limited depth)?

I was talking about unbounded n.

If you know the maximum value of n, then you need to do a platform-specific
check.

Thiago Macieira

unread,
Dec 3, 2017, 9:43:56 PM12/3/17
to std-dis...@isocpp.org
On Sunday, 3 December 2017 17:53:12 PST FrankHB1989 wrote:
> You need to study the subject a little more... Traditional call/cc would
> need to copy the *whole *stack without something more magical
> <http://wiki.c2.com/?ContinuationImplementation>