Do we need getcontext / setcontext (or an equivalent) in C?

[Philipp Klaus Krause]

POSIX used to have the functions getcontext and setcontext. Most Unices
still provide them.

They worked like setjmp / longjmp except that longjmp can only be called
from the function that called setjmp, while getcontext / setcontext has
no such restriction.

From an implementation side that means that setjmp / longjmp can be
implemented much more efficiently, with a small jmp_buf, while
getcontext / setcontext handles a copy of the whole stack in their
jmp_buf equivalent.

On the other hand, the functionality provided by getcontext / setcontext
is available in virtually all hardware, has important use cases (as can
be seen by the various uses of call/cc in functional languages, with
efficient translation of functional languages to C being another use
case) and is currently not exposed in C.

The only problem I see is that it cannot be implemented by
implementations that do not use a stack for local variables (but those
are already non-conforming by not allowing recursion), but since all
those I know of are freestanding, it could be worked around by only
requiring the etcontext / setcontext-like functionality for hosted
implementations.

Philipp

[David Brown]

What purpose would these functions serve? I haven't had any use for
these kinds of thing, so I could well be missing something. But it
appears to be a way to do manual thread switching within a single
process - a throwback from the days before *nix systems commonly had
good thread support, and from when multi-core hardware was rare and
expensive.

Why would you use these functions rather than multiple threads?

[James Kuyper]

On my desktop system, "man getcontext" displays a description that
includes a "CONFORMING TO" section which says:

"SUSv2, POSIX.1-2001. POSIX.1-2008 removes the specification of
getcontext(), citing portability issues, and recommending that
applications be rewritten to use POSIX threads instead."

I'd recommend following that recommendation.

[Casper H.S. Dik]

I do not remember any implementation which saved the whole stack;
they are often used by the implementation of segjmp/longjmp and
when returning from a signal handler. (Signal handler can
run on a separate stack.) It generally had the same restrictions
as longjmp: the part where we are jumping to must be "live": i.e.,
not unwounded.

>I'd recommend following that recommendation.

Indeed!

Casper

[Jakob Bohm]

Perhaps the common use is CoRoutines (not threads).

Another use mentioned by some is to "longjump" out of a signal handler
(those are generally invoked in response to a per thread event such as
SIGILL).



Enjoy

Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded

[David Brown]

I was thinking about that too. (Or, roughly equivalently, cooperative
multi-threading.) A quick search shows that the GNU Portable Threads
library for implementing fibers uses getcontext, setcontext and swapcontext.


I am not sure there would be much use in standardising something as
low-level as setcontext/getcontext. I think it would be better to
standardise fiber or coroutine support, similar to the C11 thread
support, and let implementations worry about whether they want to use OS
kernel support, setcontext/getcontext, assembly code, or whatever. But
first it would make sense to see how C++20 coroutines play out in
practice - let C++ go first, and learn from what works or does not work.

[Keith Thompson]

Philipp Klaus Krause <p...@spth.de> writes:
> POSIX used to have the functions getcontext and setcontext. Most Unices
> still provide them.
>
> They worked like setjmp / longjmp except that longjmp can only be called
> from the function that called setjmp, while getcontext / setcontext has
> no such restriction.

setjmp and longjmp also have no such restriction.

Here's an example from the standard, N1570 7.13.2.1p5. (The example is
about possibly squandering memory associated with a VLA.)

#include <setjmp.h>
jmp_buf buf;
void g(int n);
void h(int n);
int n = 6;

void f(void)
{
int x[n]; // valid: f is not terminated
setjmp(buf);
g(n);
}
void g(int n)
{
int a[n]; // a may remain allocated
h(n);
}
void h(int n)
{
int b[n]; // b may remain allocated
longjmp(buf, 2); // might cause memory loss
}

--
Keith Thompson (The_Other_Keith) Keith.S.T...@gmail.com
Working, but not speaking, for Philips Healthcare
void Void(void) { Void(); } /* The recursive call of the void */

[Philipp Klaus Krause]

Am 18.02.20 um 19:52 schrieb Keith Thompson:

> Philipp Klaus Krause <p...@spth.de> writes:
>> POSIX used to have the functions getcontext and setcontext. Most Unices
>> still provide them.
>>
>> They worked like setjmp / longjmp except that longjmp can only be called
>> from the function that called setjmp, while getcontext / setcontext has
>> no such restriction.
>
> setjmp and longjmp also have no such restriction.
>
> Here's an example from the standard, N1570 7.13.2.1p5. (The example is
> about possibly squandering memory associated with a VLA.)
>

longjmp is still called from f in your example, indirectly.

But full support for continuations would allow the equivalent of longjmp
after the function that called setjmp returns.

Philipp

[Philipp Klaus Krause]

Am 18.02.20 um 18:40 schrieb David Brown:

>
> I was thinking about that too.  (Or, roughly equivalently, cooperative
> multi-threading.)  A quick search shows that the GNU Portable Threads
> library for implementing fibers uses getcontext, setcontext and
> swapcontext.
>
>
> I am not sure there would be much use in standardising something as
> low-level as setcontext/getcontext.  I think it would be better to
> standardise fiber or coroutine support, similar to the C11 thread
> support, and let implementations worry about whether they want to use OS
> kernel support, setcontext/getcontext, assembly code, or whatever.  But
> first it would make sense to see how C++20 coroutines play out in
> practice - let C++ go first, and learn from what works or does not work.
>

IMO, C is a high-level language that, where it makes sense, provides
access to low-level functionality. Given that continuations are provided
in many functional high-level languages (e.g. Scheme), and used there,
it IMO makes sense to also expose this functionality in C.

Philipp

[Philipp Klaus Krause]

Am 18.02.20 um 11:07 schrieb David Brown:

>
> What purpose would these functions serve? I haven't had any use for
> these kinds of thing, so I could well be missing something.

One important use case is to make translation of functional languages to
C easier.

E.g. a Scheme-to-C compiler is quite straightforward (both in
implementation and in how the generated C code looks), as long as
call/cc is not fully supported.
But once full support for call/cc is implemented, the compiler has to
emit nonstandard C or generate C code that looks nothing like what a c C
programmer would write or what the original scheme looked like.
Supporting call/cc usign stadnard C usually also results in using the C
heap to emulate the Scheme stack, while without full call/cc one can use
the C stack (which is more intuitive and much faster).

Philipp

[David Brown]

I am not familiar with Scheme, and while I have used functional
programming languages, I have never translated them to C. Are you
talking about something like lazy evaluation, implemented using
coroutines? I would expect setcontext/getcontext to be very inefficient
for such purposes.

Efficient coroutine support is going to need serious compiler support.

[Ben Bacarisse]

David Brown <david...@hesbynett.no> writes:

> On 19/02/2020 11:03, Philipp Klaus Krause wrote:
>> Am 18.02.20 um 11:07 schrieb David Brown:
>>>
>>> What purpose would these functions serve? I haven't had any use for
>>> these kinds of thing, so I could well be missing something.
>>
>>
>> One important use case is to make translation of functional languages to
>> C easier.
>>
>> E.g. a Scheme-to-C compiler is quite straightforward (both in
>> implementation and in how the generated C code looks), as long as
>> call/cc is not fully supported.

This surprises me. Surely even something a simple as a full closure
value will give rise to strange-looking C.

>> But once full support for call/cc is implemented, the compiler has to
>> emit nonstandard C or generate C code that looks nothing like what a c C
>> programmer would write or what the original scheme looked like.
>> Supporting call/cc usign stadnard C usually also results in using the C
>> heap to emulate the Scheme stack, while without full call/cc one can use
>> the C stack (which is more intuitive and much faster).
>

> I am not familiar with Scheme, and while I have used functional
> programming languages, I have never translated them to C. Are you
> talking about something like lazy evaluation, implemented using
> coroutines?

No, Scheme is not lazy, but it has a built-in function call/cc (call
with current continuation) that, to cut a very long story short, calls a
function with the possibility of immediately "coming back here" with a
result. Combined with the other features of a functional language, this
allows almost any computational pattern to be implemented.

> I would expect setcontext/getcontext to be very inefficient
> for such purposes.

I doubt that efficiency is the main concern here. I think keeping the C
looking like normal C is the main concern. That said, given that
setcontext/getcontext are not seen in normal C, and they behave in a way
that many C programmer would find unusual, I don't think they would
really acheiv`qe the desired result.

> Efficient coroutine support is going to need serious compiler support.

--
Ben.

[David Brown]

I should probably read the Wikipedia article on Scheme - if you think
that would be more useful than writing a fuller reply, please do so.
But isn't it a feature of normal functions that they usually "come back
here with a result" ? Or do you mean they can return immediately with a
"future" type whose real value will be filled in asynchronously from the
calling code? (Or do you mean something else entirely?)

>
>> I would expect setcontext/getcontext to be very inefficient
>> for such purposes.
>
> I doubt that efficiency is the main concern here. I think keeping the C
> looking like normal C is the main concern. That said, given that
> setcontext/getcontext are not seen in normal C, and they behave in a way
> that many C programmer would find unusual, I don't think they would
> really acheiv`qe the desired result.
>

Agreed. I think if you are willing to generate getcontext/setcontext,
you should be willing to generate just about anything! There is some
sense in generating C code that is easy to follow and matches the source
language - it makes it simpler to find where things have gone wrong (for
the person writing the generator) and where things are unexpectedly
inefficient. But for the most part, someone using a Scheme-to-C
generator would be doing so because they want to program in Scheme, not
because they want to read C code. Also, being (relatively) simple to
read and understand the C code does not correlate directly with being
"normal C code". It can be perfectly clear and correspond well to the
original code while simultaneously being very far from the kind of code
anyone would write manually.

[Ben Bacarisse]

Something else entirely. You can "come back here" even from a function
called by the function that is called with the current continuation.
And code can end up with multiple continuations in scope and so can
decide where to "go back to".

>>> I would expect setcontext/getcontext to be very inefficient
>>> for such purposes.
>>
>> I doubt that efficiency is the main concern here. I think keeping the C
>> looking like normal C is the main concern. That said, given that
>> setcontext/getcontext are not seen in normal C, and they behave in a way
>> that many C programmer would find unusual, I don't think they would

>> really [acheive] the desired result.

>
> Agreed. I think if you are willing to generate getcontext/setcontext,
> you should be willing to generate just about anything! There is some
> sense in generating C code that is easy to follow and matches the source
> language - it makes it simpler to find where things have gone wrong (for
> the person writing the generator) and where things are unexpectedly
> inefficient.

And in this regard setcontext/getcontext would work because, whilst not
normal C, they would allow the translation to stick closer to the
source.

On rereading the original post, I think I should not have read "what a C
programmer would write" as "normal C". I think the intent was "what a C
programmer would write iof they had to reflect this unusual (for C)
control flow".

<cut>
--
Ben.

[Philipp Klaus Krause]

Am 18.02.20 um 10:17 schrieb Philipp Klaus Krause:

Actually, I am not sure getcontext / setcontext really work the way I
thought (after discussion on comp.unix.programmer).

So C++ callcc()
(https://www.boost.org/doc/libs/1_72_0/libs/context/doc/html/context/cc.html
and http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0534r3.pdf)
would probably be a better example than getcontext() for the
functionality I'd like to see in C.

[Philipp Klaus Krause]

Am 20.02.20 um 09:20 schrieb Philipp Klaus Krause:

>
> Actually, I am not sure getcontext / setcontext really work the way I
> thought (after discussion on comp.unix.programmer).
>
> So C++ callcc()
> (https://www.boost.org/doc/libs/1_72_0/libs/context/doc/html/context/cc.html
> and http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0534r3.pdf)
> would probably be a better example than getcontext() for the
> functionality I'd like to see in C.
>

On second look, I am not sure C++ callcc() is good enough either. While
very close to Schem call/cc in syntax, it still seems less powerful.

[David Brown]

This is clearly an interesting concept that I currently know very little
about. I could ask you more, but it would be totally unfair (as well as
off-topic for the group). You have given me some pointers and ideas
here, for which you have my thanks, but I am going to have to read up
more on the topic before I can usefully contribute more in this thread.
Maybe I'll be back when I am more knowledgable and more sleep-deprived
from getting lost on the net again...

[Florian Weimer]

* Philipp Klaus Krause:

> POSIX used to have the functions getcontext and setcontext. Most Unices
> still provide them.

These facilities are very restricted nowadays because they do not
allow resuming a computation on another thread, different from the
thread on which it was suspended. Thread-local variables did not
exist (at least not in wide use and efficient implementations) when
getcontext/setcontext were first conceived, so it's no wonder there
are severe interoperability problems.

C++ coroutines do not have this problem because they are stackless.
Coroutines still might not get the set of TLS variables they expect
after resumption, but at least there is no immediate undefined
behavior if thread switching is involved.

[Florian Weimer]

* David Brown:

> I am not familiar with Scheme, and while I have used functional
> programming languages, I have never translated them to C. Are you
> talking about something like lazy evaluation, implemented using
> coroutines? I would expect setcontext/getcontext to be very inefficient
> for such purposes.

Yes, setcontext/getcontext are basically used as a fallback if you
want to implement context switching without an existing assembler
implementation. They were specified to save and restore the process
signal mask, which makes them quite slow. Perhaps slightly faster
than a signal through a condition variable, but probably not by much.

The trouble is anyone can write assembler code for coroutine
suspend/resume, but the tricky part is making sure that it's actually
correct in the presence of compiler optimizations. (We just went
through that with the context switching implementation in C++ for a
new architecture.) Using setcontext/getcontext is generally a safe
bet (if you always resume on the same thread), but it's really quite
slow.

> Efficient coroutine support is going to need serious compiler support.

Yes, and there could be ABI implications for TLS if the goal is to
suspend with non-coroutine functions on the stack (as it is possible
with setcontext/getcontext, but not with C++ coroutines).

[Jakob Bohm]

On 2020-03-10 22:24, Florian Weimer wrote:
> * David Brown:
>
>> I am not familiar with Scheme, and while I have used functional
>> programming languages, I have never translated them to C. Are you
>> talking about something like lazy evaluation, implemented using
>> coroutines? I would expect setcontext/getcontext to be very inefficient
>> for such purposes.
>
> Yes, setcontext/getcontext are basically used as a fallback if you
> want to implement context switching without an existing assembler
> implementation. They were specified to save and restore the process
> signal mask, which makes them quite slow. Perhaps slightly faster
> than a signal through a condition variable, but probably not by much.
>

The cost of changing the process(thread) signal mask depends heavily
on the implementation of that mask, specifically if it involves a
kernel transition or other slow serialization. It is certainly
conceivable to design a POSIX system that stores the signal mask in
a libc global/thread variable and checks it in an async libc function
invoked upon signal reception (that libc function would then decide
if it should call a handler, queue the signal, abnormally terminate
the process or simply ignore the signal).

> The trouble is anyone can write assembler code for coroutine
> suspend/resume, but the tricky part is making sure that it's actually
> correct in the presence of compiler optimizations. (We just went
> through that with the context switching implementation in C++ for a
> new architecture.) Using setcontext/getcontext is generally a safe
> bet (if you always resume on the same thread), but it's really quite
> slow.
>
>> Efficient coroutine support is going to need serious compiler support.
>
> Yes, and there could be ABI implications for TLS if the goal is to
> suspend with non-coroutine functions on the stack (as it is possible
> with setcontext/getcontext, but not with C++ coroutines).
>

What is even more important is to make a cross-platform API decision
if setcontext/getcontext should change the API level TLS context or
leave that context as referring to the thread that actually runs the
code on both sides of the context switch.

Knowing which it is would allow setcontext/getcontext programs to
know for sure which behavior to expect and program for.

Unless there is a widely implemented existing rule, it would be
preferable to avoid the common POSIX interpretation mistake of making
things per process simply because the sentence mentioning "process"
was written before threads were part of the overall concept.

[Spiros Bousbouras]

It is less powerful. Page 3 of ...p0534r3.pdf says

Continuations that can be called multiple times are named full
continuations.
One-shot continuations can only resumed once: once resumed, a one-shot
continuation is invalidated.
Full continuations are not considered in this proposal because of their
nature, which is problematic in C++. Full continuations would require
copies of the stack (including all stack variables), which would violate
C++'s RAII pattern.

Scheme continuations can be called as many times as one wants.