Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Practical examples of coroutines being useful?
47 views
Skip to first unread message
Juha Nieminen
Jun 28, 2022, 9:56:16 AM6/28/22
to
One of the strangest and perhaps most surprising (and even controversial)
new features of C++20 are coroutines. This post is not about bashing them,
criticizing them, questioning the motivation for adding them to the
already complex language, or anything like that. Let's put all that aside
for a moment, and just focus on what they can actually be useful for.
Practical examples.
I had never heard of "coroutines" before, and in fact it took me quite
a while to understand what exactly they are, and what they are used for.
They are often described as kind of emulating cooperative multitasking
(similarly to how multitasking worked in Windows 3.x and MacOS prior
to X.)
I would, however (and from what I understand), describe them more like a
way to more easily store the state of a function (or group of functions)
so that the function can be exited at any point that it wants, and then
execution resumed later from that point (when the rest of the code
tells it to resume execution.)
(Well, I suppose that's exactly what "cooperative multitasking" means,
but I think describing it as I did is much clearer than using that term.)
Of course doing this was already possible even before C++20, but it
required a lot more manual labor, a lot more coding. The coroutines
help automatize the process and make it much easier.
So where is this actually useful? I can think of two examples.
Are there more?
Example 1:
You are writing a function that decompresses some compressed input format.
The decompressed data should be written to a buffer of a given size.
Whenever the buffer gets full, this decompressor function should return
it to the calling code for it to consume it. After the calling code has
consumed it, the decompression should continue from where it left.
Since compression formats are complicated there may be literally dozens
of places in the decompression algorithm where the buffer may get full,
in very varied situations, sometimes even inside deeply nested loops,
coroutines make this process a lot easier because the code can "yield"
at any point, anywhere (even inside deeply nested loops), and then
continue from that point as if nothing has happened (other than the
buffer having been emptied).
Example 2:
A class/module that parses command-line arguments. Since the arguments may
appear in many different forms, the parser is done so that it will be
called with individual command-line parameters (which may end in the null
character, or a whitespace character, or something else). In other words
for each individual parameter the calling code calls the parser giving it
that parameter.
However, not all command-line parameters are singular, as in consisting
of only one "word". A parameter may be followed by one or more strings
(such as file names, etc.) The calling code doesn't know if a particular
parameter is just data for the previous parameter or not, it just feeds
them one by one to the parser.
Thus, when the parser encounters a parameter that expects further
parameters, it needs to remember its state between calls. (It needs to
remember which particular command-line parameter it's currently
parsing, and any additional parameters given to it already.)
(This, of course, can be done even with C++98, but with coroutines this
"storing the current state of the parser" becomes much easier.)
new features of C++20 are coroutines. This post is not about bashing them,
criticizing them, questioning the motivation for adding them to the
already complex language, or anything like that. Let's put all that aside
for a moment, and just focus on what they can actually be useful for.
Practical examples.
I had never heard of "coroutines" before, and in fact it took me quite
a while to understand what exactly they are, and what they are used for.
They are often described as kind of emulating cooperative multitasking
(similarly to how multitasking worked in Windows 3.x and MacOS prior
to X.)
I would, however (and from what I understand), describe them more like a
way to more easily store the state of a function (or group of functions)
so that the function can be exited at any point that it wants, and then
execution resumed later from that point (when the rest of the code
tells it to resume execution.)
(Well, I suppose that's exactly what "cooperative multitasking" means,
but I think describing it as I did is much clearer than using that term.)
Of course doing this was already possible even before C++20, but it
required a lot more manual labor, a lot more coding. The coroutines
help automatize the process and make it much easier.
So where is this actually useful? I can think of two examples.
Are there more?
Example 1:
You are writing a function that decompresses some compressed input format.
The decompressed data should be written to a buffer of a given size.
Whenever the buffer gets full, this decompressor function should return
it to the calling code for it to consume it. After the calling code has
consumed it, the decompression should continue from where it left.
Since compression formats are complicated there may be literally dozens
of places in the decompression algorithm where the buffer may get full,
in very varied situations, sometimes even inside deeply nested loops,
coroutines make this process a lot easier because the code can "yield"
at any point, anywhere (even inside deeply nested loops), and then
continue from that point as if nothing has happened (other than the
buffer having been emptied).
Example 2:
A class/module that parses command-line arguments. Since the arguments may
appear in many different forms, the parser is done so that it will be
called with individual command-line parameters (which may end in the null
character, or a whitespace character, or something else). In other words
for each individual parameter the calling code calls the parser giving it
that parameter.
However, not all command-line parameters are singular, as in consisting
of only one "word". A parameter may be followed by one or more strings
(such as file names, etc.) The calling code doesn't know if a particular
parameter is just data for the previous parameter or not, it just feeds
them one by one to the parser.
Thus, when the parser encounters a parameter that expects further
parameters, it needs to remember its state between calls. (It needs to
remember which particular command-line parameter it's currently
parsing, and any additional parameters given to it already.)
(This, of course, can be done even with C++98, but with coroutines this
"storing the current state of the parser" becomes much easier.)
Bonita Montero
Jun 28, 2022, 10:47:17 AM6/28/22
to
If you don't like coroutines, don't use them.
Cholo Lennon
Jun 28, 2022, 11:09:18 AM6/28/22
to
On 6/28/22 11:47 AM, Bonita Montero wrote:
> If you don't like coroutines, don't use them.
Troll
> If you don't like coroutines, don't use them.
Alf P. Steinbach
Jun 28, 2022, 11:20:46 AM6/28/22
to
On 28 Jun 2022 15:56, Juha Nieminen wrote:
> One of the strangest and perhaps most surprising (and even controversial)
> new features of C++20 are coroutines. This post is not about bashing them,
> criticizing them, questioning the motivation for adding them to the
> already complex language, or anything like that. Let's put all that aside
> for a moment, and just focus on what they can actually be useful for.
> Practical examples.
>
> I had never heard of "coroutines" before, and in fact it took me quite
> a while to understand what exactly they are, and what they are used for.
> They are often described as kind of emulating cooperative multitasking
> (similarly to how multitasking worked in Windows 3.x and MacOS prior
> to X.)
>
> I would, however (and from what I understand), describe them more like a
> way to more easily store the state of a function (or group of functions)
> so that the function can be exited at any point that it wants, and then
> execution resumed later from that point (when the rest of the code
> tells it to resume execution.)
>
> (Well, I suppose that's exactly what "cooperative multitasking" means,
> but I think describing it as I did is much clearer than using that term.)
>
> Of course doing this was already possible even before C++20, but it
> required a lot more manual labor, a lot more coding. The coroutines
> help automatize the process and make it much easier.
>
> So where is this actually useful? I can think of two examples.
> Are there more?
Well, the Jackson structure mismatch problem.
> One of the strangest and perhaps most surprising (and even controversial)
> new features of C++20 are coroutines. This post is not about bashing them,
> criticizing them, questioning the motivation for adding them to the
> already complex language, or anything like that. Let's put all that aside
> for a moment, and just focus on what they can actually be useful for.
> Practical examples.
>
> I had never heard of "coroutines" before, and in fact it took me quite
> a while to understand what exactly they are, and what they are used for.
> They are often described as kind of emulating cooperative multitasking
> (similarly to how multitasking worked in Windows 3.x and MacOS prior
> to X.)
>
> I would, however (and from what I understand), describe them more like a
> way to more easily store the state of a function (or group of functions)
> so that the function can be exited at any point that it wants, and then
> execution resumed later from that point (when the rest of the code
> tells it to resume execution.)
>
> (Well, I suppose that's exactly what "cooperative multitasking" means,
> but I think describing it as I did is much clearer than using that term.)
>
> Of course doing this was already possible even before C++20, but it
> required a lot more manual labor, a lot more coding. The coroutines
> help automatize the process and make it much easier.
>
> So where is this actually useful? I can think of two examples.
> Are there more?
And any problem where a Python programmer would use a generator.
But in general, a coroutine provides an "in control" view of things, an
"I call you", while allowing a consumer of its results to also be an "in
control" design, "I call you".
Thus with coroutines an explicit state consumer, "I'm called", can be
rewritten, perhaps more clearly, as a consumer with state implied by the
execution position, "I call you".
A problem with C++ coroutines is that they're stackless, which
essentially means that a transfer of control must happen in the function
body, and not in some called function. E.g. providing range based `for`
support for iterating over a tree structure, is probably not well suited
for C++ coroutines, because I believe they can't be recursive or
transfer in the middle of a recursive call. Disclaimer: I haven't
actually used C++ coroutines, only old general Modula-2 coroutines.
Bonita Montero
Jun 28, 2022, 11:22:56 AM6/28/22
to
Am 28.06.2022 um 17:20 schrieb Alf P. Steinbach:
> A problem with C++ coroutines is that they're stackless, ...
Without being stackless they would be by fare less useful.
> A problem with C++ coroutines is that they're stackless, ...
Without being stackless they would be by fare less useful.
Alf P. Steinbach
Jun 28, 2022, 11:58:45 AM6/28/22
to
David Brown
Jun 28, 2022, 12:21:09 PM6/28/22
to
co-routines can be much more efficient than co-routines with their own
stack, and simpler to implement. If each co-routine has its own stack
then you need some way to determine how big the stack should be, some
way to get that space (i.e., heap allocation), and switching between
stacks.
Stackless co-routines all run in the threads single stack. In effect,
the compiler will generate a dispatcher function which is a big switch
statement determining which co-routine to run at any given point, and
with each co-routine having a struct containing all its local state data
that needs to be preserved between calls - these state structs are then
local variables in the dispatcher function.
This gives you something with minimal overhead and complications, while
saving the programmer from writing this boilerplate state control manually.
However, it very clearly also reduces the flexibility and use-cases for
co-routines, compared to separate stacks. They are not a full
asynchronous cooperative multi-tasking solution.
Stackless co-routines have been used in C and C++ for decades, including
on tiny microcontrollers (see Adam Dunkels' "Protothreads"). Typically,
they are implemented using some hideous macros that hide switch
statements, and you need to make the state-preserving structs manually.
C++ language co-routines are, AFAICS, designed to give you the
benefits of such coding structures without the manual work.
I haven't tried them myself as yet, but I will do some day.
Siri Cruise
Jun 28, 2022, 1:19:12 PM6/28/22
to
In article <t9f67t$147la$1...@dont-email.me>,
functions to traverse. Coroutines act sort of as mutual recursion
which smashes the stack flat on each recursion.
--
:-<> Siri Seal of Disavowal #000-001. Disavowed. Denied. Deleted. @
'I desire mercy, not sacrifice.' /|\
Discordia: not just a religion but also a parody. This post / \
I am an Andrea Chen sockpuppet. insults Islam. Mohammed
"Alf P. Steinbach" <alf.p.s...@gmail.com> wrote:
> But in general, a coroutine provides an "in control" view of things, an
> "I call you", while allowing a consumer of its results to also be an "in
> control" design, "I call you".
>
> Thus with coroutines an explicit state consumer, "I'm called", can be
> rewritten, perhaps more clearly, as a consumer with state implied by the
> execution position, "I call you".
Complicated structures sometimes need mutually recursive
> But in general, a coroutine provides an "in control" view of things, an
> "I call you", while allowing a consumer of its results to also be an "in
> control" design, "I call you".
>
> Thus with coroutines an explicit state consumer, "I'm called", can be
> rewritten, perhaps more clearly, as a consumer with state implied by the
> execution position, "I call you".
functions to traverse. Coroutines act sort of as mutual recursion
which smashes the stack flat on each recursion.
--
:-<> Siri Seal of Disavowal #000-001. Disavowed. Denied. Deleted. @
'I desire mercy, not sacrifice.' /|\
Discordia: not just a religion but also a parody. This post / \
I am an Andrea Chen sockpuppet. insults Islam. Mohammed
Scott Lurndal
Jun 28, 2022, 2:07:59 PM6/28/22
to
Siri Cruise <chine...@yahoo.com> writes:
>In article <t9f67t$147la$1...@dont-email.me>,
> "Alf P. Steinbach" <alf.p.s...@gmail.com> wrote:
>
>> But in general, a coroutine provides an "in control" view of things, an
>> "I call you", while allowing a consumer of its results to also be an "in
>> control" design, "I call you".
>>
>> Thus with coroutines an explicit state consumer, "I'm called", can be
>> rewritten, perhaps more clearly, as a consumer with state implied by the
>> execution position, "I call you".
>
>Complicated structures sometimes need mutually recursive
>functions to traverse. Coroutines act sort of as mutual recursion
>which smashes the stack flat on each recursion.
>
Coroutines in PDP-11 assembler were pretty simple.
>In article <t9f67t$147la$1...@dont-email.me>,
> "Alf P. Steinbach" <alf.p.s...@gmail.com> wrote:
>
>> But in general, a coroutine provides an "in control" view of things, an
>> "I call you", while allowing a consumer of its results to also be an "in
>> control" design, "I call you".
>>
>> Thus with coroutines an explicit state consumer, "I'm called", can be
>> rewritten, perhaps more clearly, as a consumer with state implied by the
>> execution position, "I call you".
>
>Complicated structures sometimes need mutually recursive
>functions to traverse. Coroutines act sort of as mutual recursion
>which smashes the stack flat on each recursion.
>
JSR coroutine
<coroutine prologue>
JSR (SP)+
<original function>
JSR (SP)+
<coroutine>
JSR (SP)+
<original function>
...
Alf P. Steinbach
Jun 28, 2022, 2:37:26 PM6/28/22
to
there were a lot of @ signs (presumably macros?), and memory mapped
registers, and later some enthusiastic articles about LSI-11, hurray,
unbelievable, a computer IN A BRIEFCASE, yay!
- Alf
Scott Lurndal
Jun 28, 2022, 3:07:54 PM6/28/22
to
(sp) references the value on the top of the stack.
@(sp) dereferences the value on the top of the stack.
@(r0)+ post-increments r0. (char *cp = string; while ((c=*cp++) != EOF) {})
Frederick Virchanza Gotham
Jun 28, 2022, 5:25:50 PM6/28/22
to
On Tuesday, June 28, 2022 at 2:56:16 PM UTC+1, Juha Nieminen wrote:
> So where is this actually useful? I can think of two examples.
> Are there more?
I wrote a short article on my website about C++20 coroutines:
> So where is this actually useful? I can think of two examples.
> Are there more?
http://www.virjacode.com/algorithms/coroutine_example.html
Snippet: "A coroutine is like a function that can be called the first time, return a value, and then when it's called the second time, it resumes its execution from where it last left off. (In terms of assembly language and machine code, the stack variables and program counter are preserved across each invocation of the coroutine)."
Bonita Montero
Jun 29, 2022, 1:22:20 AM6/29/22
to
a function lives the whole thing wouldn't make sense. With conventional
state machines you usually store your state on the heap as well.
Bonita Montero
Jun 29, 2022, 1:23:31 AM6/29/22
to
Am 28.06.2022 um 18:20 schrieb David Brown:
> I don't know what Bonita is talking about (I rarely do), but stackless
> co-routines can be much more efficient than co-routines with their own
> stack, and simpler to implement. If each co-routine has its own stack
> then you need some way to determine how big the stack should be, some
> way to get that space (i.e., heap allocation), and switching between
> stacks.
It's not about the size of the storage but the state of a stackless
> I don't know what Bonita is talking about (I rarely do), but stackless
> co-routines can be much more efficient than co-routines with their own
> stack, and simpler to implement. If each co-routine has its own stack
> then you need some way to determine how big the stack should be, some
> way to get that space (i.e., heap allocation), and switching between
> stacks.
coroutine can live longer than the creating function.
Frederick Virchanza Gotham
Jun 29, 2022, 2:46:23 AM6/29/22
to
On Wednesday, June 29, 2022 at 6:22:20 AM UTC+1, Bonita Montero wrote:
> Coroutines are state machines and if their state would live as long as
> a function lives the whole thing wouldn't make sense. With conventional
> state machines you usually store your state on the heap as well.
I've written loads of state machines that can be in any kind of memory, depending on where/how you define them, e.g.:
> Coroutines are state machines and if their state would live as long as
> a function lives the whole thing wouldn't make sense. With conventional
> state machines you usually store your state on the heap as well.
class StateMachine {
int units[18u];
};
StateMachine g_this_one_is_in_static_memory;
void Func(void)
{
StateMachine this_one_is_on_the_stack;
}
int main(void)
{
unique_ptr<StateMachine> this_one_is_on_the_heap( new StateMachine );
}
0 new messages