[boost] boost.spawn: review requested

[Oliver Kowalke via Boost]

Hi,
I'd like to propose the new library boost.spawn (https://olk.github.io/)
for review.
boost.asio's boost::asio::spawn() uses the deprecated and inefficient
boost.coroutine library.
boost.spawn provides the same functionality while using boost.context's
fiber directly.



Oliver

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Andrzej Krzemienski via Boost]

Hi Oliver,
Thank you for the submission.

Some quick high-level feedback, after reading the docs.

Is Boost.Spawn intended to be useful on its own, or is it only usable in
Boost.ASIO?

The provided link points to the docs. Is there an associated source code?
Or are you just checking if there is sufficient interest?

The "Effects" clause of the third overload has " on behalf of spawned
spawn", which is supposed to be " on behalf of spawned fiber", I guess.

Regards,
&rzej;

niedz., 28 lis 2021 o 07:50 Oliver Kowalke via Boost <bo...@lists.boost.org>
napisał(a):

[Oliver Kowalke via Boost]

>
>
> Is Boost.Spawn intended to be useful on its own, or is it only usable in
> Boost.ASIO?
>

It can only be used with boost .asio (or with the forthcoming C++
std::io_context).
The lib is intended to replace the boost::asio:::spawn().

> The provided link points to the docs. Is there an associated source code?
> Or are you just checking if there is sufficient interest?
>

Oh, I forgot the link: https://github.com/olk/spawn

> The "Effects" clause of the third overload has " on behalf of spawned
> spawn", which is supposed to be " on behalf of spawned fiber", I guess.
>

right


regards, Oliver

[Peter Dimov via Boost]

Oliver Kowalke wrote:
> Hi,
> I'd like to propose the new library boost.spawn (https://olk.github.io/) for
> review.
> boost.asio's boost::asio::spawn() uses the deprecated and inefficient
> boost.coroutine library.
> boost.spawn provides the same functionality while using boost.context's fiber
> directly.

The documentation of spawn_fiber often says

"must have signature void(basic_yield_context<Handler>)"

on signatures where there's no Handler.

[Andrey Semashev via Boost]

On 11/28/21 16:27, Oliver Kowalke via Boost wrote:
>>
>> Is Boost.Spawn intended to be useful on its own, or is it only usable in
>> Boost.ASIO?
>
> It can only be used with boost .asio (or with the forthcoming C++
> std::io_context).
> The lib is intended to replace the boost::asio:::spawn().

In this case why should it be a separate library and not just part of
Boost.ASIO? What would be the value for Boost users or other Boost
libraries?

[Oliver Kowalke via Boost]

Am So., 28. Nov. 2021 um 16:36 Uhr schrieb Peter Dimov via Boost <
bo...@lists.boost.org>:

>
> The documentation of spawn_fiber often says
> "must have signature void(basic_yield_context<Handler>)"
> on signatures where there's no Handler.
>

void do_echo(boost::spawn::yield_context yield) {
try {
char data[128];
for (;;) {
std::size_t length =
my_socket.async_read_some(boost::asio::buffer(data), yield);
boost::asio::async_write(my_socket,
boost::asio::buffer(data, length), yield);
}
} catch (std::exception const& e) {
// ...
}}boost::spawn_fiber(my_strand, do_echo);

do_echo() is the handler that is passed to spawn_fiber()

[Oliver Kowalke via Boost]

Am So., 28. Nov. 2021 um 17:34 Uhr schrieb Andrey Semashev via Boost <
bo...@lists.boost.org>:

> In this case why should it be a separate library and not just part of
> Boost.ASIO?

I tried to pass a fix to boost.asio that was not taken up by the author and
unfortunately no response...

> What would be the value for Boost users or other Boost
> libraries?
>

spawn_fiber() is more efficient than boost::asio::spawn()
spawn_fiber() is prepared to work with the forthcoming std::io_context
async. operations are written as synchronous code.

[Richard Hodges via Boost]

On Sun, 28 Nov 2021 at 17:34, Andrey Semashev via Boost <
bo...@lists.boost.org> wrote:

> On 11/28/21 16:27, Oliver Kowalke via Boost wrote:
> >>
> >> Is Boost.Spawn intended to be useful on its own, or is it only usable in
> >> Boost.ASIO?
> >
> > It can only be used with boost .asio (or with the forthcoming C++
> > std::io_context).
> > The lib is intended to replace the boost::asio:::spawn().
>
> In this case why should it be a separate library and not just part of
> Boost.ASIO? What would be the value for Boost users or other Boost
> libraries?
>

Getting code into Asio would require Chris Kohlhoff's buy-in.

> I tried to pass a fix to boost.asio that was not taken up by the author
and
unfortunately no response...

You need to submit the PR here:

https://github.com/chriskohlhoff/asio/

[Andrey Semashev via Boost]

On 11/28/21 19:53, Richard Hodges via Boost wrote:
> On Sun, 28 Nov 2021 at 17:34, Andrey Semashev via Boost <
> bo...@lists.boost.org> wrote:
>
>> On 11/28/21 16:27, Oliver Kowalke via Boost wrote:
>>>>
>>>> Is Boost.Spawn intended to be useful on its own, or is it only usable in
>>>> Boost.ASIO?
>>>
>>> It can only be used with boost .asio (or with the forthcoming C++
>>> std::io_context).
>>> The lib is intended to replace the boost::asio:::spawn().
>>
>> In this case why should it be a separate library and not just part of
>> Boost.ASIO? What would be the value for Boost users or other Boost
>> libraries?
>>
>
> Getting code into Asio would require Chris Kohlhoff's buy-in.

Using this library in Boost.ASIO also requires Chris' cooperation,
doesn't it? And if this library is not useful outside Boost.ASIO and
Chris is not interested, I just don't see the point.

[Oliver Kowalke via Boost]

Am So., 28. Nov. 2021 um 17:53 Uhr schrieb Richard Hodges via Boost <
bo...@lists.boost.org>:

> On Sun, 28 Nov 2021 at 17:34, Andrey Semashev via Boost <
> bo...@lists.boost.org> wrote:
>
> > On 11/28/21 16:27, Oliver Kowalke via Boost wrote:
> > >>
> > >> Is Boost.Spawn intended to be useful on its own, or is it only usable
> in
> > >> Boost.ASIO?
> > >
> > > It can only be used with boost .asio (or with the forthcoming C++
> > > std::io_context).
> > > The lib is intended to replace the boost::asio:::spawn().
> >
> > In this case why should it be a separate library and not just part of
> > Boost.ASIO? What would be the value for Boost users or other Boost
> > libraries?
> >
>
> Getting code into Asio would require Chris Kohlhoff's buy-in.
>
> > I tried to pass a fix to boost.asio that was not taken up by the author
> and
> unfortunately no response...
>
> You need to submit the PR here:
>
> https://github.com/chriskohlhoff/asio/

I wrote an email... unfortunately no response

[Oliver Kowalke via Boost]

Am So., 28. Nov. 2021 um 18:18 Uhr schrieb Andrey Semashev via Boost <
bo...@lists.boost.org>:

> On 11/28/21 19:53, Richard Hodges via Boost wrote:
> > On Sun, 28 Nov 2021 at 17:34, Andrey Semashev via Boost <
> > bo...@lists.boost.org> wrote:
> >
> >> On 11/28/21 16:27, Oliver Kowalke via Boost wrote:
> >>>>
> >>>> Is Boost.Spawn intended to be useful on its own, or is it only usable
> in
> >>>> Boost.ASIO?
> >>>
> >>> It can only be used with boost .asio (or with the forthcoming C++
> >>> std::io_context).
> >>> The lib is intended to replace the boost::asio:::spawn().
> >>
> >> In this case why should it be a separate library and not just part of
> >> Boost.ASIO? What would be the value for Boost users or other Boost
> >> libraries?
> >>
> >
> > Getting code into Asio would require Chris Kohlhoff's buy-in.
>
> Using this library in Boost.ASIO also requires Chris' cooperation,
> doesn't it?

no response to email

> And if this library is not useful outside Boost.ASIO and
> Chris is not interested, I just don't see the point.
>

Don't get it ... 'not useful outside boost.asio' ...
You can create asynchronous code that looks synchronous on behalf of
boost::asio::io_context or the forthcoming C++ std::io_context.

[Oliver Kowalke via Boost]

Am So., 28. Nov. 2021 um 18:18 Uhr schrieb Andrey Semashev via Boost <
bo...@lists.boost.org>:

> And if this library is not useful outside Boost.ASIO and
> Chris is not interested, I just don't see the point.
>

another point:
boost.asio still uses boost.coroutine which is deprecated and I really want
to remove it from boost
but is not possible because of boost::asio::spawn()
boost.spawn is the alternative to asio's spawn() (which could be removed
from boost.asio in favour of boost.spawn)

[Richard Hodges via Boost]

On Sun, 28 Nov 2021 at 18:30, Oliver Kowalke via Boost <
bo...@lists.boost.org> wrote:

> Am So., 28. Nov. 2021 um 18:18 Uhr schrieb Andrey Semashev via Boost <
> bo...@lists.boost.org>:
>
> > And if this library is not useful outside Boost.ASIO and
> > Chris is not interested, I just don't see the point.
> >
>
> another point:
> boost.asio still uses boost.coroutine which is deprecated and I really want
> to remove it from boost
> but is not possible because of boost::asio::spawn()
> boost.spawn is the alternative to asio's spawn() (which could be removed
> from boost.asio in favour of boost.spawn)
>

Asio's spawn is extremely popular and in use in a number of code bases. I
really don't think removing from boost is a sensible option unless there is
a clear, simple replacement that works for C++03.

[Oliver Kowalke via Boost]

> Asio's spawn is extremely popular and in use in a number of code bases.

I know but it still uses the deprecated boost.coroutine

> I really don't think removing from boost is a sensible option unless there
> is
> a clear, simple replacement that works for C++03.
>

boost.spawn can be extended to support C++03 too

[Robert Ramey via Boost]

Couldn't one achieve the desired effect by just updating the
implementation of boost.coroutine with the implementation of boost.spawn?

Robert Ramey

[Oliver Kowalke via Boost]

Robert Ramey via Boost <bo...@lists.boost.org> schrieb am So., 28. Nov.
2021, 20:54:

> Couldn't one achieve the desired effect by just updating the
> implementation of boost.coroutine with the implementation of boost.spawn?
>

No, I've already provided boost.coroutine2 - that's why boost.context was
deprecated. Unfortunately we can't remove it form the boost collection
because boost.asio has a dependency to boost.coroutine.
The coroutine libraries are too much heavyweight for the purpose of spawn()
(for instance push_type and pull_type etc.).
boost.spawn uses boost.context's fiber directly - without the boilerplate
code required by coroutine libs (take a look at boost/spawn/impl/spawn.hpp).

[Niall Douglas via Boost]

On 28/11/2021 13:27, Oliver Kowalke via Boost wrote:
>>
>>
>> Is Boost.Spawn intended to be useful on its own, or is it only usable in
>> Boost.ASIO?
>>
>
> It can only be used with boost .asio (or with the forthcoming C++
> std::io_context).
> The lib is intended to replace the boost::asio:::spawn().

For this to be valid as a standalone Boost library, in my opinion it
needs to also support at the very least C++ coroutines if on C++ 20.

(FYI std::io_context is currently up in the air at WG21, something quite
different to what ASIO does may be coming instead)

Niall

[Boris Kolpackov via Boost]

Oliver Kowalke via Boost <bo...@lists.boost.org> writes:

> I'd like to propose the new library boost.spawn [...]

FWIW (and I may be out of context), but "spawn" seems like a
too generic of a name. I, for example, thought this is about
posix_spawn(). Maybe call it spawn_fiber or some such?

[Oliver Kowalke via Boost]

Am Mo., 29. Nov. 2021 um 15:49 Uhr schrieb Niall Douglas via Boost <
bo...@lists.boost.org>:

>
>
> For this to be valid as a standalone Boost library, in my opinion it
> needs to also support at the very least C++ coroutines if on C++ 20.
>

Sorry, but I think this does not make sense because boost::asio::spawn() a
boost.spawn do only support stackful context switching.
It was long discussed in the C++ standardisation meetings if a unified
syntax for stackless and stackful 'coroutines' is possible - the outcome
was, no its impossible (think on function coloring).

[Vinnie Falco via Boost]

On Sat, Nov 27, 2021 at 10:50 PM Oliver Kowalke via Boost
<bo...@lists.boost.org> wrote:
> I'd like to propose the new library boost.spawn (https://olk.github.io/)
> for review.
> boost.asio's boost::asio::spawn() uses the deprecated and inefficient
> boost.coroutine library.
> boost.spawn provides the same functionality while using boost.context's
> fiber directly.

I like the direction this library takes, and I have a use for
something like this myself in Boost.JSON. Specifically, to implement a
generator which does pull parsing of JSON. This will make the
implementation of parsing directly into structs more simple and
robust. I also like the simplification over push/pull, which never
really made sense to me.

However, this library should not depend on Asio. The fiber
functionality provided by boost.spawn is useful enough that people
will want to use it in more situations than just Asio. What will
standalone Asio use? It is worth considering a standalone version of
this library (or perhaps just not depend on any other Boost
libraries).

Have you considered just rolling this into Boost.Context? That library
is sort of not particularly useful on its own. I would like to see an
example of a generator using boost.spawn. A simple fibonacci producer
would suffice. This library also needs more examples that don't
require Asio.

Thanks

[Niall Douglas via Boost]

On 29/11/2021 15:03, Oliver Kowalke via Boost wrote:
> Am Mo., 29. Nov. 2021 um 15:49 Uhr schrieb Niall Douglas via Boost <
> bo...@lists.boost.org>:
>
>>
>>
>> For this to be valid as a standalone Boost library, in my opinion it
>> needs to also support at the very least C++ coroutines if on C++ 20.
>>
>
> Sorry, but I think this does not make sense because boost::asio::spawn() a
> boost.spawn do only support stackful context switching.
> It was long discussed in the C++ standardisation meetings if a unified
> syntax for stackless and stackful 'coroutines' is possible - the outcome
> was, no its impossible (think on function coloring).

Y'see, I'd look at this as an opportunity to extend C++ Coroutines with
stackfulness emulation, and that would be exactly the value in a
standalone Boost library here.

But otherwise I gotta agree with Vinnie here, if you require
Boost.Context and Boost.ASIO and you only work with those and Chris K
isn't keen on adding this into Boost.ASIO, the next most logical place
for something as small as this is Boost.Context surely?

Niall

[Oliver Kowalke via Boost]

Am Mo., 29. Nov. 2021 um 16:56 Uhr schrieb Vinnie Falco via Boost <
bo...@lists.boost.org>:

> However, this library should not depend on Asio. The fiber
> functionality provided by boost.spawn is useful enough that people
> will want to use it in more situations than just Asio. What will
> standalone Asio use? It is worth considering a standalone version of
> this library (or perhaps just not depend on any other Boost
> libraries).
>

I'm confused by the request for a standalone version.
The spawned fibers must be scheduled for execution. boost.fiber does this
without the need of boost.asio.
boost.spawn itself relies on boost::asio's io_context (demultiplexer, event
scheduler) and the async result/handler framework.

>
> Have you considered just rolling this into Boost.Context?

Yes, I had a commit that has been reverted because of dependency cycles
between the boost libs.

> That library
> is sort of not particularly useful on its own.

boost.context must be considered as a foundation library for other
high-level constructs (like boost.coroutine2 or boost.fiber).

> I would like to see an
> example of a generator using boost.spawn. A simple fibonacci producer
> would suffice. This library also needs more examples that don't
> require Asio.
>

I think this is a misunderstanding of what boost.spawn is intended...

[Vinnie Falco via Boost]

On Mon, Nov 29, 2021 at 8:09 AM Oliver Kowalke via Boost
<bo...@lists.boost.org> wrote:
> I think this is a misunderstanding of what boost.spawn is intended...

Upon reviewing Boost.Coroutine2, Boost.Fiber, and taking another look
at Boost.Spawn, I believe you are correct.

We have a problem with not being able to deprecate Boost.Coroutine
because of Asio. The Asio author is silent on pull requests and
emails. Boost.Spawn appears to be a mechanism for facilitating
progress on this matter, in a fashion similar to adding an improved
string_view to Boost.Utility.

In light of this, I endorse Boost.Spawn for review.

Thanks

[Glen Fernandes via Boost]

On Mon, Nov 29, 2021 at 11:30 AM Vinnie Falco wrote:
> Upon reviewing Boost.Coroutine2, Boost.Fiber, and taking another look
> at Boost.Spawn, I believe you are correct.
>
> We have a problem with not being able to deprecate Boost.Coroutine
> because of Asio. The Asio author is silent on pull requests and
> emails. Boost.Spawn appears to be a mechanism for facilitating
> progress on this matter, in a fashion similar to adding an improved
> string_view to Boost.Utility.
>
> In light of this, I endorse Boost.Spawn for review.

Boost.Coroutine is deprecated, but cannot be removed because of Asio.
The discussion above implies that Boost.Spawn allows Boost.Coroutine
to be removed - but I don't understand how because Boost.Asio still
depends on Boost.Coroutine

And Boost.Asio cannot depend on Boost.Spawn since the latter depends
on Boost.Asio, no? According to Oliver's e-mail above Spawn uses
io_context from Asio, and that would result in a dependency cycle.

What am I missing?

Glen

[Oliver Kowalke via Boost]

Am Mo., 29. Nov. 2021 um 18:09 Uhr schrieb Glen Fernandes via Boost <
bo...@lists.boost.org>:

> Boost.Coroutine is deprecated, but cannot be removed because of Asio.
> The discussion above implies that Boost.Spawn allows Boost.Coroutine
> to be removed - but I don't understand how because Boost.Asio still
> depends on Boost.Coroutine
>

My suggestion is that we convince Chris to remove boost::asio::spawn and
add a notice/link to boost.spawn in the asio documentation.

>
> And Boost.Asio cannot depend on Boost.Spawn since the latter depends
> on Boost.Asio, no? According to Oliver's e-mail above Spawn uses
> io_context from Asio, and that would result in a dependency cycle.
>
> What am I missing?
>

I previously made a commit in boost.context that implemented spawn_fiber()
(but introduced a dependency to boost.asio because of io_context).
Via boost::asio::spawn() boost.asio has itself a dependency to
boost.coroutine.
Boost.coroutine depends on boost.context as a foundation for context
switching.

[Glen Fernandes via Boost]

On Mon, Nov 29, 2021 at 12:17 PM Oliver Kowalke wrote:

>
> Am Mo., 29. Nov. 2021 um 18:09 Uhr schrieb Glen Fernandes wrote:
> > Boost.Coroutine is deprecated, but cannot be removed because of Asio.
> > The discussion above implies that Boost.Spawn allows Boost.Coroutine
> > to be removed - but I don't understand how because Boost.Asio still
> > depends on Boost.Coroutine
> >
>
> My suggestion is that we convince Chris to remove boost::asio::spawn and
> add a notice/link to boost.spawn in the asio documentation.
>

Understood - thanks.

Glen

[Casey Bodley via Boost]

On Sun, Nov 28, 2021 at 1:50 AM Oliver Kowalke via Boost
<bo...@lists.boost.org> wrote:
>
> Hi,

> I'd like to propose the new library boost.spawn (https://olk.github.io/)
> for review.
> boost.asio's boost::asio::spawn() uses the deprecated and inefficient
> boost.coroutine library.
> boost.spawn provides the same functionality while using boost.context's
> fiber directly.
>
>
>

> Oliver

>
> _______________________________________________
> Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
>

thanks for proposing this, Oliver!

as a satisfied user, i'd like to express my support. the Ceph project
has been maintaining a fork of this work at
https://github.com/ceph/spawn/, and it's been running in production
for nearly 2 years now in our Ceph Object Gateway [1] component

for us, the crucial feature of this fork is its ability to customize
the StackAllocator. the original boost::asio::spawn() offers limited
customization through boost::coroutines::attributes [2], but that only
covers the stack size and unwind behavior. Oliver's fork allows you to
pass in any object meeting the requirements of boost::context's
StackAllocator concept [3], and the context library provides several
options here. you can read a bit more about our use case in the "rgw:
coroutine stack protection" thread [4] on the Ceph development mailing
list

another minor feature that this adds over boost::asio::spawn() is
support for completion handlers with more than one non-error-code
argument. for example, when a handler's signature is (error_code, int,
std::string), the async initiating function will return those
non-error-code values as a std::tuple<int, std::string>

[1] https://docs.ceph.com/en/pacific/radosgw/
[2] https://www.boost.org/doc/libs/1_77_0/libs/coroutine/doc/html/coroutine/attributes.html
[3] https://www.boost.org/doc/libs/1_77_0/libs/context/doc/html/context/stack.html
[4] https://www.spinics.net/lists/dev-ceph/msg00704.html

[Andrey Semashev via Boost]

On 11/29/21 17:54, Boris Kolpackov via Boost wrote:
> Oliver Kowalke via Boost <bo...@lists.boost.org> writes:
>
>> I'd like to propose the new library boost.spawn [...]
>
> FWIW (and I may be out of context), but "spawn" seems like a
> too generic of a name. I, for example, thought this is about
> posix_spawn(). Maybe call it spawn_fiber or some such?

+1

[Vinnie Falco via Boost]

On Mon, Nov 29, 2021 at 10:52 AM Andrey Semashev via Boost
<bo...@lists.boost.org> wrote:
> > "spawn" seems like a too generic of a name.

How about Boost.Asio.Spawn? This is more correcteror.

Thanks

[Oliver Kowalke via Boost]

Andrey Semashev via Boost <bo...@lists.boost.org> schrieb am Mo., 29. Nov.
2021, 19:52:

> On 11/29/21 17:54, Boris Kolpackov via Boost wrote:
> > Oliver Kowalke via Boost <bo...@lists.boost.org> writes:
> >
> >> I'd like to propose the new library boost.spawn [...]
> >
> > FWIW (and I may be out of context), but "spawn" seems like a
> > too generic of a name. I, for example, thought this is about
> > posix_spawn(). Maybe call it spawn_fiber or some such?
>
> +1
>

As I answered posix_spawn() is wrong because the lib supports Windows too.
Boost.spawn already names the function spawn_fiber() ...
I don't get your point or what I've to change.

[Oliver Kowalke via Boost]

Vinnie Falco via Boost <bo...@lists.boost.org> schrieb am Mo., 29. Nov.
2021, 20:02:

> On Mon, Nov 29, 2021 at 10:52 AM Andrey Semashev via Boost
> <bo...@lists.boost.org> wrote:
> > > "spawn" seems like a too generic of a name.
>
> How about Boost.Asio.Spawn? This is more correcteror.
>

I think he means the function name.

[Vinnie Falco via Boost]

On Mon, Nov 29, 2021 at 11:23 AM Oliver Kowalke via Boost
<bo...@lists.boost.org> wrote:
> I think he means the function name.

We shouldn't use types from Asio's detail namespace:

<https://github.com/olk/spawn/blob/216f23a11bf4fe854c105fa2eadfb0ba998529bc/include/boost/spawn.hpp#L80>

Regards

[Oliver Kowalke via Boost]

Am Mo., 29. Nov. 2021 um 20:31 Uhr schrieb Vinnie Falco via Boost <
bo...@lists.boost.org>:

> On Mon, Nov 29, 2021 at 11:23 AM Oliver Kowalke via Boost
> <bo...@lists.boost.org> wrote:
> > I think he means the function name.
>
> We shouldn't use types from Asio's detail namespace:
>
> <
> https://github.com/olk/spawn/blob/216f23a11bf4fe854c105fa2eadfb0ba998529bc/include/boost/spawn.hpp#L80
> >
>

Which one? yield_context and basic_yield_context reside in namespace
boost::spawn?

[Vinnie Falco via Boost]

On Mon, Nov 29, 2021 at 12:05 PM Oliver Kowalke via Boost
<bo...@lists.boost.org> wrote:
> Which one? yield_context and basic_yield_context reside in namespace
> boost::spawn?

I misread, detail::net::executor is yours not asio's (via detail/net.hpp)

Thanks

[Niall Douglas via Boost]

On 29/11/2021 16:29, Vinnie Falco via Boost wrote:
> On Mon, Nov 29, 2021 at 8:09 AM Oliver Kowalke via Boost
> <bo...@lists.boost.org> wrote:
>> I think this is a misunderstanding of what boost.spawn is intended...
>
> Upon reviewing Boost.Coroutine2, Boost.Fiber, and taking another look
> at Boost.Spawn, I believe you are correct.
>
> We have a problem with not being able to deprecate Boost.Coroutine
> because of Asio. The Asio author is silent on pull requests and
> emails. Boost.Spawn appears to be a mechanism for facilitating
> progress on this matter, in a fashion similar to adding an improved
> string_view to Boost.Utility.
>
> In light of this, I endorse Boost.Spawn for review.

It seems to me unwise to create standalone libraries just to shim around
a process and coordination problem.

As an alternative, how about announcing the complete removal of
Boost.Coroutine with two major release notice (including this release)?
That should spur Boost.ASIO being upgraded to Boost.Coroutine2.

Niall

[Oliver Kowalke via Boost]

Am Mo., 29. Nov. 2021 um 21:30 Uhr schrieb Niall Douglas via Boost <
bo...@lists.boost.org>:

>

> As an alternative, how about announcing the complete removal of
> Boost.Coroutine with two major release notice (including this release)?
> That should spur Boost.ASIO being upgraded to Boost.Coroutine2.
>

That had already happen some years ago - including warnings during
compilations - but didn't help. Ask Vini...
At the end I was forced to remove the compiler (preprocessor) warnings.

[Niall Douglas via Boost]

On 29/11/2021 20:38, Oliver Kowalke via Boost wrote:
> Am Mo., 29. Nov. 2021 um 21:30 Uhr schrieb Niall Douglas via Boost <
> bo...@lists.boost.org>:
>
>>
>> As an alternative, how about announcing the complete removal of
>> Boost.Coroutine with two major release notice (including this release)?
>> That should spur Boost.ASIO being upgraded to Boost.Coroutine2.
>>
>
> That had already happen some years ago - including warnings during
> compilations - but didn't help. Ask Vini...
> At the end I was forced to remove the compiler (preprocessor) warnings.

Alas I'm probably one of the very few here who would be absolutely fine
with removing completely Boost.Coroutine after two major releases, and
whatever didn't fix itself and broke with that sufficient amount of
warning well that's on them in my book.

I assume there's no way you can shim Coroutine's API into a Coroutine2
implementation, then you can drop Coroutine?

Niall

[Oliver Kowalke via Boost]

Niall Douglas via Boost <bo...@lists.boost.org> schrieb am Mo., 29. Nov.
2021, 21:47:

> On 29/11/2021 20:38, Oliver Kowalke via Boost wrote:
> > Am Mo., 29. Nov. 2021 um 21:30 Uhr schrieb Niall Douglas via Boost <
> > bo...@lists.boost.org>:
> >
> >>
> >> As an alternative, how about announcing the complete removal of
> >> Boost.Coroutine with two major release notice (including this release)?
> >> That should spur Boost.ASIO being upgraded to Boost.Coroutine2.
> >>
> >
> > That had already happen some years ago - including warnings during
> > compilations - but didn't help. Ask Vini...
> > At the end I was forced to remove the compiler (preprocessor) warnings.
>
> Alas I'm probably one of the very few here who would be absolutely fine
> with removing completely Boost.Coroutine after two major releases, and
> whatever didn't fix itself and broke with that sufficient amount of
> warning well that's on them in my book.
>

I think this not a good idea... boost::asio:::spawn() seams to be often
used.

I assume there's no way you can shim Coroutine's API into a Coroutine2

> implementation, then you can drop Coroutine2
>

> Unfortunately no, boost.coroutine is C++03 while boost.coroutine2 requires
> C++11.

[Vinnie Falco via Boost]

On Mon, Nov 29, 2021 at 1:50 PM Oliver Kowalke via Boost
<bo...@lists.boost.org> wrote:
> I think this not a good idea... boost::asio:::spawn() seams to be often
> used.

Yep. Unfortunately, Beast uses it in several examples:

<https://github.com/boostorg/beast/blob/94e6f143295e38f6e7e7d2f60937d4f76ac1d244/example/http/server/coro/http_server_coro.cpp#L337>

Thanks

[Boris Kolpackov via Boost]

Oliver Kowalke <oliver....@gmail.com> writes:

> Am Mo., 29. Nov. 2021 um 15:54 Uhr schrieb Boris Kolpackov <
> bo...@codesynthesis.com>:

>
> > FWIW (and I may be out of context), but "spawn" seems like a
> > too generic of a name. I, for example, thought this is about
> > posix_spawn(). Maybe call it spawn_fiber or some such?
> >
>

> boost.asio named the function spawn() - boost.spawn named it spawn_fiber()

Sorry, looks I wasn't clear enough: I am talking about the library
name being too generic: Boost.Spawn can mean spawning all kinds of
things (processes, threads, fibers, etc). The fact that you called
the function spawn_fiber() in a sense confirms this.

[Andrzej Krzemienski via Boost]

pon., 29 lis 2021 o 21:39 Oliver Kowalke via Boost <bo...@lists.boost.org>
napisał(a):

> Am Mo., 29. Nov. 2021 um 21:30 Uhr schrieb Niall Douglas via Boost <
> bo...@lists.boost.org>:
>
> >
> > As an alternative, how about announcing the complete removal of
> > Boost.Coroutine with two major release notice (including this release)?
> > That should spur Boost.ASIO being upgraded to Boost.Coroutine2.
> >
>
> That had already happen some years ago - including warnings during
> compilations - but didn't help. Ask Vini...
> At the end I was forced to remove the compiler (preprocessor) warnings.
>

Please correct me if I am wrong, but my understanding is that Boost.ASIO
works under C++03, and so does Boost.Coroutine. Whereas Boost.Coroutine2
requires C++11. If Boost.ASIO were to switch from BoostCoroutine to
Boost.Coroutine2, it would break its C++03 guarantee. (Some teams in my
work still have to use C++03.)

"Boost.Spawn" could be an addition to Boost.ASIO, but not a replacement,
right?

Regards,
&rzej;

[Oliver Kowalke via Boost]

Andrzej Krzemienski <akrz...@gmail.com> schrieb am Di., 30. Nov. 2021,
20:21:

>
>
> pon., 29 lis 2021 o 21:39 Oliver Kowalke via Boost <bo...@lists.boost.org>
> napisał(a):
>
>> Am Mo., 29. Nov. 2021 um 21:30 Uhr schrieb Niall Douglas via Boost <
>> bo...@lists.boost.org>:
>>
>> >
>> > As an alternative, how about announcing the complete removal of
>> > Boost.Coroutine with two major release notice (including this release)?
>> > That should spur Boost.ASIO being upgraded to Boost.Coroutine2.
>> >
>>
>> That had already happen some years ago - including warnings during
>> compilations - but didn't help. Ask Vini...
>> At the end I was forced to remove the compiler (preprocessor) warnings.
>>
>
> Please correct me if I am wrong, but my understanding is that Boost.ASIO
> works under C++03, and so does Boost.Coroutine. Whereas Boost.Coroutine2
> requires C++11. If Boost.ASIO were to switch from BoostCoroutine to
> Boost.Coroutine2, it would break its C++03 guarantee. (Some teams in my
> work still have to use C++03.)
>
> "Boost.Spawn" could be an addition to Boost.ASIO, but not a replacement,
> right?
>

boost.spawn can not replace boost.asio - only it's functionality
boost::asio::spawn()

[Peter Dimov via Boost]

Andrzej Krzemienski wrote:
> Please correct me if I am wrong, but my understanding is that Boost.ASIO
> works under C++03, and so does Boost.Coroutine. Whereas Boost.Coroutine2
> requires C++11. If Boost.ASIO were to switch from BoostCoroutine to
> Boost.Coroutine2, it would break its C++03 guarantee. (Some teams in my
> work still have to use C++03.)

It's a bit more complex than that. Boost.Coroutine depends on Boost.Context,
which is C++11. However, in order to not break C++03, Coroutine doesn't use
the public API of Context, but the private "fcontext" parts that are kept C++03
compatible only because of that. This incidentally means that if you compile
Context with the `ucontext` backend instead of the `fcontext` one,
Coroutine breaks.

So it's basically a mess and needs to be cleaned up. Since asio::spawn is a
separate part of Asio, moving just it to C++11 wouldn't be as disruptive a
change as requiring C++11 for all of Asio.

[Andrey Semashev via Boost]

On 11/30/21 22:28, Peter Dimov via Boost wrote:
> Andrzej Krzemienski wrote:
>> Please correct me if I am wrong, but my understanding is that Boost.ASIO
>> works under C++03, and so does Boost.Coroutine. Whereas Boost.Coroutine2
>> requires C++11. If Boost.ASIO were to switch from BoostCoroutine to
>> Boost.Coroutine2, it would break its C++03 guarantee. (Some teams in my
>> work still have to use C++03.)
>
> It's a bit more complex than that. Boost.Coroutine depends on Boost.Context,
> which is C++11. However, in order to not break C++03, Coroutine doesn't use
> the public API of Context, but the private "fcontext" parts that are kept C++03
> compatible only because of that. This incidentally means that if you compile
> Context with the `ucontext` backend instead of the `fcontext` one,
> Coroutine breaks.
>
> So it's basically a mess and needs to be cleaned up. Since asio::spawn is a
> separate part of Asio, moving just it to C++11 wouldn't be as disruptive a
> change as requiring C++11 for all of Asio.

I wonder how much of Boost.Context actually requires C++11. From my
admittedly naive perspective, it should mostly be a bunch of assembler
bits and/or interface glue for system APIs like ucontext, none of which
inherently requires C++11.

[Andrey Semashev via Boost]

I don't think Andrzej was suggesting that.

I think the question is fair - if Boost.Spawn (though I'd really prefer
a more specific name) is C++11, how Boost.ASIO can switch to it while
still supporting C++03?

And if Boost.Spawn is updated to also support C++03, how is it different
from Boost.Coroutine?

[Oliver Kowalke via Boost]

Andrey Semashev via Boost <bo...@lists.boost.org> schrieb am Di., 30. Nov.
2021, 21:00:

It's described in the C++ proposal ...
lifecycle, stack must be managed + ownership and state (for instance
resuming an already running fiber).

[Andrzej Krzemienski via Boost]

wt., 30 lis 2021 o 20:28 Peter Dimov via Boost <bo...@lists.boost.org>
napisał(a):

> Andrzej Krzemienski wrote:
> > Please correct me if I am wrong, but my understanding is that Boost.ASIO
> > works under C++03, and so does Boost.Coroutine. Whereas Boost.Coroutine2
> > requires C++11. If Boost.ASIO were to switch from BoostCoroutine to
> > Boost.Coroutine2, it would break its C++03 guarantee. (Some teams in my
> > work still have to use C++03.)
>
> It's a bit more complex than that. Boost.Coroutine depends on
> Boost.Context,
> which is C++11. However, in order to not break C++03, Coroutine doesn't use
> the public API of Context, but the private "fcontext" parts that are kept
> C++03
> compatible only because of that. This incidentally means that if you
> compile
> Context with the `ucontext` backend instead of the `fcontext` one,
> Coroutine breaks.
>
> So it's basically a mess and needs to be cleaned up. Since asio::spawn is a
> separate part of Asio, moving just it to C++11 wouldn't be as disruptive a
> change as requiring C++11 for all of Asio.
>

Thanks for this explanation.
I am not familiar with the details of the issue, so I basically work based
on the information in this thread.
I am trying to figure out whether the proposed Boost.Spawn better fits
inside Boost.ASIO or in a separate library.

My current understanding is that a C++03 user that does not use the
`ucontext` backend can successfully use `boost::asio:::spawn()` today, and
if the implementation of `boost::asio:::spawn()` were simply upgraded to
use the proposed implementation, the said C++03 user would be hurt. Is this
correct?

If this is correct, it means that the new implementation cannot fully
replace `boost::asio:::spawn()`, but it can be used as an addition, so that
the users can be offered two functions:

boost::asio::spawn_fiber() // preferred if you are C++11 or higher
boost::asio::spawn() // dangerous, but works with C++03

Would this be a correct summary?

Regards,
&rzej;

[Andrey Semashev via Boost]

IMHO, fiber execution, and the fiber class itself, should be a separate
library, which may require C++11. It was my understanding that this is
what Boost.Coroutine and Boost.Coroutine2 are for. Boost.Context, on the
other hand, would only contain low level code for saving and restoring
thread context and nothing else. This kind functionality doesn't seem
like dependent on C++11 features.

[Christopher Kohlhoff via Boost]

On Sun, Nov 28, 2021, at 5:49 PM, Oliver Kowalke via Boost wrote:
> Hi,

> I'd like to propose the new library boost.spawn

> (https://olk.github.io/) for review. boost.asio's boost::asio::spawn()
> uses the deprecated and inefficient boost.coroutine library.
> boost.spawn provides the same functionality while using
> boost.context's fiber directly.

In a nutshell: I am not going to remove spawn() from Asio, nor am I going to keep it in its current form forever. This boost.spawn library might well have utility for some folks right now, but it doesn't reflect the design direction that I have in mind for asio::spawn().

Specifically, the current problem is the code breakage inherent in upgrading asio::spawn() from coroutine-v1 to anything else. This breakage relates to:

- C++03 support
- coroutine-v1 structures that are exposed through the API

I have opted to defer that breakage for as long as possible, until I decide on the form, function, language requirement, and level of backward compatibility of a post-breakage asio::spawn() API.

The proposal in this thread is to work around me by providing a library as an alternative to asio::spawn(), because I have not yet upgraded asio::spawn(). As part of providing this alternative library, the proposal is that I should also remove asio::spawn()... and in doing so break user code.

If there is to be breakage in asio::spawn(), then that breakage creates the opportunity for introducing improvements. I do have some ideas of what these will be... performance, usability, consistency and integration with more recent additions to Asio. Of course this entails spending some time on design work and implementation, but I intend to do this and it will be happening at some point relatively soon. A one-to-one substitution of boost.context in place of boost.coroutine-v1 won't achieve this.


Chris

[Oliver Kowalke via Boost]

Am Mi., 1. Dez. 2021 um 14:31 Uhr schrieb Christopher Kohlhoff via Boost <
bo...@lists.boost.org>:

>
> The proposal in this thread is to work around me by providing a library as
> an alternative to asio::spawn(), because I have not yet upgraded
> asio::spawn().
>

Well I tried to solve this with you in 2017 ...
My intend is to get boost.coroutine removed from the boost libs.

[Oliver Kowalke via Boost]

Am Mi., 1. Dez. 2021 um 14:31 Uhr schrieb Christopher Kohlhoff via Boost <
bo...@lists.boost.org>:

>
>

When do you plan to push your boost.asio commit?