[boost] ASIO in the Standard (was Re: C++ committee meeting report)

[Dean Michael Berris]

Cross-posting to netwo...@isocpp.org and c++std-lib-ext.

On Wed, Jun 25, 2014 at 12:47 AM, Beman Dawes <bda...@acm.org> wrote:
>
> * The Library Evolution Working Group (LEWG) voted to base the Networking
> TS on Boost.ASIO. That was a major surprise, as a Networking Study Group
> had been trying to build up a Networking TS from many small proposals. But
> in LEWG polls, support for ASIO was overwhelming, with no one at all voting
> against. This was a major vote of confidence for Boost.ASIO and Chris
> Kohlhoff. And several ASIO supporters were not even in the room at the time
> because Filesystem issues were being worked on elsewhere.
>

FWIW, I was in the room when the Boost.Asio proposal was being
discussed way back in Kona. I haven't seen the most recent update to
the proposal, but if my original concerns were addressed, mainly:

- The io_service type was trying to be too many things at the same
time (executor, ran in thread-pools, implementing IO-specific hooks,
etc.) and was too prominent in the interfaces of primitive types (i.e.
sockets, etc.).

- There were too many leaking abstractions crossing boundaries --
being able to reach in to get native handles, buffer lifetimes
crossing in-and-out, etc.

- The fact that it seems the only implementation of the networking
parts (and inter-operating bits implied by the interface) would be
through a proactor -- how the API was only proactive, no support for
just purely reactive applications.

- There's no means for doing "zero-copy" I/O which is a
platform-specific but modern way of doing data transfer through
existing interfaces.

... then this I think is *good news*!

Having worked with quite a few network applications now in the past
few years, I'm happy with what Boost.Asio provides. However I think
there are other systems/libraries we can learn from to inform the
design here (I tried getting a comprehensive survey paper in, but that
proved a little harder than I originally thought and ended up with
just doing a poor job for HTTP).

A few that come to mind for potential approaches here are:

- Transport-level abstractions. Considering whether looking at
policy-driven transport protocol abstractions (say, an http
connection, a raw TCP connection, an SCTP connection with multiple
streams, etc.) would be more suitable higher-level abstractions than
sockets and write/read buffers.

- Agent-based models. More precisely explicitly having an agent of
sorts, or an abstraction of a client/server where work could be
delegated, composed with executors and schedulers.

I admit I haven't exactly been following through on attending the
Networking SG meetings and sending in papers myself. I sincerely hope
though that our collective love for Boost.Asio doesn't preclude us
from thinking at higher levels of abstractions and in broadening our
view/understanding of the network-programming landscape.

I say this with all due respect -- while network programming sounds
like it's all about sockets, buffers, event loops, and such there are
the "boring" bits like addressing, network byte ordering,
encoding/decoding algorithms (for strings and blobs of data), framing
algorithms, queueing controls, and even back-off algorithms,
congestion control, traffic shaping. There's more things to think
about too like data structures for efficiently representing frames,
headers, network buffers, routing tables, read-write buffer halves, ip
address tries, network topologies, protocol encodings (ASN.1, MIME and
friends), and a whole host of network-related concepts we're not even
touching in the networking discussions.

Having said this, I'm happy that we're willing to start with Boost.Asio. :)

> The next C++ committee meeting is November 3rd to 8th in Urbana-Champaign,
> IL, USA. As always, Boosters welcome.
>

Would really love to go, or at least be able to send comments to
papers in time for that one.

Thanks for the report Beman!

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Niall Douglas]

On 3 Jul 2014 at 23:16, Dean Michael Berris wrote:

> FWIW, I was in the room when the Boost.Asio proposal was being
> discussed way back in Kona. I haven't seen the most recent update to
> the proposal, but if my original concerns were addressed, mainly:
>
> - The io_service type was trying to be too many things at the same
> time (executor, ran in thread-pools, implementing IO-specific hooks,
> etc.) and was too prominent in the interfaces of primitive types (i.e.
> sockets, etc.).

Some would call that the low-level interfacing necessary to achieve
bare metal performance.

> - There were too many leaking abstractions crossing boundaries --
> being able to reach in to get native handles, buffer lifetimes
> crossing in-and-out, etc.

Ditto.

> - The fact that it seems the only implementation of the networking
> parts (and inter-operating bits implied by the interface) would be
> through a proactor -- how the API was only proactive, no support for
> just purely reactive applications.

Kinda hard to do true async without.

> - There's no means for doing "zero-copy" I/O which is a
> platform-specific but modern way of doing data transfer through
> existing interfaces.

I don't know what you refer to here. ASIO doesn't copy data being
sent or received. It passes through literally what it receives from
the OS.

> A few that come to mind for potential approaches here are:
>
> - Transport-level abstractions. Considering whether looking at
> policy-driven transport protocol abstractions (say, an http
> connection, a raw TCP connection, an SCTP connection with multiple
> streams, etc.) would be more suitable higher-level abstractions than
> sockets and write/read buffers.
>
> - Agent-based models. More precisely explicitly having an agent of
> sorts, or an abstraction of a client/server where work could be
> delegated, composed with executors and schedulers.

These are all outside the remit of a core C++ networking library.

> I say this with all due respect -- while network programming sounds
> like it's all about sockets, buffers, event loops, and such there are
> the "boring" bits like addressing, network byte ordering,
> encoding/decoding algorithms (for strings and blobs of data), framing
> algorithms, queueing controls, and even back-off algorithms,
> congestion control, traffic shaping. There's more things to think
> about too like data structures for efficiently representing frames,
> headers, network buffers, routing tables, read-write buffer halves, ip
> address tries, network topologies, protocol encodings (ASN.1, MIME and
> friends), and a whole host of network-related concepts we're not even
> touching in the networking discussions.

I'm personally not unsympathetic to this sentiment. However, it would
surely need POSIX to move on it first before C++ could.

Niall

--
ned Productions Limited Consulting
http://www.nedproductions.biz/
http://ie.linkedin.com/in/nialldouglas/

[Dean Michael Berris]

On Fri, Jul 4, 2014 at 9:18 PM, Niall Douglas <s_sour...@nedprod.com> wrote:
> On 3 Jul 2014 at 23:16, Dean Michael Berris wrote:
>
>> FWIW, I was in the room when the Boost.Asio proposal was being
>> discussed way back in Kona. I haven't seen the most recent update to
>> the proposal, but if my original concerns were addressed, mainly:
>>
>> - The io_service type was trying to be too many things at the same
>> time (executor, ran in thread-pools, implementing IO-specific hooks,
>> etc.) and was too prominent in the interfaces of primitive types (i.e.
>> sockets, etc.).
>
> Some would call that the low-level interfacing necessary to achieve
> bare metal performance.
>

But, there's no reason you couldn't decompose this and not make it
part of the invariants of the objects associated with them. For
example, when you create a socket object, can you not do that by
acquiring it from some function that internally would determine how
it's wired?

auto connection = establish_connection(remote_address, my_executor);

>> - There were too many leaking abstractions crossing boundaries --
>> being able to reach in to get native handles, buffer lifetimes
>> crossing in-and-out, etc.
>
> Ditto.
>
>> - The fact that it seems the only implementation of the networking
>> parts (and inter-operating bits implied by the interface) would be
>> through a proactor -- how the API was only proactive, no support for
>> just purely reactive applications.
>
> Kinda hard to do true async without.
>

Can you expand this a little?

>> - There's no means for doing "zero-copy" I/O which is a
>> platform-specific but modern way of doing data transfer through
>> existing interfaces.
>
> I don't know what you refer to here. ASIO doesn't copy data being
> sent or received. It passes through literally what it receives from
> the OS.
>

Right.

Does it support direct-memory I/O -- where I write to some memory
location instead of scheduling a write? See RDMA, and on Linux
vmsplice(...). Can I "donate" the memory I have from user-space to the
kernel, which just remaps the memory onto a device's memory buffer?
How do I achieve "raw" buffer performance if I have to provide a
user-space allocated buffer for data to be written into from the
kernel, and then write out data if the data was in a user-space buffer
that gets passed to the kernel?

We've moved on to better interfaces in the lower levels of the
networking stack, and it would be a shame if we precluded this from
being adopted by a standard library specification.

>> A few that come to mind for potential approaches here are:
>>
>> - Transport-level abstractions. Considering whether looking at
>> policy-driven transport protocol abstractions (say, an http
>> connection, a raw TCP connection, an SCTP connection with multiple
>> streams, etc.) would be more suitable higher-level abstractions than
>> sockets and write/read buffers.
>>
>> - Agent-based models. More precisely explicitly having an agent of
>> sorts, or an abstraction of a client/server where work could be
>> delegated, composed with executors and schedulers.
>
> These are all outside the remit of a core C++ networking library.
>

Why?

>> I say this with all due respect -- while network programming sounds
>> like it's all about sockets, buffers, event loops, and such there are
>> the "boring" bits like addressing, network byte ordering,
>> encoding/decoding algorithms (for strings and blobs of data), framing
>> algorithms, queueing controls, and even back-off algorithms,
>> congestion control, traffic shaping. There's more things to think
>> about too like data structures for efficiently representing frames,
>> headers, network buffers, routing tables, read-write buffer halves, ip
>> address tries, network topologies, protocol encodings (ASN.1, MIME and
>> friends), and a whole host of network-related concepts we're not even
>> touching in the networking discussions.
>
> I'm personally not unsympathetic to this sentiment. However, it would
> surely need POSIX to move on it first before C++ could.
>

Why?

[Niall Douglas]

On 4 Jul 2014 at 21:59, Dean Michael Berris wrote:

> > Some would call that the low-level interfacing necessary to achieve
> > bare metal performance.
>
> But, there's no reason you couldn't decompose this and not make it
> part of the invariants of the objects associated with them. For
> example, when you create a socket object, can you not do that by
> acquiring it from some function that internally would determine how
> it's wired?

I agree this is what should be done in a ground up refactor. But that
isn't what's on the table. What is proposed is to standardise a
subset of the common practice.

> >> - The fact that it seems the only implementation of the networking
> >> parts (and inter-operating bits implied by the interface) would be
> >> through a proactor -- how the API was only proactive, no support for
> >> just purely reactive applications.
> >
> > Kinda hard to do true async without.
>
> Can you expand this a little?

http://www.boost.org/doc/libs/1_55_0/doc/html/boost_asio/overview/core/async.html

http://www.artima.com/articles/io_design_patterns2.html

You may of course not meant reactor when you wrote "reactive".

For me personally, how QNX (and I assume Hurd) do async i/o is the
gold standard. The NT kernel does a very faithful emulation of true
async. BSD at least provides a usable aio_* POSIX API. The rest
really aren't great on async.

> >> - There's no means for doing "zero-copy" I/O which is a
> >> platform-specific but modern way of doing data transfer through
> >> existing interfaces.
> >
> > I don't know what you refer to here. ASIO doesn't copy data being
> > sent or received. It passes through literally what it receives from
> > the OS.
>
> Right.
>
> Does it support direct-memory I/O -- where I write to some memory
> location instead of scheduling a write? See RDMA, and on Linux
> vmsplice(...). Can I "donate" the memory I have from user-space to the
> kernel, which just remaps the memory onto a device's memory buffer?
> How do I achieve "raw" buffer performance if I have to provide a
> user-space allocated buffer for data to be written into from the
> kernel, and then write out data if the data was in a user-space buffer
> that gets passed to the kernel?
>
> We've moved on to better interfaces in the lower levels of the
> networking stack, and it would be a shame if we precluded this from
> being adopted by a standard library specification.

Yeah ... well, actually no, I wouldn't call vmsplice() or any of its
ilk as anything deserving the label "better". That whole way of
making DMA possible is a hack forced by Linux's networking stack
being incapable of using DMA automatically.

After all, BSD and Windows manage DMA with socket i/o automatically
and transparently. No magic syscalls needed. Linux should up its game
here instead of syscall hacks.

Even if splice() et all were a good idea to encourage, they aren't
standardised by POSIX and therefore are out of scope for
standardisation by C++. ISO standards are there to standardise
established common practice, not try to design through central
committee.

> >> A few that come to mind for potential approaches here are:
> >>
> >> - Transport-level abstractions. Considering whether looking at
> >> policy-driven transport protocol abstractions (say, an http
> >> connection, a raw TCP connection, an SCTP connection with multiple
> >> streams, etc.) would be more suitable higher-level abstractions than
> >> sockets and write/read buffers.
> >>
> >> - Agent-based models. More precisely explicitly having an agent of
> >> sorts, or an abstraction of a client/server where work could be
> >> delegated, composed with executors and schedulers.
> >
> > These are all outside the remit of a core C++ networking library.
>
> Why?

ISO standards are there to standardise established common practice,
not try to design through central committee.

Also, think in terms of baby steps. Start with a good solid low level
async networking library which is tightly integrated into threading,
executors and the rest of the STL async facilities. That already will
be hideously hard to do. For the next major C++ standard build on
that with better abstractions.

> >> I say this with all due respect -- while network programming sounds
> >> like it's all about sockets, buffers, event loops, and such there are
> >> the "boring" bits like addressing, network byte ordering,
> >> encoding/decoding algorithms (for strings and blobs of data), framing
> >> algorithms, queueing controls, and even back-off algorithms,
> >> congestion control, traffic shaping. There's more things to think
> >> about too like data structures for efficiently representing frames,
> >> headers, network buffers, routing tables, read-write buffer halves, ip
> >> address tries, network topologies, protocol encodings (ASN.1, MIME and
> >> friends), and a whole host of network-related concepts we're not even
> >> touching in the networking discussions.
> >
> > I'm personally not unsympathetic to this sentiment. However, it would
> > surely need POSIX to move on it first before C++ could.
> >
>
> Why?

Once again: ISO standards are there to standardise established common
practice, not try to design through central committee. If all the
platforms did something like managing routing interfaces identically,
we could argue in favour of it on the case of merit. But they don't,
so we can't.

One would also be extremely hesitant to standardise anything which
hasn't been given full and proper deliberation by the ISO working
group responsible for it. I feel little love for how the AWG see the
world personally (I find interacting with them deeply frustrating),
but they have their role in ISO and they haven't done too bad a job
of things in the wider picture.

[Niklas Angare]

"Niall Douglas" wrote:
> For me personally, how QNX (and I assume Hurd) do async i/o is the
> gold standard.

As a QNX user I'm curious about what you're referring to here.

Regards,

Niklas Angare

[Niall Douglas]

On 6 Jul 2014 at 16:20, Niklas Angare wrote:

> "Niall Douglas" wrote:
> > For me personally, how QNX (and I assume Hurd) do async i/o is the
> > gold standard.
>
> As a QNX user I'm curious about what you're referring to here.

All micro kernel OSs act as if all syscalls are non-blocking, even
async. I cannot speak for the Hurd, but for QNX the central scheduler
works by dispatching coroutines which when they block due to a
message send they are saved and the next unblocked coroutine is
continued.

For example, if you try writing a byte to a fd, under the bonnet in
the write() implementation that turns into a MsgSend which sends the
byte to the channel corresponding to that fd which will appear in the
resource manager's channel for that fd. The sending coroutine is then
marked as blocked on message send, is suspended and the next
coroutine executed for the next message arriving or in the queue.

What this turns into is that you can pretty much skip threads
entirely in most QNX programs - you use threads solely for having
work executed on multiple CPU cores concurrently, not for concurrency
in kernel space as you get for free concurrency in "kernel space" all
the time (there is almost no kernel space in QNX, device drivers or
handling page faults or interrupt handling are just another message
dispatch and handle). As all syscalls can be scheduled and completed
later on the same thread via coroutines, that means you can issue a
read gather across multiple fds and as the results complete you'll
receive messages with the results which appear to your code as the
API returning (note you can bypass this or override it to your
heart's content, QNX lets you hack the message dispatch very easily).

Obviously such a facility would be extremely useful to something like
ASIO because one can dispense entirely with the select()/epoll()
machinery at the heart of the io_service which dispatches completions
to the next available thread worker. In QNX an ASIO equivalent is
already there at the heart of the OS - in fact, in QNX, the message
passing infrastructure is pretty much the only thing there is, as
everything from handling page faults in memory mapped files through
to device drivers is implemented with the same core message passing
infrastructure.

QNX is a neat system. Once you've written a device driver for it
you'll never enjoy writing one for Linux ever again. Linux looks
quite stone age and backwards in comparison, though in fairness QNX
struggles to scale as Android can. In particular, QNX copies a
*tremendous* amount of memory around and will eliminate any
advantages of L1/L2 caches very quickly indeed. BB10 runs at main
memory speed a lot of the time as the L2 cache might as well not be
there under message passing load.