Mojo socket APIs

[Yuzhu Shen]

Greetings, net-dev.

As per Ryan's suggestion, I am sending this mail to let you know that we are going to add some Mojo socket APIs. In the short term, these APIs are only used internally. But eventually we probably will consider stabilizing them and exposing them to the public.

The bug to track this work: https://code.google.com/p/chromium/issues/detail?id=402671

I just drafted the UDP API: https://codereview.chromium.org/590263002/

Ryan raised the question whether we should exposed "low level" sockets (users are responsible for DNS, IPv4/IPv6 handling, socket re-use, proxy establishment) or "high level" (it just works).

IMO, the two categories are useful for different scenarios. Some applications may want "high level" APIs (say, WebSocket) so they don't need to worry too much. On the other hand, some applications do require the ability to access "low level" APIs. And it seems reasonable that they need to pay the price of extra work (IPv4/IPv6 handling, proxy establishment, etc.).

The UDP socket that I am adding, falls in the category of "low level". The capability of this API is similar to net::UDPServerSocket.

Please let me know if you have any concerns / suggestions. Thanks!

--
Best regards,
Yuzhu Shen.

[Ryan Sleevi]

Adding Paul, Thomas, and David directly to this thread, as I'm hoping they can share some of their insights into real-world UDP-using applications (e.g. a hypothetical DNS client, as one example), and how well this API would or would not meet their needs.

For UDP, I think we're inherently at a 'low level' nature (or at least, proxies are unquestionably out of the question), and I think it makes sense to expose the degree of control.

We do want to make sure that the API itself ensures good practices - see https://groups.google.com/a/chromium.org/d/msg/net-dev/U2ecAARKvAg/vEd0VJ4C4WsJ for a recent discussion about some of the tradeoffs between API and usability.

[William Chan (陈智昌)]

Not to pick on Yuzhu, but I'm minorly grumpy that net-dev@ doesn't get
consulted about new network APIs before they land. I feel like the
process is broken. Thank you Ryan for catching this.

> --
> You received this message because you are subscribed to the Google Groups
> "net-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to net-dev+u...@chromium.org.
> To post to this group, send email to net...@chromium.org.
> To view this discussion on the web visit
> https://groups.google.com/a/chromium.org/d/msgid/net-dev/CACvaWvady_%2B%3D7PcN-jDdahO-J7d-gWQLeje8NGgM3wtUhDR5_w%40mail.gmail.com.

[Darin Fisher]

Note: This isn't a feature of Chrome. It is important to have low-level socket (and related) APIs available to sandboxed code. I think it is also useful to design higher level APIs that are simpler to use correctly, etc.

-Darin

[Ryan Sleevi]

Sure, even if it's not Chrome, it still has the same concerns.

It's effectively a client networking API as part of a platform.

Chrome networking deals with a lot of client networking APIs that are parts of platforms - namely, everywhere Chrome runs.

Like Will, not trying to block any efforts, but just echoing the sentiment that we're here to help, to share the pain points we've felt, and to ideally avoid them in new APIs. We're happy to provide that feedback whether you're Microsoft or Apple trying to replace the BSD APIs everyone knows and loves or you're Google trying to work on Android or (some other platform here). :)

[Darin Fisher]

I'd love for you guys to be way more involved with Mojo :-)

-Darin

[Yuzhu Shen]

On Tue, Sep 30, 2014 at 12:33 PM, Ryan Sleevi <rsl...@chromium.org> wrote:

Sure, even if it's not Chrome, it still has the same concerns.

It's effectively a client networking API as part of a platform.

Chrome networking deals with a lot of client networking APIs that are parts of platforms - namely, everywhere Chrome runs.

Like Will, not trying to block any efforts, but just echoing the sentiment that we're here to help, to share the pain points we've felt, and to ideally avoid them in new APIs. We're happy to provide that feedback whether you're Microsoft or Apple trying to replace the BSD APIs everyone knows and loves or you're Google trying to work on Android or (some other platform here). :)

Sorry. I didn't mean to break the process. And I really appreciate all your help. :)

I mistakenly thought that the network team was aware of the bug (since networking label was added). And I should have added networking people to review my API.

[William Chan (陈智昌)]

The problem I have with this implication of how the API development
process should work, is it implies we should subscribe to all the
different mailing lists to track their networking related APIs.
Extensions, apps, pepper, Mojo, yada yada. I don't think that's
feasible for us, because of discoverability issues (we don't know that
we need to care if we don't know of a list we should be monitoring)
and email overload (processing the emails to figure out if we should
care). It seems more reasonable for team X, if they are doing
something network related, to loop in net-dev@ in case we have an
opinion.

Again, I don't mean to pick on Yuzhu. But this is a trend, and we get
stuck with unfortunate choices that may have been avoided if we had
been looped in earlier. So I'm calling it out.

[Colin Blundell]

As net-dev@ is relatively new, maybe there's more publicization that needs to be done for Chromium folks to get in their heads "If I'm doing something network-related, I should get the opinion of the net-dev@ folks" in the same way that e.g. people understand looping in security folks today?

[Ryan Sleevi]

It's not really all that new.

What stands out here (and in other cases that Will mentioned) is that people are implementing APIs without really talking to anyone who works on the //net stack (and if I'm wrong in that, then people in the //net stack aren't really forwarding on to net-dev@, and that's something we need to work on)

Whether it's the mailing list or code reviews or simple IM pings, we definitely are having a breakdown occuring.

But yes, there's been lots of ideas floated on how to resolve this.

[Ryan Sleevi]

Related, here's a Mojo TCP socket API proposal

https://codereview.chromium.org/613683006/

I have not taken a look at it yet, but net-dev@, please comment and review ASAP :)

[William Chan (陈智昌)]

What's the evolution plan for this UDP API? Is it imagined that
someone would be able to implement a high performance UDP based
protocol within a Mojo app? One thing that immediately catches my eye
is the use of byte arrays in the mojom files, which seems to imply
buffer copies to me.

On Tue, Sep 30, 2014 at 10:57 AM, Ryan Sleevi <rsl...@chromium.org> wrote:

[William Chan (陈智昌)]

Oh, and will there eventually be a Bring Your Own Buffer API?
OnReceived() does not seem to provide for that.

[Yuzhu Shen]

Hi, Will.

I created a CL so that we can more easily review the API:

https://codereview.chromium.org/612403003/

(The initial patch only contains whitespace changes since the API definition has been landed.) 

Besides, the implementation CL is here: https://codereview.chromium.org/596383002/

Yes, there exists buffer copies. My experience with Pepper suggests that the performance is not too bad. (Recently I adjusted the buffering strategy for Pepper UDP and did some measurement.)

It is possible that an implementation using Mojo DataPipe can achieve better performance. I do plan to compare the performance of the two approaches and switch if necessary. (As I said, this API is in "draft" stage.)

Thanks!

[William Chan (陈智昌)]

Great, I'll send future comments there, thanks!

Lastly though on the buffer copies, I want to make expressly clear
that it is important for a certain category of applications. You
simply cannot author as performant an application as an nginx,
varnish, etc with this sort of API, since most of what those apps do
is just read/write buffers, therefore they are super sensitive to
forced buffer copies. But you could definitely perform as well as
node.js and its ilk. So, the question is, what type of applications do
we want to support? I defer to the Mojo folks on this, and just want
to emphasize that this performance characteristic is a hard
requirement for certain apps. But maybe Mojo doesn't care to support
those apps (which is absolutely fine!).

Similarly, if you care about those apps that I've described, you need
a BYOB API. They require that level of control over buffer layout.

[Yuzhu Shen]

On Tue, Sep 30, 2014 at 2:27 PM, William Chan (陈智昌) <will...@chromium.org> wrote:

Great, I'll send future comments there, thanks!

Lastly though on the buffer copies, I want to make expressly clear
that it is important for a certain category of applications. You
simply cannot author as performant an application as an nginx,
varnish, etc with this sort of API, since most of what those apps do
is just read/write buffers, therefore they are super sensitive to
forced buffer copies. But you could definitely perform as well as
node.js and its ilk. So, the question is, what type of applications do
we want to support? I defer to the Mojo folks on this, and just want
to emphasize that this performance characteristic is a hard
requirement for certain apps. But maybe Mojo doesn't care to support
those apps (which is absolutely fine!).

Similarly, if you care about those apps that I've described, you need
a BYOB API. They require that level of control over buffer layout.

Thanks for the detailed explanation. I will keep this in mind.

I think it is reasonable to optimize the API so that it can support a wide range of applications.

Once the facility of sending structural data over Mojo DataPipe is ready, I will experiment with it which is supposed to have less (0?) buffer copies.

IMO, the current Mojo UDP socket is useful in that:

- it can be used as a baseline to measure performance gain of different approaches.

- people can try it out and give early feedback.

[William Chan (陈智昌)]

Thanks for clarifying, and that totally makes sense to me. But I'll
point out that unless you have someone trying to write something as
high throughput as HAProxy or Varnish, you won't be able to fully tell
how the performance works. You should be very careful about system
calls and buffer copies. Here's a useful link to check out:
http://www.haproxy.org/#perf. Here are some choice quotes:

* "Single-buffering without any data copy between reads and writes
whenever possible. This saves a lot of CPU cycles and useful memory
bandwidth. Often, the bottleneck will be the I/O busses between the
CPU and the network interfaces. At 10-100 Gbps, the memory bandwidth
can become a bottleneck too."
* "Zero-copy forwarding is possible using the splice() system call
under Linux, and results in real zero-copy starting with Linux 3.5.
This allows a small sub-3 Watt device such as a Seagate Dockstar to
forward HTTP traffic at one gigabit/s."

And then you should check out the graphs at
http://www.haproxy.org/10g.html and
http://www.haproxy.org/10g-20080419.html. TL;DR: Buffer sizing and
avoiding buffer copies is very important.

I'd quote stuff from the Google frontend team, but that's confidential
so I can't share that information. But the overall message is the
same: avoid buffer copies.

And it's not just highly scalable servers that care about this. Mobile
apps will want to have this level of control over memory use. This is
why a BYOB API is critical. Then you can use a slab allocator and
backpressure signals to tweak memory consumption accordingly. But this
is something I imagine a mobile app author can tell you.

[Tim Steele]

On Tue, Sep 30, 2014 at 3:44 PM, William Chan (陈智昌) <will...@chromium.org> wrote:

On Tue, Sep 30, 2014 at 3:25 PM, Yuzhu Shen <yzs...@google.com> wrote:
>
>
> On Tue, Sep 30, 2014 at 2:27 PM, William Chan (陈智昌) <will...@chromium.org>
> wrote:
>>
>> Great, I'll send future comments there, thanks!
>>
>> Lastly though on the buffer copies, I want to make expressly clear
>> that it is important for a certain category of applications. You
>> simply cannot author as performant an application as an nginx,
>> varnish, etc with this sort of API, since most of what those apps do
>> is just read/write buffers, therefore they are super sensitive to
>> forced buffer copies. But you could definitely perform as well as
>> node.js and its ilk. So, the question is, what type of applications do
>> we want to support? I defer to the Mojo folks on this, and just want
>> to emphasize that this performance characteristic is a hard
>> requirement for certain apps. But maybe Mojo doesn't care to support
>> those apps (which is absolutely fine!).

FWIW the use case we've been using for now to prototype structured (or "framed") data flowing over mojo DataPipe so far was multimedia based; e.g compressed audio/video buffers for decoding and rendering. Performance is definitely a consideration there (especially video), but we still don't quite have a working system to play with.  Anyway just thought I'd mention for context.  (Happy to provide links / more context if you're interested.)

[Aaron Boodman]

On Tue, Sep 30, 2014 at 2:27 PM, William Chan (陈智昌) <will...@chromium.org> wrote:

Great, I'll send future comments there, thanks!

Lastly though on the buffer copies, I want to make expressly clear
that it is important for a certain category of applications. You
simply cannot author as performant an application as an nginx,
varnish, etc with this sort of API, since most of what those apps do
is just read/write buffers, therefore they are super sensitive to
forced buffer copies. But you could definitely perform as well as
node.js and its ilk. So, the question is, what type of applications do
we want to support? I defer to the Mojo folks on this, and just want
to emphasize that this performance characteristic is a hard
requirement for certain apps. But maybe Mojo doesn't care to support
those apps (which is absolutely fine!).

Similarly, if you care about those apps that I've described, you need
a BYOB API. They require that level of control over buffer layout.

The philosophy of Mojo is to expose the metal. I think wherever possible we should expose as low-level an API as we possibly can and leave it to higher layers to add the sugar.