Mojo for UDP socket - alternatives?

[Victor Boivie]

WebRTC lives in the renderer process, and all receiving and sending of UDP data is done through Mojo IPC, for security reasons. For normal media, the network throughput is generally quite limited (a HD stream is typically a few MBps), but WebRTC also have a concept called "Data Channels", which is a peer-to-peer general purpose socket - similar to TCP.

The issue we're facing is that the throughput on that DataChannel socket is quite low, and CPU limited. On my Macbook, it's limited to 12MB/s to another tab (i.e: No actual network sending) and uses ~100% CPU, and I can get to 20MB/s on my high performance Linux desktop.

Doing a pprof[1] shows that a large part of this time is spent in Mojo, with IPC serialization. The pprof shown here highlights the sending - receiving also has the same overhead. 

My question: Is there another way that this can be done more efficiently? Reading the docs seems to indicate that one could perhaps pass shared memory buffers instead?

The rationale is that applications that use DataChannels - even with low throughput - are experiencing too high CPU usage, and also that Chrome can't saturate the actual network bandwidth, which it's suppose to be able to do.

Any ideas other comments are also welcome,

Thanks,

Victor

[1] http://shortn/_l77R7sRU3W (Google internal)

[K. Moon]

Looking at the profile, this seems to be the Mojo definition of interest:

https://source.chromium.org/chromium/chromium/src/+/main:services/network/public/mojom/p2p.mojom;l=55;drc=b1574057ffa55353dec871efc7e77320a1509855

--
You received this message because you are subscribed to the Google Groups "chromium-mojo" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-moj...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-mojo/458df553-905f-4846-bf3d-204ce475c07fn%40chromium.org.

[Ken Rockot]

Couple of observations:

• A good chunk of time is spent allocating and copying packet data in the message - you could avoid this by sending the packet data over a Mojo data pipe or shared memory
  
• Looks like you also have per-packet metadata that frames each packet, and that is (slightly) dominating the serialization cost. This is mostly from P2PPacketInfo serialization.
• P2PPacketInfo is declared in mojom as [Native] which means it uses legacy IPC serialization. This suffers additional overhead compared to plain mojom structs, so you could shave some cost by defining a proper P2PPacketInfo mojom struct.
• A bigger chunk of cost here is from the I/O rather than from serialization. Sending fewer IPCs is likely to make the biggest difference here.

Here's what I'd recommend as a starting point:

• Batch the I/O. Allow Send() to send multiple packets in a single call, and stagger your calls by batching locally up to some small time/space limit.
• Use a data pipe for the packet data rather than sending it inline with Send(). Batch the data pipe writes at the same time as the Send() calls.
• As a potential follow-up also consider serializing the P2PPacketInfo inline in the data pipe to frame each packet there, which can be done easily with the help of generated code if P2PPacketInfo is given a proper mojom struct definition. This would allow you to avoid P2PSocket altogether for packet I/O.

Even just starting with some naive batching might make a big difference.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-mojo/CACwGi-4bcrfMJebL8ejO8T9yN6y4ZUgfUD2obnDo6MQKEgcK%3Dw%40mail.gmail.com.

[Victor Boivie]

Thanks Ken,

Batching packets can be straight-forward in some situations - especially for data channels, which use SCTP as underlying protocol, which has a flow control algorithm similar to TCP where generally it sends three packets each time an acknowledgment is received. For media traffic, there is a packet pacer in front of it, to ensure that packets are actually spread out in time, which directly contradicts batching. I'll definitely see how I can adapt data channels to batch packets.

The other ideas about using data pipes and mojoification of P2PPacketInfo sound also promising, and I'll try to find some existing code to get inspired by. 

I'll reach out with more questions as I learn.

Thanks,

Victor