Using I/O completion ports in TCPSocketWin - Request for reviewer

[Francois Pierre Doray]

The Catan team looks at data (traces, stack samples, histograms…) on a regular basis to understand Chrome performance. As part of that work, we found that net::TCPSocketWin’s usage of base::ObjectWatcher, to be notified of completed read/writes, was among the top sources of context switches in local browsing sessions (bug). We took special interest in this because:

• We have a long time suspicion, based on system traces analysis, that heavy context switching negatively affects Chrome performance.

• Context switches induced by net::TCPSocketWin’s usage of base::ObjectWatcher can be avoided by using I/O Completion ports instead of events to be notified of completed read/writes.

The context switches of interest occur:

• When an event is signaled, a thread controlled by Windows posts to the IO thread [code].

• The IO thread then unregisters the wait, which often involves being descheduled [code].

We would like to study the effect of eliminating the context switches induced by net::TCPSocketWin’s usage of base::ObjectWatcher on Chrome’s top level metrics (LCP, INP) and low level metrics (Net.HttpTimeToFirstByte, Net.HttpJob.TotalTimeSuccess) via a field experiment. If the effect is positive, we can consider shipping the change to make Chrome faster. For my team, it would also mean that investing further in eliminating context switches is worth it. To conduct this experiment, we implemented an alternative version of net::TCPSocketWin which uses I/O completion ports instead of events to be notified of completed read/writes:

• Pre-work to facilitate adding a new implementation:

• 5725524: Make SocketDataPump work when TCPSocket::ReadIfReady isn't implemented. 

• 5797014: [net] Add TCPSocketWin::connect_callback_.

• 5627341: [net] Refactor TCPSocketWin to allow alternative read/write impls.

• Implementation:

• 5627052: [net] Add TCPSocketWin implementation based on IO completion port.

• 5796670: [net] Handle synchronous write completion in TcpSocketIoCompletionPortWin.

• Testing config:

• 5716856: [net] Add TcpSocketIoCompletionPortWin to testing config. 

Which //net OWNER could review the implementation? A Windows expert in the Catan team (grt@) pre-reviewed the implementation to make the OWNER review less time-consuming.

Note: We are aware that a previous version of TCPSocketWin used overlapped reads (https://chromiumcodereview.appspot.com/22188002). However, it used an event instead of an I/O Completion port to be notified of completed reads, so it did not eliminate the context switches from base::ObjectWatcher. A new experiment is therefore still interesting for us (also, a lot has changed in Chrome, Windows and the Web since then).

Thanks a lot in advance!

For the large context, see go/working-with-catan (Google-internal)