net.Conn.Read() performance

[anton....@gmail.com]

Hello,

I'm trying to understand whether net.Conn.Read() is expected to use a lot more CPU than reading using the lower level socket interface or if I am doing something wrong(or perhaps comparing apples to oranges)?

Consumer and producer apps are running on two different Linux machines, in "top" the net.Conn.Read() app is showing 20-30% higher CPU than the app using Recvfrom()

Thanks

Anton

producer.go

import "net"

func main() {

l, _ := net.Listen("tcp", "192.168.1.7:3333")

conn, _ := l.Accept()

for {

conn.Write([]byte("abcdefgh"))

}

}

consumer_recvfrom.go

import (

"golang.org/x/sys/unix"

"net"

)

func main() {

var addr [4]byte

copy(addr[:], net.ParseIP("192.168.1.7").To4())

fd, _ := unix.Socket(unix.AF_INET, unix.SOCK_STREAM, 0)

unix.Connect(fd, &unix.SockaddrInet4{Port: 3333, Addr: addr})

b := make([]byte, 65536, 65536)

for {

unix.Recvfrom(fd, b, 0)

}

}

consumer_netread.go

import "net"

func main() {

conn, _ := net.Dial("tcp", "192.168.1.7:3333")

b := make([]byte, 65536, 65536)

for {

conn.Read(b)

}

}

consumer_recvfrom:

$ vmstat 10 10

procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----

 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st

 3  0    864 1568372    332 2072064    0    0     0     0    2    1  0  0 100  0  0

 0  0    864 1568616    332 2071980    0    0     0     0 14950 10844  0  9 91  0  0

 0  0    864 1568832    332 2071976    0    0     0     0 14307 12735  0  8 92  0  0

 0  0    864 1568616    332 2071976    0    0     0     0 15106 10634  0  9 90  0  0

 0  0    864 1568708    332 2071980    0    0     0     0 15214 10839  0  9 90  0  0

 1  0    864 1568900    332 2071880    0    0     0     0 15466 13163  0  8 91  0  0

 0  0    864 1568572    332 2072116    0    0     0     0 14579 11268  0  9 91  0  0

 1  0    864 1568788    332 2071992    0    0     0     0 15278 11370  0  9 91  0  0

 1  0    864 1568884    332 2071912    0    0     0     0 16468 11986  0  9 91  0  0

 1  0    864 1568756    332 2071880    0    0     0     0 15769 11242  0  9 90  0  0

TX eth0: 126469 pkts/10s RX eth0: 558847 pkts/10s

TX eth0: 127229 pkts/10s RX eth0: 530672 pkts/10s

TX eth0: 113907 pkts/10s RX eth0: 572163 pkts/10s

TX eth0: 114638 pkts/10s RX eth0: 519287 pkts/10s

TX eth0: 119010 pkts/10s RX eth0: 553152 pkts/10s

consumer_netread:

$ vmstat 10 10

procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----

 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st

 2  0    864 1570996    332 2071956    0    0     0     0    2    2  0  0 100  0  0

 1  0    864 1571004    332 2072132    0    0     0     0 28082 32770  2 12 86  0  0

 2  0    864 1571108    332 2071924    0    0     0     0 28370 29395  2 12 86  0  0

 2  0    864 1571808    332 2071924    0    0     0     0 28847 29513  2 12 86  0  0

 2  0    864 1571196    332 2071892    0    0     0     0 28499 31063  2 13 86  0  0

 1  0    864 1571136    332 2071892    0    0     0     0 30216 33711  2 12 86  0  0

 2  0    864 1570920    332 2071892    0    0     0     0 28224 30675  2 12 86  0  1

 2  0    864 1570860    332 2071892    0    0     0     0 26453 29701  2 12 86  0  0

 1  0    864 1571112    332 2071892    0    0     0     0 26917 29510  2 12 86  0  0

 0  0    864 1571128    332 2071892    0    0     0     0 27030 29683  2 13 86  0  0

TX eth0: 80310 pkts/10s RX eth0: 584112 pkts/10s

TX eth0: 88469 pkts/10s RX eth0: 512453 pkts/10s

TX eth0: 89665 pkts/10s RX eth0: 512174 pkts/10s

TX eth0: 84497 pkts/10s RX eth0: 480085 pkts/10s

TX eth0: 92261 pkts/10s RX eth0: 540044 pkts/10s

[anto...@gmail.com]

Any thoughts, anyone?

[James Bardin]

Conn.Read is a much higher level construct the using the syscalls directly.  The largest part of the overhead is coordinating with the network poller. There's also a slight overhead of using the net.Conn interface, so asserting the value to a *net.TCPConn may save you a little (though I'm not certain how much of a difference that makes in recent releases).

Using the syscalls, you're essentially dedicating a single OS thread to handle that socket's IO. 

On Wednesday, November 30, 2016 at 2:13:10 PM UTC-5, anto...@gmail.com wrote:

Any thoughts, anyone?