Go UDP performance

[Tharaneedharan Vilwanathan]

Hi All,

I am trying to send a lot of UDP packets from Go code but I realized UDP performance is too low. The max I was able to do is about 160Mbps. This is in Ubuntu 16.10 on x86_64 (i7-6700HQ).

I tried to google on this and it looks like this is about the performance we can get. I am a bit surprised.

Am I missing something? Any suggestions on how to improve the performance?

Thanks

dharani

[Dave Cheney]

Can you share some more details

1. which version of Go

2. which operating system

3. where are you sending from / to, is it over localhost, does the other side care about acknowledging receipt

4. can you show your code

5. have you profiled your code? What resource is limiting the throughput of your application? CPU, Operating system, network driver?

[Rich]

I would wireshark the data coming in to both sides so that you can see when the packet was transmitted, and when it was received by the other side. That way you can isolate if it's network or Go.

[Marcus Franke]

Hi,

an additional note, don't forget to monitor the netstat udp counter on both servers.

% netstat -auns | grep -A 7 "Udp:"

Udp:

    9381 packets received

    0 packets to unknown port received

    0 packet receive errors

    1009 packets sent

    0 receive buffer errors

    0 send buffer errors

    IgnoredMulti: 1264

Maybe your client is dropping the packets and it is not your sender.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[anupam...@gmail.com]

>>>>> "Franke" == Marcus Franke writes:

,----[ Franke ]

| an additional note, don't forget to monitor the netstat udp counter on both
| servers.
|
| % netstat -auns | grep -A 7 "Udp:"
| Udp:
| 9381 packets received
| 0 packets to unknown port received
| 0 packet receive errors
| 1009 packets sent
| 0 receive buffer errors
| 0 send buffer errors
| IgnoredMulti: 1264
|
| Maybe your client is dropping the packets and it is not your sender.

`----
moreover, depending on how your datagrams looks like (src+dst ip, and
src+dst port), kernel might be distributing each rx-queue of your nic to
the same cpu core. perhaps, SO_REUSEPORT can be used to alleviate this ?

--
kind regards
anupam

[Konstantin Khomoutov]

You could estimate what a "dumb sender written in C" could produce:

On the receiver box, do:

# apt install netcat-openbsd
$ nc -k -u -l 6666 >/dev/null

On the sender box, do:

# apt install netcat-openbsd pv
$ pv </dev/zero | nc -u receiver_ip 6666

and see what speed pv will show to you.

On my old Intel E8400 Core2Duo machine I get circa 900MiB when
transferring between a two endpoints on the localhost using a pair of
netcats.

[Tharaneedharan Vilwanathan]

Hi All,

Sorry for the delay.

I remembered I got better UDP performance in another system. So I had to track it.

First, please find the code at the end of this mail.

- Version: I used go 1.7.1

- Platform: Ubuntu 16.10 on x86_64 (I used two setups. In one, I used i7 Skull Canyon NUC and in the other the two systems are E5-2650 v4 based 2U servers with 10Gb and 1Gb links)

- Model: One side just sends and the other side just receives pkts.

- I didnt profile the code yet.

Now, based on my tests, this is what I see:

- in the Xeon based servers, I get about 2.6Gbps (0.0Kbps reverse comm).

- The same setup has 1Gb links. I get about 956Mbps. This is good too!

- With localhost, I get about 2.7Gbps.

Now, in i7 NUC:

- With localhost, I get about 4.6Gbps.

- With a peer IP address, I get only about 220Mbps. This is where I got stuck.

There are some things to note. First, there is no peer. It just sends to the other side. This leads to ICMP packets in the reverse side, though low (5kbps).

What is more, I simply sent the packets to 192.168.1.1 which is my home router/gateway, Linksys WRT610N. Probably, this is my bottleneck.

I still dont have proof that the ethernet port on the NUC is good enough. I will connect a Mac soon and find it out.

Hope this helps.

Thanks

dharani

------------------------------------------

package main

import (

    "fmt"

    "net"

    "os"

    "time"

)

var buffer []byte

var displayReq bool

var pkts int

var bytes int

var dest string

func send() {

    conn, err := net.Dial("udp", dest)

    if err != nil {

        fmt.Printf("Dial() failed. Error %s\n", err)

        os.Exit(1)

    }

    defer conn.Close()

    pkts = 0

    bytes = 0

    for {

        n, err := conn.Write(buffer[0:1472])

        if err != nil {

            //fmt.Printf("Write failed! Error: %s\n", err)

            //os.Exit(1)

        }

        pkts++

        bytes += n

    }

}

func recv() {

    pc, err := net.ListenPacket("udp", ":45000")

    if err != nil {

        fmt.Printf("Read failed! Error: %s\n", err)

        os.Exit(1)

    }

    defer pc.Close()

    pkts = 0

    bytes = 0

    for {

        n, addr, err := pc.ReadFrom(buffer)

        if err != nil {

            fmt.Printf("Read failed! Error: %s\n", err)

            os.Exit(1)

        }

        if false {

            fmt.Printf("Got a pkt from addr: %s\n", addr)

        }

        pkts++

        bytes += n

    }

}

func main() {

    if len(os.Args) != 3 {

        fmt.Printf("Usage: %s [send|recv] addr:port\n", os.Args[0])

        os.Exit(1)

    }

    mode := os.Args[1]

    dest = os.Args[2]

    buffer = make([]byte, 1600)

    go displayTimerProc()

    if mode == "send" {

        send()

    } else {

        recv()

    }

}

func displayTimerProc() {

    for {

        displayReq = true

        time.Sleep(time.Second)

        fmt.Printf("Pkts %d, Bytes %d, rate %d mbps\n",

            pkts, bytes, bytes*8/(1000*1000))

        pkts = 0

        bytes = 0

    }

}

------------------------------------------