concurrent execution of WriteToUDP()

[igna...@gmail.com]

HI, I am writing udp relay. Relay listens two ports - one for clients, second for incoming stream,So I have only two instances of net.UDPConn. Incoming stream is relayed to every client.

Looks like WriteToUDP() rather slow and there is some mutex construction inside which doesn't allow to use it simultaneously; -blockprofile:
     flat  flat%   sum%        cum   cum%
 9.41days 87.55% 87.55%   9.41days 87.55%  runtime.selectgo

 1.33days 12.40% 99.95%   1.33days 12.40%  net.(*fdMutex).RWLock

Also my benchmark for WriteToUDP() shows average time of the network card's writing

func BenchmarkUDP(b *testing.B) {

        addr := &net.UDPAddr{IP: net.IP{192, 168, 1, 35}}

        var bytesNum int64 = 256

        bConn, err := net.ListenUDP("udp", addr)

        if err != nil {

               b.Fatal(err)

       }  

        bConn.SetWriteBuffer(4096 * 1024)

        src := make([]byte, bytesNum)

        io.ReadFull(rand.Reader, src)

        b.ResetTimer()

        for i := 0; i < b.N; i++ {

               _, err := bConn.WriteToUDP(src, addr)

                if err != nil {

                       b.Fatal(err)

               }  

       }  

       b.SetBytes(bytesNum)

}

PASS

BenchmarkUDP-8   500000       8705 ns/op   69.08 MB/s

So average speed is 3705ns for packet of 256 bytes(+ip header), Can I speed up(for given size)?

Or maybe there is a way to use different independent net.UDPConn instances for every client? 

[igna...@gmail.com]

So average speed is *8705ns 

[James Bardin]

On Thursday, January 14, 2016 at 3:11:11 PM UTC-5, igna...@gmail.com wrote:

So average speed is *8705ns 

You need to serialize the writes somewhere, so removing the lock on fdMutex only moves that contention elsewhere.

8705/call comes out to 115k  pps, which isn't horrible. You could bring that down a little by using a connected udp socket and calling Write on the file descriptor directly (though you have to sacrifice an OS thread per connection for that)

Do you really need to move more packets than this?

Have you tried using a separate conn for each client?

[Matt Harden]

Writes to a packet-oriented socket have to be serialized? Why?

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[James Bardin]

On Thu, Jan 14, 2016 at 4:47 PM, Matt Harden <matt....@gmail.com> wrote:
> Writes to a packet-oriented socket have to be serialized? Why?

I just mean they must be serialized at some level, IIRC the behavior
concurrent calls to write is unspecified, but send/sendto for a
SOCK_DGRAM is safe.

Granted the serialization done by the kernel may be faster than what
is done in Go ;)

[igna...@gmail.com]

Situation looks great with individuals conns,

      flat  flat%   sum%        cum   cum%

  7.46days   100%   100%   7.46days   100%  runtime.selectgo

But under identical conditions, packets loss on clients becomes much bigger when using conn per client against one conn for all clients.

Separate conns require additional conn per client on server, and additional conn on every client. And all my clients are on one machine, maybe it's the cause of packet's loss, hard to tell.

Could you point to the some example of using "connected udp socket and calling Write on the file descriptor directly"?

четверг, 14 января 2016 г., 23:23:30 UTC+3 пользователь James Bardin написал:

[James Bardin]

On Fri, Jan 15, 2016 at 6:54 AM, <igna...@gmail.com> wrote:
> Could you point to the some example of using "connected udp socket and
> calling Write on the file descriptor directly"?

You would use DialUDP to connect, take the *os.File from conn.File(),
and call its Write method. You actually can't do this though if want
to send to multiple clients from one socket. But, you could still get
an FD for the connection, and use syscall.Sendto directly.

(I'm not advocating either of these methods.)

[igna...@gmail.com]

I've made simple benchmarks in order to check every possible way of writing to the NIC https://github.com/Kaign/benchs/blob/master/udpbenchs_test.go

results like this:

BenchmarkListen-4         200000       9631 ns/op   3.32 MB/s

BenchmarkListenSock-4     200000       6264 ns/op   5.11 MB/s

BenchmarkDial-4           200000       6327 ns/op   5.06 MB/s

BenchmarkDialSock-4       200000       6340 ns/op   5.05 MB/s

Looks like WriteToUDP() is the slowest one, these are really good news. Also using net.Conn's Write() method and os.File's Write() are almost the same (in the first just added checking if connection and file descriptor !=nil). 

Thanks for the tips

пятница, 15 января 2016 г., 17:09:36 UTC+3 пользователь James Bardin написал: