How to manage lots of net.Conn for golang server?

[ZPL]

Hi gophers:

I'm now making a golang server, holding lots of tcp connection, each client will login to the server,and send message to other client.

Server allocate an unique int64 sequce number to a client.

here's some part of code:

// current max sequense number

var seq int64 = 1

// lock for clients map

var connLock sync.RWMutex

var clients map[int64] *Connection = make(map[int64] *Connection, CONNSIZE)

type Connection struct {

seq      int64

conn     net.Conn

e        *codec.Encoder

d        *codec.Decoder

}

func addClient(conn net.Conn) *Connection {

connLock.Lock()

defer connLock.Unlock()

for {

seq++

if seq == 0 {

seq = 1

}

if clients[seq] == nil {

break

}

}

clients[seq] = &Connection{

seq:      seq,

conn:     conn,

}

return clients[seq]

}

func delClientBySeq(seq int64) {

connLock.Lock()

defer connLock.Unlock()

delete(clients, seq)

}

func sendToClient(seq int64, msg interface{}) {

var c *Connection

var sessNull bool

connLock.RLock()

defer connLock.RUnlock()

// finding connections:

...

// Encode will send to net.Conn

c.e.Encode(msg)

}

I have to lock everywhere:

1. when connection is made, we call addClient FROM EACH CONNECTION gorouting, hold rw lock

2. when connection is closed, we call delClientBySeq FROM EACH CONNECTION  gorouting, hold rw lock

3. when client send message to other, call sendToClient FROM EACH CONNECTION  gorouting, hold read-only lock

This may not cause a performance issue,because i have test it with 30000 connections and watch the the server with go tool pprof, sync.RWMutex Lock(),Unlock()... is not showing.

I'm not sure this is the right model for me, but this server is too much common, and i was wondering if there is a lock free style of this server? 

[Tamás Gulácsi]

The shown code is just the clients accounting - you may use a slice instead of the map, but you'll have to synchronize access to the central directory of the connections.

You may split the locks to sublocks by partitioning the clients using the ids (hash or modulo), but it's not needed till you see perf. issue.

Does the clients come and go a lot?

[Charles Haynes]

If you want lock free I'd suggest just allocating random 64-bit (or larger) client ids and inserting them into a map. By making the client id large enough you can reduce the likelihood of a collision to be lower than the likelihood of an uncorrected hardware failure, but I think 64-bit random will be enough.

You also make it harder for an adversary to guess your client ids, if you care.

-- Charles

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Kamil Kisiel]

You can't omit locks around a global map you write to concurrently just because the keys are random. 

[Charles Haynes]

Ah true! Good point. If you know the maximum number of concurrent clients in advance you could implement a non-copying map but that seems like a lot of trouble.

-- Charles

[ZPL]

About 20 connection coming and lost per second, server is now holding 15k connections, we use python for current server, converting to golang right now!

在 2015年10月26日星期一 UTC+8下午11:07:36，Tamás Gulácsi写道：

[ZPL]

I agree!

在 2015年10月27日星期二 UTC+8上午6:22:37，Kamil Kisiel写道：