How to do a lock free for collecting multiple clients request and dispatch to another server?

[dlin]

There are C1,C2,... Cn  TCP Clients. I have a CommServer and a JobServer program.   All of them are on separated machines.

CommServer receives request from C1,C2,...Cn  and mark a sequence number on the request, then transfer to JobServer.

JobServer process the request and send the result back to CommServer, then CommServer send back to C1,C2,...Cn.

This is similar to map-reduce.  For the Clients to Commserver require tech like 'reduce'.

For the CommServer back Clients require tech like 'map'.

If in C, I can use 'select()' system call to do the 'reduce'.

For the 'map', I use a hash table to keep which response will transfer to which client.

In my old C implement, it require lock.  

The client count increases or reduces dynamically.

I always back to use sync.RWMutex to lock a "map[key] Client"  structure.

I'm wonder what's the best method to implement in Go?

[Dmitry Vyukov]

Protecting a map by sync.RWMutex is a good solution.

If you can assign dense id's to clients, then you can use an array to store per-client data. An array does not need external protection.

 

[dlin]

I'm trying convert to "Share Memory By Communicating", http://golang.org/doc/codewalk/sharemem/

But, I haven't experience of CSP  programming.

[Dmitry Vyukov]

You can have a dedicated routing goroutine that accesses the map.
I do not understand what exactly you are trying to implement, so can't
suggest something more concrete.

> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[John Nagle]

On 5/14/2013 5:59 PM, dlin wrote:
> There are C1,C2,... Cn TCP Clients. I have a CommServer and a JobServer
> program. All of them are on separated machines.
> CommServer receives request from C1,C2,...Cn and mark a sequence number on
> the request, then transfer to JobServer.
> JobServer process the request and send the result back to CommServer, then
> CommServer send back to C1,C2,...Cn.

In Go, you may be better off having a goroutine for each
JobServer. Each goroutine gets work units from a queue channel,
makes the request of its JobServer, and waits for the reply.
When a reply comes, it's sent back to whereever it's supposed to
go.

If the reply doesn't come or times out, indicating trouble
at a JobServer, you can send the work unit back to the
queue channel, so another JobServer will do it.
Then that goroutine can report trouble, test the job server
to see if it's up, but not request another work unit until
its job server recovers. It's important to get the
machine failure handling right, or the whole system will
lock up whenever any server fails.

Go's RPC package provides the necessary primitives.
Or you can run a web server on the JobServer machines and
do all this with REST requests.

John Nagle