How to implement zero-copy between two socket?

[zuxiong lin]

I have one  scenario like this:   

copy( UnixConn , TCPConn)

copy( TCPConn , UnixConn)

My Agent perform the transfer role with other things.  I need zero-copy to decrease the response time.

Client    --UnixConn-->       Agent  --TCPConn-->   RealServer

Client    <--UnixConn--       Agent  <--TCPConn--   RealServer

So    Client(ClientSide) <-> Agent(ServerSide) .    Agent(ClientSide) <--> RealServer(ServerSide). 

Please give me  help!

Thanks.

[James Bardin]

As Brad mentioned in the other thread, the only facility available for a zero copy transfers are syscalls like splice on Linux. There's nothing you can do in Go directly.

I worked tcp splicing into a Go proxy as a proof of concept once, but it wasn't really worth it. You have to write the copying code in C of course, and pass in dupes of the socket FDs via cgo. This forces each transfer into its own system thread, which wasn't acceptable for the case I was testing. It also only improves performance with specific network drivers and kernels (I'm not sure if it can help at all from within a VM).

If you want to look into this, the HAProxy source is a good reference for using splice and pipes between sockets. 

Have you tested a Go solution to determine that it's inadequate?

[Ian Lance Taylor]

On Thu, Feb 5, 2015 at 6:28 PM, James Bardin <j.ba...@gmail.com> wrote:
>
> As Brad mentioned in the other thread, the only facility available for a
> zero copy transfers are syscalls like splice on Linux. There's nothing you
> can do in Go directly.

It seems to me that we could modify net/sendfile_linux.go to use
splice when writing to a socket. It would be awkward because as far
as I can tell it would have to create a pipe, then splice one socket
to the pipe, then splice the pipe to the other socket. I don't know
that it would be a win overall.

Ian

[James Bardin]

On Fri, Feb 6, 2015 at 12:29 AM, Ian Lance Taylor <ia...@golang.org> wrote:

It seems to me that we could modify net/sendfile_linux.go to use
splice when writing to a socket.  It would be awkward because as far
as I can tell it would have to create a pipe, then splice one socket
to the pipe, then splice the pipe to the other socket.  I don't know
that it would be a win overall.

I don't think the pipe is a problem, it's akin to creating the buf in io.Copy.

The splice call will however immediately return EAGAIN, because the sysfds are non-blocking by default. I tested it that it at least works by setting the sysfds to blocking. I don't have a setup handy to verify that it can truly improve performance, but throughput was comparable if not faster on the VM I used for testing.

[zuxiong lin]

My environment is Centos 6.5.
So far , I have no idea to implement it.

> --
> You received this message because you are subscribed to a topic in the
> Google Groups "golang-nuts" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/golang-nuts/Fa6AM71CsEg/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> golang-nuts...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--
林足雄

QQ: 534439207 / fjjiaboming
Email： linzuxi...@gmail.com
Mobile: 18688162114
Weibo： fjjiaboming