Any way to set custom TCP Socket OPS?

[steview]

Hi,

I am probably being a bit dense but I cannot see any obvious way to set socket options that are not predefined as methods on a TCPConn (SetLinger, SetKeepAlive etc). All the predefined methods delegate to private methods that act on the internal FD which also appears to be private. It does not appear possible to do this without effectively copying the whole network stack as the FD is created at the internetSocket method in tcpsock which is also private.

For example I am interested in testing out the SO_REUSEPORT networking option palnned for inclusion in the 3.9 kernel:

http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=c617f398edd4db2b8567a28e899a88f8f574798d

Can anyone point me in the right direction or point out the obvious solution I have missed, or alternatively

would it be too difficult to add a method to the network interface for TCP to enable new socket options to be set without re-implementing the entire network stack.

[Péter Szilágyi]

Hi,

  Imho, this is a non-trivial issue. If you just want to play around and test a new socket option, then the simplest may be to just clone the Go repository and add the extra function. It could eventually even end up in the official library. The problem I see with its inclusion in the libs is exactly that it's too new. The goal of a lib is to ensure cross compatibility; but having a bleeding edge OS construct may do more harm than good.

  But let's wait for someone more qualified to answer this question :)

Cheers,

  Peter

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[steview]

I am not asking to add a new method to support this option per se.

Rather if anything it would be good to add something like a generic SetSocketOptionInt(opt,val) on TCPConn  that one could use as a pass through to syscall.SetsockoptInt(...

So that any new options could be added without needing to wait for a core change. I do not see that this is particularly tricky or indeed has much impact on the language per se it just removes a restriction that currently seemingly makes it very difficult to deal with these sorts of changes and requires the dev just to be a little bit more careful on the val passed in.

cheers

steve

[Péter Szilágyi]

Yes, but the problem is that it makes it a lot easier to shoot yourself in the foot. The simplest use-case is a developer with a shiny new Linux deploying to an older production server. The whole thing will come crumbling down and will require either a redesign or a server upgrade. This is what I meant by non-trivial. I'd argue that one of the major goals of Go is to remain a powerful but high-level language. If you allow such things that developers need to be aware and take care of, you're re-introducing the complexities of C.

But do prove me wrong, I'm happy to change my mind :)

All the best,

  Peter

[steview]

Forgive me for saying so but that is a simplistic strawman argument. Given  Go's target as a system language one would expect more flexibility in the lower levels of OS integration than in a higher level scripting language. 

Socket Options are hardly introducing the complexity of C - even Python and Java allow you to do this. 

Its about not making things impossible when there is no need to for the sake of wanting to protect the developers from themselves. Productive languages are about flexibility in dealing with unexpected changes not having some ideal abstraction.

[steview]

Ok there seems no way to do this without a change to the core packages. The duplicate Fd functionality is not really useful in this respect as the socket is already created and passed in to the listen call.

I am thinking along the lines of a patch consisting of:

maybe FdTCP() and FdUnix()  functions on the TCP and Unix socket files (which are the only ones that support File() AFAIK ) which take the same args as the current ListenXXX calls.

This would allow access to the Fd before any other action enabling customisation before usage. Each method would basically be the first half of each of the equivalent listen methods. 

A problem then there would need to be another method to set up the listener with the fd as its unexported so TCPListener cannot be constructed from outside the package, which seems a bit rubbish. Maybe another interface which seems clunky.

Any other suggestions as to how to set up a  simple fd with customisation and use this to construct a Listener  would be welcome. 

I must say the lack of access to the fd and lack of control of the socket setup seems a bit strange given the target goals. But maybe there is an easy way to do this I am missing.

[Kyle Lemons]

You can set the socket up yourself and call FileConn / FileListener

[steview]

Hi Kyle,

Thanks for the pointer - I had missed that function completely and its no too far off how I was thinking.

However, it did turn out quite a bit more work than I would have expected mainly due to package private stuff in net and the ipv4/6 differences (even though i cut down a couple of bits).

I would also need to split up some stuff for windows/unix differently which would be yet more work and redo the backlog stuff.

So basically to call a custom socket option on a listener socket before binding it I needed to copy and paste from the core libraries for a minimum of something like this:

-----------------------------------------------------------------------

package main

import (

"log"

"net"

"os"

"syscall"

)

// copy from ipsock_posix.go

func ipToSockaddr(family int, ip net.IP, port int) (syscall.Sockaddr, error) {

switch family {

case syscall.AF_INET:

if len(ip) == 0 {

ip = net.IPv4zero

}

if ip = ip.To4(); ip == nil {

return nil, net.InvalidAddrError("non-IPv4 address")

}

s := new(syscall.SockaddrInet4)

for i := 0; i < net.IPv4len; i++ {

s.Addr[i] = ip[i]

}

s.Port = port

return s, nil

case syscall.AF_INET6:

if len(ip) == 0 {

ip = net.IPv6zero

}

// IPv4 callers use 0.0.0.0 to mean "announce on any available address".

// In IPv6 mode, Linux treats that as meaning "announce on 0.0.0.0",

// which it refuses to do.  Rewrite to the IPv6 unspecified address.

if ip.Equal(net.IPv4zero) {

ip = net.IPv6zero

}

if ip = ip.To16(); ip == nil {

return nil, net.InvalidAddrError("non-IPv6 address")

}

s := new(syscall.SockaddrInet6)

for i := 0; i < net.IPv6len; i++ {

s.Addr[i] = ip[i]

}

s.Port = port

return s, nil

}

return nil, net.InvalidAddrError("unexpected socket family")

}

// copy from ipsock_posix.go

func probeIPv6Stack() (supportsIPv6, supportsIPv4map bool) {

var probes = []struct {

la net.TCPAddr

ok bool

}{

// IPv6 communication capability

{net.TCPAddr{IP: net.ParseIP("::1")}, false},

// IPv6 IPv4-mapped address communication capability

{net.TCPAddr{IP: net.IPv4(127, 0, 0, 1)}, false},

}

for i := range probes {

s, err := syscall.Socket(syscall.AF_INET6, syscall.SOCK_STREAM, syscall.IPPROTO_TCP)

if err != nil {

continue

}

defer closesocket(s)

syscall.SetsockoptInt(s, syscall.IPPROTO_IPV6, syscall.IPV6_V6ONLY, 0)

sa, err := ipToSockaddr(syscall.AF_INET6, probes[i].la.IP, probes[i].la.Port)

if err != nil {

continue

}

err = syscall.Bind(s, sa)

if err != nil {

continue

}

probes[i].ok = true

}

return probes[0].ok, probes[1].ok

}

// copy from ipsock_posix.go

var supportsIPv6, supportsIPv4map = probeIPv6Stack()

// copy from fd_unix.go - this is different for windows and should support both

func closesocket(s int) error {

return syscall.Close(s)

}

// copy from ipsock_posix.go

func isWildcard(ip net.IP) bool {

if ip == nil {

return true

}

return ip.IsUnspecified()

}

// copy from tcpsock_posix.go

func family(ip net.IP) int {

if len(ip) <= net.IPv4len || ip.To4() != nil {

return syscall.AF_INET

}

return syscall.AF_INET6

}

// copy from ipsock_posix.go

func favouriteAddrFamily(net string, laddr *net.TCPAddr, mode string) (int, bool) {

switch net[len(net)-1] {

case '4':

return syscall.AF_INET, false

case '6':

return syscall.AF_INET6, true

}

if mode == "listen" && isWildcard(laddr.IP) {

if supportsIPv4map {

return syscall.AF_INET6, false

}

return family(laddr.IP), false

}

return family(laddr.IP), false

}

// copy from sock_cloexec.go

func sysSocket(f, t, p int) (int, error) {

s, err := syscall.Socket(f, t|syscall.SOCK_NONBLOCK|syscall.SOCK_CLOEXEC, p)

// The SOCK_NONBLOCK and SOCK_CLOEXEC flags were introduced in

// Linux 2.6.27.  If we get an EINVAL error, fall back to

// using socket without them.

if err == nil || err != syscall.EINVAL {

return s, err

}

// See ../syscall/exec_unix.go for description of ForkLock.

syscall.ForkLock.RLock()

s, err = syscall.Socket(f, t, p)

if err == nil {

syscall.CloseOnExec(s)

}

syscall.ForkLock.RUnlock()

if err != nil {

return -1, err

}

if err = syscall.SetNonblock(s, true); err != nil {

syscall.Close(s)

return -1, err

}

return s, nil

}

// copy from sockopt_linux.go - different for windows

func setDefaultSockopts(s, f int, ipv6only bool) error {

switch f {

case syscall.AF_INET6:

if ipv6only {

syscall.SetsockoptInt(s, syscall.IPPROTO_IPV6, syscall.IPV6_V6ONLY, 1)

} else {

// Allow both IP versions even if the OS default

// is otherwise.  Note that some operating systems

// never admit this option.

syscall.SetsockoptInt(s, syscall.IPPROTO_IPV6, syscall.IPV6_V6ONLY, 0)

}

}

// Allow broadcast.

err := syscall.SetsockoptInt(s, syscall.SOL_SOCKET, syscall.SO_BROADCAST, 1)

if err != nil {

return os.NewSyscallError("setsockopt", err)

}

return nil

}

// copy from sockopt_linux.go - different for windows

func setDefaultListenerSockopts(s int) error {

// Allow reuse of recently-used addresses.

err := syscall.SetsockoptInt(s, syscall.SOL_SOCKET, syscall.SO_REUSEADDR, 1)

if err != nil {

return os.NewSyscallError("setsockopt", err)

}

return nil

}

// set up our custom socket

func CreateListenerSocket(netType string, laddr *net.TCPAddr, maxbacklog int) (net.Listener, error) {

var socketAddr syscall.Sockaddr

var s int

var err error

family, ipv6only := favouriteAddrFamily(netType, laddr, "listen")

if socketAddr, err = ipToSockaddr(family, laddr.IP, laddr.Port); err != nil {

return nil, err

}

log.Printf("Found local socket address of [%v]", socketAddr)

if s, err = sysSocket(family, syscall.SOCK_STREAM, 0); err != nil {

return nil, err

}

// set the socket options we want

if err = setDefaultSockopts(s, family, ipv6only); err != nil {

closesocket(s)

return nil, err

}

// set the reuse addr

if err = setDefaultListenerSockopts(s); err != nil {

closesocket(s)

return nil, err

}

// set the custom reusePort option

/* if err = syscall.SetsockoptInt(s,syscall.SOL_SOCKET,15,1); err != nil {

closesocket(s)

return 0, err

}

*/

// call bind

if err = syscall.Bind(s, socketAddr); err != nil {

closesocket(s)

return nil, err

}

// set up the listener - would have to reimplement  maxbacklog  here for real code

if err = syscall.Listen(s, maxbacklog); err != nil {

closesocket(s)

return nil, err

}

file := os.NewFile(uintptr(s), "listener-"+laddr.String())

var socketListener net.Listener

if socketListener, err = net.FileListener(file); err != nil {

file.Close()

return nil, err

}

log.Printf("Got file  handle %s", file.Name())

file.Close()

return socketListener, nil

}

------------------------------------------------

If there is a simpler way to get hold of an unbound socket while keeping the caller's behaviour similar to how the core packages treat the ip versioning that would be really useful. Unless Kyle wants to step in and point out another file that does this stuff  that I have overlooked again.