[go-nuts] Creating a TCP proxy

[James Fisher]

Hi.  I'm trying to get my head around how to create a simple TCP proxy using the `net` package.  My intention is to create a small binary that, when run, (1) starts up another HTTP-serving binary, (2) starts listening on another TCP port, then (3) routes all incoming connections to the HTTP server, and returns all replies back.  I've had a couple of attempts at this but they've been failures.

Could someone kindly show me a snippet that:

* listens on one TCP port,
* for every incoming connection, open a connection with another server and route the request and response,
* has the ability to execute a function immediately before setting up each the connection with the server (i.e., allows me to control the through-flow and do stuff based on the incoming-connection event).
* no need for any HTTP abilities, just (for the moment) deals blindly with data over TCP.

(Apologies if I've misunderstood any aspects of TCP; I haven't dabbled in it much.)

My problem is just that this involves many interfaces, possibly generic readers/writers, concurrency, event-based stuff, and I'm in over my head for my second day with the language. :)


James

[roger peppe]

something like this perhaps?

package main

import (
"net"
"fmt"
"io"
"os"
)

func main() {
if len(os.Args) != 3 {
fatal("usage: netfwd local remote")
}
localAddr := os.Args[1]
remoteAddr := os.Args[2]
local, err := net.Listen("tcp", localAddr)
if local == nil {
fatal("cannot listen: %v", err)
}
for {
conn, err := local.Accept()
if conn == nil {
fatal("accept failed: %v", err)
}
go forward(conn, remoteAddr)
}
}

func forward(local net.Conn, remoteAddr string) {
remote, err := net.Dial("tcp", "", remoteAddr)
if remote == nil {
fmt.Fprintf(os.Stderr, "remote dial failed: %v\n", err)
return
}
go io.Copy(local, remote)
go io.Copy(remote, local)
}

func fatal(s string, a ... interface{}) {
fmt.Fprintf(os.Stderr, "netfwd: %s\n", fmt.Sprintf(s, a))
os.Exit(2)
}


--
Subscription settings: http://groups.google.com/group/golang-nuts/subscribe?hl=en

[⚖ Alexander "Surma" Surma]

Posts like this amuse and impress me everytime. Nice one!

[James Fisher]

Right, that was lovely, Roger.  It does exactly what I asked -- only, it turns out that what I asked for wasn't exactly what I wanted.  (At least a learnt a couple of things.)

Turns out my understanding of HTTP is stuck at 1.0, and with 1.1 the same TCP connection can be kept open between any two hosts for the whole session.  So I can't detect individual "HTTP requests" on the connection, which is necessary.  I'm therefore now using the http package to do essentially the same thing.  I've worked out most of it:

package main

import (
  "http"
  "net"
)

type handler struct { }

func (h *handler) ServeHTTP(conn *http.Conn, req *http.Request) {
  client_tcp_conn, _ := net.Dial("tcp", "", "127.0.0.1:9999")   // Open new TCP connection to the server
  client_http_conn := http.NewClientConn(client_tcp_conn, nil)  // Start a new HTTP connection on it
  client_http_conn.Write(req)                                   // Pass on the request
  resp, _ := client_http_conn.Read()                            // Read back the reply
  resp.Write(conn)                                              // Write the reply to the original connection
  client_http_conn.Close()                                      // Close the connection to the server
  }

func main() {
  local, _  :=  net.Listen("tcp", "0.0.0.0:9998")
  h := new(handler)
  http.Serve(local, h)
  }



The one, probably trivial, problem with this is that the HTTP response is written to the connection as a response *body* -- i.e., my browser displays the raw HTTP response rather than parsing it :)

The problem presumably is that I'm doing "resp.Write(conn)" on a Conn from the http package, not tcp.  I just can't find how to get at that underlying TCP connection.

[James Fisher]

Btw, I kinda solved this, but it's a hack.  I had to write my own method for http.Response so it could be written to an http.Conn; functionality that IMO should really be in that library:

func (r *http.Response) WriteToHttpConn(conn http.Conn) os.Error {
  /* In the mysterious absence of the ability to access an `http.Conn`'s  underlying net.Conn,
   * or alternatively something like http.Conn.WriteResponse(http.Response),
   * this allows the Response struct instance to transfer data to an http.Conn.
   */
  for k,v := range r.Header {
    conn.SetHeader(k, v)
    }
  body, _ := ioutil.ReadAll(resp.Body)
  conn.Write(body)

[Frederic Marand]

Nice use of IO.Copy ! FWIW, this still works on Go 1.3, and just solved my need for a generic TCP proxy today. 

[Naoki INADA]

Very nice example.

But does this example close socket as soon as connection closed?

[James Bardin]

I wish people wouldn't dig up these old (4+ years) threads.

No, it's not a perfect example, and definitely doesn't shutdown cleanly. The basics however are the same, two io.Copy calls to splice the connections together. 

There are probably much better examples of proxying TCP connections go floating around now.