Handling timeouts

[Sam Crawford]

Hello,

I'm just getting started with Go (love it so far), so bear with me
please! I'm doing a lot of work with network I/O, so obviously
handling timeouts and errors is important to me.

I've read the blog post
(http://blog.golang.org/2010/09/go-concurrency-patterns-timing-out-and.html)
around concurrency, and have got that all working fairly nicely with
the code below (irrelevant parts snipped out). However, I have a
couple of questions:

1) Is this the right way to handle timing out when working with
blocking system calls (such as net.Conn.Read)? I appreciate that
setting a socket read timeout would be better, but that's not an
option with this application I'm afraid (the handshake has a timeout,
but everything after that must not have a fixed timeout).

2) It would be nice in getRequestInternal if I could return a
meaningful error back to the calling function (at the moment I'm just
passing back nil if there was an error). If I were calling this method
synchronously then I'd use the multiple return feature, but how can
this same effect be achieved if I'm returning the result back via a
channel? I could create a new type that enclosed a Request and an
os.Error, but that seems inelegant. Any suggestions?

Thanks,

Sam

// The publicly exposed GetRequest function
// Note that the structure of Request is irrelevant (it just has a few
strings and int32s)
func GetRequest(c net.Conn, timeout int64) (req *Request) {
creq := make(chan *Request, 1)
cto := make(chan bool, 1)
go func() {
time.Sleep(timeout)
cto <- true
}()
go getRequestInternal(c, creq)

select {
case req = <- creq:
fmt.Fprintf(os.Stderr, "Got request item back\n")
case <- cto:
fmt.Fprintf(os.Stderr, "Hit timeout\n")
}
return req
}

func getRequestInternal(c net.Conn, cr chan *Request) {
req := new(Request)
// parse the request from c.Read(...)
if ok {
c <- req
} else {
c <- nil
}
}

[Andrew Gerrand]

On 7 December 2010 10:34, Sam Crawford <samcr...@gmail.com> wrote:
> Hello,
>
> I'm just getting started with Go (love it so far), so bear with me
> please! I'm doing a lot of work with network I/O, so obviously
> handling timeouts and errors is important to me.
>
> I've read the blog post
> (http://blog.golang.org/2010/09/go-concurrency-patterns-timing-out-and.html)
> around concurrency, and have got that all working fairly nicely with
> the code below (irrelevant parts snipped out). However, I have a
> couple of questions:
>
> 1) Is this the right way to handle timing out when working with
> blocking system calls (such as net.Conn.Read)? I appreciate that
> setting a socket read timeout would be better, but that's not an
> option with this application I'm afraid (the handshake has a timeout,
> but everything after that must not have a fixed timeout).

Can't you just set the timeout before the handshake, and then re-set
it afterward?

Andrew

[samcr...@gmail.com]

Hi Andrew,

Thanks for the quick reply.

Whilst changing the read timeout on the connection will work, it won't do the same as the code below. The timeout in the code below applies to the *complete* handshake, whereas setting a socket read timeout would apply to as little as a single byte.

Specifically, a socket read timeout wouldn't be triggered if the client was trickling bytes to the server program. My protocol has a similar header exchange to HTTP, so with 4096 bytes max per header line, 1024 max lines, and say a 5 second socket read timeout, that's an awfully long time that a malicious client could tie up a server connection. Multiply this by a few thousand clients and you've got a significant DoS attack with very little effort from the attackers.

Thanks

Sam

Sent from my BlackBerry® wireless device

[Dave Cheney]

Then I would suggest something like this (excuse the untested pseudocode)

func readRequestOrTimeout(socket *net.TCPConn) *http.Request {
timeout := time.Tick(10e9) // ten seconds to comply
c := make(chan *http.Request)
select {
case request := <- readRequest(socket, c):
return request
case <- timeout:
socket.Close()
return nil
}
}

func readRequest(socket *net.TCPConn) *http.Request {
for {
// read socket
// loop if not enough data
}
}

Cheers

Dave

[Dave Cheney]

>
> func readRequest(socket *net.TCPConn) *http.Request {
>   for {
>      // read socket
>      // loop if not enough data
>    }
> }

oops that should have been

func readRequest(socket *net.TCPConn, c chan *http.Request) [

for {
// read socket
// loop if not enough data
}

c <- request

[Sam Crawford]

Dave,

Thanks, but unless I'm missing something that's fundamentally the same
code as I included in my original message. The key thing I am looking
for is how to sensibly return an os.Error from the readRequest
function (to use your code as an example). Simply returning nil via
the channel is not that great - it doesn't tell the calling function
anything about the nature of the issue - was it a EOF (i.e. clean
close) or some socket error?

Thanks,

Sam

[Dave Cheney]

Hi Sam,

It does look like I ended up implementing your initial program in an
attempt to solve your previous response.

If the goal is to avoid being Solaris'ed does the response state
matter? Either a full request was returned within the timeout window,
or the connection closed and the resources released. If your goal is
to track the detailed error code, then at the moment you'll probably
have to implement a tuple struct to track both the response and an
possible error code.

Another possibility might be returning the tuple at your top level
function, something like this.

func GetRequest(c net.Conn, timeout int64) (req *Request, err os.Error) {
...

select {
case req = <- creq:
fmt.Fprintf(os.Stderr, "Got request item back\n")
case <- cto:

err = fmt.Errorf("Timeout waiting for requiest")

fmt.Fprintf(os.Stderr, "Hit timeout\n")
}
return
}

as the <- cto channel will only fire if there is a timeout.

Cheers

Dave

[Sam Crawford]

Ah yes, I'd forgotten the term "Solaris'ed"! The goal is to return a
Request object to the calling function within the specified timeout,
or if some error occurs, then return a sensible error message.

So yes, it sounds like I'll have to use a tuple struct to wrap the
Request and os.Error objects before passing the result down the
channel. Of course, I'd unwrap these in the GetRequest function and
then make use of multiple return.

I was hoping there may be a more generic way of achieving this, as I
can see myself repeating this pattern in numerous places (basically
anywhere where I need to use I/O timeouts), but it's no major
headache.

Thanks,

Sam

[Matt Joiner]

What does this term Solaris'ed mean? I'll guess it's something to do with support being dropped, or something being completely inferior...?

[samcr...@gmail.com]

There was a denial of service tool called Sloworis (not Solaris) that received quite a bit of publicity a year or so back. This would open a TCP connection to a webserver and send an HTTP request very slowly (e.g. One byte every few seconds, just enough to stop a socket read timeout), and it would also send a lot of them.

So even if you were serving static HTML content, an attacker could send tens of thousands of these slow requests to Apache, and exhaust its resources - without requiring much bandwidth at all.

See http://ha.ckers.org/slowloris/ for more.

Thanks

Sam

Sent from my BlackBerry® wireless device

---

From: Matt Joiner <anac...@gmail.com>

Date: Wed, 8 Dec 2010 01:11:44 +1100

To: Sam Crawford<samcr...@gmail.com>

Cc: Dave Cheney<da...@cheney.net>; Andrew Gerrand<a...@golang.org>; <a...@google.com>; golang-nuts<golan...@googlegroups.com>

[Matt Joiner]

Hehehe... Thanks for explaining.

[Andrew Gerrand]

On 8 December 2010 00:04, Sam Crawford <samcr...@gmail.com> wrote:
> Dave,
>
> Thanks, but unless I'm missing something that's fundamentally the same
> code as I included in my original message. The key thing I am looking
> for is how to sensibly return an os.Error from the readRequest
> function (to use your code as an example). Simply returning nil via
> the channel is not that great - it doesn't tell the calling function
> anything about the nature of the issue - was it a EOF (i.e. clean
> close) or some socket error?

You could make the channel a chan interface{}, and then send either a
request value or an os.Error value.

On the other side, you can use a type switch to determine which event
has occurred:

v := <-ch
switch u := v.(type) {
case *Request:
// do something with the request, u
case os.Error:
// do something with the error, u
}

Andrew

[samcr...@gmail.com]

Thanks Andrew, that sounds like a neater solution, I'll try that out.

Thanks

Sam

Sent from my BlackBerry® wireless device

-----Original Message-----
From: Andrew Gerrand <a...@google.com>
Date: Wed, 8 Dec 2010 08:50:08
To: Sam Crawford<samcr...@gmail.com>
Cc: Dave Cheney<da...@cheney.net>; golang-nuts<golan...@googlegroups.com>
Subject: Re: [go-nuts] Handling timeouts