Clean shutdown when blocked on I/O

[Brian Candler]

I am looking for the safe way to do a clean shutdown when blocked on I/O.  Here is a basic example:

package main

import (

"encoding/json"

"fmt"

"io"

"os"

"os/signal"

"syscall"

)

func main() {

rx := json.NewDecoder(os.Stdin)

chanTERM := make(chan os.Signal, 1)

signal.Notify(chanTERM, syscall.SIGTERM)

for {

select {

case <-chanTERM:

fmt.Println("Shutdown requested via signal")

os.Exit(0)

default:

}

var msg interface{}

err := rx.Decode(&msg)

if err != nil {

if err == io.EOF {

os.Exit(0)

}

fmt.Printf("Error: %v\n", err)

os.Exit(1)

}

fmt.Printf("Message: %v\n", msg)

}

}

This sort-of works, except that once the code is blocked in rx.Decode(&msg), the termination signal is not handled until another message arrives.  (Actually I'm using sockets - os.Stdin is just for example here).  I'd like it to shutdown immediately if it's not in the middle of doing something.

The question is then how to unblock this reader.

1. Is there a way to link an io.Reader to a context, so I can just send the ctx.Done() signal?  If so, I couldn't find it.

(I found this post, but the library has the same issue: the context is checked before the read, but a cancel won't unblock the read)

2. I can just close the input stream from another goroutine.  This seems fairly brutal.  Also, to distinguish this condition from an actual error, it seems I need to parse the error message and look for the text "use of closed network connection" (github: #4373)

3. I can move the rx.Decode(&msg) into a goroutine which passes a message over a channel, and just let it lock up if there's no data coming in.  This isn't a problem here where I want to exit the entire program.  If this was a network server with many connections and I wanted to disconnect just one, then I think I'd end up having to close the channel anyway to disconnect the client.

Is there another option I should be looking at?

Thanks,

Brian.

[Ian Davis]

Can you write your own ContextReader that checks ctx.Done in its Read method?

--

You received this message because you are subscribed to the Google Groups "golang-nuts" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/3f3b6f98-ff9c-4829-844b-a0d0abc5d753o%40googlegroups.com.

[Ross Light]

See https://golang.org/issue/20280

[Brian Candler]

On Monday, 6 July 2020 14:53:55 UTC+1, Ian Davis wrote:

Can you write your own ContextReader that checks ctx.Done in its Read method?

Inside a ContextReader I can check ctx.Done *before* calling the wrapped Read method - but if the context isn't done at that point and I called the wrapped Read, it will be blocked at that point.

[Robert Engels]

You need to close the socket from another Go routine. Otherwise the Reader api would need to be changed to always take a Context. 

On Jul 6, 2020, at 12:36 PM, Brian Candler <b.ca...@pobox.com> wrote:



On Monday, 6 July 2020 14:53:55 UTC+1, Ian Davis wrote:

Can you write your own ContextReader that checks ctx.Done in its Read method?

Inside a ContextReader I can check ctx.Done *before* calling the wrapped Read method - but if the context isn't done at that point and I called the wrapped Read, it will be blocked at that point.

--

You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/5069054f-19db-4bbc-89b4-c2a6ff58a270o%40googlegroups.com.

[Ian Lance Taylor]

On Mon, Jul 6, 2020 at 10:57 AM Robert Engels <ren...@ix.netcom.com> wrote:
>
> You need to close the socket from another Go routine. Otherwise the Reader api would need to be changed to always take a Context.

Either close the socket or set the deadline to zero.

Ian

> On Jul 6, 2020, at 12:36 PM, Brian Candler <b.ca...@pobox.com> wrote:
>
> 
> On Monday, 6 July 2020 14:53:55 UTC+1, Ian Davis wrote:
>>
>> Can you write your own ContextReader that checks ctx.Done in its Read method?
>>
>
> Inside a ContextReader I can check ctx.Done *before* calling the wrapped Read method - but if the context isn't done at that point and I called the wrapped Read, it will be blocked at that point.
>
> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/5069054f-19db-4bbc-89b4-c2a6ff58a270o%40googlegroups.com.
>
> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/75342356-0313-4CF9-BC27-F03871A4F2C8%40ix.netcom.com.

[robert engels]

If you set the deadline to zero, don’t you need to use another flag to know that it “should be terminated”, otherwise you can’t know if the deadline exceeded was regular - I guess, unless you do a “get deadline” on the error and check for 0, but that seems racy ?

[Ian Lance Taylor]

On Mon, Jul 6, 2020 at 4:23 PM robert engels <ren...@ix.netcom.com> wrote:
>
> If you set the deadline to zero, don’t you need to use another flag to know that it “should be terminated”, otherwise you can’t know if the deadline exceeded was regular - I guess, unless you do a “get deadline” on the error and check for 0, but that seems racy ?

You're shutting down anyhow, I'm not sure it makes much difference.
You already need some way to know that you are shutting down, such as
checking a context or whatever.

But it's not a big deal. I was only making the point that setting the
deadline to zero will wake up the goroutine.

[Robert Engels]

I only meant that the “socket closed” error is a good (perfect?) signal that the Go routine should exit rather than using an additional state var.

> On Jul 6, 2020, at 6:43 PM, Ian Lance Taylor <ia...@golang.org> wrote:

[Brian Candler]

Thank you, that was what I was looking for.  I had forgotten about deadlines, and I didn't realise that you could change a deadline even while a read was in progress.

In case it's helpful to anyone, here's a proof-of-concept.  It doesn't work on play.golang.org because of the network communication.

package main

import (

"context"

"fmt"

"io"

"net"

"os"

"time"

)

type ConnWithContext struct {

net.Conn

Ctx context.Context

}

func NewConnWithContext(ctx context.Context, conn net.Conn) *ConnWithContext {

go func() {

<-ctx.Done()

conn.SetDeadline(time.Now())

}()

return &ConnWithContext{

Conn: conn,

Ctx:  ctx,

}

}

func (c *ConnWithContext) Read(b []byte) (n int, err error) {

return c.Conn.Read(b)

}

func main() {

ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)

defer cancel()

conn, err := net.Dial("tcp", "smtp.gmail.com:25")

if err != nil {

fmt.Println(err)

return

}

cwc := NewConnWithContext(ctx, conn)

n, err := io.Copy(os.Stdout, cwc)

fmt.Println(n, err)

cwc.Close()

}

[Andrei Tudor Călin]

I think this works for some cases, but it is potentially wasteful (and even leaky) in terms of resource usage.

For example, if ctx is context.Background(), it leaks a goroutine for every connection. It also keeps the additional goroutine around for the entire lifetime of the connection. I'd like to present a more involved, but more robust and precise approach: https://play.golang.org/p/OgsE3erjJTK

I have not tested the code on anything real, but I hope it's correct, and that it helps. You might not need the entire machinery, but maybe other readers of this thread do.

--

You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/c7d926b1-7b05-47a0-9eb3-f0e1533b81a6o%40googlegroups.com.

--

Andrei Călin

[Brian Candler]

On Wednesday, 8 July 2020 11:17:56 UTC+1, Andrei Tudor Călin wrote:

I think this works for some cases, but it is potentially wasteful (and even leaky) in terms of resource usage.

For example, if ctx is context.Background(), it leaks a goroutine for every connection

If this is a network server, that's easily fixed: you make a new ctx for each connection (derived from the main one), and cancel it just before the connection is closed.  I think it's useful anyway to have a context which covers the lifetime of that connection.

>  It also keeps the additional goroutine around for the entire lifetime of the connection.

Goroutines are cheap.

How about the following?

type Deadliner interface {

        SetDeadline(t time.Time) error

}

func addContext(ctx context.Context, d Deadliner) {

        go func() {

                <-ctx.Done()

                d.SetDeadline(time.Now())

        }()

}

func handleConnection(ctx context.Context, conn net.Conn) {

    ctx, cancel := context.WithCancel(ctx)

    defer cancel()

    addContext(ctx, conn)

    ... rest of code goes here

}

That seems pretty clean to me.  My only concern is there's a possibility of calling SetDeadline on an already-closed connection, and whether that would cause a panic.

[Andrei Tudor Călin]

Indeed, servers are easier, and if all you're looking to do is to issue a cancellation signal which immediately closes every active connection, the code you posted is perfectly adequate.

I don't expect calling SetDeadline to ever cause a panic, regardless of the state of the connection.

--

You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/aa0eaf7a-27a8-4dc0-a6d7-245317cc4187o%40googlegroups.com.

--

Andrei Călin

[Brian Candler]

Cool, thanks.

One thing I also need to unblock is the Listen loop.  Unfortunately, net.Listener doesn't have a SetDeadline call, so I'll just have to Close it anyway.

[Olivier Mengué]

I have implemented such Reader/Writer in my package github.com/dolmen-go/contextio. Doc: https://pkg.go.dev/github.com/dolmen-go/contextio/?tab=doc

Olivier. 

[Olivier Mengué]

Oh, I just noticed that you had already refered to my package in your original question.

Calling SetDeadline is in fact the answer, but the problem becomes "when to call d.SetDeadline(time.Now()) ?". The answer to that question is "just when you cancel the context".

So a simpler answer is to just enrich the cancel function returned by context.WithCancel to handle

func handleConnection(ctx context.Context, conn net.Conn) {

    ctx, cancelCtx := context.WithCancel(ctx)

    cancel := func() {

        cancel()

        conn.SetDeadline(time.Now())

    }

    defer cancel()

    defer conn.Close()

...

Olivier

[Brian Candler]

On Monday, 13 July 2020 13:10:41 UTC+1, Olivier Mengué wrote:

Calling SetDeadline is in fact the answer, but the problem becomes "when to call d.SetDeadline(time.Now()) ?". The answer to that question is "just when you cancel the context".

So a simpler answer is to just enrich the cancel function returned by context.WithCancel to handle

func handleConnection(ctx context.Context, conn net.Conn) {

    ctx, cancelCtx := context.WithCancel(ctx)

    cancel := func() {

        cancel()

        conn.SetDeadline(time.Now())

    }

    defer cancel()

    defer conn.Close()

Presumably you mean cancelCtx() not cancel() inside the inner func?

AFAICS, your code will only set the deadline if cancel() is explicitly called, or when the outer function terminates (due to the "defer cancel()")

What I want is when either the inner or outer context is cancelled, the deadline is set, so as to break a read() loop.  But as far as I can see, the only way to do this is by selecting on <-ctx.Done() in a goroutine.