Manually interrupt io.Copy goroutine

[kay ru]

Hi,

There are many use cases when you have to interrupt go func() {io.Copy(dst,src)}(). I've googled but still didn't find any solution on how to implement that.

In my case I close writer: myWriter.Close() at the end of function, but io.Copy(myWriter,os.Stdin) still works. Don't know how to interrupt it.

Thanks in advance.

[Manlio Perillo]

Il giorno mercoledì 13 aprile 2016 18:00:00 UTC+2, kay ru ha scritto:

Hi,

There are many use cases when you have to interrupt go func() {io.Copy(dst,src)}(). I've googled but still didn't find any solution on how to implement that.

You can not interrupt a goroutine like you can interrupt, as an example, a process.

 

In my case I close writer: myWriter.Close() at the end of function, but io.Copy(myWriter,os.Stdin) still works. Don't know how to interrupt it.

Can you post your code?

This, as an example, works as expected:

https://play.golang.org/p/U9zUR9LkD8

It prints on stderr:

Copy(): 0 bytes copied, write /dev/stdout: bad file descriptor

Note that in my example I close Stdout from a  *different* goroutine.

Also note that the code I posted is probably not concurrent safe: there are better methods to interrupt a goroutine, but they require special coding for synchronization.

Manlio

[Konstantin Khomoutov]

io.Copy should stop if either of the two conditions hold:

* The nearest read from `src` hits the EOF condition or other error.

* The nearest write to `dst` hits the error.

Hence it might be an error in the implementation of myWriter since it
might not signal io.ErrShortWrite (or io.ErrClosedPipe) to the client
when a write is attempted to it after it was closed.

Another case I might fathom is the case of your writer being os.Stdout
or os.Stderr, and the possible resulting interaction with the EPIPE
errors and the SIGPIPE signal.

I'd say we need more details including your OS and Go version.

[Zippoxer]

Make your own Copy method that can be interrupted. Let me know if you want a snippet.

[Konstantin Khomoutov]

On Wed, 13 Apr 2016 10:03:32 -0700 (PDT)
Zippoxer <zipp...@gmail.com> wrote:

> Make your own Copy method that can be interrupted. Let me know if you
> want a snippet.

I wonder how would you implement one taking into account that either
reading from `src` or writing to `dst` might legitimately sleep in a
syscall -- blocking the host OS thread.

[Zippoxer]

You could only stop read and writing chunks once you received a cancel signal. This has the advantage of the copy exiting right after it's current read or write operation.

You cannot interrupt a single read or write operation as far as I know, unless the reader/writer allows you similarly to net.Conn's Close.

[kay ru]

Sure, I'd see how should it look like.

[anton....@coreos.com]

Here is how I've tried to implement my own io.Copy: http://play.golang.org/p/fHxdtSnjsI

But it doesn't work. It is blocked by "nr, er := teein.Read(buf)" and even sending fake data to the os.Stdin doesn't help.

It is being unblocked only if you press any key in the terminal.

[kay ru]

Looks like it is possible to implement something like "readerCloser := ioutil.NopCloser(os.Stdin)" which could be closed. But even when call readerCloser.Close() it still doesn't work as expected and you have to type something in console manually.

On Wednesday, April 13, 2016 at 7:03:32 PM UTC+2, Zippoxer wrote:

[Skip Tavakkolian]

perhaps you could use timeouts to abandon reads that take too long.  here's a timed copy:

https://gist.github.com/9nut/b874afee6a3cae629c9adc689d195884

so long as there's keyboard input (terminated by newline) it will loop; otherwise it times out. closing stdin before timeout also works.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[kay ru]

Your solution doesn't work. You have to type enter in the terminal. Test is here:

http://play.golang.org/p/pnFctQa-Zl

[C Banning]

stdin buffers for a NL (Enter) key stroke unless it's in raw mode

[kay ru]

And how to avoid this buffering?

[C Banning]

Your program has to set the terminal in "raw" mode - "man stty" should help.

[kay ru]

Too hacky. There should be a way to interrupt Read. It is interrupted when you exit the process, so I hope there should be possibility to do that manually.

[Konstantin Khomoutov]

On Fri, 15 Apr 2016 06:47:31 -0700 (PDT)
C Banning <clba...@gmail.com> wrote:

> Your program has to set the terminal in "raw" mode - "man stty"
> should help.

See [1] for already existing pure-Go solution to do this.

1. https://github.com/creack/termios

[Konstantin Khomoutov]

On Fri, 15 Apr 2016 06:55:49 -0700 (PDT)
kay ru <kay....@gmail.com> wrote:

> Too hacky. There should be a way to interrupt Read. It is interrupted
> when you exit the process, so I hope there should be possibility to
> do that manually.

There are two ways: close the FD or send a signal to the thread which
is waiting in the syscall (and it has to have that signal unmasked and
its disposition not set to SA_RESTART) so that the syscall is
interrupted returning EINTR. Since Go has to fiddle with signal
handling itself to make its runtime happy, I'd say you're left with the
first option.

Considering "interrupted when you exit the process" is in vain because
when you exit the process the kernel cleans after for your process
itself so it's free to do with the state of your process whatever it
wants.

[...]

[kay ru]

Closing of the FD doesn't allow you to use stdin in next iteration. Even if you'll try to open it again through: "newStdin := os.NewFile(uintptr(syscall.Stdin), "/dev/stdin")"

[jark...@gmail.com]

You could wrap src with this: https://play.golang.org/p/Du9_jgwNuY and call the Cancel method on the returned struct to make future reads fail with io.EOF, which would stop an in progress copy

[Konstantin Khomoutov]

On Fri, 15 Apr 2016 10:10:39 -0700 (PDT)
jark...@gmail.com wrote:

> You could wrap src with this: https://play.golang.org/p/Du9_jgwNuY
> and call the Cancel method on the returned struct to make future
> reads fail with io.EOF, which would stop an in progress copy

Can you explain how it works?

[Alex Bligh]

On 15 Apr 2016, at 18:10, jark...@gmail.com wrote:

> You could wrap src with this: https://play.golang.org/p/Du9_jgwNuY and call the Cancel method on the returned struct to make future reads fail with io.EOF, which would stop an in progress copy

How does that work?

Firstly the 'default:' and 'case <-r.interupt:' appear to be inverted.

Secondly, sending a cancel is not going to stop an inprogress (i.e. blocked) r.r.Read(p) as far as I can tell. It will merely skip subsequent reads.

--
Alex Bligh

[Lars Seipel]

On Fri, Apr 15, 2016 at 07:13:49AM -0700, kay ru wrote:
> Closing of the FD doesn't allow you to use stdin in next iteration. Even if
> you'll try to open it again through: "newStdin :=
> os.NewFile(uintptr(syscall.Stdin), "/dev/stdin")"

Actually, this doesn't open the file, but only creates an os.File
structure for the (closed, or worse) file descriptor 0 (syscall.Stdin).

[Matt Harden]

If you dup stdin and read from the dup'ed fd, you should be able to close that one and interrupt the read without closing stdin itself.

[Matt Harden]

Apparently you can't interrupt the read syscall on a terminal by closing the fd if it's in line mode. The close returns but the read doesn't. I suspect even the select syscall wouldn't help here.

[kay ru]

Can you provide an example on how to avoid stdin buffering using this package?

[Konstantin Khomoutov]

On Tue, 19 Apr 2016 10:31:56 -0700 (PDT)
kay ru <kay....@gmail.com> wrote:

> Can you provide an example on how to avoid stdin buffering using this
> package?

Combine [2] and [3] I suppose.

2. https://github.com/creack/termios/blob/master/raw/raw_test.go#L58
3. https://golang.org/pkg/os/#File.Fd

[...]
> > 1. https://github.com/creack/termios
[...]