io.Reader

[John]

Hey guys,

So, I'm trying to use the exec library, but have run into a problem
with dealing with io.Reader.

So, I build a cmd like so:

cmd := exec.Command(`ssh antartica.mtv`)
cmd.Start()

stdin, err := cmd.StdinPipe()
if err != nil {
fmt.Println(err)
return
}
stdout, err := cmd.StdoutPipe()
if err != nil {
fmt.Println(err)
return
}
=======

Now I want to be able to read from the stdout and either break after a
specified time period or see something I want and then write to stdin.

But when I do:

store := [1]byte{}
storeSlice := store[:]
_, err := e.stdout.Read(storeSlice)

I get hung on Read() (it never returns).

I tried switching out the cmd.Stdout for bytes.Buffer, but it looks
like calling cmd.Start() does an assigment to cmd.Stdout(since buffer
gives me much more control).

Anyone know how to do this?

[Ian Lance Taylor]

Use one goroutine to read from the pipe and send the results on a
channel. Use another goroutine to select between that channel and a
time.After.

Ian

[John Doak]

What happens if the Read() never comes back in the go routine? Do you
end up with an orphaned coroutine mapped to a thread that sits there
forever?

--
John Doak
http://flavors.me/johnsiilver

[Ian Lance Taylor]

John Doak <johns...@gmail.com> writes:

> What happens if the Read() never comes back in the go routine? Do you
> end up with an orphaned coroutine mapped to a thread that sits there
> forever?

Yes, but it's worse than that: you must have a child process sitting
around that never exits. Because if the process exited, then the Read
would return with an error and the goroutine would stop. If you don't
want the child process to linger forever, perhaps you need another
goroutine that sleeps for a while and then kills the child process.

Ian

[John Doak]

Am I crazy to think this is a severe limitation of the io.ReadCloser
being used for exec? I'd switch out stdin/stdout for
bytes.Buffer(which has a lot more features to utilize), but it looks
like Start() is what creates the stdin/stdout/stderr, and if you
replace it after calling start() you would lose all the data
inbetween(plus I doubt that is thread safe).

I was building something akin to tcl/Expect for input(and one of the
main input's would of course be exec.Cmd). But I'm thinking that if I
have to create goroutines to kill child processes, then either I'm
doing something terribly wrong or exec.Cmd is not flexible enough.
Should I look at using a lower level library instead of cmd.Exec? Or
really, any advice on what I *should* be doing would be welcome.

[roger peppe]

On 21 June 2012 07:18, John Doak <johns...@gmail.com> wrote:
> Am I crazy to think this is a severe limitation of the io.ReadCloser
> being used for exec?  I'd switch out stdin/stdout for
> bytes.Buffer(which has a lot more features to utilize), but it looks
> like Start() is what creates the stdin/stdout/stderr, and if you
> replace it after calling start() you would lose all the data
> inbetween(plus I doubt that is thread safe).

No, you can do that just fine if you want, although I don't think
it will solve your problem, as you have to wait for the command
to terminate before using the bytes.Buffer.

> I was building something akin to tcl/Expect for input(and one of the
> main input's would of course be exec.Cmd).  But I'm thinking that if I
> have to create goroutines to kill child processes, then either I'm
> doing something terribly wrong or exec.Cmd is not flexible enough.

I think exec.Cmd is flexible enough. What you need, I think, is something
that layers on top of an io.Reader and provides timeout functionality
along the same lines as net.Conn.

Something like this would probably do the job:

http://play.golang.org/p/yaiDx0zdW4

[John Doak]

On Thu, Jun 21, 2012 at 1:08 AM, roger peppe <rogp...@gmail.com> wrote:
> On 21 June 2012 07:18, John Doak <johns...@gmail.com> wrote:
>> Am I crazy to think this is a severe limitation of the io.ReadCloser
>> being used for exec?  I'd switch out stdin/stdout for
>> bytes.Buffer(which has a lot more features to utilize), but it looks
>> like Start() is what creates the stdin/stdout/stderr, and if you
>> replace it after calling start() you would lose all the data
>> inbetween(plus I doubt that is thread safe).
>
> No, you can do that just fine if you want, although I don't think
> it will solve your problem, as you have to wait for the command
> to terminate before using the bytes.Buffer.

I'm curious to why this is? Wouldn't the command be writing to the
buffer and it streaming through so another goroutine would be able to
see it? I assume bytes.Buffer has locks around reading/writing.

>
>> I was building something akin to tcl/Expect for input(and one of the
>> main input's would of course be exec.Cmd).  But I'm thinking that if I
>> have to create goroutines to kill child processes, then either I'm
>> doing something terribly wrong or exec.Cmd is not flexible enough.
>
> I think exec.Cmd is flexible enough. What you need, I think, is something
> that layers on top of an io.Reader and provides timeout functionality
> along the same lines as net.Conn.
>
> Something like this would probably do the job:
>
> http://play.golang.org/p/yaiDx0zdW4

Thanks for pointing this out. This could help.

--John

[Ian Lance Taylor]

John Doak <johns...@gmail.com> writes:

> I assume bytes.Buffer has locks around reading/writing.

That turns out not to be the case.

Ian

[John Doak]

So the reason I'd have to wait for the program to exit is because I'd have a thread safety issue otherwise?

[Kyle Lemons]

Even beyond that, you'd be lacking a way to communicate that more data has arrived.

[Matthew R Chase]

I'm not sure that timeoutReader works.  I ran it, but it crashed with a nil memory address (or some-such) error when receiving data from the reader.  Furthermore, the example instates one reader which continuously reads; the timeout does not affect the reader but just the blocking while listening for the reader.  I found the concept to be misleading.

I've been doing some serial programming in go, and can offer the following condensed example:

package main

import(

    "github.com/tarm/goserial"

    "log"

    "runtime"

    "time"

)

func main() {

    runtime.GOMAXPROCS(runtime.NumCPU())

    

    log.Print("Opening serial port")

    c := &serial.Config{Name: "/dev/ttyUSB0", Baud: 9600}

    s, err := serial.OpenPort(c)

    if err != nil {

        log.Fatal(err)

    }

    

    type readResult struct{

        b []byte

        err error

    }

    rc := make(chan *readResult)

    

    go func(){

        for{

            buf := make([]byte, 128)

            n, err := s.Read(buf)

            log.Print("Reading from serial port")

            rc <- &readResult{buf[:n], nil}

            if err != nil { return }

        }

    }()

    for {

        timeout := time.NewTicker(2 * time.Second)

        defer timeout.Stop() //is this necessary?

        

        select{

        case got := <-rc:

            log.Print("got a result")

            switch{

            case got.err != nil:

                //Catching an EOF error here can indicate the port was disconnected.

                // -- if using a USB to serial port, and the device is unplugged 

                //    while being read, we'll receive an EOF.

                log.Fatal("  error:" + got.err.Error())

            default:

                log.Print(got.b)

            }

        case <-timeout.C:

            //stop waiting for the reader to send something on channel rc

        }

        log.Print("Writing to serial port")

        _, err = s.Write([]byte("test"))

        if err != nil {

            log.Fatal("Could not write to port: " + err.Error())

        }

        

        time.Sleep(2 * time.Second) //stutter the infinite loop.

    }

    

}

[roger peppe]

On Jun 23, 2012 6:43 PM, "Matthew R Chase" <ma...@chasefox.net> wrote:
>
> I'm not sure that timeoutReader works.  I ran it, but it crashed with a nil memory address (or some-such) error when receiving data from the reader.  Furthermore, the example instates one reader which continuously reads; the timeout does not affect the reader but just the blocking while listening for the reader.  I found the concept to be misleading.

I'm not surprised it crashed - it was only half an hour's proof-of-concept (although I'd like to see the code that crashed for you, as I may end up using the code some time in the future).

Perhaps the name is misleading, but it is not possible to time out reads on a pipe AFAIK, so the only alternative is to leave the reader around indefinitely, and use the substitute "timeout reader" whenever you might wish to use the original.

[Matthew R Chase]

Sorry, I didn't keep that example and I am unable to reproduce my error.  A non-working example is below (for all it's worth).  It isn't reading at all; I'm not sure what I'm doing different/wrong this time.  The example below doesn't appear to be reading the port because the deadline isn't applied.  Looping is done every second; if it was pausing to read from the port I would expect it to loop every 6 seconds.

This test was performed with two PCs connected via serial.  one running the routine below and a windows PC running procomm.  the PC running procomm receives the "test" sent from the below routine.  This is the same environment I used when timeoutreader crashed for me previously.  If I sent nothing from procomm, a routine similar in concept to the one below would just keep reading nothing.  But when I sent anything (by pressing a key in the procomm terminal), it would crash.

package main

import(

    "timeoutreader"

    "github.com/tarm/goserial"

    "log"

    "runtime"

    "time"

)

func main() {

    runtime.GOMAXPROCS(runtime.NumCPU())

    

    c := &serial.Config{Name: "/dev/ttyUSB0", Baud: 9600}

    port, err := serial.OpenPort(c)

    if(err != nil){ log.Fatal("Could not open port") }

    reader := timeoutreader.NewTimeoutReader(port)

    deadline := time.Now()

    for {

        deadline = deadline.Add(5 * time.Second)

        reader.SetReadDeadline(deadline)

        

        _, err = port.Write([]byte("test"))

        if err != nil { log.Fatal("Could not write to port: ", err) }

        

        log.Print("reading")

        length, err := reader.Read(nil)

        if(err != nil){ log.Fatal("Could not read from port:", err) }

        if(length > 0){ log.Print("Received result of length:", length) }

        

        time.Sleep(1 * time.Second) //stutter the infinite loop

[roger peppe]

You are trying to read no bytes. It will always return immediately.