STDOUT of worker process / Gtk based REPL

[Tobi]

Hi,

is there a way to obtain the console output of a worker process? I am experimenting with a Gtk Julia REPL and the best approach seems to be to pass all executions to a worker process so that GUI and REPL process are distinct.

Onother question: Does someone know if REPL.jl supports running the commands in a worker process?

My first poor mans approach for a Gtk based REPL is to read the command from a GtkEntry, eval it in a worker and then print the output (when I have the workers redirect) to a GtkTextView. It would be much nicer to somehow "run" REPL.jl in an GtkTextView. But I still do not know how the interface between REPL.jl and GtkTextView has to look like. Would be happy about input if someone has an idea (Keno?)

Thanks

Tobi

[Amit Murthy]

A lot of the parallel code assumes that process 1, i.e. the current REPL is the driver process and only uses workers (i.e., pids > 1) for computation. 

Would it work for you to start the GUI in a separate worker instead? Maybe we should anyway provide a means of "reserving" certain workers (either the GUI or the REPL process), so that they are used in any parallel work distribution. 

 

The "create_worker" function in multi.jl currently redirects all output from the worker to the clients stdout. We will have to make this step optional based on a keyword arg to `addprocs` and also provide a means of getting the stream handle given a worker id.

[Amit Murthy]

In my previous comment, the "reserved" worker processes should NOT be used in parallel work distribution.

[Tobi]

Thanks! Whats the difference between start_worker and addprocs? When I use addprocs the stdout seems not to be redirected. start_worker only seems to hang.

[Amit Murthy]

start_worker is an internal function - just mentioned it in case you wanted to know what is happening under the hood.

addprocs does redirect stdout of the workers. Juts try remotecall_fetch(2, ()->println("Hello World!") and you will see its output in the REPL with a prefix identifying the worker. 

[Tobi]

Ok I see. But I use redirect_stdout() in order to redirect the stdout of the master process into a GtkTextView. And in this setting I don't get output from the worker while the redirection of the masters output works.

The code looks like this:

  rd, wr = redirect_stdout()

  function redirectToWindow(timer,::Int32)
    @async begin
      response = readavailable(rd)
      if !isempty(response)
        insert!(textView,string(response,"\n"))
      end
    end
  end

  timeout = Base.TimeoutAsyncWork(redirectToWindow)
  start_timer(timeout,1e-1,5e-3)

[Amit Murthy]

I don't know what is happening in your case, but the below code does print output from the worker after calling redirect_stdout()

addprocs(1)

rd, wr = redirect_stdout()

@schedule begin

    while(true)

        response = readavailable(rd)

        println(STDERR, response)

    end

end

remotecall_fetch(2, ()->println("Hello World from $(myid())"))

It may still not serve your need since you will see the output from the worker prefixed with the text "From worker 2:"....

[Amit Murthy]

Ah! I think you are missing a while(true) loop in your @async block. Can you also do away with the timer and just use a scheduled block like my example?

[Tobi]

Thanks I try that.

[Tobi]

This works better but not completely. println("Hello") works but print("Hello") does not work. There seems to be a flushing issue.

[Tobi]

And redirection of stderr using

  rderr, wrerr = redirect_stderr()
  @schedule begin
    while(true)
       response = readavailable(rderr)
       if !isempty(response)
         insert!(textView,string(response))
       end
    end
  end

seems to not work. It still writes to the console (currently on windows 32bit Julia 0.2)

[Amit Murthy]

For that you will have to redirect stderr to stdout on the worker ...

[Tobi]

Thanks, that works.