How to fetch the output of ipython kernel background thread

[Jeff Zhang]

I would submit one piece of python to ipython kernel. And this piece of kernel would run a thread to generate output continuously. Then What kind of ipython api I can use to get the output of this daemon thread ? Thanks

[MinRK]

IPython uses global state to identify which request produced which output. This is the `parent_header` in the kernel. The result is that, as far as the messages are concerned, output produced in a background thread will always be associated with the most recently executed cell.

If you know what you want, you can, however, explicitly associate the outputs of a thread with a particular parent. Doing so, however, may result in a race condition where it appears that the thread owns output produced in the main thread, depending on how output is captured or redirected.

-Min

On Tue, Oct 10, 2017 at 8:33 AM, Jeff Zhang <zjf...@gmail.com> wrote:

I would submit one piece of python to ipython kernel. And this piece of kernel would run a thread to generate output continuously. Then What kind of ipython api I can use to get the output of this daemon thread ? Thanks

--
You received this message because you are subscribed to the Google Groups "Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jupyter+unsubscribe@googlegroups.com.
To post to this group, send email to jup...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/31687feb-9acf-4d9a-b7d1-e02e43df5894%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jeff Zhang]

Thanks Min. Here's the sample code I use ipython kernel api.

self._km, self._kc = jupyter_client.manager.start_new_kernel(kernel_name='python')

reply = self._kc.execute_interactive(request.code,
                                output_hook=_output_hook,
                                timeout=TIMEOUT)

Do you think what kind of api I can use to get the background thread output after `execute_interactive` is finished ? Thanks

在 2017年10月10日星期二 UTC+8下午5:11:31，Min RK写道：

IPython uses global state to identify which request produced which output. This is the `parent_header` in the kernel. The result is that, as far as the messages are concerned, output produced in a background thread will always be associated with the most recently executed cell.

If you know what you want, you can, however, explicitly associate the outputs of a thread with a particular parent. Doing so, however, may result in a race condition where it appears that the thread owns output produced in the main thread, depending on how output is captured or redirected.

-Min

On Tue, Oct 10, 2017 at 8:33 AM, Jeff Zhang <zjf...@gmail.com> wrote:

I would submit one piece of python to ipython kernel. And this piece of kernel would run a thread to generate output continuously. Then What kind of ipython api I can use to get the output of this daemon thread ? Thanks

--
You received this message because you are subscribed to the Google Groups "Project Jupyter" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jupyter+u...@googlegroups.com.

[MinRK]

On Wed, Oct 11, 2017 at 4:08 AM, Jeff Zhang <zjf...@gmail.com> wrote:

Thanks Min. Here's the sample code I use ipython kernel api.

self._km, self._kc = jupyter_client.manager.start_new_kernel(kernel_name='python')

reply = self._kc.execute_interactive(request.code,
                                output_hook=_output_hook,
                                timeout=TIMEOUT)

Do you think what kind of api I can use to get the background thread output after `execute_interactive` is finished ? Thanks

You have `output_hook`, which can be used to capture all of the outputs. You can do whatever you want with them. You can chose to redisplay the outputs when the final result is finished, as they arrive (which can get messy when it comes to associating with particular parents), or capture them and redisplay them explicitly from the main thread via a `.redisplay_outputs(request)` API.

-Min

 

在 2017年10月10日星期二 UTC+8下午5:11:31，Min RK写道：

IPython uses global state to identify which request produced which output. This is the `parent_header` in the kernel. The result is that, as far as the messages are concerned, output produced in a background thread will always be associated with the most recently executed cell.

If you know what you want, you can, however, explicitly associate the outputs of a thread with a particular parent. Doing so, however, may result in a race condition where it appears that the thread owns output produced in the main thread, depending on how output is captured or redirected.

-Min

On Tue, Oct 10, 2017 at 8:33 AM, Jeff Zhang <zjf...@gmail.com> wrote:

I would submit one piece of python to ipython kernel. And this piece of kernel would run a thread to generate output continuously. Then What kind of ipython api I can use to get the output of this daemon thread ? Thanks

--
You received this message because you are subscribed to the Google Groups "Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jupyter+u...@googlegroups.com.
To post to this group, send email to jup...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/31687feb-9acf-4d9a-b7d1-e02e43df5894%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Project Jupyter" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jupyter+unsubscribe@googlegroups.com.

To post to this group, send email to jup...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/de0ef6ac-a779-4283-9302-97aec00c3b71%40googlegroups.com.

[Jeff Zhang]

Hi Min,

I do it like following, could still could not get the output of the backend daemon thread (I only see the "hello world" output). Could you guide me how to do that ? Thanks

import jupyter_client

import threading

km, kc = jupyter_client.manager.start_new_kernel(kernel_name='python')

code = """\

import time

import threading

def worker():

  for i in range(10):

    time.sleep(1)

    print(i)

t = threading.Thread(name="ConsumerThread", target=worker)

t.start()

print("hello world*********************")

"""

def _output_hook(msg):

  print("****************")

  print(msg)

reply = kc.execute_interactive(code, output_hook=_output_hook, timeout=100)

print(reply)

import time

time.sleep(10)

在 2017年10月11日星期三 UTC+8下午5:56:48，Min RK写道：