How to handle async IPython magic when using nbconvert with --execute?

[Matthias Geier]

Dear list.

I've written an IPython cell magic which runs some computation in a
background thread (via concurrent.futures.ThreadPoolExecutor) and
asynchronously updates a code output cell (via
IPython.display.DisplayHandle.update() called with the help of
add_done_callback()) when it's done.

This works nicely in JupyterLab (and in the Classic Notebook), but
when I run the notebook with nbconvert's --execute flag, the final
outputs don't have their updated values.

Can I do something in the implementation of my cell magic to wait for
all asynchronous updates at the end of the notebook?

Is there some hook function for this?

Or can I somehow check (from within the magic code) if nbconvert (or
another non-interactive tool) is running?
I could easily switch to synchronous processing in that case.

Or is there another way to solve this (that keeps the async behavior
in the interactive case)?

cheers,
Matthias

[René Fritze]

Dear Matthias, dear List,

I wonder if you ever found a solution to your problem, as I am currently facing a similar one. I'm using asyncio to load additional data while plotting in a notebook cell. If I run this through nbconvert I want to wait on all data being loaded before the cell finishes.

Cheers,

René

[Matthias Geier]

On Fri, Oct 4, 2019 at 10:28 AM René Fritze wrote:
>
> Dear Matthias, dear List,
>
> I wonder if you ever found a solution to your problem,

Well, not a good one.

I've found an existing issue: https://github.com/jupyter/nbconvert/issues/913

And, as mentioned in the issue, I've found an ugly work-around:

get_ipython().get_parent()['content']['allow_stdin']

It would be great to have a proper API for that, though.
And my work-around can break any time when a new version comes out.

cheers,
Matthias

> --
> You received this message because you are subscribed to the Google Groups "Project Jupyter" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to jupyter+u...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/70967489-dcbd-4ae6-80dc-06aaa2cfc3ce%40googlegroups.com.

[René Fritze]

Hey Matthias,

Thanks for the answer. I've opted for setting a environment var through our CI, but that check you came up with would be an OK fallback for now.

Best,

René

> To unsubscribe from this group and stop receiving emails from it, send an email to jup...@googlegroups.com.