Jupyter moves to next cell before rendering is finished

[Wim Bogaerts]

I use VPython in a Jupyter environment, using a Python2 kernel (same happens with a VPython kernel)

I have a function thet renders a complex scene (~50 extrusion objects). When I call the function in a cell, it renders correctly, and returns the canvas object, so I can use it in the next cell to manipulate it.

However, when I perform a 'run all cells' in my notebook, jupyter does not wait for the cell with my function to finish, and already starts with the next cell, where the canvas object is obviously not available yet:

[4]   my_canv = draw_my_scene(...)

  

[3]   my_canv.camera.pos = ...

 # this generates an error, because the cell above was not yet finished 

I tried using canv.waitfor("draw_complete") in the draw_my_scene function, but this does not help. 

Is there a way to make jupyter wait until the rendering is complete?

 

[Bruce Sherwood]

The situation is truly bizarre, and at the moment I have no clue what the cause is or how to fix it.

I put the following in cell 1:

print('start')

print('cell 1')

and in cell 2 I put this:

print('cell 2')

When I run all cells, I of course see the first two prints followed by the second print.

Next I change cell 1 to look like this:

print('start')

from vpython import *

print('cell 1')

When I run all cells, the print statements come out in the following order:

start

cell 2

cell 1

Moreover, the cells are now labeled In [2]: and In [1]: instead of In [1]: and In [2]:!

Perhaps John Coady will have some insight on how this can occur. It is certainly the case that importing vpython does some tricky things to set up communications between the local server and the browser, but I would naively think that Jupyter wouldn't advance to cell 2 without finishing cell 1, including the print('cell 1').

I also often see the error "ERROR! Session/line number was not unique in database" when I also see the cells run out of order and renumbered.

Bruce

[John]

This may be due to calling the routine do_one_iteration() when we import with "from vpython import *". We call do_one_iteration() in the rate() routine to retrieve data sent from the front end. I think we are calling it at another place in the code when we import vpython but it might only be necessary to call it in the rate routine.

John

[Bruce Sherwood]

I checked, and do_one_iteraction() is called in just one place, sendtofrontend(), a method of _RateKeeper2(), so part of the rate machinery.

Bruce

[John]

I double checked and it is called on line 66 of with_notebook.py file.

John

[Bruce Sherwood]

Thanks for catching my mistake, John. I did the simple experiment of commenting out the line in with_notebook.py, and the result is that the sequence of operations now works properly, though with the first cell comes reproducibly the "ERROR! Session/line number was not unique in database". Also, the cells are renumbered as before, with the first cell now being cell 2 and the second cell being cell 1.

So you're right, there's something going on in the do_one_iteration area, but simply eliminating the call in with_notebook.py doesn't fix the problems.

Bruce

[John]

As a workaround you could try inserting a blank cell after the cell that contains the "from vpython import * " statement or before the cell containing your code "my_canv.camera.pos = ..."  and then try "run all cells" and see if that works for you.

John

[Wim Bogaerts]

I tried adding the blank cell. It doesn't help. It still executes out of order. 

[John]

I have a potential fix for your problem but I need to test it out further before I update the vpython code.

John

[Bruce Sherwood]

Sounds promising, John.

Bruce

[John]

The fix for this problem has been checked in to jupyter vpython on github. It will be part of vpython version 7.1.6 .

On Wednesday, November 22, 2017 at 10:17:32 PM UTC-8, Bruce Sherwood wrote:

Sounds promising, John.

Bruce

[Bruce Sherwood]

That's wonderful, John! Thanks!

Bruce

[Wim Bogaerts]

Thanks John. I tried this, but it still gives me the same out-of-order execution of the cells.

Wim

[John]

If you are calling rate() routine in a cell then in this case other cells will get executed. Typically a vpython program will contain an infinite loop with a rate statement to run the simulation indefinitely.

  while True:

      rate()

      .....

If a cell contained this code then the cell would never complete execution. But vpython needs to be interactive to allow data to be sent back and forth between the python kernel and the javascript code running in the web browser front end and this occurs when the rate() routine is called. As a consequence of this, other cells will be allowed to execute.

If you don't want other cells to be executed then perhaps you can order your cells in the notebook so that the cell with the rate() statement is executed last. 

This fix that I put in handled the other cases where we were calling rate() internally in vpython waiting for the communication channel between vpython kernel and front end to be set up.

[Wim Bogaerts]

Bizarre. I did not make any call to this 'rate' function. I just create a scene with some meshes in it.

[John]

Hi Wim

   You mentioned earlier the following

"I tried using canv.waitfor("draw_complete") in the draw_my_scene function, but this does not help. "

Are you still calling waitfor routine or any other wait routine? It looks like they call rate() function internally. If you are calling any wait routine then perhaps you can try running without them. Also, if you can post your notebook on github or 

https://nbviewer.jupyter.org/

that reproduces the problem then I can get a better understanding of what is going on.

John

[Wim Bogaerts]

Hi John, 

It took me a bit of time to extract the code from the libraries to get a minimal self-contained example. I have attached the notebook.

Wim

[Wim Bogaerts]

In that particular example, 'rate' is invoked 8 times.