removing computation from the main thread? :-/

[Brendan Colloran]

Hamilton did some research into using webworkers as "compute kernels" today. His initial impressions is that there are some major obstacles to having this be usable, particularly WRT dom access and manipulation. Also, it turns out that code evaled in a worker has to be written in strict mode, which is another smaller but non-negligible usability wrinkle. This is preliminary (I think Hamilton had a couple other avenues he wanted to look into) so I'll allow Hamilton to expand on what he's found as needed.

So, let's start to discuss the _user_ facing needs and user stories that we are implicitly trying to address with moving compute out of the main thread so that we assess the cost of failing to address those needs, and even better, so that we can begin to think through other mitigation strategies (since initial indications are that webworkers are less promising than hoped).

First, to reiterate the core value add of the Iodide notebook: we want to give users the ability to seamlessly manipulate the DOM in the course of their workflow, b/c we believe rich media and interactivity opens up new ways of communicating scientific and technical concepts. (and this is why webworkers aren't looking promising ATM).

## user stories

So that said, I think these are the user stories we are trying to address when we talk about moving computation off the main thread. please add if there are more--

1) user starts a long computation. thread is blocked and the notebook is unresponsive, so until the computation completes the user cannot perform any notebook actions, edit text blocks, etc.

- Expectation: the user expects to be able to keep working on other parts of the notebook while a computation runs.
- Jupyter parity: the NB never hangs even when the kernel is occupied.

- Current solution: you just wait

2) user starts a computation that hangs, of a long running computation that they want to abort (probably so that they can tweak their code and try again)

- Expectation: the user wants to be able to bail out on an evaluation in progress without losing the work they've done so far (their global variables).

- Jupyter parity: at least with the python kernels, jupyter can easily interrupt evaluation without losing the global environment b/c python support "KeyboardInterrupt". This may or may not be supported for all kernels, I don't know.

- current solution: if the notebook hangs, you have to reload the whole page. Work in progress is lost -- minimally the global environment, but possibly code edits as well.

[aside: It should go without saying that anything that results in people losing work instantly disqualifies a product from being desirable. So I'll plant this as an MVP Criteria Flag: even in the worst case, iodide must make work recoverable, and should do so with minimal friction. https://github.com/iodide-project/iodide/issues/503 is a step on this path, but not enough]

Those are the two main user needs that i have in mind around this. What am I missing?

## mitigation strategies

A) using other threads -- we may be able to solve some of this by putting computation in something other than the main content thread.

A.1) something with workers. this does not look promising based on current research. but workers are non-blocking (story 1), and can be killed (story 2)

A.2) something with extensions? i think that web extensions run in a separate thread from page content (but i haven't found a reference on that so i'm not 100% sure). if this is true, maybe there is some way of having the react/redux app live in an extension and do the app-level dom manipulation while the evaluation happens in the main content thread (or vice versa). something like this could potentially solve story 1 if the UI manipulation is on a different thread than the computation. It's unclear if this would allow for interrupt (story 2), but we might be able to simulate that with some combination of something like this and the ideas below.

B) without using other threads -- if we cannot come up with a good way of putting UI and Science computation in actually different threads -- we may at least be able to come up with mitigation strategies that make the worst case scenarios less bad.

B.1) So if we can't address story (1) by default, what can we do to make the experience less annoying. some ideas:

B.1.1) We can have better (i.e.: any) UI around running computations -- something that let's a user know that the they cannot interact with the NB because a big computation is running.

B.1.2) Some better UI around that allows heavy computation to be put into web workers. This forces a lot more complexity onto the user (and presumably adds complexity to the app too), but it might be worth it for some use cases

B.2) a few things we can do to make the experience of reloading a hung notebook less terrible:

B.2.1) As mentioned in https://github.com/iodide-project/iodide/issues/503 we can always update the autosave before evaling. We could also have an alert like "Iodide has detected a more recent autosave of this notebook, do you want to use that version?"

B.2.2) Perhaps we can also stash user-defined variables in local storage and make it available when restoring autosaves (this might be tough with functions). As a subset of this, we could store variables that will be "frozen" for export (though of course we haven't worked out how freezing will work yet...)

B.2.3) We could implement the ideas above with some kind of webextension that has an "interrupt computation" button that behind the scenes just reloads the page, restores the latest autosave, and injects the user-defined variables back into the environment (or something along those lines)

Those are my ideas so far, but I know there must be other things we can do. Please jump in and let's brainstorm more ideas, and also discuss advantages/disadvantages of each.

-brendan

[Dhiraj Barnwal]

So I was researching more on web workers and how we can use it for our case. Since web workers do not run on the main UI thread, they do not have access to the DOM. I was thinking about the possibility of having a Virtual DOM for the web worker. We can make the changes on the Virtual DOM and send a copy to the main thread where the differences get executed. Also, as we are using React we already have a Virtual DOM with us to work with. I did a bit of googling and found that there are some basic implementations of the above concept. I am posting some of the links which I found out during my research. If this looks promising, maybe we can dig deeper into what it can offer for Iodide's case.

http://blog.nparashuram.com/2016/02/using-webworkers-to-make-react-faster.html
https://stackoverflow.com/questions/34324843/rendering-react-components-with-webworkers
https://github.com/theefer/virtualdom-in-web-worker

Alternatively, we can use JsDOM. JsDOM also works with web workers and has a research paper published on this. There is one caveat though. It's browser implementation is still weak (https://github.com/jsdom/jsdom#running-jsdom-inside-a-web-browser). We need to browserify it before it can be used (issue tracked in https://github.com/jsdom/jsdom/issues/1284).

I am not fully aware of the implications and the implementation part but this could serve as a starting point for us.

[Brendan Colloran]

This is great research Dhiraj. Some really interesting ideas to consider in here.

--
You received this message because you are subscribed to the Google Groups "iodide-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to iodide-dev+unsubscribe@googlegroups.com.
To post to this group, send email to iodid...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/iodide-dev/e59a574e-4ed9-403e-9e2d-2bb014d57601%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[David Sarma]

this problem (long-running computation blocking the main thread) seems to be what this stopify project is intending to address: http://www.stopify.org/

[Brendan Colloran]

Ah that looks cool, thanks for the reference David. Kind of similar to some other things we looked at a while back that act as wrappers on the interpreter, but this might be a bit smarter.

Hamilton -- might be something interesting to do here with using stopify of one of those other things we looked at for a "debug run" mode that allows you to step through code. Who knows what kinds of gotchas and dragons might be turned up, but something to keep in mind for later perhaps.

--
You received this message because you are subscribed to the Google Groups "iodide-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to iodide-dev+...@googlegroups.com.

To post to this group, send email to iodid...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/iodide-dev/7ce2d527-674a-4d59-860f-0be625747f78%40googlegroups.com.