Executing script statement-by-statement

30 views
Skip to first unread message

Tristan Zajonc

unread,
Oct 13, 2016, 3:57:53 AM10/13/16
to Project Jupyter
Has anybody considered extending the jupyter messaging protocol to allow kernels to optionally implement statement-by-statement execution?  This would allow users to create notebooks from raw scripts, akin to RStudio.  The behavior of the parser would be up to the kernel, but typically it would parse a code string into statements that could be executed individually.

I'd be interested in potentially adding this feature -- I've done so out-of-band for R and Python -- but am curious if there's a preferred path.  I could imagine either extending execute_request to take a "split" parameter, although that may get confusing since there would then be multiple execute_results.  The alternative would be to implement a new message type, execute_statements, then would parse the code string and issue multiple execute_requests.  This should be backward compatible since the output messages would be unchanged.

This could be combined with something like: jupyter notebook run analysis.py --output=results.ipynb

Is this worth pursuing as part of an enhancement proposal?

Tristan

MinRK

unread,
Oct 13, 2016, 8:52:38 AM10/13/16
to Project Jupyter
I don't believe any protocol changes are needed for this. You can already run a script through Jupyter, breaking it up however you want. The trick would be writing a runner for a given script (you would need to write a different one for every language to do the parsing correctly), and then submit the chunks as you see fit. Alternatively, you could rely on simple markers in comments to identify the chunks to split on.

There is [a PR](https://github.com/jupyter/jupyter_client/pull/184) adding a `jupyter run` command that executes a script as a whole file. It doesn't record the output into a notebook, but a similar script that does would not be a large project.

What do you see as a needed change to the protocol that isn't served by what we already have?

-Min

--
You received this message because you are subscribed to the Google Groups "Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jupyter+unsubscribe@googlegroups.com.
To post to this group, send email to jup...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/a3340660-4eb2-4079-b7ee-df7dc4962a35%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Tristan Zajonc

unread,
Oct 13, 2016, 2:39:33 PM10/13/16
to jup...@googlegroups.com
Min,

What you suggest is exactly what we do currently.  The downside is that we need to implement parsers for each language and put another proxy in front of the kernels or kernel gateway and ensure that our parsers are using the same underlying runtime as the kernel if they parse a proper AST. So the advantage of having a feature like this would be that kernels could implement this behavior directly, against the same runtime, and everybody could share this implementation easily across both commands like jupyter run and kernel gateway.

They very real downside, which you allude to, is there are many approaches to chunking a file.  For instance, one might want to use something like Rmd or allow some output control in comment pragmas.  This could be accommodated by having a style parameter.

Tristan

--
You received this message because you are subscribed to a topic in the Google Groups "Project Jupyter" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jupyter/SM68P70XzW4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jupyter+unsubscribe@googlegroups.com.

To post to this group, send email to jup...@googlegroups.com.

Matthias Bussonnier

unread,
Oct 13, 2016, 5:47:48 PM10/13/16
to jup...@googlegroups.com
What I would suggest if for you to come up with an "API" by modifying
Jupyter/IPython adding endpoints or if possible as an extension.

I would go for an api that get a document "content" and return a list
of positions that are block of execution code, then see if we can
iterate on it.
That would allow the frontend to show visual indication of what these
"statements" are.

I would also look at how Hydrogen (the Jupyter plugin for atom) is
doing it, as AFAICT it auto-detects blocks.

--
M
>>> email to jupyter+u...@googlegroups.com.
>>> To post to this group, send email to jup...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/jupyter/a3340660-4eb2-4079-b7ee-df7dc4962a35%40googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "Project Jupyter" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/jupyter/SM68P70XzW4/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>> jupyter+u...@googlegroups.com.
>> To post to this group, send email to jup...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/jupyter/CAHNn8BUbvc2-EyX1VvSLxdcrDH6OwqounZt_Swdbbhz%3De-P8Rg%40mail.gmail.com.
>>
>> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Project Jupyter" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jupyter+u...@googlegroups.com.
> To post to this group, send email to jup...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jupyter/CAPVVPNyW-YUPJMP7G5HYLMEsiULDyEtUXdH-CB2nwsMnMqAGJw%40mail.gmail.com.
Reply all
Reply to author
Forward
0 new messages