The design of Leo+Ipython+Jupyter+Lit-computing

[Edward K. Ream]

In this post, I'll discuss how Leo can integrate with IPython and Jupyter notebooks so as to become a premier literate computing environment, as Offray uses the term.

This is a design document.  I'll focus only on what the user sees and experiences, not on how Leo might actually get the job done. tl:dr:  Read the summary.

IPython and Jupyter will remain essentially unchanged

I assume that both IPython and Jupyter will not change in their essential qualities.  That is, IPython and Jupyter are fixed (constant).  In particular, I assume that for the foreseeable future that Jupyter notebooks will be a linear list of cells, not a tree of cells. In any case, the following design would not change much if Jupyter notes ever do support trees of cells.

Implications

1. As a direct consequence of this assumption, only Leo will explicitly deal with outlines.

2. There must be some way of converting (parts of) Leo outlines to a list of .ipynb files. There is considerable room for creativity in this conversion process.  That is, we want to keep tree-to-notebooks process in the back of our mind as we think about Leo trees.

The Jupyter design

The jupyter notebook handles cells in a simple, intuitive manner as follows:

1. Running a code cell produces output in that cell.  That is, the output is attached to the cell itself, not in a Qt popup window.

2. It's possible to switch the types of Jupyter cells from a dropdown menu. The cell types are Code, Markdown, Raw NB Convert and Heading.

3. There are Jupyter commands to run some or all cells, and to insert, delete and move cells.

A preliminary design

Imo, we probably want similar (or improved) behavior in Leo cells:

- It should not be necessary to open a view-rendered window to see the output of a cell.

- Cells should show the In and Out numbers, just like in IPython or Jupyter.

- Leo directives will indicate the type of cells.  Some of these directives will apply to descendant nodes.

- Leo will provide commands to run all cells, selected cells, selected subtrees and marked cells.

- Existing Leo commands will suffice to insert, delete, move, clone, promote and demote cells.

New directives

@jupyter-notebook: explicitly denotes a notebook.  All descendants are IPython cells. Useful when converting a Leo outline to one or more Jupyter notebooks.

@cells: All descendants are IPython cells.

@cell: A single IPython cell.

Within a cell, @language denotes the type of cell.

@language markdown denotes a Jupyter Markdown cell.  There probably is not great need for Jupyter Heading cells: they may come free with Leo's outline structure.

@language python (or julia, or r, etc) denotes a code cell.

Behavior

There is lots of room for experimentation re behavior and rendering.  In general, we want cells to work just like the corresponding Jupyter cells, but enhanced, when possible, by Leo's outline structure.

For example, the rending of @markdown cells might include the rendering of all descendant @markdown cells, thereby making Jupyter Heading cells unnecessary with Leo itself.  Otoh, Leo will "reconstitute" Heading cells when exporting @jupyter-notebook trees.

Summary

1. New Leo directives will designate subtrees as containing IPython cells. Such cells will look and work like individual IPython cells.  They will show In and Out numbers, and will show output directly, without the need for a viewrendered pane.

2. Leo's existing @language directives will indicate the type of cell. Such directives could affect which Jupyter kernel evaluates Code nodes.

3. Wherever possible, Jupyter commands and cells will be enhanced by Leo's outline structure. For example, the rendering of an @markdown tree may make Jupyter Heading cells unnecessary.

4. Converting an outline to one or more .ipynb files might allow easy cross-file references in the resulting Jupyter notebooks.  A table-of-contents notebook would likely be easy to create.

All comments welcome.

Edward

P. S. To prepare for this post, I have carefully read all the comments and references in the Leo + PlantUML workflow thread. Naturally, I may have misunderstood something. Please correct me if so.

EKR

[Edward K. Ream]

On Tuesday, January 10, 2017 at 2:55:34 AM UTC-5, Edward K. Ream wrote:

Summary

1. New Leo directives will designate subtrees as containing IPython cells.

...

2. Leo's existing @language directives will indicate the type of cell.

Leo's existing infrastructure is already remarkably useful.  For example, we can use nested cells in two ways: either as filing cabinets or for composing longer scripts.  A node without @others or section references acts as a filing cabinet.  Otherwise, the node creates a script when run with execute-script.

And now that we have @cells, execute-script can determine whether the contents of a node should be sent to a Jupyter kernel for execution.

EKR

[Edward K. Ream]

On Tuesday, January 10, 2017 at 9:56:48 AM UTC-5, Edward K. Ream wrote:

Leo's existing infrastructure is already remarkably useful.

As a further example, Leo's uA's easily suffice to hold all data in .ipynb files, which have .json format.

EKR

[Offray Vladimir Luna Cárdenas]

Hi,

On 10/01/17 02:55, Edward K. Ream wrote:

[...]

New directives

@jupyter-notebook: explicitly denotes a notebook.  All descendants are IPython cells. Useful when converting a Leo outline to one or more Jupyter notebooks.

@cells: All descendants are IPython cells.

@cell: A single IPython cell.

Within a cell, @language denotes the type of cell.

@language markdown denotes a Jupyter Markdown cell.  There probably is not great need for Jupyter Heading cells: they may come free with Leo's outline structure.

@language python (or julia, or r, etc) denotes a code cell.

Behavior

There is lots of room for experimentation re behavior and rendering.  In general, we want cells to work just like the corresponding Jupyter cells, but enhanced, when possible, by Leo's outline structure.

For example, the rending of @markdown cells might include the rendering of all descendant @markdown cells, thereby making Jupyter Heading cells unnecessary with Leo itself.  Otoh, Leo will "reconstitute" Heading cells when exporting @jupyter-notebook trees.

Seems like a good approach. I would give priority to @cell directive and interaction with (I)python kernels (maybe via yoton), even if other @-directives for the notebook are not supported at the beginning. Having compatibility to import and export Jupyter notebooks is important, but the core concerns should be interactivity "inside" Leo, by making code nodes behave like code cells. I imagine some kind of default behavior that keeps you in the flow. For example I have some @icell (for interactive cell) and Leo presumes that the contents are python and let them to be executed with a shortcut and to import the output into the cell (defining another @language inside the @icell will change the kernel for the proper language).

Having this  @icell talking with the python/ipython kernel, will bring the core of live coding experience, even if we use the Leo tree for making other experiments, that are not properly exported to the jupyter format. For example, I have some Leo scripts to traverse a Leo tree, ignore "invisible" nodes and export all that to markdown (that was before the markdown support) to create PDFs with pandoc, all that without referring to the Jupyter format, just from Leo -> pandoc -> LaTeX -> PDF, but what I was missing was interactivity on the Leo nodes, that I have with IPython/Jupyter notebooks.

In summary:

The core experience of Jupyter's literate computing is on weaving, in a document, prose with "live coding", by providing interactive cells. Such core experience can be bring to Leo by supporting @icell that, with proper defaults, would made the content of such Leo node behave like an IPython/Python "code cell", that can talk with a python kernel (or others) and import the result back to the node. This open a lot of possibilities to Leo in the scenario of literate computing, to go even beyond of what is possible with Jupyter now (without damaging compatibility with it), using Leo's superior programmable tree format and capabilities: You can have complete books inside a single tree (instead of split in several ipynb files and directories), you can export directly to PDF/LaTeX/HTML, just by traversing the tree with your own  @directives, or you can go back to jupyter notebook format when needed. For this, what is needed is to give priority to @icell and its behavior, to combine the best of both worlds: interactivity and a self referential programmable tree.

Cheers,

Offray

[Offray Vladimir Luna Cárdenas]

Hi,

On 10/01/17 12:14, Offray Vladimir Luna Cárdenas wrote:
> For example I have some @icell (for interactive cell) and Leo presumes
> that the contents are python and let them to be executed with a
> shortcut and to import the output into the cell (defining another
> @language inside the @icell will change the kernel for the proper
> language).
>

I meant: "For example, I would have some @icell..."

Cheers,

Offray

[Mike Hodson]

I must ask, with all this extending to great big new things, all I've wanted from Leo for over a year now is the ability for it to save a file without causing the entire user interface to redraw itself.

how hard is it to decouple the UI from the gears behind it, vs adding all this new cruft?

[Offray Vladimir Luna Cárdenas]

No idea. You could open a new thread on that one with specific details. See you there ;-).

Cheers,

Offray

On 10/01/17 13:23, Mike Hodson wrote:

I must ask, with all this extending to great big new things, all I've wanted from Leo for over a year now is the ability for it to save a file without causing the entire user interface to redraw itself.

how hard is it to decouple the UI from the gears behind it, vs adding all this new cruft?

--
You received this message because you are subscribed to the Google Groups "leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email to leo-editor+...@googlegroups.com.
To post to this group, send email to leo-e...@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

[Terry Brown]

Just for context, am I remembering correctly you wanted this for some sort of auto-save function, so an auto-save could happen without impacting the user's focus etc.?  Or if not that, at least some issue where you didn't want focus disrupted during a (timed?) save?

Is there a GitHub issue for this? https://github.com/leo-editor/leo-editor/issues

Cheers -Terry

---

From: Mike Hodson <mys...@gmail.com>
To: leo-e...@googlegroups.com
Sent: Tuesday, January 10, 2017 12:23 PM
Subject: Re: The design of Leo+Ipython+Jupyter+Lit-computing

I must ask, with all this extending to great big new things, all I've wanted from Leo for over a year now is the ability for it to save a file without causing the entire user interface to redraw itself.

how hard is it to decouple the UI from the gears behind it, vs adding all this new cruft?

[Edward K. Ream]

On Tue, Jan 10, 2017 at 1:23 PM, Mike Hodson <mys...@gmail.com> wrote:

I must ask, with all this extending to great big new things, all I've wanted from Leo for over a year now is the ability for it to save a file without causing the entire user interface to redraw itself.

​The redraw happens because icons must be redrawn. This is done via a call to c.redraw_after_icons_changed() in fc.save.​

However, the todo.py plugin does cause a full redraw on each save, via the todoController.redraw method. Perhaps that method could do the following instead:

    c.redraw_after_icons_changed()
    c.outerUpdate()

​Tracing confirms that no other redrawing happens as the result of a save command.​

​

​

how hard is it to decouple the UI from the gears behind it,

​The two are decoupled already.​ ​

vs adding all this new cruft?

​You can always complain about performance issues without having to denigrate new directions. Bug fixes and new work proceed on a relatively equal footing.

Edward

[Offray Vladimir Luna Cárdenas]

Hi,

Thanks Edward for your rational and calm answer, even when such attempts to diminish the quality of conversation or new directions arise.

Cheers,

Offray

[Edward K. Ream]

On Tuesday, January 10, 2017 at 12:14:44 PM UTC-5, Offray Vladimir Luna Cárdenas wrote:

I would give priority to @cell directive and interaction with (I)python kernels (maybe via yoton), even if other @-directives for the notebook are not supported at the beginning.

I agree.  
 

Having compatibility to import and export Jupyter notebooks is important, but the core concerns should be interactivity "inside" Leo, by making code nodes behave like code cells.

Yes.  A conversation with Rebecca clarified my thinking and allowed this design to flow.  The main idea is that I can't change either Jupyter or IPython, so I have to focus on what Leo can do.  Import/export is an afterthought.  In fact, the proper strategy is to support cells even better than pyzo does.
 

I imagine some kind of default behavior that keeps you in the flow. For example I have some @icell (for interactive cell) and Leo presumes that the contents are python and let them to be executed with a shortcut and to import the output into the cell (defining another @language inside the @icell will change the kernel for the proper language).

Yes.  This behavior is implicit in how Leo handles @language.  At any node, there is a default @language in effect, which the @icell (@cell for short) will use.

Having this  @icell talking with the python/ipython kernel, will bring the core of live coding experience,

I agree.  Part of this "talk" will be receiving results and rendering them, pretty much exactly as is done in a Jupyter notebook.  This means rendering them in the same @icell in such a way that the @icell remains "live".  In other words, the user must be able to re-execute the  @icell.
 

even if we use the Leo tree for making other experiments, that are not properly exported to the jupyter format.

Yes.
 

For example, I have some Leo scripts to traverse a Leo tree, ignore "invisible" nodes and export all that to markdown (that was before the markdown support) to create PDFs with pandoc, all that without referring to the Jupyter format, just from Leo -> pandoc -> LaTeX -> PDF, but what I was missing was interactivity on the Leo nodes, that I have with IPython/Jupyter notebooks.

That is surely the key feature.  Interaction with one or more Jupyter kernels is the heart of this feature.

In summary:

The core experience of Jupyter's literate computing is on weaving, in a document, prose with "live coding", by providing interactive cells. Such core experience can be bring to Leo by supporting @icell that, with proper defaults, would made the content of such Leo node behave like an IPython/Python "code cell", that can talk with a python kernel (or others) and import the result back to the node. This open a lot of possibilities to Leo in the scenario of literate computing, to go even beyond of what is possible with Jupyter now (without damaging compatibility with it), using Leo's superior programmable tree format and capabilities: You can have complete books inside a single tree (instead of split in several ipynb files and directories), you can export directly to PDF/LaTeX/HTML, just by traversing the tree with your own  @directives, or you can go back to jupyter notebook format when needed. For this, what is needed is to give priority to @icell and its behavior, to combine the best of both worlds: interactivity and a self referential programmable tree.

I agree completely.  Making @icell work is a major endeavor. It's worth any amount of work.  Perhaps yoton will help, but this project must succeed whatever the cost.

Edward

[Thomas Passin]

If I could step in here some months later, and move to a higher level of conversation, I think that there are several levels of engagement with Jupyter that we could contemplate.  For example, we could display a notebook, graphics output and all, in a separate pane.  ViewRendered does that pretty well.  But there's not much point, because it's static and we can't change the notebook.  The notebook can already be viewed in a web browser, and looks better or at least it can be larger.

To be useful, I think the minimum engagement level is:

1. Be able to import a Jupyter notebook into a Leo tree, and have it look like a normal Leo tree (possibly with some @jupyter-notebook nodes).
2. Leo cells would not have to have code outputs necessarily, just static images and text.
3. Changes to the Leo tree can be exported back to the Jupyter notebook, which would then work right in a Jupyter application.

Anything less than this, I don't see as worth spending time on.

At a higher level of engagement, we would add  the ability for code output and graphics to be captured and exported to the notebook.  I think that Ed may be thinking about some version of this, if I'm understanding him right.

At a higher level yet, a Jupyter server could run in Leo's namespace and Leo could interact with its API directly.  This would let us have dynamic graphics that the user can interact with, just as Jupyter notebook has already.   For example, I'm thinking about Holoview interactive graphs.  I believe that the graph has to be able to communicate with the server for this to work (though I'm not completely sure).

As a fallout of this work, it would be useful if the machinery for putting a plot in a cell, putting code output in a cell, etc., could be abstracted so that in the future it would be easier to plug in something else besides a Jupyter Notebook with having to reinvent and re-code everything.

I wonder if using QML widgets would make this easier...and if that would be feasible with Leo's current design.

[Terry Brown]

On Sat, 10 Mar 2018 17:41:47 -0800 (PST)
Thomas Passin <tbp1...@gmail.com> wrote:

> If I could step in here some months later, and move to a higher level
> of conversation, I think that there are several levels of engagement
> with Jupyter that we could contemplate.

I have a long term goal of doing something on the spectrum you describe
as part of this project:
https://github.com/leo-editor/leo-editor/issues/778

But don't hold your breath. Although seeing that project and this one
https://github.com/leo-editor/leo-editor/blob/92ed9d33ac54210de5d97a4612bba2f1fb8f67d5/branches.md#qdock-layout
are related, and the above is finally moving again, maybe?

Cheers -Terry

[Thomas Passin]

Do you happen to know if it's feasible to use QML widgets in Leo?  I don't know either Qt or QML, but much of the PyQt code I see in Leo looks very painful, a steep learning curve to climb.  I remember when I wrote my Matplotlib graphics plotting and calculator program, finding out how to do even simple things in Matplotlib gave me fits.  It's mostly because there is so much *stuff* in there, it's hard to find what you really need to do and how to do the details.  It's obviously similar for Qt, and for similar reasons.

Maybe QML would reduce that learning and coding effort.  OTOH, it looks like it's much easier to use QML with PySide rather than PyQt.  I have no idea if QML widgets constructed using PySide would be compatible with Qt widgets instantiated with PyQt.

[Terry Brown]

On Sat, 10 Mar 2018 18:46:12 -0800 (PST)
Thomas Passin <tbp1...@gmail.com> wrote:

> Do you happen to know if it's feasible to use QML widgets in Leo? I
> don't know either Qt or QML, but much of the PyQt code I see in Leo
> looks very painful, a steep learning curve to climb. I remember when
> I wrote my Matplotlib graphics plotting and calculator program,
> finding out how to do even simple things in Matplotlib gave me fits.
> It's mostly because there is so much *stuff* in there, it's hard to
> find what you really need to do and how to do the details. It's
> obviously similar for Qt, and for similar reasons.

I think a long time ago Ville wrote something in QML in Leo but I'm not
sure. It might have been a sort of continuous scrolling column of body
editors that I remember being a bit more reactive / animated than
regular Qt UI elements.

Being fairly used to Qt I don't find it difficult to code in. OTOH
maybe a modern browser based GUI system - well, really "HTML" CSS is a
lot better than Qt's Stylesheet system I think. With Qt 5.8 breaking
the ~= operator it's hard to even emulate CSS classes well.

Cheers -Terry

[Thomas Passin]

I haven't worked on anything much in-browser since before html5 came out.  So I didn't know anything about "~=", for example.  Even then I tried to only work with the simpler constructs (both javascript and css), so I probably wouldn't have used that particular construct anyway.

On Saturday, March 10, 2018 at 11:13:33 PM UTC-5, Terry Brown wrote:

...Being fairly used to Qt I don't find it difficult to code in.  OTOH

maybe a modern browser based GUI system - well, really "HTML" CSS is a
lot better than Qt's Stylesheet system I think.  With Qt 5.8 breaking

the ~= operator it's hard to even emulate CSS classes well....

[Terry Brown]

On Sun, 11 Mar 2018 07:05:54 -0700 (PDT)
Thomas Passin <tbp1...@gmail.com> wrote:

> I haven't worked on anything much in-browser since before html5 came
> out. So I didn't know anything about "~=", for example. Even then I
> tried to only work with the simpler constructs (both javascript and
> css), so I probably wouldn't have used that particular construct
> anyway.

The ~= operator is a Qt Stylesheet thing that let's Qt stylesheets act
like HTML CSS with classes. So having a class is probably the most
fundamental selector in CSS, ignoring ID, but it's a second class
citizen in Qt Stylesheets, implemented by ~= which is "attribute
contains". So

HTML CSS

p.foo { }

Qt Stylesheet

p[style_class ~= 'foo'] { }

where style_class is arbitrary, but what Leo uses.

But I'd take second class citizen over broken, which is what I think it
is in Qt >= 5.8, unless it got fixed and they never cleared the bug
report.

Cheers -Terry

[Thomas Passin]

Looks like it works pretty much as expected, if you know the right classes to use.  I inserted  a QML view into a regular PyQt container, no problem.  The view was a QDeclarativeView imported from PyQt4.QtDeclarative.  I don't know if they have all the widgets we'd want to use as QDeclaratives ...