ENB: Why deleting cell markup won't work

[Edward K. Ream]

This Engineering Notebook post explains why removing and restoring cell markup can not work reliably. The idea seems tempting. Why put up with all that cruft?

But this scheme has a fatal flaw: there is no dead easy way of restoring markup:

- Cluttering headlines with the required data won't be pleasant.

- Manually inserting such data (when creating new nodes) will be unreliable.

- Using uAs won't work because they won't exist for newly-created nodes.

The intuition is this. Leo converts an .ipynb file to an outline only once. But Leo will write the converted outline back to the .ipynb file many times.

Summary

Cell markup is essential. Without them, Leo can't reliably write outlines back to their .ipynb files.

Edward

[Thomas Passin]

I don't see this at all.  .ipyb files contain a flat sequence of cells.  Having Leo create an indented tree is a Leonine convenience for users.  Users use heading levels to indicate implied nesting to users, but nesting isn't, as far as I know, a feature of the Jupyter model.

Given that we have a user who wants to use nested levels while working in Leo on .ipynb files, it makes a lot of sense to me to not force someone to manually recreate the nesting each time a file is imported, which will happen, e.g. each time the file is changed outside of Leo. That will get old very fast.  IMO, round-trip fidelity means more than that a file opened in Jupyter looks the same after being opened and re-written by Leo.  It also implies that a .ipynb as seen in Leo will re-open and have the same structure after passing through a no-op in Jupyter. If the nested structure of Leo nodes is lost on each round-trip, then we won't have round-trip fidelity.

Leo can reliably write back to the .ipynb file by writing a sequence of cells in order.  The original heading levels in the markdown of each cell need to be preserved for round-trip fidelity and that is easy because we know from the nesting level what those heading levels should be.  It doesn't affect the order of the cells in the output.

This mixing up of semantic with syntactic levels bites all the time with rst3 files. Forget what nesting symbol to use for a level N heading and docutils will barf and quit, and the right symbol to use is dynamic, not fixed by RsT. It's a design flaw but with markdown and jupyter files, getting the headline levels right is easy to handle. The heading level equals the position's level.  Code cells don't get a headline in the output.

If we don't do a good job about the matter of nesting, headlines, readability, and ease of use, there will be no reason for anyone to want to use Leo to work with Jupyter outlines. The VS Code plugin is already very good. It's very readable and has a WYSIWYG cell editor, cell execution, and results, including animations, get inserted live into the outline. It has an outline view.  What it doesn't have is the ability to work directly with the outline, and it doesn't have indentation levels in the outline to help the notebook user/reader know how cells are related and to help the user re-organize the notebook.  Leo can supply those outline features, but if too many of the other desirable qualities get lost, there's no point.

[Edward K. Ream]

On Wed, Oct 30, 2024 at 6:44 AM Edward K. Ream <edre...@gmail.com> wrote:

> Cell markup is essential. Without them, Leo can't reliably write outlines back to their .ipynb files.

True, but there remains the question of creating new cells. I'll discuss this in a new ENB.

Edward

[Edward K. Ream]

On Wed, Oct 30, 2024 at 8:12 AM Thomas Passin <tbp1...@gmail.com> wrote:

>> Cell markup is essential. Without them, Leo can't reliably write outlines back to their .ipynb files.

> I don't see this at all.

I'm not going to discuss this question now. It's too late to make major changes to the PR. The PR gives Leonistas at least 85% of their dreams.

But don't despair. A new ENB will discuss possibilities for abbreviations, @button nodes and plugins. Spoiler: The PR will:

- add a new event: after-insert-node.

- will trigger the existing after-reading-external-file and before-writing-external-file events.

Supporting these events will let you play with whatever schemes you like.

Edward

[Edward K. Ream]

On Wed, Oct 30, 2024 at 9:38 AM Edward K. Ream wrote:

The PR will:

- add a new event: after-insert-node.

Heh. Leo already triggers the undocumented `create-node` event. The PR will document it.

- will trigger the existing after-reading-external-file and before-writing-external-file events.

Done at rev 9e891e2

Edward

[Edward K. Ream]

On Wed, Oct 30, 2024 at 10:59 AM Edward K. Ream <edre...@gmail.com> wrote:

> Heh. Leo already triggers the undocumented `create-node` event. The PR will document it.

Actually, 'create-node' does appear in LeoDocs.leo. However, it doesn't appear on the web.

I've made a note to myself to make sure the next build of Leo's docs includes the 'create-node' event.

Edward