Leo's jupytext code works!

[Edward K. Ream]

PR #4119 has reached an epic milestone!

- Leo now seamlessly interacts with Jupyter notebooks.

- Leo reads @jupytext trees using the @clean upgrade algorithm.

- Leo updates @jupytext trees when the Jupyter changes the corresponding .ipynb file.

Leo now deletes the paired .py files when writing .ipynb files. As a result, you must reload the Jupyter page after changing a .ipynb file from Leo.

It would be easy to retain paired .py files, but they seem to confuse Jupyter. I welcome testing, comments and suggestions! It's still early days for this project.

Edward

P.S. The next step will be to provide, as an option, the ability to read and write @jupytext trees as if they were @file trees. @file has significant advantages over @clean, but of course @file writes Leo sentinels. We shall see whether those sentinels are compatible with Jupyter and bearable for Jupyter users.

EKR

[Edward K. Ream]

On Thursday, October 24, 2024 at 8:14:20 AM UTC-5 Edward K. Ream wrote:

> P.S. The next step will be to provide, as an option, the ability to read and write @jupytext trees as if they were @file trees.

On second thought, adding Leo sentinels to .ipynb files seems like a bad idea.

People using Jupyter won't (and shouldn't!) add sentinels. Leo's @file read code will probably struggle to read .ipynb files containing a mixture of nodes with and without sentinels.

In short, adding sentinels to .ipynb files will likely be buggy and confusing. I am going to abandon this idea for now.

Edward

[Edward K. Ream]

On Thursday, October 24, 2024 at 8:14:20 AM UTC-5 Edward K. Ream wrote:

> PR #4119 has reached an epic milestone!

The only remaining items on my list are to add two new settings, something like the following, with the given defaults:

  @bool jupytext-delete-paired-python-file = True

  @string jupytext-read-format = 'py:percent'

  @string jupytext-write-format = 'py:percent'

Expect these later today.

This PR should have no effect whatever on people who don't use @jupytext nodes, so unless I hear objections I plan to merge this PR tomorrow!

Edward

[Thomas Passin]

I wonder if all the work with the percent-encoded files couldn't be done with StringIO in-memory streams.

[Edward K. Ream]

On Thu, Oct 24, 2024 at 11:47 AM Thomas Passin <tbp1...@gmail.com> wrote:

I wonder if all the work with the percent-encoded files couldn't be done with StringIO in-memory streams.

What percent encoded files? And why should I care?

Edward

[Thomas Passin]

" Leo now deletes the paired .py files when writing .ipynb files. "

Those are the files I meant.  I thought they were called "percent-encoded":

"A Python notebook encoded in the py:percent format has a .py extension" - from https://jupytext.readthedocs.io/en/latest/

but whatever they are called, they are what I meant.

[Edward K. Ream]

On Thu, Oct 24, 2024 at 12:53 PM Thomas Passin <tbp1...@gmail.com> wrote:

" Leo now deletes the paired .py files when writing .ipynb files. "

I'm learning about jupytext and jupyter options. Stay tuned.

Edward

[Thomas Passin]

A very small addition to VR3's code will allow the notebook to be rendered, and even executed.

However, I have found at least a couple of errors in JupyterText's conversion of a Jupyter notebook file to its .py format, so that's something to keep in mind.  OTOH, the .ipynb file is probably fairly old and maybe it's got an error or obsolete syntax.  The JupyterText conversion had a few places where a markdown cell that should have been a one-line commentary got included into the code of the next cell.

[Edward K. Ream]

On Thu, Oct 24, 2024 at 6:14 PM Thomas Passin <tbp1...@gmail.com> wrote:

A very small addition to VR3's code will allow the notebook to be rendered, and even executed.

Wow!

However, I have found at least a couple of errors in JupyterText's conversion of a Jupyter notebook file to its .py format, so that's something to keep in mind.  OTOH, the .ipynb file is probably fairly old and maybe it's got an error or obsolete syntax.  The JupyterText conversion had a few places where a markdown cell that should have been a one-line commentary got included into the code of the next cell.

I'm not concerned about this at present.

Thanks for your testing!

Edward

[HaveF HaveF]

A very small addition to VR3's code will allow the notebook to be rendered, and even executed.

Hi, Thomas, I believe the rendering is great --- although I can't imagine how this thing works with the @jupytext node.

I think execution may be a bit troublesome, because my usage scenario is that ipynb will be used in different virtual environments, and it will be executed in Leo without corresponding dependencies. If you really need to execute it, you may write some simple scripts, switch to the corresponding environment first, and execute it. But I may still be used to executing it in jupyter.

 

[Thomas Passin]

Well, yes, execution ability will be limited, that's right. VR3 won't know about magics, and all the needed libraries will have to be imported in the outline. Also graphics that Jupyter would embed won't work.  So execution capabilities would be limited, but there will be some outlines that could be executed.

[HaveF HaveF]

Well, yes, execution ability will be limited, that's right. VR3 won't know about magics, and all the needed libraries will have to be imported in the outline. Also graphics that Jupyter would embed won't work.  So execution capabilities would be limited, but there will be some outlines that could be executed.

I already saw your screenshot in the PR. I like it! You and Edward are awesome, this feature will save me a lot of time in future. Once @jupytext is stable I will definitely show off our usage to the jupytext guys :-D Hahahaha

[Thomas Passin]

Don't get too set on the way @jupytext works right now.  It's not actually very Leonistic.  We can do better.  I will talk about why I say that and how to do better in an upcoming post.

[Edward K. Ream]

On Saturday, October 26, 2024 at 6:11:40 AM UTC-5 Thomas wrote:

Don't get too set on the way @jupytext works right now.  It's not actually very Leonistic.

The correct adjective is "Leonine" :-) And I couldn't agree less. Thomas, you will have difficulty convincing me that @jupytext can be improved significantly. The present @jupytext works with astonishingly minimal code. I won't add significant complexity to suit you or anyone else.

Edward

[Thomas Passin]

On Thursday, October 24, 2024 at 7:14:08 PM UTC-4 Thomas Passin wrote:

A very small addition to VR3's code will allow the notebook to be rendered, and even executed.

Here's an example of VR3 rendering a converted jupyter notebook (I did by hand what that "small addition to VR3's code" would do).