ENB: Félix, leojs should still use the bridge

[Edward K. Ream]

This  Engineering Notebook post is primarily for Félix. I'm making this post public because I like to avoid important conversations.

As I said yesterday, we want to avoid forks in Leo's code. For leojs, that means basing transliterated ts code on Leo's official code, not some leojs-specific reworking.

This morning I made a list of Leo's crown jewels, Leo's essential features. They all should eventually become part of leoInteg. The list is long, and the code is complex:

- Leo's find commands, in leo.core.leoFind.

- @clean and @find, in leo.core.leoAtFile.

- The @clean update algorithm, in leo.core.leoShadow.

- Leo's importers, in leo.core.leoImport, leo.plugins.importers, and leo.plugins.writers.

- Leo's Position and VNode classes, in leo.core.leoNodes.

- Leo's generators, in leo.core.leoCommands.

Imo, that's too much to transliterate!  Instead, I recommend:

    Use the bridge to create trees.

In other words, transliterate only the code that must be fast:

- Leo's interactive find commands, in leo.core.leoFind.

- Leo's Position and VNode classes, in leo.core.leoNodes.

- Leo's generators, in leo.core.leoCommands.

Summary

leojs should use ts only for those commands that should execute synchronously for performance reasons. There aren't many because vs-code will do editing.

Commands that create trees of nodes should use Leo's bridge.

Transliterating too much would fork Leo's code and waste our time.

Félix, your comments please, Amigo.

Edward

[Edward K. Ream]

On Saturday, January 16, 2021 at 5:17:08 AM UTC-6 Edward K. Ream wrote:

This  Engineering Notebook post is primarily for Félix. I'm making this post public because I like to avoid important conversations.

Hehe, I meant to say, "I like to avoid private conversations.

Edward

[Edward K. Ream]

On Saturday, January 16, 2021 at 5:18:36 AM UTC-6 Edward K. Ream wrote:

>> I'm making this post public because I like to avoid important conversations.

> Hehe, I meant to say, "I like to avoid private conversations.

This was a last-minute editing blunder.

I now have a new game: getting the free version of Grammarly to report no problems. It's kinda like a crossword puzzle, which for the first time in my life I have started to do. I'm starting with the New York Times mini puzzles :-)

Anyway, the free version of Grammarly shows where the paid version of Grammarly would make suggestions. It's fun and useful to clear all warnings without seeing the hints.

In this last post, Grammarly upped the "engagement" rating from "a bit bland" to "very engaging". I agree. Discovering the improvements myself was a fun and useful learning exercise.

In short, I recommend the free version of Grammarly :-) You'll learn more about writing that way.

Edward

[Félix]

Hey Y'all! 

Heheh ... "avoid important conversations" what a funny blooper ! :D

Yes, after pondering on your reflections, I do believe that the concept you propose, to have a minimal subset of the VNodes and Positions functionality while keeping a bridge to the original python Leo codebase would be optimal for things such as:

 - Complete compatibility with any 'real' Leo file. Including all their scripts and non-ui-plugins. (legacy wise)

 - Performance for tree navigation, and even maybe simple tree node operations (move, copy paste, etc.)

- And obviously, time and effort considerations of not translating all of Leo's source!

It led me to consider that the plan we had setup in our last zoom meeting was still a valid one:

 1- finish enough of python-to-ts translation to get Leo file to load, and browsing its outline. (in leojs project)

 2- finish redoing the body pane to workaround the API changes of vscode since last autumn.. (in morejs project)

 3- Apply the body pane fix to leoInteg to have at least a workable version to release before spring.

(4-The ideas you have brought forth in this thread come into play after this point)

Maybe it's just "project-naming" and it's totally secondary, but It would incite me to do it (naming wise) towards 'leoInteg' more than 'leojs' ... As it is still an integration, with a super-suped-up front-end that is optimized for performance with a subset of Leo's classes implemented in ts. But again, this is just project-nomenclature and is not really relevant to the great concepts you proposed in here.

As I envisioned the projects, leojs was simply an implementation of https://github.com/leo-editor/leo-editor/issues/1025 strongly and statically typed in typescript. scriptable in javascript and not usable with legacy Leo files that would rely on running python scripts and leo's official plugins, and the like...  (To be continued or abandoned - who knows! )

So maybe leojs will be just like morejs: a side-project used to isolate a particular aspect of functionality, and implement it, to later be incorporated in a bigger project. 

(btw you should merge https://github.com/leo-editor/leo-editor/issues/1025 with the crown jewels you just defined up there in a more comprehensive, detailed and more permanent list of feature) 

Thanks to all Leonistas who contribute code and ideas, and please feel free to comment on what I say here or ask questions :)

Félix

[Edward K. Ream]

On Mon, Jan 18, 2021 at 8:01 PM Félix <felix...@gmail.com> wrote:

Hey Y'all! 

Hi Félix. Thanks for your reply. It has changed my thinking, as follows.

Heheh ... "avoid important conversations" what a funny blooper! :D

Glad to add a bit of humor :-)

Yes, after pondering on your reflections, I do believe that the concept you propose, to have a minimal subset of the VNodes and Positions functionality while keeping a bridge to the original python Leo codebase would be optimal for things such as:

 - Complete compatibility with any 'real' Leo file. Including all their scripts and non-ui-plugins.

 - Performance for tree navigation, and even maybe simple tree node operations

- And obviously, time and effort considerations of not translating all of Leo's source!

Heh. Let me raise some doubts, and suggest one overarching goal and one fundamental strategy...

Goal

I suggest only one long-term goal for leoInteg:

   Support Leo's crown jewels, as compatibly as possible.

I am going to stop talking about forks. Imo, it's better to focus on what we do want, namely compatibility between leoInteg and Leo. 

As you suggested, I have updated #1025. Note these changes:

• (Deprecated) @test & @suite.  (But see below!)
  
• Much of Leo's source code, and most plugins.
• Leo's API that controls gui: virtual wrappers and widgets.

Strategy

I regret putting "should" in the title of this thread. Whether leoInteg uses the bridge is an open question!

I suggest only one fundamental strategy:

    Simplify short, medium, and long-term work

Imo, you, Félix, should be the primary judge of what's best, because you understand vs-code and LeoInteg far better than I. Some comments:

1. There isn't all that much code in the crown jewels. Conceivably, you could translate all the importer/exporter code. Indeed, leo.plugins.linescanner does almost all the heavy lifting.

2. unitTest.leo contains reasonable tests of all importer plugins. Aha: one way to keep Leo and LeoInteg in sync will be to run unitTest.leo from LeoInteg.

3. None of Leo's existing gui plugins will work in the vs-code environment. But as you say, Leo has several important plugins that are gui independent.

About using Leo's bridge

Using Leo's bridge seems like the best course provided that doing so doesn't cause it's own problems.  I want to emphasize the conditional nature of the last sentence.

Félix, please do whatever seems best/simplest to get leoInteg 1.0 out the door. Later, we can worry about long-term compatibility between Leo and leoInteg!

If (eventually) leoInteg doesn't use Leo's bridge, then we'll need a way of syncing Leo and leoInteg. Running tests in unitTest.leo will help, but existing tests don't provide enough guarantees. leoInteg will likely have to track Leo's PR's.

It led me to consider that the plan we had setup in our last zoom meeting was still a valid one:

 1- finish enough of python-to-ts translation to get Leo file to load, and browsing its outline. (in leojs project)

 2- finish redoing the body pane to workaround the API changes of vscode since last autumn.. (in morejs project)

 3- Apply the body pane fix to leoInteg to have at least a workable version to release before spring.

(4-The ideas you have brought forth in this thread come into play after this point)

I agree. Let's simplify short-term work as much as possible, always keeping in mind upcoming work in the medium and long terms. Yes, this sounds simplistic, but I think dead simple strategies are the best strategies!

Maybe it's just "project-naming" and it's totally secondary, but It would incite me to do it (naming wise) towards 'leoInteg' more than 'leojs'

I agree. leoInteg is the product. leojs is the strategy, or as you say, a side project.

Summary

Goal: Support leoInteg Leo's crown jewels, as compatibly as possible.

Strategy: Simplify short, medium, and long-term work.

- It is an open question whether leoInteg should use Leo's bridge.

- Félix, please do whatever seems best/simplest to get leoInteg 1.0 out the door.

- Let's keep long-term compatibility in the back of our minds at all times.

Edward

[Edward K. Ream]

On Tuesday, January 19, 2021 at 5:18:45 AM UTC-6 Edward K. Ream wrote:

> Let's simplify short-term work as much as possible, always keeping in mind upcoming work in the medium and long terms. Yes, this sounds simplistic, but I think dead simple strategies are the best strategies!

...

I spend so much time on planning because planning saves so much time! Otoh, at some point we just have to write code and see what happens.

> Félix, please do whatever seems best/simplest to get leoInteg 1.0 out the door.

Why do I keep repeating this? Because leoInteg 1.0 will allow you to change leoInteg much more easily! This is one of Leo's killer features. I've just add these words to #1025:

    Leo's defining features embolden programmers.

    Programmers can refactor and re-imagine their code

    much more easily than with any other editor.

> It is an open question whether leoInteg should use Leo's bridge.

leoFind.py illustrates the choices.  Félix, please base your work on the code in the ekr-find branch. Many choices remain, regardless of whether you transliterate leoFind.py into ts. Here are some notes about leoFind.py:

When I return from "vacation" I'll update the code so that all the do_* methods return descriptions of what vs-code should do. However, I just realized that this is not the whole story:

- leoFind.py calls w.getAllText(), w.getSelection() and w.getInsertPoint(), where w is either c.frame.body.wrapper or c.tree.editWidget().

Somehow, leoInteg will have to discover a) whether the headline is being edited, b) the text, selection range and insert point of w. leoFind.py handles the case where focus is in the tree but no headline is being edited.

- leoFind calls w.setAllText() and w.setSelection() to set the text, selection range and the insert point of w. leoInteg can ignore those calls, using the returned descriptions instead.

- Important: leoInteg will have to implement all the interactive_* methods in leoFind.py with vs-code. In other words, use the interactive_* methods only as a hint about vs-code should do. Don't bother transliterating them ts. For leoInteg, the do_* methods are the proper entry points.

Summary

leoInteg 1.0 will make it much easier to change leoInteg. At first, medium and long-term considerations aren't as important as one might think. Whatever works is good enough!

Implementing leoFind.py in leoInteg will require substantial inventions, regardless of whether leoInteg transliterates leoFind.py into ts or instead uses Leo's bridge.

Edward

[tbp1...@gmail.com]

Remember that you can expect to re-implement at least twice before you get something really good...

On Tuesday, January 19, 2021 at 7:50:21 AM UTC-5 Edward K. Ream wrote:

<snip>

[Edward K. Ream]

On Tue, Jan 19, 2021 at 6:50 AM Edward K. Ream <edre...@gmail.com> wrote:

> Félix, please base your work on the code in the ekr-find branch. Many choices remain, regardless of whether you transliterate leoFind.py into ts.

Some more notes:

1. I am going to honor a promise to a friend and refrain from writing code for Leo this next week. After my vacation, I'll likely add a leo_integ switch, indicating code that vs-code's api should probably handle:

leo_integ = False

...

if not leo_integ:  # pragma: no cover

    @others

2. The new leoFind.py is on a firm foundation just as it is. After the vacation I'll merge ekr-find into devel. Thereafter, any bug fixes or tweaks will result in small PR's.

Félix, you and I can then add any kind of experimental code we like, marked with the leo_integ switch. Now is the time to make changes. The new code is already as clean as I can make it, and I understand every line of code.

Edward