About PR #3215: unbreakable unls!

[Edward K. Ream]

PR #3215 changes many files in complex ways. See detailed comments in the first comment of the PR.

This PR started as a fix for a minor bug, but work expanded in unexpected directions.

New, unbreakable, unls

The status line now reports unls of the form: unl:gnx:<gnx>.

After copying this string to any body text:

- Leo's syntax coloring code will show this as any other url.

- Control-clicking this url will take you to the first position of the outline having the given <gnx>.

These unls won't break unless you delete the original node (including all its clones)!

Summary

This PR is a milestone in Leo's history. How did we ever live without unbreakable unl links?

Internally, Leo now uses only these new unls. The old unls are gone. This PR should simplify leoJS.

I'll merge this PR into devel in a day or three. I won't wait for a code review. The code involved is too tricky to understand in an hour or five.

I'll take full responsibility for any problems that may arise. We aren't going back.

All comments are welcome. There is plenty of time for tweaks.

Edward

[Thomas Passin]

I would rather have created a new gnx:// type and left existing unls alone.  Will existing UNL syntax and methods still  work?

[jkn]

Yes, I was a bit surprised this wasn't your approach as well. Looking forward to the results regardless...

[Robert-Felix]

Thanks for this simplification! supporting Leo's unl links is on my todo list for leojs (and leointeg!)

Félix

--
You received this message because you are subscribed to the Google Groups "leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email to leo-editor+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/leo-editor/bfd2fee7-7792-45b8-8964-ff18881d4ac9n%40googlegroups.com.

[Edward K. Ream]

On Sun, Jun 25, 2023 at 11:49 AM Thomas Passin <tbp1...@gmail.com> wrote:

> I would rather have created a new gnx:// type and left existing unls alone.

I understand your desire for compatibility, but the old url/gnx code was unbearably complicated.

I have spent the last week wrestling with the issues. I don't think any better way forward is possible.

> Will existing UNL syntax and methods still  work?

Syntax: The old syntax  <file-name>#<list of headlines, separated by "-->"> will not work.

The PR contains an item: Write a script to convert from old gnx form to new.  I'll complete and test that script before merging the PR.

Methods: the answer is too complicated to discuss in detail:

- Some methods remain unchanged. Others have changed significantly.

- g.findUNL is now in the attic. I see no reason to retain it.

- Few scripts are likely to use the existing url/unl methods. It is impossible to provide complete compatibility.

Summary

A script will convert from old (breakable) unls to new unls.

We all should test the PR for several weeks once it becomes part of devel.

Edward

[Edward K. Ream]

On Sun, Jun 25, 2023 at 1:44 PM Robert-Felix <felix...@gmail.com> wrote:

Thanks for this simplification! supporting Leo's unl links is on my todo list for leojs (and leointeg!)

You're welcome. I'm glad you like the new scheme. I think you'll like the simpler code.

Edward

[Thomas Passin]

So now my bookmarks manager will not work, something I use every day and depend on.  I suppose I will have to pull out code from the attic and put it into my own module.  Or recreate the pathlike part of the functionality, anyway.

I'm unclear as to whether the gnx string for a given node - that is, p.gnx - will now be in a different format. If that has been changed, it will break the zettelkasten code too, and destroy all the zettel and geneological data I have collected and organized.

"Few scripts are likely to use the existing url/unl methods. It is impossible to provide complete compatibility."  It can be done by using new names for the new methods, etc., and leaving the old ones in place.  When Microsoft bit the bullet and started to support Unicode strings, they didn't change the old string methods, they added the "Wide" series of strings and string functions.

[Edward K. Ream]

On Sun, Jun 25, 2023 at 4:48 PM Thomas Passin <tbp1...@gmail.com> wrote:

So now my bookmarks manager will not work, something I use every day and depend on.  I suppose I will have to pull out code from the attic and put it into my own module.  Or recreate the pathlike part of the functionality, anyway.

Don't panic. I'll work with you on this.

I've just finished the convert-unls command. Retro-fitting the bookmarks and backlinks plugins is next.

Edward

[Edward K. Ream]

On Sun, Jun 25, 2023 at 4:48 PM Thomas Passin <tbp1...@gmail.com> wrote:

I'm unclear as to whether the gnx string for a given node - that is, p.gnx - will now be in a different format.

p.gnx can never ever change.  But p.get_GNX() now returns f"unl:gnx:{self.gnx}".

Yes, this is confusing, but it shouldn't cause great consternation.  If it does, I'll fix it.

Edward

[Thomas Passin]

On Sunday, June 25, 2023 at 10:36:32 PM UTC-4 Edward K. Ream wrote:

On Sun, Jun 25, 2023 at 4:48 PM Thomas Passin <tbp1...@gmail.com> wrote:

So now my bookmarks manager will not work, something I use every day and depend on.  I suppose I will have to pull out code from the attic and put it into my own module.  Or recreate the pathlike part of the functionality, anyway.

Don't panic. I'll work with you on this.

I have just completed taking the currently-existing unl handling functions p.get_UNL(), g.findUNL(), and g.handleUnl() from the core - the ones scheduled to go to the attic or be changed - and turned them into stand-alone functions in my bookmarks manager. I pass p and g as function arguments instead of using instance method calls.  That turned out to be easy since I used them in only a few places, and in a well-defined way that was easy to adapt.  The revised code is working.  So that's a relief.

Next I need to see if I can do the same for the zettelkasten code.

However, I feel strongly that p.get_GNX() should return the position's actual gnx string, which is what the name says it does..  There should be another method, say p.get_unl_gnx() to return that new unl:gnx: string.  Let's have these method names reflect what they actually do! I assume that p.v.gnx will still be the actual gnx string, is that right?

[Edward K. Ream]

On Sun, Jun 25, 2023 at 10:44 PM Thomas Passin <tbp1...@gmail.com> wrote:

I have just completed taking the currently-existing unl handling functions p.get_UNL(), g.findUNL(), and g.handleUnl() from the core - the ones scheduled to go to the attic or be changed - and turned them into stand-alone functions in my bookmarks manager.

I hope you won't have to do this. I'm going to restore g.findUNL() and hook it back into g.handleUnl().

Restoring the old code and the legacy operation should be complete sometime today. At that point your bookmarks and zettelkasten should work as before in the  ekr-3181-mypy-links branch. Let me know if they don't.

Actually, bookmarks and zettelkasten should work better than before, provided:

- they use the new p.get_UNL() and

- they work with both legacy and new unls, which they should if they use the new g.handleUnl().

Edward

P.S. Restoring g.findUNL() will also improve the convert-unls command.

EKR

[Edward K. Ream]

On Sun, Jun 25, 2023 at 10:44 PM Thomas Passin <tbp1...@gmail.com> wrote:

> However, I feel strongly that p.get_GNX() should return the position's actual gnx string, which is what the name says it does.

p.gnx is p's gnx. That hasn't changed. p.get_GNX() is something completely different.

The name is not as important as how Leo's code (including plugins) use the method.

Edward

[Edward K. Ream]

On Monday, June 26, 2023 at 6:34:57 AM UTC-5 Edward K. Ream wrote:

> I'm going to restore g.findUNL() and hook it back into g.handleUnl().

Done, and fairly cleanly too in g.handleUnl().

Please test the ekr-3181-mypy-links branch and report any problems.

test.leo now contains working clickable unls (in the body pane) in a variety of formats.  Please let me know if there are other formats that should be tested.

Edward

P.S. The old/unchanged g.findUNL ignores file names and finds (legacy) unls only in the present outline. I plan to extend the search to all open outlines, as in g.findGNX.

EKR

[Thomas Passin]

The bookmark manager needs to 1) flatten the paths of nested nodes (to provide labels like "software/python/frameworks")  and 2) make each step of those flattened labels clickable so that a user can navigate to their node.  Gnxs can't provide that capability, whether or not they are called "new style unls".  Pre-change unls, OTOH, are ideal and require almost no tinkering with to achieve those requirements.  If Leo won't support those capabilities, the the bookmarks manager needs to compute the equivalent, and the obsoleted Leo methods already do that.  So it's easier for me to use them than to re-create their capability from scratch.

The zettelkasten machinery runs on gnxs and should be less affected if at all.  But I have a host of nodes that include unl links to other outlines, and none of those links would work any more.  Some supporting commands do things like getting the unl of a node and copying it to the clipboard.  Maybe those would still work, maybe they wouldn't.  I'll have to see.

[Edward K. Ream]

On Mon, Jun 26, 2023 at 9:21 AM Thomas Passin <tbp1...@gmail.com> wrote:

The bookmark manager needs to 1) flatten the paths of nested nodes (to provide labels like "software/python/frameworks")  and 2) make each step of those flattened labels clickable so that a user can navigate to their node.  Gnxs can't provide that capability, whether or not they are called "new style unls".  Pre-change unls, OTOH, are ideal and require almost no tinkering with to achieve those requirements.  If Leo won't support those capabilities, the the bookmarks manager needs to compute the equivalent, and the obsoleted Leo methods already do that.  So it's easier for me to use them than to re-create their capability from scratch.

The zettelkasten machinery runs on gnxs and should be less affected if at all.  But I have a host of nodes that include unl links to other outlines, and none of those links would work any more.  Some supporting commands do things like getting the unl of a node and copying it to the clipboard.  Maybe those would still work, maybe they wouldn't.  I'll have to see.

Thanks for the clarification. All your existing unls should now work. Please test the branch and report any problems.

About the name, p.get_UNL

The more I think about this question, the more tricky it becomes.

The PR previously did use the scheme you proposed. But I want p.get_UNL to change so that legacy code will use the new unls.

In other words, the underlying issue is about compatibility, not names.

New methods

I'll create two new methods for use by plugins such as yours:

p.get_legacy_UNL will return what g.get_UNL used to return.

p.get_short_legacy_UNL will return a headline-oriented unl, omitting the file name component (everything up to and including the '#').

Note: g.findUNL never did anything useful with the file name!

Better searching

The new g.findGNX searches all open outlines for nodes with the given gnx.  g.findUNL can use the same scheme!

Summary

p.get_UNL should return gnx-oriented unls so that existing code will automatically use non-breakable gnxs.

I'll add p.get_legacy_UNL and p.get_short_legacy_UNL.

I'll add expanded searching to g.findUNL, just as in g.findGNX.

Please test your plugins. I'll fix any compatibility problems before merging the PR.

Edward

[Edward K. Ream]

On Monday, June 26, 2023 at 10:02:40 AM UTC-5 Edward K. Ream wrote:

> I'll add p.get_legacy_UNL and p.get_short_legacy_UNL.

> I'll add expanded searching to g.findUNL, just as in g.findGNX.

Both done.

Notes

1. Warning: Plugins that call g.findUNL directly must not assume that the returned position is in the same outline! Do something like this:

p = g.findUNL(unlList, c)

# Do not assume that p is in c.
if p:
    c2 = p.v.context
    c2.redraw(p)

2. g.handleUnl contains code similar to that shown above, so the warning applies only to writers of plugins.

3. g.findUNL no longer does any gui-related operations. It was never a clever idea.

4. No, I won't add a kwarg to g.findUNL that suppress other-outline searches.

EKR

[Thomas Passin]

After checking out the branch for this PR:

1. CTRL-clicking on an old-style unl pointing into the same outline navigates to the right node; for a unl pointing into a different outline it does not navigate to the right place or even the right outline.

2. Both the bookmark manager and the zettelkasten apps partially work but not completely.   I will look at using these new "legacy"  methods next.

[Edward K. Ream]

On Mon, Jun 26, 2023 at 12:50 PM Thomas Passin <tbp1...@gmail.com> wrote:

1. CTRL-clicking on an old-style unl pointing into the same outline navigates to the right node; for a unl pointing into a different outline it does not navigate to the right place or even the right outline.

Thanks for testing.  We'll make everything right.

g.findGNX searches all open windows.

The legacy version of g.handleUnl contained Leo-specific code which I deleted. Afterwards, the code called g.openWithFileName to open a window, then searched in the window.

One path was g.app.homeDir, so maybe part of the code should be reinstated. I'll investigate.

2. Both the bookmark manager and the zettelkasten apps partially work but not completely.   I will look at using these new "legacy"  methods next.

Let me know if I can help.

Edward

[Thomas Passin]

On Monday, June 26, 2023 at 2:26:36 PM UTC-4 Edward K. Ream wrote:

On Mon, Jun 26, 2023 at 12:50 PM Thomas Passin <tbp1...@gmail.com> wrote:

g.findGNX searches all open windows.

The legacy version of g.handleUnl contained Leo-specific code which I deleted. Afterwards, the code called g.openWithFileName to open a window, then searched in the window.

Yes, since the UNL contained the path to the outline, Leo could open a non-open outline and navigate to the right node.  I always thought that was a wonderful, even almost miraculous, capability.  We shouldn't lose it.  Furthermore, having the path visible in the status bar could be really useful  For example, a search might return several hits for a def.  You navigate to one or another of them.  But wait! It might be in a plugin instead of the core.  You can't tell by looking at the tree because too many nodes are open and you can't see what file or class you are in.  But a glance at the (legacy) unl in the status line would tell you.  The new way loses that capability.  To me, that's a real loss.

[Thomas Passin]

Something close to seven plugins use unls, including mod-http and quickMove.  They should all be checked to see if they will still work (I don't know which of them still work apart from unl changes).

[Edward K. Ream]

On Mon, Jun 26, 2023 at 1:50 PM Thomas Passin <tbp1...@gmail.com> wrote:

Yes, since the UNL contained the path to the outline, Leo could open a non-open outline and navigate to the right node.  I always thought that was a wonderful, even almost miraculous, capability.  We shouldn't lose it.

I've just added this (experimental!) item to the PR's checklist.

Furthermore, having the path visible in the status bar could be really useful 

But it hasn't been useful.  Alt-N (goto-next-clone) and cff are easy, existing, ways of navigating.

In contrast, show-clone-ancestors and show-clone-parents are well worth removing.

Edward

[Edward K. Ream]

On Mon, Jun 26, 2023 at 2:01 PM Thomas Passin <tbp1...@gmail.com> wrote:

Something close to seven plugins use unls, including mod-http and quickMove.  They should all be checked to see if they will still work.

Good idea. I've added this item to the PRs list.

Edward

[Edward K. Ream]

On Monday, June 26, 2023 at 1:50:28 PM UTC-5 tbp1...@gmail.com wrote:

> Furthermore, having the path visible in the status bar could be really useful.

Let's not argue about preferences. I've just added this checklist item to the PR:

Add a setting that determines whether the status area will show legacy or new unls.

Edward

[Thomas Passin]

In the zettelkasten app, using the new p.get_legacy_UNL() doesn't work right.  I think it's because I had to change the ">" to "%3E" to get QT to pass it to my code when I click on these links (these paths appear in an anchor element).  So there must be another of the changed methods that doesn't quite play right with this.  I'll try to track it down tomorrow.

[Edward K. Ream]

On Mon, Jun 26, 2023 at 10:44 PM Thomas Passin <tbp1...@gmail.com> wrote:

In the zettelkasten app, using the new p.get_legacy_UNL() doesn't work right.  I think it's because I had to change the ">" to "%3E" to get QT to pass it to my code when I click on these links (these paths appear in an anchor element).  So there must be another of the changed methods that doesn't quite play right with this.  I'll try to track it down tomorrow.

Thanks for this.  It's an important clue.

Edward