Capabilities of the 'restart-leo' command ?

[Viktor Ransmayr]

Hello Edward & Community,

Is this command designed / prepared to be used to update the current instance of Leo after a 'git pull' of the associated Leo repository & branch ?

Context: During my tests for the new / revised layout system I tried this use-case with different repositories & branches in a Debian & a Fedora VM - and - it did not work for me ...

I always had to explicitly close & re-open the updated Leo instance in that repository & branch !

With kind regards,

Viktor

[Thomas Passin]

I don't think existing commands and menus get updated by closing and re-opening an outline. Commands and Menus created in a settings tree in the outline might update, but I'm not sure about that.

[Viktor Ransmayr]

Hello Thomas,

I guess my question & the description of the context was not clear enough. - Let me try again:

If  Leo's code has been updated, e.g. via 'git pull', can I use this command to apply this update without exiting the running Leo instance itself ?

The help info for this command is very short :

Restart Leo, reloading all presently open outlines.

With kind regards,

Viktor

[Thomas Passin]

I don't think you can use the changed code without restarting Leo.  There is a lot of code initialized and running in Leo that's needed just to open an outline. That wouldn't pick up the changes when an outline gets opened. Leo itself has to be restarted, as you've noticed.

[Edward K. Ream]

On Fri, Oct 11, 2024 at 4:17 PM Thomas Passin <tbp1...@gmail.com> wrote:

I don't think you can use the changed code without restarting Leo. 

And that's what restart-leo does.

Edward

[Edward K. Ream]

On Thu, Oct 10, 2024 at 3:40 PM Thomas Passin <tbp1...@gmail.com> wrote:

I don't think existing commands and menus get updated by closing and re-opening an outline. Commands and Menus created in a settings tree in the outline might update, but I'm not sure about that.

restart-leo reports (in the console) what it does. For example:

os.chdir(C:\Repos\leo-editor)
subprocess.run([
  C:\Python\Python3.12\python.exe,
  C:\Repos\leo-editor\launchLeo.py,
  C:\Users\Dev\ekr.leo,
  --no-splash
])

In other words, restart-leo completely restarts Leo.

Edward

[Viktor Ransmayr]

Hello Edward,

I knew that this command restarts Leo - but - I was not sure which use-cases it supports ...

I'm taking your answer answer as: No limitations / restrictions are expected / known.

As I stated in my initial message: This is not true, at least not for the setup in my Debian Linux - & Fedora Linux VMs.

Below you find the logs for one of several attempts of restarting Leo after a 'git pull' of the 'devel' branch of Leo's repository:

• Before the 'git pull' operation - Log-001.
• During & after executing 'restart-leo' command at the console - Log-002.
• During & after executing 'restart-leo' command at the log panel - Log-003.
• After having closed & restarted Leo ~ manually ~ via the GUI - Log-004.
  

Do you consider this a bug ?

With kind regards,

Viktor

### Log-001 from Fedora - Before the 'git pull' operation ...

    Leo Log Window
    Leo 6.8.2-devel, devel branch, build bb7c9c1d95 <- !
    2024-10-08 15:08:38 -0500
    Python 3.12.6, PyQt version 6.7.1
    linux
    setting leoID from os.getenv('USER'): 'user'
          home: /home/user
    leo-editor: /home/user/PyVE/GitHub/Leo/leo-editor
          load: /home/user/PyVE/GitHub/Leo/leo-editor/leo/core
        config: /home/user/PyVE/GitHub/Leo/leo-editor/leo/config
    reading settings in /home/user/PyVE/GitHub/Leo/leo-editor/leo/config/leoSettings.leo
    reading settings in /home/user/.leo/myLeoSettings.leo
    reading settings in /home/user/PyVE/GitHub/Leo/leo-editor/leo/themes/tbp_dark_solarized.leo
    VR3 -- no asciidoc processor
    VR3 -- no asciidoc3 processor
    Can't find /home/user/.leo/vr3/vr3_config.ini so VR3 cannot execute non-Python code
    reading settings in /home/user/Documents/SL2024.leo
    read outline in 0.27 seconds

### Log-002 from Fedora - During & after executing 'restart-leo' command at the console ...

    Restarting Leo...
    os.chdir(/home/user/PyVE/GitHub/Leo/leo-editor)
    subprocess.run([
      /home/user/PyVE/GitHub/Leo/bin/python,
      /home/user/PyVE/GitHub/Leo/leo-editor/leo/core/runLeo.py
    ])

    setting leoID from os.getenv('USER'): 'user'
    Leo 6.8.0-b1, master branch <- !
    Python 3.12.6, PyQt version 6.7.1
    linux

### Log-003 from Fedora - During & after executing 'restart-leo' command at the log panel ...

    Leo Log Window
    Leo 6.8.0-b1, master branch <- !
    Python 3.12.6, PyQt version 6.7.1
    linux
    setting leoID from os.getenv('USER'): 'user'
          home: /home/user
    leo-editor: /home/user/PyVE/GitHub/Leo/lib64/python3.12/site-packages
          load: /home/user/PyVE/GitHub/Leo/lib64/python3.12/site-packages/leo/core
        config: /home/user/PyVE/GitHub/Leo/lib64/python3.12/site-packages/leo/config
    reading settings in /home/user/PyVE/GitHub/Leo/lib64/python3.12/site-packages/leo/config/leoSettings.leo
    reading settings in /home/user/.leo/myLeoSettings.leo
    reading settings in /home/user/PyVE/GitHub/Leo/lib64/python3.12/site-packages/leo/themes/tbp_dark_solarized.leo
    VR3 -- no asciidoc processor
    VR3 -- no asciidoc3 processor
    Can't find /home/user/.leo/vr3/vr3_config.ini so VR3 cannot execute non-Python code
    reading settings in /home/user/Documents/SL2024.leo
    read outline in 0.26 seconds

### Log-004 from Fedora - After having closed & restarted Leo ~ manually ~ via the GUI ...

    Leo Log Window
    Leo 6.8.2-devel, devel branch, build cb37d41c50 <- !
    2024-10-10 10:03:11 -0500
    Python 3.12.6, PyQt version 6.7.1
    linux
    setting leoID from os.getenv('USER'): 'user'
          home: /home/user
    leo-editor: /home/user/PyVE/GitHub/Leo/leo-editor
          load: /home/user/PyVE/GitHub/Leo/leo-editor/leo/core
        config: /home/user/PyVE/GitHub/Leo/leo-editor/leo/config
    reading settings in /home/user/PyVE/GitHub/Leo/leo-editor/leo/config/leoSettings.leo
    reading settings in /home/user/.leo/myLeoSettings.leo
    reading settings in /home/user/PyVE/GitHub/Leo/leo-editor/leo/themes/tbp_dark_solarized.leo
    VR3 -- no asciidoc processor
    VR3 -- no asciidoc3 processor
    Can't find /home/user/.leo/vr3/vr3_config.ini so VR3 cannot execute non-Python code
    reading settings in /home/user/Documents/SL2024.leo
    read outline in 0.26 seconds

[Thomas Passin]

I understand log 003 well enough.  The standard-issue Leo install is being executed instead of the one in your clone repo. How do you start Leo normally, e.g., for log 001?

I think the restart-leo command has a weakness - it's going to start Leo using a shell inheriting its owner's environment. If that original environment doesn't include a PYTHONPATH variable or hasn't been cd'ed to the leo-editor directory, it will find the pip-installed version instead of the git version.

I just tried the restart-leo command in one of my Linux VMs.  It's also running in a venv.  The restart worked as intended.  however, I start Leo with the git repo clone version of Leo by using a shell script that firsts sets PYTHONPATH to point to the leo-editor directory in my git repo.  Then it sources the venv.  So when the restart-leo command runs, the new shell inherits the right environment.

If this is right, the command will be unreliable for anyone who doesn't start the original Leo session the "right" way. The user might not even notice they are running the wrong version of Leo.

Here is my script:

#! /bin/bash
if [ "$PYTHONPATH" == "" ]; then
    export PYTHONPATH=~/git/leo-editor
    echo "Using PYTHONPATH:" $PYTHONPATH
else
    echo "PYTHONPATH already set to" $PYTHONPATH
fi

source ~/venv/leo/bin/activate && python3 -m leo.core.runLeo $*

The simplest fix would probably be for the restart-leo command to find the leo-editor directory and have the subprocess command cd to it before calling runLeo.  This should (but untested!) work for Leo running in any environment, whether it be a venv, a git repo, or wherever.  Of course, if there is no leo-editor directory, it should omit the cd command.

[Viktor Ransmayr]

Hello Thomas,

tbp1...@gmail.com schrieb am Samstag, 12. Oktober 2024 um 14:42:04 UTC+2:

I understand log 003 well enough.  The standard-issue Leo install is being executed instead of the one in your clone repo. How do you start Leo normally, e.g., for log 001?

[user@fedora-leo-study-vm ~]$ cd PyVE/GitHub/Leo/
[user@fedora-leo-study-vm Leo]$ source bin/activate
(Leo) [user@fedora-leo-study-vm Leo]$ cd leo-editor/
(Leo) [user@fedora-leo-study-vm leo-editor]$ python -m leo.core.runLeo &

 

I think the restart-leo command has a weakness - it's going to start Leo using a shell inheriting its owner's environment. If that original environment doesn't include a PYTHONPATH variable or hasn't been cd'ed to the leo-editor directory, it will find the pip-installed version instead of the git version.

My environment does not include a PYTHONPATH variable - but - as you can see above I am starting Leo from the leo-editor directory ...

 

I just tried the restart-leo command in one of my Linux VMs.  It's also running in a venv.  The restart worked as intended.  however, I start Leo with the git repo clone version of Leo by using a shell script that firsts sets PYTHONPATH to point to the leo-editor directory in my git repo.  Then it sources the venv.  So when the restart-leo command runs, the new shell inherits the right environment.

If this is right, the command will be unreliable for anyone who doesn't start the original Leo session the "right" way. The user might not even notice they are running the wrong version of Leo.

...

Let's see, what Edward thinks about this issue / problem ?

Thanks a lot for your detailed explanation !

 

With kind regards,

Viktor

[Edward K. Ream]

On Sat, Oct 12, 2024 at 7:04 AM Viktor Ransmayr <viktor....@gmail.com> wrote:

> I knew that this command restarts Leo - but - I was not sure which use-cases it supports ...

As you imply, restart-leo should "just work." However, there may be platform-dependent limitations.

I will consider PRs that remove such limitations, but everything works for me on Windows.

Edward

[Thomas Passin]

Yes, and I've been finding that Linux is a different beast.  Things can be different on different distros - and with the various user configurations of them.

[Thomas Passin]

I have just confirmed that the restart-leo command does work on my OpenSUSE VM.  Leo is not being run in a venv on this one. I use the same launch command as I posted above except it doesn't open a venv.  This is with a recent version of devel (yesterday's).

On Saturday, October 12, 2024 at 12:26:24 PM UTC-4 Edward K. Ream wrote:

[Thomas Passin]

... But when I start it Viktor's way the restart-leo command opens the pip-installed version.  IOW

cd ~/git/leo-editor

python3 -m leo.core.runLeo

leads to restarting the pip-installed version instead of the git.leo-editor version. I have found out the difference, though not yet why the difference happens.  When you start Leo in the git/leo-editor directory, you can display the Python path be running this in Leo's console tab:

import sys

print('\n'.join(sys.path))

This will show that the first location on the search path is git/leo-editor, as it should be.

When you restart it and the wrong version opens, the first location on my system is git/leo-editor/leo/core. Of course, Python won't find leo.core.runLeo there so it looks in site-packages instead and finds the pip-installed version.

So, progress.

[Thomas Passin]

Leo restart on Windows suffers from the same problem. If I start it with my batch file that sets PYTHONPATH, it restarts the git/leo-editor version.  If instead I cd to git/leo-editor and start with py -m leo.core.runLeo, then issue restart-leo, the pip-installed version gets started.  So it's the same problem on Windows.  It's just easy to overlook. OTOH, if I start Leo with the launchLeo.py script, the restart-leo command works as intended (only tested on Linux so far).  Very mysterious.

[Viktor Ransmayr]

Hello Edward & Thomas,

tbp1...@gmail.com schrieb am Samstag, 12. Oktober 2024 um 20:28:22 UTC+2:

Leo restart on Windows suffers from the same problem. If I start it with my batch file that sets PYTHONPATH, it restarts the git/leo-editor version.  If instead I cd to git/leo-editor and start with py -m leo.core.runLeo, then issue restart-leo, the pip-installed version gets started.  So it's the same problem on Windows.  It's just easy to overlook. OTOH, if I start Leo with the launchLeo.py script, the restart-leo command works as intended (only tested on Linux so far).  Very mysterious.

I have created a GitHub issue, so this restriction is not forgotten.

With kind regards,

Viktor

[Edward K. Ream]

I have created a GitHub issue, so this restriction is not forgotten.

Thanks.  I have assigned it to Leo 6.8.3, but I'll merge it into Leo 6.8.2 if Thomas solves it.

Edward

[Edward K. Ream]

On Friday, October 18, 2024 at 4:23:21 AM UTC-5 viktor....@gmail.com wrote:

Leo restart on Windows suffers from the same problem. If I start it with my batch file that sets PYTHONPATH, it restarts the git/leo-editor version.  If instead I cd to git/leo-editor and start with py -m leo.core.runLeo, then issue restart-leo, the pip-installed version gets started.

I've just taken another look at 'restart-leo'.

After shutting down Leo, the command begins the restart process like this:

os.chdir(g.app.initial_cwd)  # The directory from which Leo started.

I thinks this is correct and I don't want to change this logic!

I am going to close the issue.

Edward

[Edward K. Ream]

On Friday, October 18, 2024 at 4:23:21 AM UTC-5 viktor....@gmail.com wrote:

Leo restart on Windows suffers from the same problem. If I start it with my batch file that sets PYTHONPATH, it restarts the git/leo-editor version.  If instead I cd to git/leo-editor and start with py -m leo.core.runLeo, then issue restart-leo, the pip-installed version gets started.

[Edward K. Ream]

On Tuesday, November 19, 2024 at 3:08:34 AM UTC-6 Edward K. Ream wrote:

I thinks [the restart-leo command] is correct and I don't want to change this logic!

Also note info issue #1183: Changing git branches can break clones.

My workflow (and strong advice) is to close Leo before switching branches.

Edward

[Thomas Passin]

On Tuesday, November 19, 2024 at 4:08:33 AM UTC-5 Edward K. Ream wrote:

On Friday, October 18, 2024 at 4:23:21 AM UTC-5 viktor....@gmail.com wrote:

Leo restart on Windows suffers from the same problem. If I start it with my batch file that sets PYTHONPATH, it restarts the git/leo-editor version.  If instead I cd to git/leo-editor and start with py -m leo.core.runLeo, then issue restart-leo, the pip-installed version gets started.

I've just taken another look at 'restart-leo'.

After shutting down Leo, the command begins the restart process like this:

os.chdir(g.app.initial_cwd)  # The directory from which Leo started.

This ought to be right but something goes wrong.  If I start Leo from the git/leo-editor directory and run the restart-leo command, the right parameters get printed:

Restarting Leo...
os.chdir(C:\Tom\git\leo-editor)
subprocess.run([
  C:\Users\tom\AppData\Local\Programs\Python\Python312\python.exe,
  C:\Tom\git\leo-editor\leo\core\runLeo.py,
  C:\Users\tom\.leo\workbook.leo
])

But the pip-installed version of Leo gets run anyway:

Leo 6.8.1  <=== should have been Leo 6.8.3-devel, tbp-vr3-restore-scroll-position branch, build 0879c45738
Python 3.12.3, PyQt version 6.7.3

[Thomas Passin]

Switching branches is not involved in this issue.  Kindly don't close the issue because it is going to confuse the heck out of someone.

[Edward K. Ream]

On Tue, Nov 19, 2024 at 6:25 AM Thomas Passin <tbp1...@gmail.com> wrote:

> Switching branches is not involved in this issue.  Kindly don't close the issue because it is going to confuse the heck out of someone.

I'm not inclined to re-open this issue.

The workarounds are obvious: devs can change the environment or their scripts. Even better: don't use restart-leo in situations where it doesn't do the right thing.

I'll be happy to reopen this issue when someone submits a PR that fixes an obvious bug in the restart-leo command. Imo, there is no such bug. As always, I could be mistaken.

Please take any further discussion offline.

Edward

[Viktor Ransmayr]

Hello Edward,

Here's the latest GitHub pull information from my Debian 12 VM:

• What ( else ) is needed to convince you that there is an issue ? - I'm not using any environment variables - just - plain 'git clone or pull' operations ...

With kind regards,

Viktor

###

    user@debian-leo-study-vm:~$
    user@debian-leo-study-vm:~$ cd PyVE/GitHub/Leo/
    user@debian-leo-study-vm:~/PyVE/GitHub/Leo$
    user@debian-leo-study-vm:~/PyVE/GitHub/Leo$ source bin/activate
    (Leo) user@debian-leo-study-vm:~/PyVE/GitHub/Leo$
    (Leo) user@debian-leo-study-vm:~/PyVE/GitHub/Leo$ cd leo-editor/
    (Leo) user@debian-leo-study-vm:~/PyVE/GitHub/Leo/leo-editor$
    (Leo) user@debian-leo-study-vm:~/PyVE/GitHub/Leo/leo-editor$ git branch
    * devel
    (Leo) user@debian-leo-study-vm:~/PyVE/GitHub/Leo/leo-editor$
    (Leo) user@debian-leo-study-vm:~/PyVE/GitHub/Leo/leo-editor$ git pull
    remote: Enumerating objects: 776, done.
    remote: Counting objects: 100% (776/776), done.
    remote: Compressing objects: 100% (234/234), done.
    remote: Total 776 (delta 581), reused 713 (delta 542), pack-reused 0 (from 0)
    Receiving objects: 100% (776/776), 1.61 MiB | 7.34 MiB/s, done.
    Resolving deltas: 100% (581/581), completed with 35 local objects.
    From https://github.com/leo-editor/leo-editor
       99b2235db..804a840d5  devel                   -> origin/devel
     * [new branch]          ekr-4179-dark-web-pages -> origin/ekr-4179-dark-web-pages
     * [new branch]          ekr-4193-execute-script -> origin/ekr-4193-execute-script
       e17e0af23..d475d8006  gh-pages                -> origin/gh-pages
    Updating 99b2235db..804a840d5
    Fast-forward
     docs/appendices.html                           |  473 ++----
     docs/cheatsheet.html                           |  198 +--
     docs/commands.html                             |   10 +-
     docs/customizing.html                          |   20 +-
     docs/debuggers.html                            |   31 +-
     docs/directives.html                           |   62 +-
     docs/emacs.html                                |  105 +-
     docs/genindex.html                             |  545 ++++---
     docs/glossary.html                             |  654 ++++----
     docs/history.html                              |    4 +-
     docs/intermediatetopics.html                   |    2 +-
     docs/leo-university.html                       |   13 +-
     docs/leoBridge.html                            |   20 +-
     docs/leo_toc.html                              |    2 +-
     docs/leoandasciidoc.html                       |    2 +-
     docs/leoandotherprograms.html                  |    2 +-
     docs/leomarkup.html                            |   23 +-
     docs/plugins.html                              |   23 +-
     docs/scripting-miscellany.html                 |  182 +--
     docs/search.html                               |    2 +-
     docs/searchindex.js                            |    2 +-
     docs/theory.html                               |    8 +-
     docs/toc-more-links.html                       |    2 +-
     docs/tutorial-basics.html                      |   65 +-
     docs/tutorial-pim.html                         |    2 +-
     docs/tutorial-rst3.html                        |   17 +-
     docs/tutorial-scripting.html                   |   20 +-
     docs/tutorial-tips.html                        |   20 +-
     docs/tutorial.html                             |    2 +-
     docs/usersguide.html                           |    2 +-
     docs/vim-theory.html                           |   20 +-
     docs/writingPlugins.html                       |   38 +-
     leo/commands/checkerCommands.py                |   17 +-
     leo/core/leoApp.py                             |   24 +-
     leo/core/leoColorizer.py                       |  320 ++--
     leo/core/leoCommands.py                        |    5 +-
     leo/core/leoFrame.py                           |    2 -
     leo/doc/LeoDocs.leo                            | 2001 ++++++++++++------------
     leo/modes/clojure.py                           |    2 +-
     leo/modes/html.py                              |    8 +-
     leo/modes/javascript.py                        |    6 +-
     leo/modes/md.py                                |   66 +-
     leo/modes/nim.py                               |    4 +-
     leo/modes/pandoc.py                            |   66 +-
     leo/plugins/freewin.py                         |   57 +-
     leo/plugins/patch_python_colorizer.py          |    1 -
     leo/plugins/viewrendered3.py                   |  379 +++--
     leo/plugins/viewrendered3/vr3-medium-black.css | 1295 +++++++++++++++
     leo/scripts/pylint_leo.py                      |   12 +-
     leo/scripts/scripts.leo                        |  195 +++
     leo/unittests/core/test_leoColorizer.py        |    2 +-
     leo/unittests/misc_tests/test_syntax.py        |   52 +
     52 files changed, 4327 insertions(+), 2758 deletions(-)
     create mode 100644 leo/plugins/viewrendered3/vr3-medium-black.css
    (Leo) user@debian-leo-study-vm:~/PyVE/GitHub/Leo/leo-editor$

    ###

    (Leo) user@debian-leo-study-vm:~/PyVE/GitHub/Leo/leo-editor$
    (Leo) user@debian-leo-study-vm:~/PyVE/GitHub/Leo/leo-editor$ which leo
    /home/user/PyVE/GitHub/Leo/bin/leo
    (Leo) user@debian-leo-study-vm:~/PyVE/GitHub/Leo/leo-editor$

    ###

    (Leo) user@debian-leo-study-vm:~/PyVE/GitHub/Leo/leo-editor$
    (Leo) user@debian-leo-study-vm:~/PyVE/GitHub/Leo/leo-editor$ leo --version
    Leo 6.8.0-b2
    Python 3.11.2
    linux
    (Leo) user@debian-leo-study-vm:~/PyVE/GitHub/Leo/leo-editor$

    ###

    (Leo) user@debian-leo-study-vm:~/PyVE/GitHub/Leo/leo-editor$
    (Leo) user@debian-leo-study-vm:~/PyVE/GitHub/Leo/leo-editor$ python3 -m leo.core.runLeo --version
    Leo 6.8.3-devel, devel branch, build 804a840d59
    2024-11-19 03:40:11 -0600
    Python 3.11.2
    linux
    (Leo) user@debian-leo-study-vm:~/PyVE/GitHub/Leo/leo-editor$

[Edward K. Ream]

On Tue, Nov 19, 2024 at 10:27 AM Viktor Ransmayr <viktor....@gmail.com> wrote:

• What ( else ) is needed to convince you that there is an issue ?

Context. I have no idea what you are doing, why you are doing it or why you are using restart-leo.

If restart-leo is causing you problems, just do something else.

Edward

[Viktor Ransmayr]

Hello Edward,

I'll take a break for a while ! - I have provided all the information I have in my previous postings ...

With kind regards,

Viktor

 

[Edward K. Ream]

On Tue, Nov 19, 2024 at 10:44 AM Viktor Ransmayr <viktor....@gmail.com> wrote:

 

I'll take a break for a while ! - I have provided all the information I have in my previous postings ...

That's fine with me. I handle many emails each day. Please don't assume I remember any details. The more succinct the better.

Edward

[Thomas Passin]

I have tracked down what is happening but I don't know why.  The problem is not with the restart command, it's with the runLeo.py module.  It runs a different version of Leo, and even a different version of Python, depending on how it is invoked even though you would swear the results should be the same. Details are in the issue I just filed:

https://github.com/leo-editor/leo-editor/issues/4199

[Thomas Passin]

On Tuesday, November 19, 2024 at 1:44:08 PM UTC-5 Thomas Passin wrote:

I have tracked down what is happening but I don't know why.  The problem is not with the restart command, it's with the runLeo.py module.  It runs a different version of Leo, and even a different version of Python, depending on how it is invoked even though you would swear the results should be the same. Details are in the issue I just filed:

https://github.com/leo-editor/leo-editor/issues/4199

The reason that this problem has been showing up with the restart-leo command is that the launch command that is captured by sys.argv is NOT always that same command that launched Leo in the first place. The case that occurs for me is that I launch Leo using

py -m leo.core.runLeo

but sys.argv reports that argument as leo\core\runLeo (IOW, no longer using -m but just sys.executable leo\core\runLeo.py).  It is exactly this difference that runLeo handles differently.

[Edward K. Ream]

On Tue, Nov 19, 2024 at 4:21 PM Thomas Passin wrote:

I have tracked down what is happening but I don't know why.  The problem is not with the restart command, it's with the runLeo.py module.  It runs a different version of Leo, and even a different version of Python, depending on how it is invoked even though you would swear the results should be the same. Details are in the issue I just filed:

https://github.com/leo-editor/leo-editor/issues/4199

The reason that this problem has been showing up with the restart-leo command is that the launch command that is captured by sys.argv is NOT always that same command that launched Leo in the first place. The case that occurs for me is that I launch Leo using

py -m leo.core.runLeo

Bingo!

My local leo.cmd is:

python C:\Repos\leo-editor\launchLeo.py %*

This tells python exactly where to find Leo's repo.  In contrast, py -m leo.core.runLeo does not.

Unless I am mistaken, this difference explains:

- All the behaviour you describe.

- The reason I have never seen the behaviors you describe.

Discussion

Using python -m is often correct, so the choice will always be a tripper.

I use this local file,  run-installed-leo.cmd, during testing, as discussed in the distribution checklist:

rem Do **NOT** run this script from the leo-editor folder.
rem pip must *not* find the leo-editor folder!
cd c:\Repos
call python -m leo-editor.leo.scripts.run_installed_leo
cd c:\Repos\leo-editor

In this script, python -m is correct.

A similar tripper?

I am having a devil of a time with #4179: use a dark theme for Leo's website.

Single-stepping through the `@button make-sphinx` script in LeoDocs.leo showed mismatches between python.exe and the sphinx libs! I'm still investigating.

I did notice an unexpected venv folder in my python folder. I foolishly ignored that surprise at the time, but I'll get back to that mystery soon enough.

Summary

py -m may have unexpected consequences that will bite devs especially.

HTH. Please let me know whether this post solves the mystery for you.

Edward

[Thomas Passin]

On Wednesday, November 20, 2024 at 5:20:12 AM UTC-5 Edward K. Ream wrote:

On Tue, Nov 19, 2024 at 4:21 PM Thomas Passin wrote:

I have tracked down what is happening but I don't know why.  The problem is not with the restart command, it's with the runLeo.py module.  It runs a different version of Leo, and even a different version of Python, depending on how it is invoked even though you would swear the results should be the same. Details are in the issue I just filed:

https://github.com/leo-editor/leo-editor/issues/4199

The reason that this problem has been showing up with the restart-leo command is that the launch command that is captured by sys.argv is NOT always that same command that launched Leo in the first place. The case that occurs for me is that I launch Leo using

py -m leo.core.runLeo

Bingo!

My local leo.cmd is:

python C:\Repos\leo-editor\launchLeo.py %*

This tells python exactly where to find Leo's repo.  In contrast, py -m leo.core.runLeo does not.

Unless I am mistaken, this difference explains:

Not really.  I usually run my git clone of Leo using a .cmd file that sets PYTHONPATH to the leo-editor directory, then runs py -m leo.core.runLeo.  This script accomplished the same thing as yours. The restart-leo command succeeds when I launch the original Leo instance with this script.  Using the script, the restart-leo command succeeds even when the working directory is leo-editor.

The experiments I posted about above did not use that script.  But the working directory was leo-editor. That *should* have caused Python to find the leo-editor files first, before the pip-installed ones. It did, but the restart-leo command manages to prevent Python using the leo-editor files even though the working directory is leo-editor.  It sometimes even runs the wrong version of Python, apparently because it calls python.exe, which is incorrect.

If one starts Leo with py -m leo.core.runLeo, without setting PYTHONPATH, one of two things should happen:

1. If the working directory is leo-editor, then the git repo's version of Leo will run.

2. If the working directory is not leo-editor, then the pip-installed version of Leo will run.

And in fact both 1) and 2) happen, even if a venv is being used.

In either case, the restart-leo command should work correctly. But it does not work right in case 1. Not working right is against all expectations of how Python should work, is non-intuitive and confusing, and it is impossible to know what will happen in advance unless one has tried all the various combinations and managed to remember them.

 

- All the behaviour you describe.

- The reason I have never seen the behaviors you describe.

Discussion

Using python -m is often correct, so the choice will always be a tripper.

python -m should work when the initial working directory is leo-editor. Something in launchLeo.py  is making an erroneous assumption or two.

 

I use this local file,  run-installed-leo.cmd, during testing, as discussed in the distribution checklist:

rem Do **NOT** run this script from the leo-editor folder.
rem pip must *not* find the leo-editor folder!
cd c:\Repos
call python -m leo-editor.leo.scripts.run_installed_leo
cd c:\Repos\leo-editor

In this script, python -m is correct.

A similar tripper?

I am having a devil of a time with #4179: use a dark theme for Leo's website.

Single-stepping through the `@button make-sphinx` script in LeoDocs.leo showed mismatches between python.exe and the sphinx libs! I'm still investigating.

As I wrote earlier, you cannot assume that "python.exe" will run the right version of python.  What will happen will depend on your installation history, your file associations, and the path. That's why the "py" launcher is so useful.  Even on Linux, "python" or "python3" may not run the desired version because they run the system's python installation.  If you want to use another, you will be out of luck. For example, one of my VMs used python 3.8 for the system, and I wanted to run Leo with python 3.11, which was the latest version at that time.  So I installed it outside of the distro's repo, since 3.11 wasn't offered yet. This worked but I had to run programs using "python3.11" instead of "python3".

 

I did notice an unexpected venv folder in my python folder. I foolishly ignored that surprise at the time, but I'll get back to that mystery soon enough.

Summary

py -m may have unexpected consequences that will bite devs especially.

HTH. Please let me know whether this post solves the mystery for you.

No it doesn't. There is nothing wrong with using the py -m command. The root cause is that py -m leo.core.runLeo does not produce the same results as python leo\core\runLeo.py, at least when the working directory is leo-editor. This difference is totally unexpected and non-intuitive, and should be fixed.

The restart-leo command constructs the command to be given to subprocess.run using sys.argv.  Even when Leo was launched with py -m leo.core.runLeo, sys,argv will contain leo\core\runLeo. This causes the restart-leo command to run the wrong version of Leo when the working directory is leo-editor.

[Edward K. Ream]

On Wed, Nov 20, 2024 at 7:17 AM Thomas Passin <tbp1...@gmail.com> wrote:

QQQ

If one starts Leo with py -m leo.core.runLeo, without setting PYTHONPATH, one of two things should happen:

1. If the working directory is leo-editor, then the git repo's version of Leo will run.

2. If the working directory is not leo-editor, then the pip-installed version of Leo will run.

QQQ

That's what happens on my installation after pip install leo:

Running from home

C:\Users\Dev>python -m leo.core.runLeo
Leo 6.8.2
...

The log shows:

Leo 6.8.2
Python 3.12.0, PyQt version 6.7.3
Windows 11 AMD64 (build 10.0.26100) SP0
      home: C:\Users\Dev
leo-editor: C:\Python\Python3.12\Lib\site-packages
      load: C:\Python\Python3.12\Lib\site-packages\leo\core
    config: C:\Python\Python3.12\Lib\site-packages\leo\config

Running from Leo's github repo

C:\Repos\leo-editor>python -m leo.core.runLeo
Leo 6.8.3-devel, ekr-726-at-language-vue2 branch, build eab3d7e0a2
2024-11-20 04:46:08 -0600

...

The log shows:

Leo 6.8.3-devel, ekr-726-at-language-vue2 branch, build eab3d7e0a2
2024-11-20 04:46:08 -0600
Python 3.12.0, PyQt version 6.7.3
Windows 11 AMD64 (build 10.0.26100) SP0
      home: C:\Users\Dev
leo-editor: C:\Repos\leo-editor
      load: C:\Repos\leo-editor\leo\core
    config: C:\Repos\leo-editor\leo\config

About runLeo.py

runLeo.py contains only three imports:

import os
import sys
import traceback

The run function is:

def run(fileName=None, pymacs: bool = None, *args, **keywords):
    """Initialize and run Leo"""
    # #1403: sys.excepthook doesn't help.
    # sys.excepthook = leo_excepthook
    assert g.app
    g.app.loadManager = leoApp.LoadManager()
    g.app.loadManager.load(fileName, pymacs)

LoadManager.load instantiates commanders.

c.initObjects contains many imports of the form:

from leo.core import leoAtFile

Finally, I do have a sitecustomize.py file, but it is a do-nothing.

Summary

I do not believe a word of your analysis. There is something wrong with your installation.

And one last question. Why are you leaving a pip-installed version of Leo lying around?

I uninstall it immediately after testing.

Edward

[Edward K. Ream]

On Wed, Nov 20, 2024 at 8:41 AM Edward K. Ream <edre...@gmail.com> wrote:

> And one last question. Why are you leaving a pip-installed version of Leo lying around?

> I uninstall it immediately after testing.

C:\Scripts>python -m pip uninstall leo
Found existing installation: leo 6.8.2
Uninstalling leo-6.8.2:
  Would remove:
    c:\python\python3.12\lib\site-packages\leo-6.8.2.dist-info\*
    c:\python\python3.12\lib\site-packages\leo\*
    c:\python\python3.12\scripts\leo-c.exe
    c:\python\python3.12\scripts\leo-console.exe
    c:\python\python3.12\scripts\leo.exe
Proceed (Y/n)? y
  Successfully uninstalled leo-6.8.2

C:\Scripts>python -m leo.core.runLeo
C:\Python\Python3.12\python.exe: Error while finding module specification for 'leo.core.runLeo' (ModuleNotFoundError: No module named 'leo')

Edward

[Thomas Passin]

I do not believe a word of your analysis. There is something wrong with your installation.

Then you had better think again.

1. The exact same behavior happens on my Linux EndeavourOS VM.  I just checked it.

2. My statement that the starting argument of -m leo.core.runLeo gets changed in sys.argv to leo\core\runLeo is a fact.  I put a print statement into the code to see (I forget whether ".py" gets appended).

3. My statement that py -m leo.core.runLeo acts differently from py leo/core/runLeo.py if the working directory is leo-editor is a fact. To illustrate using that same Linux VM, and running in a venv:

(leo) [tom@tom-endeavour leo-editor]$ python3 leo/core/runLeo.py

Leo 6.8.0
Python 3.12.7, PyQt version 6.7.2
linux

(leo) [tom@tom-endeavour leo-editor]$ python3 -m leo.core.runLeo

Leo 6.8.3-devel, devel branch, build 9862614477
2024-11-16 08:56:31 -0600
Python 3.12.7, PyQt version 6.7.2
linux

4. The restart-leo command uses sys.argv to construct its subprocess.run() arguments.

These things are facts, not my analysis, and not my installation quirks.

[Edward K. Ream]

On Wed, Nov 20, 2024 at 9:35 AM Thomas Passin <tbp1...@gmail.com> wrote:

I do not believe a word of your analysis. There is something wrong with your installation.

Then you had better think again.

1. The exact same behavior happens on my Linux EndeavourOS VM.  I just checked it.

2. My statement that the starting argument of -m leo.core.runLeo gets changed in sys.argv to leo\core\runLeo is a fact. 

It seems that we are discussing two different issues: 'restart-leo' vs starting Leo.

I'll continue to investigate.

Edward

[Thomas Passin]

They are closely related.  restart-leo cannot know if Leo was started using py -m or not.  Using py -m leo.core.runLeo gives a different result than py leo/core/runLeo.py if the working directory is leo-editor. The restart-leo command can't know which way Leo was started based solely on sys.argv. So it constructs the arguments for subprocess.run in a way that unfortunately causes runLeo.py to misbehave in the case that the starting directory is leo-editor. Therefore runLeo.py should be changed so that it no longer misbehaves in that particular case.

Now that was analysis based on facts.

[Edward K. Ream]

On Wed, Nov 20, 2024 at 9:37 AM Edward K. Ream <edre...@gmail.com> wrote:

> It seems that we are discussing two different issues: 'restart-leo' vs starting Leo.

Here are my recent tests.

Run python -m leo.core.runLeo from my home directory

C:\Users\Dev>python -m leo.core.runLeo

C:\Python\Python3.12\python.exe: Error while finding module specification for 'leo.core.runLeo' (ModuleNotFoundError: No module named 'leo')

This is the expected result. I'll show the result after pip install leo later.

Run python -m leo.core.runLeo from Leo's github repo

c:\Repos\leo-editor>python -m leo.core.runLeo

Leo 6.8.3-devel, ekr-726-at-language-vue2 branch, build eab3d7e0a2
2024-11-20 04:46:08 -0600

Python 3.12.0, PyQt version 6.7.3
Windows 11 AMD64 (build 10.0.26100) SP0

wrote recent file: C:/Users/Dev/.leo/.leoRecentFiles.txt

This is the expected result.

Now run restart-leo

Restarting Leo...
os.chdir(c:\Repos\leo-editor)  <===
subprocess.run([
  C:\Python\Python3.12\python.exe,
  c:\Repos\leo-editor\leo\core\runLeo.py   <===
])

Leo 6.8.3-devel, ekr-726-at-language-vue2 branch, build eab3d7e0a2
2024-11-20 04:46:08 -0600
Python 3.12.0, PyQt version 6.7.3
Windows 11 AMD64 (build 10.0.26100) SP0

wrote recent file: C:/Users/Dev/.leo/.leoRecentFiles.txt

This is the expected result.

restart-leo always shows these arguments. No additional traces needed.

Now pip install leo

c:\Repos\leo-editor>python -m pip install leo
Collecting leo
  Using cached leo-6.8.2-py3-none-any.whl.metadata (4.5 kB)

[big snip]

Installing collected packages: leo
  WARNING: The scripts leo-c.exe, leo-console.exe and leo.exe are installed in 'C:\Python\Python3.12\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed leo-6.8.2

Note that my workflow never pollutes sys.path.

Run Leo from home (using pip-installed leo)

C:\Users\Dev>python -m leo.core.runLeo
Leo 6.8.2

Python 3.12.0, PyQt version 6.7.3
Windows 11 AMD64 (build 10.0.26100) SP0

wrote recent file: C:/Users/Dev/.leo/.leoRecentFiles.txt

Restarting Leo...
os.chdir(C:\Users\Dev)  <===
subprocess.run([
  C:\Python\Python3.12\python.exe,
  C:\Python\Python3.12\Lib\site-packages\leo\core\runLeo.py  <===
])

Leo 6.8.2
Python 3.12.0, PyQt version 6.7.3
Windows 11 AMD64 (build 10.0.26100) SP0

wrote recent file: C:/Users/Dev/.leo/.leoRecentFiles.txt

Once again, this is as expected.

Summary

python -m leo.core.runLeo works as expected from all locations.

restart-leo works as expected at all times.

Edward

[Thomas Passin]

On Wednesday, November 20, 2024 at 12:18:26 PM UTC-5 Edward K. Ream wrote:

Summary

python -m leo.core.runLeo works as expected from all locations.

restart-leo works as expected at all times.

Edward

I agree that py -m leo.core.runLeo works as expected in all the cases.  My tests all show the same.  What doesn't work as expected is running py leo\core\runLeo.py when the working directory is leo-editor. And that's the command that restart-leo issues, not py -m.  In addition, on my Windows system where "python.exe" runs Python 3.9.9, the latter seems to get launched as well, not the original sys.executable. I'm not completely sure when this happens but happen it does.

Remember, I got the same differences between running runLeo.py with py -m leo.core.runLeo vs py leo\core\runLeo.py on both my Windows computer and my Linux VM.  So it's not some quirk of my Windows installation that's in play here. (yes, having python.exe and the .py file association linked to Python 3.9.9 is a quirk of my machine but it's not the main issue).

[Later] I removed Leo from my Python 3.9.9 site-packages directory, and after this the command does run the intended version of Leo, but still with the wrong executable:

C:\Tom\git\leo-editor>py leo\core\runleo.py  %USERPROFILE%\.leo\workbook.leo

Leo 6.8.3-devel, devel branch, build 99593cba0c
2024-11-19 16:37:27 -0600
Python 3.9.9, PyQt version 6.6.2

After more experimenting, I find that the right version of Python gets run if I specify it explicitly (e.g., py -3.12) but if I use just "py", runLeo launches "python.exe". So there is some weird interaction between runLeo and the py launcher, since all other programs I run with a bare "py" run the latest installed version.

[Later] sys.argv for py -m ...:

C:\Tom\git\leo-editor>py -m leo.core.runLeo %USERPROFILE%\.leo\workbook.leo

sys.argv=['C:\\Tom\\git\\leo-editor\\leo\\core\\runLeo.py', 'C:\\Users\\tom\\.leo\\workbook.leo']

sys.argv for py ...:

C:\Tom\git\leo-editor>py leo\core\runleo.py  %USERPROFILE%\.leo\workbook.leo

sys.argv=['leo\\core\\runleo.py', 'C:\\Users\\tom\\.leo\\workbook.leo']

[Thomas Passin]

Let's continue this investigation on the GitHub issue thread, shall we?

[Edward K. Ream]

On Wed, Nov 20, 2024 at 12:48 PM Thomas Passin <tbp1...@gmail.com> wrote:

Let's continue this investigation on the GitHub issue thread, shall we?

I'd rather not.

I agree that py -m leo.core.runLeo works as expected in all the cases.  My tests all show the same.  What doesn't work as expected is running py leo\core\runLeo.py when the working directory is leo-editor.

By "py" I'm assuming that you mean Python's launcher, py.exe.

We are back to the bingo. 

Leo's devs should use their own local python script: python.cmd or a python script or alias on linux/MacOS machines. Either way, Leo's devs are responsible for choosing the correct python.

People who pip install leo shouldn't have problems. None have been reported.

Leo works. If devs have a problem, they are responsible for the workarounds. I can't debug devs' installations.

Edward