Location of 'leoserver.py' & 'leoclient.py' in Leo's directory structure?

[Viktor Ransmayr]

Hello Edward & Félix,

Looking into the sources I also noticed, that we'll run into a similar problem as already documented in issue #603.

In other words, both files should be moved from the top-level, i.e.

* https://github.com/leo-editor/leo-editor/blob/devel/leoserver.py
* https://github.com/leo-editor/leo-editor/blob/devel/leoclient.py

to an existing / new directory somewhere in

* https://github.com/leo-editor/leo-editor/tree/devel/leo

Otherwise I don't see how it can be part of a stable version of Leo uploaded to PyPI.

With kind regards,

Viktor

--

[1] https://github.com/leo-editor/leo-editor/issues/603

[Edward K. Ream]

On Sun, May 16, 2021 at 2:21 AM Viktor Ransmayr <viktor....@gmail.com> wrote:

Hello Edward & Félix,

Looking into the sources I also noticed, that we'll run into a similar problem as already documented in issue #603.

In other words, both files should be moved from the top-level, i.e.

Thanks for this.  Félix, do you have any comments?

Edward

[Félix]

No biggie :)

I'll just add a little text 'tip' that hints that the leoserver.py is just inside the '/leo' subfolder of the main Leo installation folder...

I don't know much at all about those python system specific issues. (although I wonder how the other python files that import parts of leo do exist in the root of the leo folder without problem, such as launchleo.py, ect.. ) So I leave it to you to make this change for leoserver and leoclient (and/or the test program for leoserver.py)

Many thanks to Viktor for spotting those issues before the general Leo's user base got stuck on those installation problems!

--

Félix

[Viktor Ransmayr]

Hello Félix

Félix schrieb am Dienstag, 18. Mai 2021 um 00:55:18 UTC+2:

I don't know much at all about those python system specific issues. (although I wonder how the other python files that import parts of leo do exist in the root of the leo folder without problem, such as launchleo.py, ect.. )

My answer to your question is: They don't exist at all, if the user has installed Leo via pip from PyPI - and - not via 'git clone' from GitHub.

I guess this is not / has not been considered a major issue, because if users are developing plugins for Leo - or - are contributing to Leo itself, it's assumed that they work in their own local repository cloned from GitHub anyway.

The problem / surprise occurs for any user, which is assuming that both installation methods provide equal functionality.

List of Python files in Leo's root folder:

* flake8-leo.py
* launchLeo.py -> similar functionality is available in .../leo/scripts/leo , leo.bat , etc. !
* launchLeo-console.py
* leoclient.py
* leoserver.py
* profileLeo.py
* pyflakes-leo.py
* pylint-leo.py
* run_pytest_test.py
* run_travis_unit_tests.py
* setup.py

All / most other Python files are needed only for Leo's developers and not for Leo's users.

I guess in a 'perfect' world the only Python file expected in the root folder is 'setup.py' [1].

I'm not an expert on this topic - so - my explanation / view may not be entirely correct.

With kind regards,

Viktor

---

[1] https://packaging.python.org/

[Félix]

Thank you Viktor! 

(glad I asked you - this was mystifying for me!)

--

Félix

[Edward K. Ream]

On Tuesday, May 18, 2021 at 3:32:23 AM UTC-5 viktor....@gmail.com wrote:

 > They [the files in leo-editor folder] don't exist at all, if the user has installed Leo via pip from PyPI - and - not via 'git clone' from GitHub.

Somehow I never understood this.

> The problem / surprise occurs for any user, which is assuming that both installation methods provide equal functionality.

> List of Python files in Leo's root folder:

* flake8-leo.py
* launchLeo.py -> similar functionality is available in .../leo/scripts/leo , leo.bat , etc. !
* launchLeo-console.py
* leoclient.py
* leoserver.py
* profileLeo.py
* pyflakes-leo.py
* pylint-leo.py
* run_pytest_test.py
* run_travis_unit_tests.py
* setup.py

Wow. This is bad. I don't want to pick and choose which files a "more important" than others.

> All / most other Python files are needed only for Leo's developers and not for Leo's users.

Hmm. If possible, I would like all these files to be part all distributions.

> I guess in a 'perfect' world the only Python file expected in the root folder is 'setup.py' [1].

> [1] https://packaging.python.org/

My eyes start to roll to the back of my head when I read this page :-)

Summary

I had no idea that `pip install leo` omits so many files. I'd like to fix this, but changing the locations of files could break existing scripts.

Edward

[Félix]

What happened to the standard 'installer' program concept... "install.msi" or .exe. or whatever? Is that still a thing? 

Leo is not a python library like 'websockets' or some other 3rd party functionality library... should it not be installed with a 'real' installation program? (on windows) and some .deb package on debian linuxes, etc...? 

I'm a newbie with regards to those things... Just asking for my own curiosity for whomever has answers to this to answer back, if any... :)

--

Félix

[tbp1...@gmail.com]

In my view, Leo *is* a 3rd party python package and should be installable via pip.  It is true that some Linux systems will tinker with the directories, paths, etc and want you to install certain packages using their own package manager.  But Leo as a project can't cater to all these possible variations.

I can see a separate pip package named, say, "leo-server" or some such that includes the parts that you want to work with that ordinary users can't.  Such a package could I'm sure be created by a script that creates a temporary directory tree with the extra parts for the packager to package up.  Or maybe the existing setup.py can be enhanced to do that.

[Félix]

First of all, thank you very much tbp1, for answering questions regarding the installation of Leo, and to shed some light on this matter. 

Now something that popped in my mind a second ago is intriguing me very much: If someone had installed Leo with pip, would then it be possible for that person to just start Leo, open leopyref.leo, hit ctrl+s, and have launchLeo.py , flake8-leo.py, launchLeo-console.py, etc. be generated in their root Leo installation folder?

Thanks to anyone who answers this question!

--

Félix

[tbp1...@gmail.com]

I don't think that you would want it to work exactly this way because that would change the action of the CTRL-S command and that would be confusing.  Instead, how about a button for leoPyRef that would do those things?

[Edward K. Ream]

On Thu, May 20, 2021 at 9:05 PM Félix <felix...@gmail.com> wrote:

> If someone had installed Leo with pip, would then it be possible for that person to just start Leo, open leopyref.leo, hit ctrl+s, and have launchLeo.py , flake8-leo.py, launchLeo-console.py, etc. be generated in their root Leo installation folder?

Yes, it would be possible, provided that these files existed as @clean or @nosent trees in LeoPyRef.leo. At present, they don't exist at all, or are @edit or @file nodes, which get their contents from an external file.  We definitely would want @clean for the complex files, including setup.py, leoclient.py and leoserver.py.

Btw, LeoPyRef.leo contains the contents of leo_to_html.xsl just as a node, not any kind of @<file> node. That looks like an oversight. See

    Code-->Other files-->In leo-editor directory

Edward

[Viktor Ransmayr]

Hello Félix, hello Edward,

Félix schrieb am Freitag, 21. Mai 2021 um 04:05:14 UTC+2:

Now something that popped in my mind a second ago is intriguing me very much: If someone had installed Leo with pip, would then it be possible for that person to just start Leo, open leopyref.leo, hit ctrl+s, and have launchLeo.py , flake8-leo.py, launchLeo-console.py, etc. be generated in their root Leo installation folder?

 I don't think this is a good idea. - Matt Wilkie described the WHY well in issue 603 [1]:

""" Basically anything in the root folder of the leo-editor code repository is not installed when using pip. This

is because they need to be relative to ./leo-editor/leo folder, which make them at the top of ./site-packages,

and thus in the global namespace and not part of leo (as far as python is concerned).  ... """

I believe it is time now to seriously discuss his other proposal on "Root source code folder cleanup: justify your existence or be banished!"" [2] as well.

With kind regards,

Viktor

--

[1] https://github.com/leo-editor/leo-editor/issues/603#issue-279265779

[2] https://github.com/leo-editor/leo-editor/issues/603#issuecomment-350173140

[Félix]

Thank you Viktor for those enlightening facts. Much appreciated input!

--

BTW Edward, please feel free to move leoserver.py and leoclient.py inside the /leo/ folder. Its not too bad still somewhat easy to browse too, for users setting up a remote client like leointeg. 

(It's still easier than the old <root>/leo/core/ that it initially was.)

I'll pick it up from there next week when I merge the leoserver / client cleanup/refactor that I started.

--

Félix

[David Szent-Györgyi]

On Thursday, May 20, 2021 at 1:39:06 PM UTC-4 tbp1...@gmail.com wrote:

In my view, Leo *is* a 3rd party python package and should be installable via pip.  It is true that some Linux systems will tinker with the directories, paths, etc and want you to install certain packages using their own package manager.  But Leo as a project can't cater to all these possible variations.

I can see a separate pip package named, say, "leo-server" or some such that includes the parts that you want to work with that ordinary users can't.  Such a package could I'm sure be created by a script that creates a temporary directory tree with the extra parts for the packager to package up.  Or maybe the existing setup.py can be enhanced to do that.

The last time I worked on an installer for Leo or any other package, Leo 4.3 was the current release. That installer was for Windows only, and used NSIS to generate the single-file setup executable. NSIS as it stood then was flexible as it saw use in many projects, but its use was somewhat arcane. My work was aimed at supporting per-user installations of CPython as well as shared installations, and installing per-user and system-wide installations of Leo on top of system-wide installations of CPython; my hope was that the two flavors of per-user support would ease the work of side-by-side testing of multiple CPython releases and multiple Leo releases. By the time I had something to share with Edward, he was already burdened by maintenance of the source code of the script that NSIS compiles. It's possible that my work only promised more installer-related work for him, but it's Edward's right to comment on that if he wishes to. 

If I recall correctly, Leo releases are less frequent than was the case when Leo 4.2, 4.3, and 4.4 were under development.  Stabilizing Leo for a release requires work. It is also the case that an installer requires a matching uninstaller. Serious effort is made to minimize the periods during which the most recent commit devel branch on GitHub is unusable; as such, the expectation is that one needs to pull from the devel branch in between releases. 

Pip is for packages; I question whether Leo with its Qt -based GUI and with its plugins is best considered as a single package. In my days working with Leo with the Tkinter-based GUI and working solely on Windows, all I needed was the the single-file installer for CPython, the single-file NSIS-based installer executable for Leo, and the single-file installer for Mark Hammond's pywin32 tools - I could fit the installers on a rewritable CD, and install them on a new machine in three minutes, and understand the process without documenting it.

Making Leo easier to install across deployment targets may increase the volume of requests for help, and my experience providing technical support for software published by my employers is that addressing deployment strategies in the software itself is critical to allowing the support staff to scale. 

Python is a great language for writing scripts and many sorts of applications, but its deployment story does not deliver the application publisher a one solution-fits-all across Windows, macOS, the various Unix distributions, and the various Linux distributions. The details of the requirements for the different deployment targets vary so greatly that I've just abandoned detailing those known to me here; a serious attempt to catalogue them for the use of Leo's developers needs to be addressed separately. 

That said, I think all those details need to be laid out for Leo's developers to consider; the rearrangement of Leo's source code for pip is only a start on the changes needed to support deployment, and I am sure that improvement of minimal support for all the targets listed in the preceding paragraph would require changes within Leo itself. 

[Félix]

Thanks to David, Edward, Victor and others participating in this particular thread about server script file location, 

To conclude and move forward with this specific (and important) issue that Victor raised:

I tough about this in the last few days and I've come to the conclusion that the server.py script (and its client/test scripts) should go back to their original location: <root-folder>/leo/core so that anyone has them regardless of the installation method that was used to install Leo.

(Edward please let me know if you can change this in the devel branch - no rush - sorry to have you revert this! )

To compensate in making the server script easier to find & help the user locate the server script, I will simply make leointeg's "server -location" setting more flexible:  I will let it accept either a string folder-path to the leo-editor root installation folder, (leointeg will figure out the specific /leo/core/server.py path from there), or, a .py file-path to a specific server script. (in case a user wants to use a specific server script).

Tada! problem solved!

--

Félix

[Edward K. Ream]

On Sat, Jun 5, 2021 at 4:23 PM Félix <felix...@gmail.com> wrote:

> I've come to the conclusion that the server.py script (and its client/test scripts) should go back to their original location: <root-folder>/leo/core so that anyone has them regardless of the installation method that was used to install Leo.

I'm good with this.  I have made the changes in the ekr-bridge branch.  See PR #1966.

> I will simply make leointeg's "server -location" setting more flexible:

Sounds like a plan.

Edward