[vim/vim] vim (and mccvim) fail to open a REPL for Lisp under SLIMV - file path errors. (Issue #11040)
kd4ttc
Steps to reproduce
With vim or MacVim installed and SLIMV installed in the ~/.vim directory the system allows users to edit .lisp files with paren management and omni completion working. All is good there.
Go into command mode and enter the comma command ,c and the error message "Vim is compiled without the Python feature or Python is not installed. Unable to run SWANK client. Press ENTER to continue".
Running :ver shows python is installed.
Checking
:execute (has('python3') ? "python3" : "python") . " import sys; print(sys.version)"
shows
E370: Could not load library /System/Library/Frameworks/Python.framework/Versions/2.7/Python: dlopen(/System/Library/Frameworks/Python.framework/Versions/2.7/Python, 0x0009): tried: '/System/Library/Frameworks/Python.framework/Versions/2.7/Python' (no such file), '/Library/Frameworks/Python.framework/Versions/2.7/Python' (no such file)
Looking in my /System/Library/Frameworks and '/Library/Frameworks show that the Python frameworks are no longer present.
Apple has discontinued bundling of the Frameworks with MacOS.
It is possible to install Python3 with brew. However, the paths don't go to the brew paths, they go to the system paths.
Recommendation:
- Expect that the user will install Python3 with brew.2.
- Let users specify the path to their python as an option probably in .vimrc.
Expected behaviour
The REPL should start.
Version of Vim
Vi IMproved 9.0 (2022 Jun 28, compiled Jul 25 2022 08:23:04)
Environment
MacOS version 12.5.1 (21G83)
Logs and stack traces
No response
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.
Yee Cheng Chin
That Stackoverflow answer by @romainl doesn't seem correct. has('python3') should work regardless of whether Python3 is compiled statically or dynamically. Your plugin is doing the correct thing.
The only issue here is that Homebrew installed Python3 3.10 here, whereas MacVim is compiled setting the path to 3.9 here. There is no bug here because Vim will never know for sure which version of Python you want to use as you could have multiple versions installed. MacVim tries to be smart and sets it to the latest Python3 version it knows of, but in this case 3.9 is no longer the latest and the auto-configuration doesn't work. It's a simple change to do set pythonthreedll=/opt/homebrew/Frameworks/Python.framework/Versions/3.10/Python in your vimrc and it will solve your issue I think (and you should undo all the other stuff you have done such as the custom patch to slimv and the symlink).
MacVim's next release will also be built against Python 3.10 anyway so the problem will go away but for now you can just set pythonthreedll.
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.
Romain Lafourcade
That Stackoverflow answer by @romainl doesn't seem correct. has('python3') should work regardless of whether Python3 is compiled statically or dynamically. Your plugin is doing the correct thing.
My answer is, sadly, 100% correct.
python3/dyn and python3 are different features, enabled with different flags, and tested with different strings. The purpose of has('python3') is not to test whether Vim can execute Python 3 code but to to test if the python3 feature was built in, which is not the same thing at all. The plugin in question, and many others I have seen in the past, uses the test incorrectly, makes incorrect assumptions and, of course, draws the wrong conclusions:
if ( !exists( 'g:slimv_python_version' ) && has( 'python3' ) ) ||
\ ( exists( 'g:slimv_python_version' ) && g:slimv_python_version == 3 )
let s:py_cmd = 'python3 ' "note space
let s:pyfile_cmd = 'py3file '
else
let s:py_cmd = 'python ' "note space
let s:pyfile_cmd = 'pyfile '
endif
Incorrect assumptions:
has('python3')tests if Vim can run Python 3 code.has('python3')is sufficient for that,- not being able to run Python 3 code implies being able to run Python code,
:python3will work as long as Vim can run Python 3 code.
Case in point, MacVim is built without the python3 feature (you know that full well because you are the one in charge, thank you by the way), therefore has('python3') returns 0. If the goal is to decide whether to use :python or :python3 to run custom commands, then that test is not sufficient at all.
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.
kd4ttc
I am the original reporter for the problem.
Here is additional data:
Running Mac OS 12.5.1
The following are in /opt/homebrew/Frameworks/Python.framework/Versions on my system.
lrwxr-xr-x 1 sholland admin 78 Aug 31 16:18 3.10 -> ../../../Cellar/pyt...@3.10/3.10.6_2/Frameworks/Python.framework/Versions/3.10
lrwxr-xr-x 1 sholland admin 9 Sep 2 15:39 3.9 -> ./Current
lrwxr-xr-x 1 sholland admin 81 Aug 31 16:18 Current -> ../../../Cellar/pyt...@3.10/3.10.6_2/Frameworks/Python.framework/Versions/Current
Due to the above comments I did further investigation.
When the 3.9 symlink is not available (by deleting the symlink 3.9 to ./Current in the directory)
: echo has('python3') -> 0
: echo has('python3_dynamic') -> 1
When the 3.9 symlink is available (having the symlink 3.9 to ./Current in the directory)
: echo has('python3') -> 1
: echo has('python3_dynamic') -> 1
So vim reports python3 available differs on whenther there is subdirectory for version 3.9 in the brew hierarchy. This is consistent with python3_dynmic having special meaning.
The vim error message that comes up when trying to get the REPL running is not fully informative. It says "Vim is compiled without the Python feature or Python is not installed. Unable to run SWANK client. Press ENTER to continue." That error message is incorrect. First, having a multiple choice for a user isn't all that helpful. Secondly, a user can hve Python3 installed. What vim really wants is to have Python3.9 installed. If a user has 3.10 installed vim will not run the REPL.
Does vim really need Python3.9 only? Is there a problem with running the current version of Python3? An error message that specifies that Python3.9 is the required version would be much better. Perhaps referring to Python3 Current version could be done. If Python changes and breaks vim then vim could start specifying a specific version of python again, with adjustment of the error message to specify a particular version of python.
I expect that when vim upgrades to version 3.10 users will find that their Lisp with REPL under vim will break. If a user upgrades from 3.9 to 3.10 of python their Lisp with REPL will break.
Steve Holland
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.
Yee Cheng Chin
That Stackoverflow answer by @romainl doesn't seem correct. has('python3') should work regardless of whether Python3 is compiled statically or dynamically. Your plugin is doing the correct thing.
My answer is, sadly, 100% correct.
python3/dynandpython3are different features, enabled with different flags, and tested with different strings. The purpose ofhas('python3')is not to test whether Vim can execute Python 3 code but to to test if thepython3feature was built in, which is not the same thing at all.
The code you linked to is not the whole story. The features that you test using has() aren't just direct correspondence to what you compile Vim with. See right below your linked code snippet under the function f_has() where it is handled in a special case (source code):
// features also in has_list[] but sometimes enabled at runtime if (x == TRUE && n == FALSE) { if (0) { // intentionally empty } // … (omitted) … #ifdef DYNAMIC_PYTHON3 else if (STRICMP(name, "python3") == 0) n = python3_enabled(FALSE); #endif }
I believe using has('python3') is the correct way to test for whether you can run Python 3 code in Vim. You can also see that by doing :help has-python as well.
Case in point, MacVim is built without the
python3feature (you know that full well because you are the one in charge, thank you by the way), thereforehas('python3')returns0. If the goal is to decide whether to use:pythonor:python3to run custom commands, then that test is not sufficient at all.
In MacVim, echo has('python3') actually does return 1 even though it's using dynamic python. It will only return 0 if pythonthreedll is pointing to a non-existing library.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.
Yee Cheng Chin
The vim error message that comes up when trying to get the REPL running is not fully informative. It says "Vim is compiled without the Python feature or Python is not installed. Unable to run SWANK client. Press ENTER to continue." That error message is incorrect. First, having a multiple choice for a user isn't all that helpful. Secondly, a user can hve Python3 installed. What vim really wants is to have Python3.9 installed. If a user has 3.10 installed vim will not run the REPL.
Does vim really need Python3.9 only? Is there a problem with running the current version of Python3? An error message that specifies that Python3.9 is the required version would be much better. Perhaps referring to Python3 Current version could be done. If Python changes and breaks vim then vim could start specifying a specific version of python again, with adjustment of the error message to specify a particular version of python.
I expect that when vim upgrades to version 3.10 users will find that their Lisp with REPL under vim will break. If a user upgrades from 3.9 to 3.10 of python their Lisp with REPL will break.
@kd4ttc Vim doesn't really care what version of Python. You can build it yourself against any Python version, or dynamically (which is what MacVim is using) which allows you to tell Vim which Python to use. You are using MacVim which is a particular distribution / build of Vim (and also technically a separate project from Vim), where we pre-build a binary for you to download. Since we build MacVim for you we just assume a particular version of Python for you so it "just works".
The vim error message that comes up when trying to get the REPL running is not fully informative. It says "Vim is compiled without the Python feature or Python is not installed. Unable to run SWANK client. Press ENTER to continue." That error message is incorrect. First, having a multiple choice for a user isn't all that helpful. Secondly, a user can hve Python3 installed. What vim really wants is to have Python3.9 installed. If a user has 3.10 installed vim will not run the REPL.
This error message is not from Vim. It's from your plugin. You can ask them to be a little more informative, or provide a way for you to diagnose your issue. Basically, the plugin is saying "Python is not configured properly in your Vim environment, you should figure out why".
Does vim really need Python3.9 only? Is there a problem with running the current version of Python3? An error message that specifies that Python3.9 is the required version would be much better.
Again, MacVim is only using 3.9 because that's what it was built against in the CI environment. Mixing versions of Python can be dicey at times but usually it will be just fine, and that's why we provide the pythonthreedll for you to specify which Python version you want to use.
Perhaps referring to Python3 Current version could be done.
I think the issue here is that I don't think there is a universal definition of what "installed" or "current version" of Python 3 means in a Unix system. Homebrew is not the only way to install Python. I guess you can search PATH for the python or python3 binary and see its version, but that's not the same as the installed library that Vim loads dynamically. It's not like there is a universal manifest that everyone can load and read the list of all installed Python libraries. That's why Vim just lets you specify the path manually.
I expect that when vim upgrades to version 3.10 users will find that their Lisp with REPL under vim will break. If a user upgrades from 3.9 to 3.10 of python their Lisp with REPL will break.
We just have to make a decision what the defaults are (and I have to stress that these are defaults. Users can always use pythonthreedll to set their own installation paths instead of "the latest Homebrew Python build"). 3.10 is the latest version of Python 3 and MacVim will just use that and shouldn't be using 3.9.
The reason why MacVim isn't built to use Python statically is that it's kind of hard for a binary release like this to know what build of Python its users want to use, and therefore we sacrifice some predictability with the flexibility for its users to choose to use their own Python installations.
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.
Yee Cheng Chin
So bottomline is, for @kd4ttc , you just need to do :set pythonthreedll=/opt/homebrew/Frameworks/Python.framework/Versions/3.10/Python and it should work without you needing to symlink anything. After the new release of MacVim, you should not need to do that anymore (since it will default to 3.10).
As for your comments about Python configuration pains, maybe others can chime in but I feel like it's kind of an old problem tbh, as it's always kind of annoying to have an external dependency that the user needs to install, and is why both Vim and Neovim are moving away from relying on builtin Python/Ruby/etc support. Maybe the error messages could be better, but I don't think we have a universal way to get this to "just works", but maybe we could make better detection mechanism like detecting Homebrew versions in macOS, apt for Debian, etc.
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.
Christian Brabandt
I am closing this, as this doesn't seem to be a vim bug.
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.
Christian Brabandt
Closed #11040 as completed.
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.
kd4ttc
Above comments noted. I’m going to modify my stackoverflow post to use the set pythonthreedll setting, but I’ll be recommending setting it to current. I’ll monitor vim and SLIMV and watch for glitches. If you could let me know (email me?) when you’re switching to 3.10 I’ll keep special watch.
I still think it’s a bug when you are ok with various Python versions but demand 3.9. Googling didn’t come up with an answer and I had to resort to a big report to get useful advice. That is, I appreciate the useful advice you gave, but not a lot of people will be going to these lengths.
I appreciate the Python problems. The user community is remarkably tolerant of Guido changing Python around. Back in the day (1970s-19980s) we expected more stability in language specs.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.
kd4ttc
Further testing confirns the following in the .vimrc file works:
set pythonthreedll=/opt/homebrew/Frameworks/Python.framework/Versions/3.10/Python
I see that the setting points to the Python binary. I know that when vim is hunting it looks into several of the brew directories. If brew isn't there I suppose the directory search pattern is different.
How does vim go about knowing where to look? Can one put in a series of directory in the .vimrc file to guide vim on the directory search?
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.
Yee Cheng Chin
As I have been trying to allude to, a lot of the behaviors you are asking about are specific to MacVim (a separate project), not Vim itself. For Vim, the Python binary path is specified when you build Vim yourself. For MacVim, that's hard-coded to /opt/homebrew/Frameworks/Python.framework/Versions/3.09/Python at build time when we make a release as mentioned (since CI for MacVim uses Homebrew to find Python to build against, and also Homebrew is the most popular option). The additional logic is in https://github.com/macvim-dev/macvim/blob/master/src/MacVim/vimrc where it just searches some common known paths:
if exists("&pythonthreedll") && exists("&pythonthreehome") && \ !filereadable(&pythonthreedll) if filereadable("/opt/local/Library/Frameworks/Python.framework/Versions/3.9/Python") " MacPorts python 3.9 set pythonthreedll=/opt/local/Library/Frameworks/Python.framework/Versions/3.9/Python elseif filereadable("/Library/Frameworks/Python.framework/Versions/3.9/Python") " https://www.python.org/downloads/mac-osx/ set pythonthreedll=/Library/Frameworks/Python.framework/Versions/3.9/Python endif endif
You can definitely do something similar in your own vimrc.
I still hesitate defaulting to searching other versions of Python, but I guess we could add that as a last resort to look for "Current" instead of 3.9 / 3.10. The issue is that there could be subtle issues that arise when you do something like this, and it won't be a blatant "Python is not installed" error.
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.
kd4ttc
When I download vim from Brew I get a binary installed and don’t have to compile it. Are the compile options the responsibility of brew or the vim team? Is the choice of python3.9 the vim team’s choice or the brew team’s choice?
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.
Yee Cheng Chin
Brew formulas are part of Homebrew. MacVim is compiled against 3.10 in https://github.com/Homebrew/homebrew-core/blob/master/Formula/macvim.rb, but as I said, the current release is looking for 3.9 in the vimrc. You could always just fix it using the pythonthreedll as I mentioned, and the next release of MacVim will automatically set it to do 3.10 anyway. So there's nothing needed to be fixed on brew side really.
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.