venv vs pipx on 0.56

[kaydeeH]

Hey gang,

I'm trying to migrate a game from 0.55 to 0.56 and it looks like installation has changed a lot. It used to be "make a virtual environment and install everything in that", but now it's "yay pipx". I'm having trouble getting 0.56 working, and now I'm wondering if it's because pipx and my former venv don't play well together. So I guess my question is, now that we have pipx for installation, should I be forgetting about running a venv, or am I missing something simple?

Thanks in advance,

Kaydee

[Brian Madden]

Good questions! You certainly don't have to use pipx for 0.56. I only learned about pipx when I was doing the new installers last summer, and I kind of got the feeling that "this is the way" moving forward for python package management. My understanding of pipx is that it's a wrapper which combines pip, venv, and pyenv (or something like that?) into a single and easier to use interface. (And I think it actually uses all three of those under the hood and just sort of coordinates everything?)

Personally I just use old fashioned pip and manually installed Python, but I somewhat know what I'm doing so can untangle things when I break them. I was thinking pipx would keep things easier for people who don't want to dive into the inner workings of python installations.

So, short answer, yes, pipx can let you forget about venv. But if you also just want to use venv + pip for MPF 0.56+ that's ok too.

I would also love any feedback on this process or the docs. Is what we recommend the right approach? Are there easier ways? I'm open to suggestions!

Thanks!

Brian

[kaydeeH]

Thanks for the reply! Got it working without using my old venv. Had to uninstall/reinstall Python/pip/pipx because I had too many older versions going and things were a mess, but once I did that, it all worked. One issue lingers though...

If I start the game with "mpf both -X", the MPF main window opens and goes to the text layout of switches, etc. and says it's waiting for the MC connection. The MC window opens for a few seconds, then closes, never to return or start and the MPF window just sits there waiting for the MC to connect. I don't see any errors in the MPF or MC logs. It just... doesn't work.

However, if I run the game as just "mpf -X", then separately run "mpf mc" in another console, the MC opens up and starts just fine.

I've replicated this over and over and it behaves this way every single time. I mean, I can just change the way I run it, but I was thinking "both" should still work, right? Thoughts?

[kaydeeH]

Okay, now I'm trying to get this working on Xubuntu 20. After some fiddling, I'm up on Python 3.9.13, I have MPF 0.56 and MPF 0.56.1 installed via pipx. I have my game installed. MPF starts just fine. The Media Controller on the other hand does not. I get the following errors:

mpfmc: Mission Pinball Framework Media Controller v0.56.1
mpfmc: Mission Pinball Framework Game Engine v0.56.0
mpfmc: Machine path: /home/jake/OutpostKodelia-DoH
mpfmc: Starting clock at 60.0Hz
kivy: Text: Provider: sdl2
kivy: Window: Provider: x11(['window_sdl2'] ignored)
kivy: Window: Unable to find any valuable Window provider. Please enable debug logging (e.g. add -d if running from the command line, or change the log level in the config) and re-run your app to identify potential causes
sdl2 - ModuleNotFoundError: No module named '_ctypes'
  File "/home/jake/.local/pipx/venvs/mpf/lib/python3.9/site-packages/kivy/core/__init__.py", line 59, in core_select_lib
    mod = importlib.__import__(name='{2}.{0}.{1}'.format(
  File "<frozen importlib._bootstrap>", line 1109, in __import__
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/home/jake/.local/pipx/venvs/mpf/lib/python3.9/site-packages/kivy/core/window/window_sdl2.py", line 28, in <module>
    from kivy.core.window._window_sdl2 import _WindowSDL2Storage
  File "kivy/core/window/_window_sdl2.pyx", line 1, in init kivy.core.window._window_sdl2
  File "/usr/local/lib/python3.9/ctypes/__init__.py", line 8, in <module>
    from _ctypes import Union, Structure, Array

x11 - ValueError: invalid literal for int() with base 10: 'False'
  File "/home/jake/.local/pipx/venvs/mpf/lib/python3.9/site-packages/kivy/core/__init__.py", line 71, in core_select_lib
    cls = cls()
  File "/home/jake/.local/pipx/venvs/mpf/lib/python3.9/site-packages/kivy/core/window/__init__.py", line 1071, in __init__
    self.create_window()
  File "kivy/core/window/window_x11.pyx", line 161, in kivy.core.window.window_x11.WindowX11.create_window
  File "/usr/local/lib/python3.9/configparser.py", line 818, in getint
    return self._get_conv(section, option, int, raw=raw, vars=vars,
  File "/usr/local/lib/python3.9/configparser.py", line 808, in _get_conv
    return self._get(section, conv, option, raw=raw, vars=vars,
  File "/usr/local/lib/python3.9/configparser.py", line 803, in _get
    return conv(self.get(section, option, **kwargs))

I did see something about installing these dependencies, but this uses pip not pipx so I figured it wasn't going to work.

python -m pip install docutils pygments pypiwin32 kivy.deps.sdl2 kivy.deps.glew;

 Any thoughts on how to run that command in a pipx world, or what else I should be looking into? I'm desperately trying to get this running today or early tomorrow, so any help is greatly appreciated!

Thanks,

Kaydee

[kaydeeH]

Since I posted, I've tried adding any dependency I could find suggested, but nothing helped. I tried dumping the pipx mpf and creating a traditional venv and installed everything there. Same exact error. So I tried uninstalling mpf/mc and installing version 0.55. Whole different set of errors, MPF can't even start to the point of writing a log out. So I uninstalled MPF again, and installed it with the zip installer package. The installer itself threw a similar error to what I was seeing in the 0.56.1 MC. So now I'm wondering if the Python 3.9.13 install had some issues. I know it gave me some grief but I thought I had gotten it up and running correctly. So I'm going to try a clean install of the OS, go with Python 3.8 since it's preinstalled and worked in the past with 0.55. More to come...

[Brian Madden]

That error is coming from kivy, (which is what MPF-MC uses as its multimedia interface). There are some simple kivy tests you can do. (I forget off the top of my head, run "python" and then it's something like "import kivy" (or something like that to import the kivy window library).

So you might try the kivy docs, support, or user forum? If you get kivy working, MPF-MC will work.

[kaydeeH]

I think I replied to author, not all on my last response, so here it is again for public view (to the best of my ability to recreate it):

After going round and round in circles with different MPF versions and installation techniques, I finally got past that error. Now I'm getting the following from MC:

2023-03-18 12:02:47,100 : SoundAsset : Load sound doh_track0 failed due to an exception - Unable to create a GStreamer pipeline: code=1 message=b'no element "uridecodebin"'
2023-03-18 12:02:47,747 : mpfmc : Shutting down due to child thread crash
2023-03-18 12:02:47,748 : mpfmc : Crash details: Traceback (most recent call last):
  File "/home/jake/.local/lib/python3.8/site-packages/mpfmc/assets/sound.py", line 618, in do_load
    self._container = self.machine.sound_system.audio_interface.load_sound_file_for_streaming(self.file)
  File "mpfmc/core/audio/audio_interface.pyx", line 705, in mpfmc.core.audio.audio_interface.AudioInterface.load_sound_file_for_streaming
  File "mpfmc/core/audio/sound_file.pyx", line 148, in mpfmc.core.audio.sound_file.SoundStreamingFile.__init__
  File "mpfmc/core/audio/sound_file.pyx", line 267, in mpfmc.core.audio.sound_file.SoundStreamingFile.load
  File "mpfmc/core/audio/sound_file.pyx", line 224, in mpfmc.core.audio.sound_file.SoundStreamingFile._construct_pipeline
mpfmc.core.audio.audio_exception.AudioException: Unable to create a GStreamer pipeline: code=1 message=b'no element "uridecodebin"'

So something isn't right in gstreamer. I'm on Python 3.8 at the moment. Could that be the issue? The MPF docs disagree with themselves about Python support. Says 0.56 only works on Python 3.9 in one place, but says 3.8 is good in another. I suspect I'm missing a dependency related to gstreamer, but not sure what to install to fix it.

[kaydeeH]

And for what it's worth, I've been through the Kivy install guide here and installed all the dependencies they recommended, including the gstreamers.

https://kivy.org/doc/stable/installation/installation-linux.html#dependencies-with-sdl2

[Brian Madden]

I'm pretty sure the MPF-MC runs on Python 3.7-3.9. The caveat is the automated tests do not actually test audio. (The cloud runners that do the builds and tests don't have sound cards, or something?)

I have absolutely no idea about that error, I don't recall seeing it ever. Those sound_file.pyx code references are from MPF (e.g. it's our code, not Kivy), but I don't know if that's an issue with the building or compiling, or Gstreamer, or the sound file itself? I wouldn't even know where to begin, so I won't be much help here. :(

[kaydeeH]

It's definitely not the audio files themselves. Not only did they work before, but I opened them up in the local media player and they played fine. They also played fine (in fact all of MPF 0.56 works great) on Windows. Just struggling with Xubuntu 20.04.

[kaydeeH]

Okay, here's where I stand.

1. Clean install of Xubuntu 20.04 (same Xubuntu I was using before) on an Up Squared Pro board (intel based, same system that worked before)
2. sudo apt-get update && sudo apt-get upgrade
3. Install all the packages and dependencies Kivy claims to need here:  https://kivy.org/doc/stable/installation/installation-linux.html#dependencies-with-sdl2 (if I skip this, it doesn't seem to have any impact on 0.56, but 0.55 won't install without it)
4. Install python3.8-venv on top of the existing Python 3.8 that came with Xubuntu.
5. Create a new venv and activate it
6. Update pip and setuptools to latest
7. Use pip to install MPF-MC 0.56.1 (auto installs MPF 0.56.0 along with it; if I install MPF first then MC, I get the error from this earlier message https://groups.google.com/g/mpf-users/c/bcAAxTcDwMI/m/JjJvm7HRBAAJ ).
8. Add my user to the dialout group so MPF can access the USB ports.
9. Install git and clone down my machine files (same files that worked before)
10. Run MPF - logs seem to indicate that it's starting up succesfully but it doesn't show the familiar text interface with all the switches/etc.
11. Run MC - I get the sound pipeline error.

So then I try to get 0.55 working.

1. Deactivate previous venv.
2. Create a new venv for 0.55 and activate it
3. Update pip and setuptools to latest
4. Use pip to install MPF and MPF-MC
5. Run MPF and/or MPF-MC, the following error is thrown:

Traceback (most recent call last):

  File "/home/jake/mpfenv55/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3105, in _dep_map
    return self.__dep_map
  File "/home/jake/mpfenv55/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2901, in __getattr__
    raise AttributeError(attr)
AttributeError: _DistInfoDistribution__dep_map

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "/home/jake/mpfenv55/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/requirements.py", line 35, in __init__
    parsed = parse_requirement(requirement_string)
  File "/home/jake/mpfenv55/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/_parser.py", line 64, in parse_requirement
    return _parse_requirement(Tokenizer(source, rules=DEFAULT_RULES))
  File "/home/jake/mpfenv55/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/_parser.py", line 82, in _parse_requirement
    url, specifier, marker = _parse_requirement_details(tokenizer)
  File "/home/jake/mpfenv55/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/_parser.py", line 120, in _parse_requirement_details
    specifier = _parse_specifier(tokenizer)
  File "/home/jake/mpfenv55/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/_parser.py", line 209, in _parse_specifier
    tokenizer.consume("WS")
  File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__
    next(self.gen)
  File "/home/jake/mpfenv55/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/_tokenizer.py", line 183, in enclosing_tokens
    self.raise_syntax_error(
  File "/home/jake/mpfenv55/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/_tokenizer.py", line 163, in raise_syntax_error
    raise ParserSyntaxError(
pkg_resources.extern.packaging._tokenizer.ParserSyntaxError: Expected closing RIGHT_PARENTHESIS
    sys-platform (=="win32") ; extra == 'angle'

Now the really strange part here... if I uninstall ONLY MPF-MC from this venv, suddenly MPF starts working. But with MPF-MC also installed, they both crash out as above.

So I'm dead in the water, with TPF coming up fast and the game heading out tomorrow. I can't find anything useful about any of these errors.

What am I doing wrong? I just need either environment working. Is there another approach someone would recommend (a different OS, different steps, different order of steps)? I'm desperate here, and it's incredibly frustrating because a year ago I had no problem getting this same setup up and running. :(

Thanks again,
Kaydee

[kaydeeH]

FYI, I have started over on a LattePanda Delta (v2) running Windows 10. 0.56.0 works the same as it did on my other windows PC, so the Kivy sound issue above seems to be some sort of bug/incompatibility in MPF-MC on debian (xubuntu), at least at first glance.

But I noticed we have a problem with the widgets when migrating our game to 0.56, so I tried to roll back to known good 0.55.0 and python 3.7. After downgrading Python from 3.9 to 3.7, I used the requirements.txt file we created back when things worked to restore everything to the known good MPF/MPF-MC/MPF-MONITOR versions 0.55.0 and......... I STILL get the error I last reported. So... different computer, different OS, same versions of MPF 0.55.0 that used to work, but now they don't work. Something is now preventing me from getting back to a stable 0.55.0 environment, even when everything is freshly installed. Once again, that error below (and this has now appeared on both Xubuntu and Windows 10, so there's something to it):

[kaydeeH]

If anyone else can try to recreate this on a clean Xubuntu 20 (or 22, both failed for me in the same way), I would really appreciate it. It's possible I'm just missing some undocumented step. Or there's a problem in the MPF/MC codebase. A second set of eyes would really help to narrow this down.

[kaydeeH]

Update: I have 0.55.0 working again! The problem was Python 3.8. I dropped back to Python 3.6 and it worked. So at least I can get to a version that works for me. Our game just isn't behaving properly with 0.56, so this is a relief. :) Still can't figure out the 0.56 on Xubuntu issues but I've been focused on Windows for the moment. 

[Brian Madden]

Oh good news! 🎉

Keep us posted if you learn anything more about 0.56. The MC is a beast to build and I'm not 100% convinced it's even building properly across all platforms, but also I'm in over my head and don't know how to proceed, so anything you figure out would be great.

[kaydeeH]

I'm going to try setting up a VM to help me iterate/experiment faster until I can hopefully pin something down.

[Todd Legere]

kaydeeH

I have 0.56 running on my Ubuntu 20.04  and Xbuntu and my virtual linux machine labs.

I am using python3.8.10

I don't use pipx nor do I use venv.

Had no problem going my existing code from 0.55 to 0.56.

What seems to be the issues you are seeing ?

[Todd Legere]

My Xbuntu is Xbuntu-20.04.04-desktop

After a fresh install of either Ubuntu or Xubuntu version 20.04 I downloaded mpf-debian-installer-0.55.x.zip

cd ~/Download

unzip mpf-debian-installer-0.55.x.zip

cd mpf-debian-installer-0.55.x

chmod +x install
sudo ./install
mpf --version
pip3 list
pip3 list |grep mpf
pip3 install mpf-monitor/  <--- I use this but optional

If you encounters errors or problems

pip3 uninstall mpf mpf-mc mpf-monitor

pip3 install mpf mpf-mc mpf-monitor

[Ralf G.]

(1st version broke my installation, so here's the 2nd version)

Hi,

I wanted to share my hard-earned success installing MPF 0.56 on a fresh 22.04 Xubuntu minimal installation:

MPF 0.56:

Download https://github.com/missionpinball/mpf/archive/refs/heads/0.56.x.zip to e.g ~/mpf
cd ~/mpf
unzip mpf-0.56.x.zip
cd 0.56.x
sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt-get update
sudo apt-get install python3.9
sudo apt-get install python3-pip
sudo apt-get install curl
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
sudo apt-get install python3.9-distutils
python3.9 get-pip.py
export PATH="/home/<yourusername>/.local/bin:$PATH"
pip install --upgrade pip setuptools wheel build
pip install -e .

MPF-MC 0.56:

Download https://github.com/missionpinball/mpf-mc/archive/refs/heads/0.56.x.zip to e.g ~/mpf-mc
cd ~/mpf-mc
unzip mpf-mc-0.56.x.zip
cd mpf-mc-0.56.x
sudo apt-get install git
git clone --recursive --branch 0.56.x https://github.com/missionpinball/mpf.git _mpf
pip install -e _mpf
sudo apt-get update
sudo apt-get -y install libunwind-dev libsdl2-dev libsdl2-image-dev libsdl2-mixer-dev gstreamer1.0-plugins-base gstreamer1.0-plugins-base gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly libgstreamer1.0-dev
pip install --upgrade setuptools wheel build
sudo apt-get install virtualenv
python3.9 -m build --sdist
cd dist
tar -xvzf mpf-mc-0.56.1.tar.gz
cd mpf-mc-0.56.1
pip3 install mpf_mc

Good luck!

Ralf

[grastle flowpat]

Thank you

[Aziz Pinball]

Ralf, thank you so much! This the first time ever that I managed to install MPF and MPF-MC flawlessly. To be fair, also the first time I used a completely clean Ubuntu installation. :)

[Aziz Pinball]

Hey,

is this -> https://groups.google.com/g/mpf-users/c/B3gC4EiwLqM a result of this kind of installation process, or is it just something on my side?

Thanks,

Sebastian

[Avery Tummons]

Your question was answered on that thread. You need to install with the all parameter. It was changed so that the text_ui is not installed by default. Since you didn’t do all, you don’t have the tool installed, and thus doesn’t show. Just install again with [all] and it should work.

Sent from my iPhone

On Oct 1, 2023, at 11:41 AM, Aziz Pinball <azizp...@posteo.de> wrote:



--
You received this message because you are subscribed to the Google Groups "MPF Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mpf-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mpf-users/45f4b491-ea9e-4732-a914-be5b3e356d8en%40googlegroups.com.

[Aziz Pinball]

Yep, it worked, thank you. I just wanted to make sure I don't break my MPF installation by doing that, because I did not install it the "standard" way.

[kaydeeH]

Hey there Ralf... thanks so much for this writeup. Everything was good up until literally the last step. When I run "pip3 install mpf_mc" from the untar'd file in dist, it goes out and installs mpf-mc 0.57.x from the internet instead of installing the locally built 0.56 files. It also upgrades mpf to 0.57 at the same time. What am I missing? Prior to running this, pip list shows mpf 0.56 installed from the local build, but after this, it's been replaced by the standard 0.57 build. Thoughts? I tried running "pip3 install -e ." just like you had us do on the mpf build, but that just errors out. Thanks in advance!

[Anthony van Winkle]

You're close! The original instructions missed the -e on the mc install, the -e means that you're going to point to a folder with the package. So if you are inside the extracted mpf-mc folder, it would be pip3 install -e . (with that trailing period meaning "the current folder"). If you are in the parent folder it would be pip3 install -e mpf-mc-0.56.1

Hope that helps!

[kaydeeH]

Thanks for the quick response Anthony! That worked. Ish... The command worked but the pip install of the local mpf-mc threw a million Cython errors of all sorts. Looks like I'm missing files in the .../dist/mpf_mc-0.56.2/mpfmc/core/audio folder, at least. I probably missed or fat-fingered a package in the following... I'll mess around with it after I have the kids fed. :)

sudo apt-get -y install libunwind-dev libsdl2-dev libsdl2-image-dev libsdl2-mixer-dev gstreamer1.0-plugins-base gstreamer1.0-plugins-base gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly libgstreamer1.0-dev

[kaydeeH]

Sadly, that wasn't it. All of those dependencies were met. Any thoughts on why I'm missing files in the audio subfolder?

[Avery Tummons]

I ran into this long ago. Or at least similar. Pip uninstall the packages. Specifically mpf-mc, then rerun pip list. Even though you installed the -e, the other is likely still there. Repeat uninstall and pop list until it doesn’t show up. Took me 3 uninstalls. Then install -e for the correct way and those errors will hopefully go away.

Sent from my iPhone

On Jun 28, 2024, at 9:12 PM, kaydeeH <kayde...@gmail.com> wrote:

Sadly, that wasn't it. All of those dependencies were met. Any thoughts on why I'm missing files in the audio subfolder?

--

You received this message because you are subscribed to the Google Groups "MPF Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mpf-users+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mpf-users/fa39b21d-6baf-4323-9c06-658211d80920n%40googlegroups.com.

[Aziz Pinball]

So this still works in 2024, however my problem is: After I do the very last step of the mpf-mc installation. It changes the mpf version from 0.56.2 to 0.57.3. How can I prevent that? Or how can I go back to 0.56.2 once that happend?

Thanks for your help,

-Seb

On Tuesday, June 13, 2023 at 8:30:33 PM UTC+2 Ralf G. wrote:

[Aziz Pinball]

Oh dang, I did not read the rest of the thread that answers my question, apologies! However, I also run into the cython errors and I'm not sure what to make of @atum 's suggestion. If mpf-mc won't even install, how would I uninstall it? And what does pip list do there?

Thanks for your help.

-Seb