Unable to instantiate generator

[Lucas Heijst]

Installed Weewx 5.0.0a17 (which show as a16) with pip3 on a raspberry pi with bullseye and configured for Simulator.

Got an error with ImageGenerator, see attachment.

My knowledge of Linux and Python is not much so probably I missed something during install.

Luc

[Vince Skahan]

Worked for me on a pi4 FWIW.

That library comes from Pillow, so perhaps that didn't install somehow.

Try "pip3 list --verbose" and look for the Pillow line.

Mine looks like:

            Pillow             9.4.0     /home/pi/.local/lib/python3.9/site-packages pip

[Lucas Heijst]

This is the list on my pi3b+

pip3 list ~verbose
Package       Version
------------- ---------
certifi       2020.6.20
chardet       4.0.0
colorzero     1.1
distro        1.5.0
gpiozero      1.6.2
idna          2.10
numpy         1.19.5
picamera2     0.3.8
pidng         4.0.9
piexif        1.1.3
Pillow        8.1.2
pip           20.3.4
python-apt    2.2.1
python-prctl  1.7
requests      2.25.1
RPi.GPIO      0.7.0
setuptools    52.0.0
simplejpeg    1.6.4
six           1.16.0
spidev        3.5
ssh-import-id 5.10
toml          0.10.1
urllib3       1.26.5
v4l2-python3  0.3.1
wheel         0.34.2

Op dinsdag 21 februari 2023 om 14:23:31 UTC-3 schreef vince...@gmail.com:

[Lucas Heijst]

Vince, I noticed on my system pip is not in directory site-packages

root@pi36:/# find -name pip
./root/.local/lib/python3.9/site-packages/wee_resources/docs/quickstarts/pip
./root/.cache/pip
./root/weewx-data/docs/quickstarts/pip
./usr/lib/python3/dist-packages/pip
./usr/bin/pip

root@pi36:/# ls -al /root/.local/lib/python3.9/site-packages
total 316
drwx------ 29 root root  4096 Feb 21 08:57 .
drwxr-xr-x  3 root root  4096 Feb 21 08:57 ..
drwxr-xr-x  8 root root  4096 Feb 21 08:57 Cheetah
drwxr-xr-x  3 root root  4096 Feb 21 08:57 configobj
drwxr-xr-x  2 root root  4096 Feb 21 08:57 configobj-5.0.8.dist-info
drwxr-xr-x  2 root root  4096 Feb 21 08:57 CT3-3.3.1.dist-info
drwxr-xr-x  5 root root  4096 Feb 21 08:57 ephem
drwxr-xr-x  2 root root  4096 Feb 21 08:57 ephem-3.7.7.0.dist-info
-rw-r--r--  1 root root 32472 Feb 21 08:57 LICENSE.txt
drwxr-xr-x  3 root root  4096 Feb 21 08:57 PIL
drwxr-xr-x  2 root root  4096 Feb 21 08:57 Pillow-9.4.0.dist-info
drwxr-xr-x  2 root root  4096 Feb 21 08:57 __pycache__
drwxr-xr-x  2 root root  4096 Feb 21 08:57 pyephem-3.7.7.0.dist-info
drwxr-xr-x  4 root root  4096 Feb 21 08:57 pymysql
drwxr-xr-x  2 root root  4096 Feb 21 08:57 PyMySQL-1.0.2.dist-info
drwxr-xr-x  2 root root  4096 Feb 21 08:57 pyserial-3.5.dist-info
drwxr-xr-x  2 root root  4096 Feb 21 08:57 pyusb-1.2.1.dist-info
-rw-r--r--  1 root root  3301 Feb 21 08:57 README.md
drwxr-xr-x  3 root root  4096 Feb 21 08:57 schemas
drwxr-xr-x  6 root root  4096 Feb 21 08:57 serial
drwxr-xr-x  4 root root  4096 Feb 21 08:57 usb
drwxr-xr-x  3 root root  4096 Feb 21 08:57 validate
drwxr-xr-x  3 root root  4096 Feb 21 08:57 weecfg
drwxr-xr-x  3 root root  4096 Feb 21 08:57 weectllib
-rwxr-xr-x  1 root root  1455 Feb 21 08:57 weectl.py
-rwxr-xr-x  1 root root 52419 Feb 21 08:57 wee_database.py
drwxr-xr-x  3 root root  4096 Feb 21 08:57 weedb
-rwxr-xr-x  1 root root 16464 Feb 21 08:57 wee_debug.py
-rwxr-xr-x  1 root root  2209 Feb 21 08:57 wee_device.py
drwxr-xr-x  3 root root  4096 Feb 21 08:57 weeimport
-rwxr-xr-x  1 root root 38964 Feb 21 08:57 wee_import.py
drwxr-xr-x  3 root root  4096 Feb 21 08:57 weeplot
-rwxr-xr-x  1 root root  5652 Feb 21 08:57 wee_reports.py
drwxr-xr-x  8 root root  4096 Feb 21 08:57 wee_resources
drwxr-xr-x  3 root root  4096 Feb 21 08:57 weeutil
drwxr-xr-x  4 root root  4096 Feb 21 08:57 weewx
drwxr-xr-x  2 root root  4096 Feb 21 08:57 weewx-5.0.0a17.dist-info
-rwxr-xr-x  1 root root  9640 Feb 21 08:57 weewxd.py
-rwxr-xr-x  1 root root 23621 Feb 21 08:57 wunderfixer.py

Op dinsdag 21 februari 2023 om 15:18:46 UTC-3 schreef Lucas Heijst:

[Tom Keffer]

Thanks for the report, Lucas.

I forgot to update the version numbers with 5.0.0a17, which is why it shows as a16. This has been corrected in a18.

I was able to reproduce your problem. It can be solved with

sudo apt install libopenjp2-7

but the library really should have been installed from the Pillow wheel. I can't quite figure out why that didn't happen.

A couple of comments:

- Bullseye comes with Pillow 8.1.2, while WeeWX requires >=8.2. So, pip should have installed a later version of Pillow when it did the weewx install.

- Because it looks like you did a "sudo pip install", you may have broken some system software. In particular, anything that depended on Pillow 8.1.2.

- In your "pip3 list" Pillow was not listed. Nor was weewx. Had you done an uninstall?

- When you use 'pip3 list', be sure to use sudo if that's how you installed weewx. Also, the flag is --verbose, not ~verbose. Here's what mine looks like after installing weewx:

sudo python3 -m pip list --verbose
Package       Version   Location                               Installer
------------- --------- -------------------------------------- ---------
certifi       2020.6.20 /usr/lib/python3/dist-packages
chardet       4.0.0     /usr/lib/python3/dist-packages
colorzero     1.1       /usr/lib/python3/dist-packages
configobj     5.0.8     /usr/local/lib/python3.9/dist-packages pip
CT3           3.3.1     /usr/local/lib/python3.9/dist-packages pip
distro        1.5.0     /usr/lib/python3/dist-packages
ephem         3.7.7.0   /usr/local/lib/python3.9/dist-packages pip
gpiozero      1.6.2     /usr/lib/python3/dist-packages
idna          2.10      /usr/lib/python3/dist-packages
numpy         1.19.5    /usr/lib/python3/dist-packages
picamera2     0.3.8     /usr/lib/python3/dist-packages
pidng         4.0.9     /usr/lib/python3/dist-packages
piexif        1.1.3     /usr/lib/python3/dist-packages
Pillow        9.4.0     /usr/local/lib/python3.9/dist-packages pip
pip           20.3.4    /usr/lib/python3/dist-packages
pyephem       3.7.7.0   /usr/local/lib/python3.9/dist-packages pip
PyMySQL       1.0.2     /usr/local/lib/python3.9/dist-packages pip
pyserial      3.5       /usr/local/lib/python3.9/dist-packages pip
python-apt    2.2.1     /usr/lib/python3/dist-packages
python-prctl  1.7       /usr/lib/python3/dist-packages
pyusb         1.2.1     /usr/local/lib/python3.9/dist-packages pip
requests      2.25.1    /usr/lib/python3/dist-packages
RPi.GPIO      0.7.0     /usr/lib/python3/dist-packages
setuptools    52.0.0    /usr/lib/python3/dist-packages
simplejpeg    1.6.4     /usr/lib/python3/dist-packages
six           1.16.0    /usr/lib/python3/dist-packages
spidev        3.5       /usr/lib/python3/dist-packages
ssh-import-id 5.10      /usr/lib/python3/dist-packages
toml          0.10.1    /usr/lib/python3/dist-packages
urllib3       1.26.5    /usr/lib/python3/dist-packages
v4l2-python3  0.3.1     /usr/lib/python3/dist-packages
weewx         5.0.0a18  /usr/local/lib/python3.9/dist-packages pip
wheel         0.34.2    /usr/lib/python3/dist-packages

More later.

--
You received this message because you are subscribed to the Google Groups "weewx-development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-developm...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-development/234260c2-812b-4b75-a6c4-5ff8f3d4050an%40googlegroups.com.

[Tom Keffer]

On Tue, Feb 21, 2023 at 10:55 AM Lucas Heijst <ljm.h...@gmail.com> wrote:

Vince, I noticed on my system pip is not in directory site-packages

How did you install pip? It does not come with bullseye. If you did "sudo apt install python3-pip" it will go into /usr/lib/python3/dist-packages, not /root/.local, nor /usr/local/lib.

-tk

[Lucas Heijst]

Tom, I installed pip with

sudo apt install gcc python3-dev python3-pip

sudo apt install libopenjp2-7 indeed solved my problem:
Feb 21 16:16:34 pi36 weewx[27305] INFO weewx.imagegenerator: Generated 60 images for report SeasonsReport in 48.30 seconds

Op dinsdag 21 februari 2023 om 16:03:03 UTC-3 schreef Tom Keffer:

[Vince Skahan]

On a current RaspiOS both 'pip' and 'pip3' are installed by the python3-pip dpkg according to "dpkg -S /usr/bin/pip3 /usr/bin/pip"

I reflashed a SD to current RaspiOS and installed 5.0.0a18 via pip --user and also recreated the issue.   Adding the libopenjp2-7 package fixed it here too.

FWIW, I had an old debian_version 11.2 pi3 SD card with Pillow 8.1 that did work ok with the v5 software, but I previously had an old v4 setup.py installation there.  My 'guess' is that old Pillow version was current at that point in time many months ago so that's why it was there.  Regardless, installing v5 on the ancient SD didn't upgrade Pillow from 8.x to 9.x if that was what was supposed to happen.

A clean RaspiOS and clean weewx os installs and runs ok, once that libopenjp2-7 package is installed on a pi3.

[Tom Keffer]

Same conclusions here. The mystery is why Pillow didn't install an appropriate version of libopenjp2. 

It seems like a fundamental problem, yet if I search the web I can only find solutions that are inappropriate or don't work. For example, one suggested removing PIL first, then installing Pillow, which I did. No relief.

--

You received this message because you are subscribed to the Google Groups "weewx-development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-developm...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-development/227ddb6e-657f-4839-80dd-7ac48724ef12n%40googlegroups.com.

[Tom Keffer]

One other comment. It's super easy to recreate the problem. No need to install WeeWX. Just do

python3 -c "from PIL import Image"

[Lucas Heijst]

Tom and Vince,

This is how I installed bulleye, pip and weewx:

I used PI imager to write bullseye to an USB stick, setup for user wx and for my wifi

After I logged in with user wx the following sequence was done:
sudo su
sudo passwd root
sudo apt update && sudo apt upgrade

sudo apt install gcc python3-dev python3-pip

sudo apt install sshpass nginx  (extra)
sudo apt autoremove (one not longer used package)
pip3 install wheel  (was already present)
pip3 install weewx --user  (version a17 presented as a16)

Luc

Op dinsdag 21 februari 2023 om 16:42:51 UTC-3 schreef Tom Keffer:

[Tom Keffer]

I was able to solve this problem by simply relaxing the version requirement for Pillow. Instead of >=8.2, I set it to >=5.2, the first version that supports Python 3.7. This will cause slight alignment problems in the images, but I don't think it will be anything serious.

So now weewx can use the native version of Pillow, and nothing is broken. 

In v5.0.0a19.

-tk

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-development/25938c44-535e-4f3c-9b0b-efdb2198ee1fn%40googlegroups.com.

[Vince Skahan]

This one might be a pi oddity.   If you install the python3-willow dpkg, it will install a dpkg variant of Pillow 8.1.2 and the pip --user installation works ok.

FWIW, this issue does not appear on vagrant/virtualbox debian-11 or fedora-37.

--

-----  vince...@gmail.com ----

[Tom Keffer]

Well, shoot. Now I remember why I required Pillow >=8.2. You get a mysterious segfault with earlier versions. The problem was fixed in Pillow 8.2.

See https://github.com/python-pillow/Pillow/issues/3066

However, there is a workaround, provided no one is trying to use a font that requires a vertical layout. I think we're safe there. v5.0.0a20 uses the workaround.