Pillow (or PIL) import issue with fresh 5.0.2 install

[Chris Richmond]

Finally trying out weewx 5, and running into python issues.  This is on a pi3 running

Devuan daedalus (Debian bookworm sans systemd).  I'd previously set up my own

stuff to run in a pyvenv, so that basic stuff worked OK.  Following the weewx install steps failed because I forgot about activating with activate.csh since I run tcsh.

Next, pillow failed to build because libjpeg headers were missing.  Fixed with:

apt install libjpeg-dev

Also, the instructions here: https://pillow.readthedocs.io/en/latest/installation.html

warned about having PIL and pillow installed, so I removed PIL from the non-pyvenv and started over (bcm2709_/root# apt remove python3-pil)

Not 100% sure now, but had to install wheel once sourced into the pyvenv:

(weewx) crichmon@bcm2709> python3 -m pip install wheel

Proceeded with: (weewx) crichmon@bcm2709> python3 -m pip install weewx

which ended with:

Successfully built Pillow
Installing collected packages: pyserial, ephem, six, pyusb, PyMySQL, Pillow, CT3, configobj, weewx
Successfully installed CT3-3.3.3 Pillow-10.2.0 PyMySQL-1.1.0 configobj-5.0.8 ephem-4.1.5 pyserial-3.5 pyusb-1.2.1 six-1.16.0 weewx-5.0.2

Created a Simulator station

Created the init.d script and weewxd was off to the races until it came time to make graphs.  Got this is the logs:
File "/usr/local/py.venv/weewx/lib/python3.11/site-packages/PIL/ImageFont.py", line 52, in <module>
 from PIL import _imagingft as core

I'd modified that import line from: "from . import" to "from PIL import" with no change.

From the pyvenv command line, this works:
bcm2709_/home/crichmon> source weewx/bin/activate.csh
(weewx) crichmon@bcm2709> python3
Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from PIL import ImageFont
>>> ImageFont
<module 'PIL.ImageFont' from '/usr/local/py.venv/weewx/lib/python3.11/site-packages/PIL/ImageFont.py'>
>>> 

Bug with newer python version or something I'm missing?

syslog attached.

Thx, Chris

[Tom Keffer]

For whatever reason, your copy of Pillow was compiled without freetype (that's what the "ft" stands for in _imagingft).

Try updating pip, then remove pillow, then reinstall.

If that doesn't work, try installing the freetype library first:

sudo apt install libfreetype6-dev

Then remove pillow, then reinstall.

If one of these works, let me know and I'll add it to our pip troubleshooting guide.

-tk

--
You received this message because you are subscribed to the Google Groups "weewx-development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-developm...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-development/bd9fc7d0-9bc6-4d0b-96c2-042674d04632n%40googlegroups.com.

[Chris Richmond]

For whatever reason, my email reply got rejected...

Excellent guess!  So…  I rarely use pip, so those bits took a few tries… 

Anyhow;

bcm2709_/root# apt install python3-pip -y

python3-pip is already the newest version (23.0.1+dfsg-1).

 

bcm2709_/root# apt install libfreetype6-dev

The following NEW packages will be installed:

  libbrotli-dev libfreetype-dev libfreetype6-dev libpng-dev libpng-tools

 

bcm2709_/root# exit

bcm2709_/home/crichmon> source weewx/bin/activate.csh

(weewx) bcm2709_/home/crichmon> python3 -m pip remove pillow

ERROR: unknown command "remove"

(weewx) bcm2709_/home/crichmon> python3 -m pip uninstall pillow

Found existing installation: pillow 10.2.0

Uninstalling pillow-10.2.0:

  Would remove:

    /usr/local/py.venv/weewx/lib/python3.11/site-packages/PIL/*

    /usr/local/py.venv/weewx/lib/python3.11/site-packages/pillow-10.2.0.dist-info/*

Proceed (Y/n)?

  Successfully uninstalled pillow-10.2.0

(weewx) bcm2709_/home/crichmon> python3 -m pip install pillow

Collecting pillow

  Using cached pillow-10.2.0-cp311-cp311-linux_armv7l.whl

Installing collected packages: pillow

Successfully installed pillow-10.2.0

(weewx) bcm2709_/home/crichmon> python3 -m pip uninstall pillow

Found existing installation: pillow 10.2.0

Uninstalling pillow-10.2.0:

  Would remove:

    /usr/local/py.venv/weewx/lib/python3.11/site-packages/PIL/*

    /usr/local/py.venv/weewx/lib/python3.11/site-packages/pillow-10.2.0.dist-info/*

Proceed (Y/n)?

  Successfully uninstalled pillow-10.2.0

 

(weewx) bcm2709_/home/crichmon> find .cache/pip/wheels/ -type f -ls

(weewx) bcm2709_/home/crichmon> rm -r .cache/pip/wheels/40

(weewx) bcm2709_/home/crichmon> python3 -m pip install pillow

  <normal download and build>

Successfully installed pillow-10.2.0

 

Restarted weewx and this was the previously broken part of the log:

2024-02-22T13:40:15.411903-07:00 bcm2709 crichmon[7216]: INFO weewx.manager: Added record 2024-02-22 13:40:00 MST (1708634400) to database 'weewx.sdb'

2024-02-22T13:40:15.458556-07:00 bcm2709 crichmon[7216]: INFO weewx.manager: Added record 2024-02-22 13:40:00 MST (1708634400) to daily summary in 'weewx.sdb'

2024-02-22T13:40:15.607406-07:00 bcm2709 crichmon[7216]: DEBUG weewx.reportengine: Running reports for latest time in the database.

2024-02-22T13:40:15.607850-07:00 bcm2709 crichmon[7216]: DEBUG weewx.reportengine: Running report 'SeasonsReport'

2024-02-22T13:40:15.648974-07:00 bcm2709 crichmon[7216]: DEBUG weewx.reportengine: Found configuration file /home/crichmon/weewx-data/skins/Seasons/skin.conf for report 'SeasonsReport'

2024-02-22T13:40:15.678583-07:00 bcm2709 crichmon[7216]: DEBUG weewx.reportengine: Running generators for report 'SeasonsReport' in directory '/home/crichmon/weewx-data/skins/Seasons'

2024-02-22T13:40:15.841151-07:00 bcm2709 crichmon[7216]: DEBUG weewx.cheetahgenerator: Using search list ['weewx.cheetahgenerator.Almanac', 'weewx.cheetahgenerator.Current', 'weewx.cheetahgenerator.DisplayOptions', 'weewx.cheetahgenerator.Extras', 'weewx.cheetahgenerator.Gettext', 'weewx.cheetahgenerator.JSONHelpers', 'weewx.cheetahgenerator.PlotInfo', 'weewx.cheetahgenerator.SkinInfo', 'weewx.cheetahgenerator.Station', 'weewx.cheetahgenerator.Stats', 'weewx.cheetahgenerator.UnitInfo']

2024-02-22T13:40:15.857779-07:00 bcm2709 crichmon[7216]: DEBUG weewx.manager: Daily summary version is 4.0

2024-02-22T13:40:19.310712-07:00 bcm2709 crichmon[7216]: INFO weewx.cheetahgenerator: Generated 8 files for report SeasonsReport in 3.49 seconds

2024-02-22T13:40:19.405933-07:00 bcm2709 crichmon[7216]: DEBUG weewx.manager: Daily summary version is 4.0

2024-02-22T13:40:23.482435-07:00 bcm2709 crichmon[7216]: INFO weewx.imagegenerator: Generated 60 images for report SeasonsReport in 4.09 seconds

2024-02-22T13:40:23.494414-07:00 bcm2709 crichmon[7216]: INFO weewx.reportengine: Copied 5 files to /home/crichmon/weewx-data/public_html

 

And all the .png files showed up in ~/weewx-data/public_html with the rest of the .html’s.

 

So, I checked several of my RPi’s, mostly running RPi-OS Bullseye, and it was a mix of what I found with:

apt search libfreetype6

 

So… it sounds like this needs to be a dependency check somewhere.  “apt search libfreetype6” certainly isn’t the fastest

to return results.  Hope that helps.

 

Thx as always, Chris

 

[Tom Keffer]

I'm a little mystified why libfreetype6 wasn't included in the Pillow wheel.

Could you check which wheel you used? You can see what's in the cache with

pip3 cache list

-tk

[Chris Richmond]

So... (from pyvenv):

(weewx) bcm2709_/home/crichmon> pip3 cache list
Cache contents:

 - CT3-3.3.3-cp311-cp311-linux_armv7l.whl (195 kB)
 - args-0.1.0-py3-none-any.whl (3.3 kB)
 - clint-0.5.1-py3-none-any.whl (34 kB)
 - ephem-4.1.5-cp311-cp311-linux_armv7l.whl (1.7 MB)
 - pillow-10.2.0-cp311-cp311-linux_armv7l.whl (1.0 MB)

My xterm history doesn't go back far enough to show the log when the pillow build failed.  It sounds like that

should have included getting libfreetype6 in the process.

As a reminder; this OS version or more likely python version doesn't support adding non-OS supported python

modules (again, error not in xterm history), so anything the OS doesn't support natively HAS to be in a pyvenv.

I'm guessing the only way to figure this out is to start from scratch.  I'm up for it if you think it's required.

And, FWIW, I tried this (outside pyvenv):

bcm2709_/home/crichmon> pip3 cache list
Cache contents:

 - CT3-3.3.3-cp311-cp311-linux_armv7l.whl (195 kB)
 - args-0.1.0-py3-none-any.whl (3.3 kB)
 - clint-0.5.1-py3-none-any.whl (34 kB)
 - ephem-4.1.5-cp311-cp311-linux_armv7l.whl (1.7 MB)
 - pillow-10.2.0-cp311-cp311-linux_armv7l.whl (1.0 MB)
bcm2709_/home/crichmon> sudo tcsh
[sudo] password for crichmon:
root@bcm2709> cd
bcm2709_/root# pip3 cache list
No locally built wheels cached.

Here's how pip3 fails outside the pyvenv:  (guessing this isn't news)

bcm2709_/home/crichmon> sudo pip3 install Adafruit_Blinka
error: externally-managed-environment

à This environment is externally managed
â°â> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.
   
    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.
   
    If you wish to install a non-Debian packaged Python application,
    it may be easiest to use pipx install xyz, which will manage a
    virtual environment for you. Make sure you have pipx installed.
   
    See /usr/share/doc/python3.11/README.venv for more information.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

Thx, Chris

[Tom Keffer]

The wheel pillow-10.2.0-cp311-cp311-linux_armv7l.whl is not listed on pypi (list). The processor "armv7" is a 32-bit processor, so I guess you're running a pretty old machine? If so, I'm not surprised that Pillow does not offer a prebuilt wheel.

So, I'm guessing that it was built on your machine, which would explain why the truetype library is not included until you installed it with apt.

I've included it in the troubleshooting guide: https://github.com/weewx/weewx/wiki/pip-troubleshooting#pillow-runtime-error

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-development/b02b57be-25b6-4970-93ad-2f33abbc3151n%40googlegroups.com.