Python 3.7.3 to 3.8.5 upgrade problem

[jerry...@gmail.com]

I upgraded Python to 3.8.5, reinstalled Weewx.  Python -V is OK. Davis loop data is ok, but now seeing errors in image generation.  Pillow and other dependencies were reinstalled.  What am I missing?  Or was Python 3.8.5 a mistake.  I have dumped and cleared memory on the Vantage Pro.

weewx[89452] INFO __main__: Initializing weewx version 4.1.1

weewx[89452] INFO __main__: Using Python 3.8.5 (default, Jul 21 2020, 10:48:26) 

[Clang 11.0.3 (clang-1103.0.32.62)]

weewx[89452] INFO __main__: Platform macOS-10.15.6-x86_64-i386-64bit

weewx[89452] INFO __main__: Locale is 'C/UTF-8/C/C/C/C'

weewx[89452] INFO __main__: Using configuration file /Users/rycweather/weewx/weewx.conf

weewx[89452] INFO __main__: Debug is 1

weewx[89452] DEBUG __main__: Initializing engine

weewx[89452] INFO weewx.engine: Loading station type Vantage (weewx.drivers.vantage)

weewx[89452] DEBUG weewx.drivers.vantage: Driver version is 3.2.1

weewx[89452] DEBUG weewx.drivers.vantage: Opened up ethernet host 192.168.145.106 on port 22222. timeout=4.0, tcp_send_delay=0.5

weewx[89452] DEBUG weewx.drivers.vantage: Gentle wake up of console successful

weewx[89452] DEBUG weewx.drivers.vantage: Hardware type is 16

weewx[89452] DEBUG weewx.drivers.vantage: ISS ID is 3

weewx[89452] DEBUG weewx.drivers.vantage: Hardware name: Vantage Pro2

weewx[89452] DEBUG weewx.engine: Loading service weewx.engine.StdTimeSynch

weewx[89452] DEBUG weewx.engine: Finished loading service weewx.engine.StdTimeSynch

weewx[89452] DEBUG weewx.engine: Loading service weewx.engine.StdConvert

weewx[89452] INFO weewx.engine: StdConvert target unit is 0x1

weewx[89452] DEBUG weewx.engine: Finished loading service weewx.engine.StdConvert

weewx[89452] DEBUG weewx.engine: Loading service weewx.engine.StdCalibrate

weewx[89452] DEBUG weewx.engine: Finished loading service weewx.engine.StdCalibrate

weewx[89452] DEBUG weewx.engine: Loading service weewx.engine.StdQC

weewx[89452] DEBUG weewx.engine: Finished loading service weewx.engine.StdQC

weewx[89452] DEBUG weewx.engine: Loading service weewx.wxservices.StdWXCalculate

weewx[89452] DEBUG weewx.manager: Daily summary version is 2.0

weewx[89452] INFO weewx.wxservices: The following values will be calculated: pressure=prefer_hardware, barometer=prefer_hardware, altimeter=prefer_hardware, windchill=prefer_hardware, heatindex=prefer_hardware, dewpoint=prefer_hardware, inDewpoint=prefer_hardware, rainRate=prefer_hardware, maxSolarRad=prefer_hardware, cloudbase=prefer_hardware, humidex=prefer_hardware, appTemp=prefer_hardware, ET=prefer_hardware, windrun=prefer_hardware

weewx[89452] INFO weewx.wxservices: The following algorithms will be used for calculations: altimeter=aaASOS, maxSolarRad=RS

weewx[89452] DEBUG weewx.engine: Finished loading service weewx.wxservices.StdWXCalculate

weewx[89452] DEBUG weewx.engine: Loading service weewx.engine.StdArchive

weewx[89452] INFO weewx.engine: Archive will use data binding wx_binding

weewx[89452] INFO weewx.engine: Record generation will be attempted in 'hardware'

weewx[89452] INFO weewx.engine: Using archive interval of 300 seconds (specified by hardware)

weewx[89452] DEBUG weewx.engine: Use LOOP data in hi/low calculations: 1

weewx[89452] DEBUG weewx.engine: Finished loading service weewx.engine.StdArchive

weewx[89452] DEBUG weewx.engine: Loading service weewx.restx.StdStationRegistry

weewx[89452] INFO weewx.restx: StationRegistry: Station will be registered.

weewx[89452] DEBUG weewx.engine: Finished loading service weewx.restx.StdStationRegistry

weewx[89452] DEBUG weewx.engine: Loading service weewx.restx.StdWunderground

weewx[89452] DEBUG weewx.restx: WU essentials: {}

weewx[89452] INFO weewx.restx: Wunderground-PWS: Data for station KCACALIF93 will be posted

weewx[89452] DEBUG weewx.engine: Finished loading service weewx.restx.StdWunderground

weewx[89452] DEBUG weewx.engine: Loading service weewx.restx.StdPWSweather

weewx[89452] INFO weewx.restx: PWSweather: Posting not enabled.

weewx[89452] DEBUG weewx.engine: Finished loading service weewx.restx.StdPWSweather

weewx[89452] DEBUG weewx.engine: Loading service weewx.restx.StdCWOP

weewx[89452] INFO weewx.restx: CWOP: Posting not enabled.

weewx[89452] DEBUG weewx.engine: Finished loading service weewx.restx.StdCWOP

weewx[89452] DEBUG weewx.engine: Loading service weewx.restx.StdWOW

weewx[89452] INFO weewx.restx: WOW: Posting not enabled.

weewx[89452] DEBUG weewx.engine: Finished loading service weewx.restx.StdWOW

weewx[89452] DEBUG weewx.engine: Loading service weewx.restx.StdAWEKAS

weewx[89452] INFO weewx.restx: AWEKAS: Posting not enabled.

weewx[89452] DEBUG weewx.engine: Finished loading service weewx.restx.StdAWEKAS

weewx[89452] DEBUG weewx.engine: Loading service user.windy.Windy

weewx[89452] DEBUG weewx.manager: Daily summary version is 2.0

weewx[89452] INFO user.windy: version is 0.7

weewx[89452] INFO user.windy: Data will be uploaded to https://stations.windy.com/pws/update

weewx[89452] DEBUG weewx.engine: Finished loading service user.windy.Windy

weewx[89452] DEBUG weewx.engine: Loading service weewx.engine.StdPrint

weewx[89452] DEBUG weewx.engine: Finished loading service weewx.engine.StdPrint

weewx[89452] DEBUG weewx.engine: Loading service weewx.engine.StdReport

weewx[89452] DEBUG weewx.engine: Finished loading service weewx.engine.StdReport

weewx[89452] INFO __main__: Starting up weewx version 4.1.1

weewx[89452] DEBUG weewx.manager: Daily summary version is 2.0

weewx[89452] DEBUG weewx.drivers.vantage: Gentle wake up of console successful

weewx[89452] INFO weewx.engine: Clock error is -3.70 seconds (positive is fast)

weewx[89452] INFO weewx.engine: Using binding 'wx_binding' to database 'weewx.sdb'

weewx[89452] INFO weewx.manager: Starting backfill of daily summaries

weewx[89452] DEBUG weewx.drivers.vantage: Getting archive packets since 2020-07-31 12:30:00 PDT (1596223800)

weewx[89452] DEBUG weewx.drivers.vantage: Gentle wake up of console successful

weewx[89452] DEBUG weewx.drivers.vantage: Retrieving 1 page(s); starting index= 1

weewx[89452] INFO weewx.manager: Added record 2020-07-31 12:35:00 PDT (1596224100) to database 'weewx.sdb'

weewx[89452] INFO weewx.manager: Added record 2020-07-31 12:35:00 PDT (1596224100) to daily summary in 'weewx.sdb'

weewx[89452] DEBUG weewx.drivers.vantage: Empty record page 0; index 2

weewx[89452] INFO weewx.engine: Starting main packet loop.

weewx[89452] INFO weewx.restx: Windy: Published record 2020-07-31 12:35:00 PDT (1596224100)

weewx[89452] INFO weewx.restx: Wunderground-PWS: Published record 2020-07-31 12:35:00 PDT (1596224100)

weewx[89452] INFO weewx.restx: StationRegistry: Published record 2020-07-31 12:35:00 PDT (1596224100)

weewx[89452] DEBUG weewx.drivers.vantage: Gentle wake up of console successful

weewx[89452] DEBUG weewx.drivers.vantage: Requesting 200 LOOP packets.

weewx[89452] DEBUG weewx.drivers.vantage: Gentle wake up of console successful

weewx[89452] DEBUG weewx.drivers.vantage: Getting archive packets since 2020-07-31 12:35:00 PDT (1596224100)

weewx[89452] DEBUG weewx.drivers.vantage: Gentle wake up of console successful

weewx[89452] DEBUG weewx.drivers.vantage: Retrieving 1 page(s); starting index= 2

weewx[89452] INFO weewx.manager: Added record 2020-07-31 12:40:00 PDT (1596224400) to database 'weewx.sdb'

weewx[89452] INFO weewx.manager: Added record 2020-07-31 12:40:00 PDT (1596224400) to daily summary in 'weewx.sdb'

weewx[89452] DEBUG weewx.drivers.vantage: Empty record page 0; index 3

weewx[89452] DEBUG weewx.restx: StationRegistry: wait interval (300 < 604800) has not passed for record 2020-07-31 12:40:00 PDT (1596224400)

weewx[89452] DEBUG weewx.reportengine: Running reports for latest time in the database.

weewx[89452] DEBUG weewx.drivers.vantage: Requesting 200 LOOP packets.

weewx[89452] DEBUG weewx.reportengine: Report 'StandardReport' not enabled. Skipping.

weewx[89452] DEBUG weewx.reportengine: Running report 'SeasonsReport'

weewx[89452] DEBUG weewx.reportengine: Found configuration file /Users/rycweather/weewx/skins/Seasons/skin.conf for report 'SeasonsReport'

weewx[89452] DEBUG weewx.cheetahgenerator: Using search list ['weewx.cheetahgenerator.Almanac', 'weewx.cheetahgenerator.Station', 'weewx.cheetahgenerator.Current', 'weewx.cheetahgenerator.Stats', 'weewx.cheetahgenerator.UnitInfo', 'weewx.cheetahgenerator.Extras']

weewx[89452] DEBUG weewx.manager: Daily summary version is 2.0

weewx[89452] INFO weewx.restx: Windy: Published record 2020-07-31 12:40:00 PDT (1596224400)

weewx[89452] INFO weewx.restx: Wunderground-PWS: Published record 2020-07-31 12:40:00 PDT (1596224400)

weewx[89452] DEBUG weewx.drivers.vantage: Gentle wake up of console successful

weewx[89452] INFO weewx.cheetahgenerator: Generated 8 files for report SeasonsReport in 1.20 seconds

weewx[89452] ERROR weewx.reportengine: Unable to instantiate generator 'weewx.imagegenerator.ImageGenerator'

weewx[89452] ERROR weewx.reportengine:         ****  No module named 'Image'

weewx[89452] ERROR weewx.reportengine:         ****  Traceback (most recent call last):

weewx[89452] ERROR weewx.reportengine:         ****    File "/Users/rycweather/weewx/bin/weeplot/genplot.py", line 15, in <module>

weewx[89452] ERROR weewx.reportengine:         ****      from PIL import Image, ImageDraw

weewx[89452] ERROR weewx.reportengine:         ****    File "/Users/rycweather/weewx/bin/PIL/Image.py", line 93, in <module>

weewx[89452] ERROR weewx.reportengine:         ****      from . import _imaging as core

weewx[89452] ERROR weewx.reportengine:         ****  ImportError: cannot import name '_imaging' from 'PIL' (/Users/rycweather/weewx/bin/PIL/__init__.py)

weewx[89452] ERROR weewx.reportengine:         ****  

weewx[89452] ERROR weewx.reportengine:         ****  During handling of the above exception, another exception occurred:

weewx[89452] ERROR weewx.reportengine:         ****  

weewx[89452] ERROR weewx.reportengine:         ****  Traceback (most recent call last):

weewx[89452] ERROR weewx.reportengine:         ****    File "/Users/rycweather/weewx/bin/weewx/reportengine.py", line 180, in run

weewx[89452] ERROR weewx.reportengine:         ****      obj = weeutil.weeutil.get_object(generator)(

weewx[89452] ERROR weewx.reportengine:         ****    File "/Users/rycweather/weewx/bin/weeutil/weeutil.py", line 1093, in get_object

weewx[89452] ERROR weewx.reportengine:         ****      mod = __import__(module)

weewx[89452] ERROR weewx.reportengine:         ****    File "/Users/rycweather/weewx/bin/weewx/imagegenerator.py", line 19, in <module>

weewx[89452] ERROR weewx.reportengine:         ****      import weeplot.genplot

weewx[89452] ERROR weewx.reportengine:         ****    File "/Users/rycweather/weewx/bin/weeplot/genplot.py", line 17, in <module>

weewx[89452] ERROR weewx.reportengine:         ****      import Image, ImageDraw

weewx[89452] ERROR weewx.reportengine:         ****  ModuleNotFoundError: No module named 'Image'

weewx[89452] ERROR weewx.reportengine:         ****  Generator ignored

weewx[89452] INFO weewx.reportengine: Copied 5 files to /Users/rycweather/www

weewx[89452] DEBUG weewx.reportengine: Running report 'SmartphoneReport'

weewx[89452] DEBUG weewx.reportengine: Found configuration file /Users/rycweather/weewx/skins/Smartphone/skin.conf for report 'SmartphoneReport'

weewx[89452] DEBUG weewx.cheetahgenerator: Using search list ['weewx.cheetahgenerator.Almanac', 'weewx.cheetahgenerator.Station', 'weewx.cheetahgenerator.Current', 'weewx.cheetahgenerator.Stats', 'weewx.cheetahgenerator.UnitInfo', 'weewx.cheetahgenerator.Extras']

weewx[89452] DEBUG weewx.manager: Daily summary version is 2.0

weewx[89452] INFO weewx.cheetahgenerator: Generated 6 files for report SmartphoneReport in 0.09 seconds

weewx[89452] ERROR weewx.reportengine: Unable to instantiate generator 'weewx.imagegenerator.ImageGenerator'

weewx[89452] ERROR weewx.reportengine:         ****  No module named 'Image'

weewx[89452] ERROR weewx.reportengine:         ****  Traceback (most recent call last):

weewx[89452] ERROR weewx.reportengine:         ****    File "/Users/rycweather/weewx/bin/weeplot/genplot.py", line 15, in <module>

weewx[89452] ERROR weewx.reportengine:         ****      from PIL import Image, ImageDraw

weewx[89452] ERROR weewx.reportengine:         ****    File "/Users/rycweather/weewx/bin/PIL/Image.py", line 93, in <module>

weewx[89452] ERROR weewx.reportengine:         ****      from . import _imaging as core

weewx[89452] ERROR weewx.reportengine:         ****  ImportError: cannot import name '_imaging' from 'PIL' (/Users/rycweather/weewx/bin/PIL/__init__.py)

weewx[89452] ERROR weewx.reportengine:         ****  

weewx[89452] ERROR weewx.reportengine:         ****  During handling of the above exception, another exception occurred:

weewx[89452] ERROR weewx.reportengine:         ****  

weewx[89452] ERROR weewx.reportengine:         ****  Traceback (most recent call last):

weewx[89452] ERROR weewx.reportengine:         ****    File "/Users/rycweather/weewx/bin/weewx/reportengine.py", line 180, in run

weewx[89452] ERROR weewx.reportengine:         ****      obj = weeutil.weeutil.get_object(generator)(

weewx[89452] ERROR weewx.reportengine:         ****    File "/Users/rycweather/weewx/bin/weeutil/weeutil.py", line 1093, in get_object

weewx[89452] ERROR weewx.reportengine:         ****      mod = __import__(module)

weewx[89452] ERROR weewx.reportengine:         ****    File "/Users/rycweather/weewx/bin/weewx/imagegenerator.py", line 19, in <module>

weewx[89452] ERROR weewx.reportengine:         ****      import weeplot.genplot

weewx[89452] ERROR weewx.reportengine:         ****    File "/Users/rycweather/weewx/bin/weeplot/genplot.py", line 17, in <module>

weewx[89452] ERROR weewx.reportengine:         ****      import Image, ImageDraw

weewx[89452] ERROR weewx.reportengine:         ****  ModuleNotFoundError: No module named 'Image'

weewx[89452] ERROR weewx.reportengine:         ****  Generator ignored

[Tom Keffer]

I see that you used the setup.py install method. When you installed Pillow, did you by any chance install it from the weewx home directory? If so, pip will see the setup.cfg file, and try to install Pillow in (in this case), /Users/rycweather/weewx, instead of in a systems directory

Use pip with the list option, and verbose, and it will show where everything is installed:

pip3 list -v

See if Pillow (or anything else), was accidentally installed in /Users/rycweather/weewx.

--tk

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/ebb4fdc5-0bc7-404a-b862-34d60d1fe0c9n%40googlegroups.com.

[jerry...@gmail.com]

Seems OK.

└─[$] <> pip3 list -v

Package    Version Location                               Installer

---------- ------- -------------------------------------- ---------

Cheetah3   3.2.5   /usr/local/lib/python3.8/site-packages pip

configobj  5.0.6   /usr/local/lib/python3.8/site-packages pip

ephem      3.7.7.0 /usr/local/lib/python3.8/site-packages pip

Pillow     7.2.0   /usr/local/lib/python3.8/site-packages pip

pip        20.2    /usr/local/lib/python3.8/site-packages pip

pyephem    3.7.7.0 /usr/local/lib/python3.8/site-packages pip

pyserial   3.4     /usr/local/lib/python3.8/site-packages pip

pyusb      1.0.2   /usr/local/lib/python3.8/site-packages pip

setuptools 49.2.0  /usr/local/lib/python3.8/site-packages

six        1.15.0  /usr/local/lib/python3.8/site-packages pip

wheel      0.34.2  /usr/local/lib/python3.8/site-packages

[Tom Keffer]

Looking back at your original post, I'm wondering if you're missing the C library _imaging. Try this:

python3 -c "from PIL import _imaging; print(_imaging.__file__)"

You should get something like

/usr/local/lib/python3.8/site-packages/PIL/_imaging.cpython-37m-darwin.so

If you get an exception, then try uninstalling both PIL (in case it is present; it shouldn't be), and Pillow, then reinstalling just Pillow:

pip3 uninstall PIL
pip3 uninstall Pillow
pip3 install Pillow

See if that helps.

-tk

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/2ee63fd4-d60e-4ebe-85bd-adf2ae3e36adn%40googlegroups.com.

[jerry...@gmail.com]

That looks ok also.

└─[$] <> python3 -c "from PIL import _imaging; print(_imaging.__file__)"

/usr/local/lib/python3.8/site-packages/PIL/_imaging.cpython-38-darwin.so

[Tom Keffer]

You've got me stumped. I would try the uninstall/install. Make sure you don't do it from the weewx home directory.

But, before you do, how about seeing what's in that home directory?

ls -l /Users/rycweather/weewx/

Then try the uninstall/install:

cd ~

pip3 uninstall PIL
pip3 uninstall Pillow
pip3 install Pillow

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/f220ff59-7d20-43c8-a024-8b860bada7ffn%40googlegroups.com.

[jerry...@gmail.com]

Solved.  There were residual py*37 files in PIL, so 

rm -rf ~/weewx/bin/PIL/*

followed by a python3 setup.py install (I'm not sure that was needed)

and it's all good.

[Tom Keffer]

Ah! So, it looks like the problem was caused by installing Pillow from the weewx home directory at some point.

I can't tell you how many times that has happened to me.

-tk

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/df2bbe7c-f403-4cb9-9a57-dfc717b7365bn%40googlegroups.com.

[jerry...@gmail.com]

Wow.  It never even occurred to me that PIL shouldn't be there at all.  The files were there from early May when I cut over to 4.1 and saw some graphics issues with an extension.  It could be even older.  I probably ran uninstall / reinstall on the Python dependencies from ~/weewx.  The native macOS Python doesn't change often, so it didn't matter.  But I just switched over to the HomeBrew version when graphics generation broke.

Is that worth a caveat on the macOS installation page?

Thanks for your help