Weewx ERROR in weewx.cheetahgenerator Weewx 4.0 Python3

[Mike Revitt]

I am having a really strange behaviour at the moment, which I thought I have fixed but it came back yesterday.

Occasionally Weewx will stop with no errors, but not be doing anything and when you do a status check it shows that the process has died.

pi@raspberrypi:/home/weewx $ sudo /etc/init.d/weewx status

● weewx.service - LSB: weewx weather system

   Loaded: loaded (/etc/init.d/weewx; generated)

   Active: active (exited) since Sat 2020-03-07 08:24:46 GMT; 24h ago

     Docs: man:systemd-sysv-generator(8)

  Process: 13209 ExecStart=/etc/init.d/weewx start (code=exited, status=0/SUCCESS)

Mar 07 11:55:22 raspberrypi weewxd[13224]: weewx[13224] INFO botocore.credentials: Found credentials in shared credentials…edentials

Mar 07 11:55:57 raspberrypi weewxd[13224]: weewx[13224] INFO user.s3backup: S3BackupGenerator: S3 copied files to S3 in 0 …7 seconds

Mar 07 12:00:16 raspberrypi weewxd[13224]: weewx[13224] INFO weewx.manager: Added record 2020-03-07 12:00:00 GMT (15835824…eewx.sdb'

Mar 07 12:00:16 raspberrypi weewxd[13224]: weewx[13224] INFO weewx.manager: Added record 2020-03-07 12:00:00 GMT (15835824…eewx.sdb'

Mar 07 12:00:18 raspberrypi weewxd[13224]: weewx[13224] INFO weewx.cheetahgenerator: Generated 7 files for report SeasonsR…0 seconds

Mar 07 12:00:23 raspberrypi weewxd[13224]: weewx[13224] INFO weewx.imagegenerator: Generated 27 images for report SeasonsR…1 seconds

Mar 07 12:00:23 raspberrypi weewxd[13224]: weewx[13224] INFO weewx.reportengine: Copied 0 files to /var/www/html/MountWeather

Mar 07 12:00:23 raspberrypi weewxd[13224]: weewx[13224] INFO botocore.credentials: Found credentials in shared credentials…edentials

Mar 07 12:00:29 raspberrypi weewxd[13224]: weewx[13224] INFO user.s3upload: S3UploadGenerator: AWS-S3 copied 34 files to S…0 seconds

Mar 07 12:00:30 raspberrypi weewxd[13224]: weewx[13224] INFO botocore.credentials: Found credentials in shared credentials…edentials

Then when I restart weewx I get the following error

pi@raspberrypi:/home/weewx $ sudo /etc/init.d/weewx status

● weewx.service - LSB: weewx weather system

   Loaded: loaded (/etc/init.d/weewx; generated)

   Active: active (running) since Sun 2020-03-08 08:45:32 GMT; 14min ago

     Docs: man:systemd-sysv-generator(8)

  Process: 29330 ExecStart=/etc/init.d/weewx start (code=exited, status=0/SUCCESS)

    Tasks: 2 (limit: 2200)

   Memory: 42.2M

   CGroup: /system.slice/weewx.service

           └─29345 /usr/bin/python /home/weewx/bin/weewxd --daemon --pidfile=/var/run/weewx.pid /home/weewx/weewx.conf

Mar 08 09:00:16 raspberrypi weewxd[29345]: weewx[29345] INFO weewx.manager: Added record 2020-03-08 09:00:00 GMT (1583658000) to daily summary in 'weewx.sdb'

Mar 08 09:00:16 raspberrypi weewxd[29345]: weewx[29345] ERROR weewx.cheetahgenerator: Generate failed with exception '<class 'TypeError'>'

Mar 08 09:00:16 raspberrypi weewxd[29345]: weewx[29345] ERROR weewx.cheetahgenerator: **** Ignoring template /home/weewx/skins/Seasons/index.html.tmpl

Mar 08 09:00:16 raspberrypi weewxd[29345]: weewx[29345] ERROR weewx.cheetahgenerator: **** Reason: '>' not supported between instances of 'NoneType' and 'float'

Mar 08 09:00:16 raspberrypi weewxd[29345]: weewx[29345] ERROR weewx.cheetahgenerator: ****  Traceback (most recent call last):

Mar 08 09:00:16 raspberrypi weewxd[29345]: weewx[29345] ERROR weewx.cheetahgenerator: ****    File "/home/weewx/bin/weewx/cheetahgenerator.py", line 322, in generate

Mar 08 09:00:16 raspberrypi weewxd[29345]: weewx[29345] ERROR weewx.cheetahgenerator: ****      unicode_string = compiled_template.respond()

Mar 08 09:00:16 raspberrypi weewxd[29345]: weewx[29345] ERROR weewx.cheetahgenerator: ****    File "_home_weewx_skins_Seasons_index_html_tmpl.py", line …, in respond

Mar 08 09:00:16 raspberrypi weewxd[29345]: weewx[29345] ERROR weewx.cheetahgenerator: ****  TypeError: '>' not supported between instances of 'NoneType' and 'float'

Mar 08 09:00:16 raspberrypi weewxd[29345]: weewx[29345] INFO weewx.cheetahgenerator: Generated 6 files for report SeasonsReport in 1.50 seconds

Mar 08 09:00:21 raspberrypi weewxd[29345]: weewx[29345] INFO weewx.imagegenerator: Generated 27 images for report SeasonsReport in 4.65 seconds

Mar 08 09:00:21 raspberrypi weewxd[29345]: weewx[29345] INFO weewx.reportengine: Copied 0 files to /var/www/html/MountWeather

Hint: Some lines were ellipsized, use -l to show in full.

I have compared index.html.tmpl to the original and it is identical as is  cheetahgenerator.py

I have got this working again in the past by simply copying my modified files back to the Raspberry Pi and restarting the process, but this doesn't make any sense as I can't find anything that is different. Currently trying to pin this down to a reproducible case but an help appreciated here as I can't find the file _home_weewx_skins_Seasons_index_html_tmpl.py which appears to be the source of the problem

[Mike Revitt]

Update

After an extensive search in my environment I have found a hidden directory that I can't cd into

weewx/bin.20200217183229/pip/_internal/utils

And this directory contains a file   packaging.py

This is the only  fine in my entire environment that contains the string NoneType.

Below is the function from that file containing g the NoneType string, hopefully this means something to a Python developer

def get_metadata(dist):

    # type: (Distribution) -> Message

    """

    :raises NoneMetadataError: if the distribution reports `has_metadata()`

        True but `get_metadata()` returns None.

    """

    metadata_name = 'METADATA'

    if (isinstance(dist, pkg_resources.DistInfoDistribution) and

            dist.has_metadata(metadata_name)):

        metadata = dist.get_metadata(metadata_name)

    elif dist.has_metadata('PKG-INFO'):

        metadata_name = 'PKG-INFO'

        metadata = dist.get_metadata(metadata_name)

    else:

        logger.warning("No metadata found in %s", display_path(dist.location))

        metadata = ''

    if metadata is None:

        raise NoneMetadataError(dist, metadata_name)

    feed_parser = FeedParser()

    # The following line errors out if with a "NoneType" TypeError if

    # passed metadata=None.

    feed_parser.feed(metadata)

    return feed_parser.close()

[Mike Revitt]

Further update

After trying all combinations of file refresh I kept getting the same error so I renamed the weewx directory to weewx.broken and attempted to reinstall weewx from my previous build, with some interesting results. Bear in mind that nothing has been changed in the environment since I installed it yesterday

pi@raspberrypi:~ $ sudo mv /home/weewx /home/weewx.broken

pi@raspberrypi:~ $ ls -al /home

total 16

drwxr-xr-x  4 root root 4096 Mar  8 10:12 .

drwxr-xr-x 21 root root 4096 Feb 13 16:32 ..

drwxr-xr-x 21 pi   pi   4096 Mar  6 18:18 pi

drwxr-xr-x  8 pi   root 4096 Mar  8 10:08 weewx.broken

pi@raspberrypi:~ $ cd weewx-4.0.0b14/

pi@raspberrypi:~/weewx-4.0.0b14 $ ./setup.py build

running build

running build_py

running build_scripts

pi@raspberrypi:~/weewx-4.0.0b14 $ sudo ./bin/weewxd weewx.conf

Traceback (most recent call last):

  File "./bin/weewxd", line 261, in <module>

    main()

  File "./bin/weewxd", line 148, in main

    engine = weewx.engine.StdEngine(config_dict)

  File "/home/pi/weewx-4.0.0b14/bin/weewx/engine.py", line 69, in __init__

    self.setupStation(config_dict)

  File "/home/pi/weewx-4.0.0b14/bin/weewx/engine.py", line 87, in setupStation

    driver = config_dict[station_type]['driver']

  File "/usr/lib/python3/dist-packages/configobj.py", line 554, in __getitem__

    val = dict.__getitem__(self, key)

KeyError: 'unspecified'

pi@raspberrypi:~/weewx-4.0.0b14 $ python -V

Python 3.7.3

So I logged out and back in, you never know it might have  been something in memory, but that made no difference

So I deleted the directory weewx-4.0.0b14 and reinstalled it from the zip file already on disk which is the one I used yesterday

pi@raspberrypi:~ $ sudo rm -fr weewx-4.0.0b14

pi@raspberrypi:~ $ tar xvfz weewx-4.0.0b14.tar.gz 

pi@raspberrypi:~ $ cd weewx-4.0.0b14/

pi@raspberrypi:~/weewx-4.0.0b14 $ ./setup.py build

running build

running build_py

creating build

pi@raspberrypi:~/weewx-4.0.0b14 $ sudo ./setup.py install

running install

running build

running build_py

running build_scripts

running install_lib

creating /home/weewx

I then copied my modified files back , started weewx and compared the broken directory to the one that is now working

pi@raspberrypi:/home $ diff -qr weewx weewx.broken/

Files weewx/archive/weewx.sdb and weewx.broken/archive/weewx.sdb differ

Only in weewx.broken/bin/user/__pycache__: s3backup.cpython-37.pyc

Only in weewx.broken/bin/user/__pycache__: s3upload.cpython-37.pyc

Only in weewx.broken/bin/user/__pycache__: stats.cpython-37.pyc

pi@raspberrypi:/home $ sudo /etc/init.d/weewx status

● weewx.service - LSB: weewx weather system

   Loaded: loaded (/etc/init.d/weewx; generated)

   Active: active (running) since Sun 2020-03-08 10:18:18 GMT; 10min ago

     Docs: man:systemd-sysv-generator(8)

  Process: 12254 ExecStart=/etc/init.d/weewx start (code=exited, status=0/SUCCESS)

    Tasks: 1 (limit: 2200)

   Memory: 37.8M

   CGroup: /system.slice/weewx.service

           └─12269 /usr/bin/python /home/weewx/bin/weewxd --daemon --pidfile=/var/run/weewx.pid /home/weewx/weewx.conf

Mar 08 10:21:20 raspberrypi weewxd[12269]: weewx[12269] INFO weewx.reportengine: Copied 22 files to /var/www/html/MountWeather

Mar 08 10:21:21 raspberrypi weewxd[12269]: weewx[12269] INFO botocore.credentials: Found credentials in shared credentials file: ~/.aws/credentials

Mar 08 10:21:46 raspberrypi weewxd[12269]: weewx[12269] INFO user.s3upload: S3UploadGenerator: AWS-S3 copied 147 files to S3 in 25.55 seconds

Mar 08 10:25:15 raspberrypi weewxd[12269]: weewx[12269] INFO weewx.manager: Added record 2020-03-08 10:25:00 GMT (1583663100) to database 'weewx.sdb'

Mar 08 10:25:15 raspberrypi weewxd[12269]: weewx[12269] INFO weewx.manager: Added record 2020-03-08 10:25:00 GMT (1583663100) to daily summary in 'weewx.sdb'

Mar 08 10:25:17 raspberrypi weewxd[12269]: weewx[12269] INFO weewx.cheetahgenerator: Generated 7 files for report SeasonsReport in 1.60 seconds

Mar 08 10:25:18 raspberrypi weewxd[12269]: weewx[12269] INFO weewx.imagegenerator: Generated 9 images for report SeasonsReport in 0.77 seconds

Mar 08 10:25:18 raspberrypi weewxd[12269]: weewx[12269] INFO weewx.reportengine: Copied 0 files to /var/www/html/MountWeather

Mar 08 10:25:18 raspberrypi weewxd[12269]: weewx[12269] INFO botocore.credentials: Found credentials in shared credentials file: ~/.aws/credentials

Mar 08 10:25:21 raspberrypi weewxd[12269]: weewx[12269] INFO user.s3upload: S3UploadGenerator: AWS-S3 copied 16 files to S3 in 3.23 seconds

And it is now running perfectly and nothing is different, although just ran another diff after it has been running for about 30 minutes and the only difference now are some cached files

pi@raspberrypi:/home $ diff -qr weewx weewx.broken/

Files weewx/archive/weewx.sdb and weewx.broken/archive/weewx.sdb differ

Only in weewx.broken/bin/user/__pycache__: s3backup.cpython-37.pyc

Files weewx/bin/user/__pycache__/s3upload.cpython-37.pyc and weewx.broken/bin/user/__pycache__/s3upload.cpython-37.pyc differ

Files weewx/bin/user/__pycache__/stats.cpython-37.pyc and weewx.broken/bin/user/__pycache__/stats.cpython-37.pyc differ

Only in weewx: #S3.last

pi@raspberrypi:/home $ 

I am hoping someone can help with this as it is stopping me going live on the Raspberry Pi, and as far as I can tell the environment that works and the one that doesn't are identical

[Andrew Milner]

You can always go live on the RPI by installing the current version of weewx - v3.9.2 I believe.  Version 4 has not yet been released, but is in beta test.  So your V4 weewx should be regarded as a beta test version (which may be being compared to a live V3.9 version), rather than your sole live version.

Well, that is my opinion anyway!!!!

[Mike Revitt]

Thanks for the suggestion, but I need Python3 for my extensions which is forcing me to Weewx v4

[Thomas Keffer]

Some comments that might help you debug.

1. The error message 

TypeError: '>' not supported between instances of 'NoneType' and 'float'

 

is not to be taken literally. It means that you are comparing two variables, one of which has Python value None, the other a floating point number. 

2. Most likely, there is some comparison in the file skins/Seasons/index.html.tmpl that is not checking first whether one of the numbers might be None. I know you say that you have not changed the templates, but a comparison of this sort must be in there, somewhere.

3. The file _home_weewx_skins_Seasons_index_html_tmpl.py is a temporary file generated by Cheetah from the template index.html.tmpl. You will not find it in your directory structure, as it is deleted by Cheetah immediately after use. 

4. The error

KeyError: 'unspecified'

is because you tried to run weewxd from the unpacked tar file. The type of weather station has not been specified yet in the tar file, hence the error. You must do "setup install" first.

-tk

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/9b44ac23-d8cf-44ec-ac17-ac654e15be6e%40googlegroups.com.

[gjr80]

Mike,

Stop looking for 'NoneType', you won't find anything that will help, its just the eror message structure. The clue is in the error message:

Mar 08 09:00:16 raspberrypi weewxd[29345]: weewx[29345] ERROR weewx.cheetahgenerator: **** Ignoring template /home/weewx/skins/Seasons/index.html.tmpl
Mar 08 09:00:16 raspberrypi weewxd[29345]: weewx[29345] ERROR weewx.cheetahgenerator: **** Reason: '>' not supported between instances of 'NoneType' and 'float'
Mar 08 09:00:16 raspberrypi weewxd[29345]: weewx[29345] ERROR weewx.cheetahgenerator: ****  Traceback (most recent call last):
Mar 08 09:00:16 raspberrypi weewxd[29345]: weewx[29345] ERROR weewx.cheetahgenerator: ****    File "/home/weewx/bin/weewx/cheetahgenerator.py", line 322, in generate
Mar 08 09:00:16 raspberrypi weewxd[29345]: weewx[29345] ERROR weewx.cheetahgenerator: ****      unicode_string = compiled_template.respond()
Mar 08 09:00:16 raspberrypi weewxd[29345]: weewx[29345] ERROR weewx.cheetahgenerator: ****    File "_home_weewx_skins_Seasons_index_html_tmpl.py", line …, in respond
Mar 08 09:00:16 raspberrypi weewxd[29345]: weewx[29345] ERROR weewx.cheetahgenerator: ****  TypeError: '>' not supported between instances of 'NoneType' and 'float'

The error indicates there is some in-line python code in the index.html.tmpl template that is comparing the python value None and a float using the > operator. Whilst the error refers to the index.html.tmpl template if you look in that template it actually includes a number of 'includes' ie the contents of other files. So you need to look through them (plus there is no > operator in index.html.tmpl).

Actually the only > operator in any of the include files is in hilo.inc and current.inc. It happens ot be the same line of code in each:

#if $day.ET.has_data and $day.ET.sum.raw > 0.0

I suspect that is the cause of your issue and its intermittent nature is due to the periodic nature of the ET calculation. In the Seasons skin directory in each of hilo.inc and current.inc try chnaging that line to:

#if $day.ET.has_data and $day.ET.sum.raw is not None and $day.ET.sum.raw > 0.0

save the files (no need for restart) and see if the problem disappears.

Gary

[Mike Revitt]

Thanks Thomas,

That gives me something to go open

I can discount the issue with index.html.tmpl because whilst I have modified it I simply copy it in from my source control so is identical every time.

Which leave the final issue, running weewxd from an unpacked tar file. But I just realised that was my making a rookie error and copying and pasting the wrong command from the installation guide, I should of course have been running sudo ./setup.py install. when I got the last error.

Which brings me back to where I started. Everything is currently running perfectly but at some point I expect the system to stop and be unable to restart with the error messages at the top of this thread

[Mike Revitt]

Thanks Gary,

There is indeed inline python code in the index.html.tmpl and I will also make your suggest change to current.inc and see if that fixes it.

But closer investigation of my code has revealed the problem. This is how I change the text in my barometer, so based on your suggestion I have changed the top if statement to detect the value None in raw.

I then copied my old directory back which caused the error to reappear, then made the change below to index.html.tmp which has indeed fixed the error.

That is something I would have never found by myself so thank you very much for your help.

So is if fair to say Python3 is more stringent when it comes to these types of comparisons?

          Trend       : #if $trend.barometer.raw is not None

                          #if $trend.barometer.raw > 2.0

                            'Rising Rapidly'

                          #else if $trend.barometer.raw >  0.7

                            'Rising Slowly'

                          #else if $trend.barometer.raw < -2.0

                            'Falling Rapidly'

                          #else if $trend.barometer.raw < -0.7

                            'Falling Slowly'

                          #else

                            'Steady'

                          #end if

                        #else

                          'Steady'

                        #end if

[Thomas Keffer]

So is if fair to say Python3 is more stringent when it comes to these types of comparisons?

Yes, and that's a good thing. Under Python 2, None always tests less than any number. Under Python 3, such a comparison is an error:

None < -1

Python 2: True

Python 3: Error

-tk