Python versions

[Colin Larsen]

Hi all

I know that this is buried somewhere in the threads of information, sorry apologies for not being able to find it.

I've just upgraded my Pi based station to Weewx 4.0 using the apt-get method. Ihas Python3 installed but is still using Python2 as can be seen here. I thought I had read that if Python3 was already installed then the update would use that version - apparently not :)

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent

permitted by applicable law.

Last login: Wed Apr 29 09:47:37 2020 from 192.168.20.45

pi@raspberrypi:~ $ ps -ef|grep weewx

pi       30253 30223  0 12:01 pts/0    00:00:00 grep --color=auto weewx

root     32235     1  5 May01 ?        02:23:06 python2 /usr/share/weewx/weewxd --daemon --pidfile=/var/run/weewx.pid /etc/weewx/weewx.conf

pi@raspberrypi:~ $ python3 --version

Python 3.5.3

pi@raspberrypi:~ $ 

Whats the trick that I've missed to tell it to use Python3?

Many thanks

Colin

[Tom Keffer]

You will have to edit whatever script you use to start weewx as a daemon.

Look in /etc/init.d/weewx. If not there, then /etc/systemd/system/weewx.service.

-tk

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/CACjxfUv_62X8mX2Q8zG7SO6-9TWXgWaFHYLRO8LTnPmrMRTJYw%40mail.gmail.com.

[Colin Larsen]

Thanks Tom

Well /etc/init.d/weewx is there and the other one isn't so I'll go with that but to be honest the innards of that file is out of my depth. 

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/CAPq0zECE9FUpRR_gZ6TCrRM6zp1msX0OhGPVpbFBgSpP93TdOA%40mail.gmail.com.

[Graham Eddy]

as others have already mentioned you can instruct use of python3 either

 (a) system-wide e.g. /etc/default or defaults

 (b) startup e.g. /etc/init.d/weewx or /Library/LaunchDaemons/com.weewx.weewxd

 (c) embedded e.g. shebang (insert #!python3 in first line of weewxd)

my platform (macos) provides python2.7 native not python3.7 and i hesitate to set system-wide in case it impacts some other platform-provided utilities. i find startup approach inconvenient on macos so i usually use embedded (shebang), in my case #!/opt/local/bin/python3.7 because that’s where macports puts python3

[Colin Larsen]

Hi Graham

Thanks for that and I understand all of what you are saying - except I have no idea whereabouts in /etc/init.d/weewx to change to make it use Python3. I thought I might find the line command line containing weewxd in there somewhere but alas no (or I've done a "boy look"). And I know enough this part of the system to be dangerous, so I'll keep fingers off until I can actually figure this out! Still learning lots here so appreciate the patience and explanations :)

Colin

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/2BE0524D-D87A-45EB-8A45-A6ED93BD087B%40gmail.com.

[Colin Larsen]

I found this /etc/default

WEEWX_PYTHON=python2

WEEWX_BINDIR=/usr/share/weewx

WEEWX_BIN=/usr/bin/weewxd

WEEWX_CFG=/etc/weewx/weewx.conf

Changing it to python3 breaks weewx so I'll just leave well enough alone

Colin

[Colin Larsen]

Whoops sorry /etc/default/weewx that should have been

[mwall]

On Saturday, May 2, 2020 at 9:12:41 PM UTC-4, Colin Larsen wrote:

Whoops sorry /etc/default/weewx that should have been

colin has it right.  one of the things we did in the weewx4 packaging for debian, redhat (and soon suse and *bsd) is to put the parameters into /etc/default/weewx (it will be /etc/defaults/weewx on *bsd)

since the weewx installation is exactly the same for both python2 and python3, all you have to do is change the interpreter that invokes weewxd, wee_config, or any other weewx entry point.

there are scripts in /usr/bin/wee* and these simply pull in the variables from /etc/default/weewx then invoke the corresponding wee* 'binaries'

the /etc/init.d/weewx scripts for debian, redhad, suse, and *bsd use this pattern

if you install from setup.py then you would have to install /etc/default/weewx yourself (it is in the util directory)

m

[mwall]

On Saturday, May 2, 2020 at 9:11:51 PM UTC-4, Colin Larsen wrote:

I found this /etc/default

WEEWX_PYTHON=python2

WEEWX_BINDIR=/usr/share/weewx

WEEWX_BIN=/usr/bin/weewxd

WEEWX_CFG=/etc/weewx/weewx.conf

Changing it to python3 breaks weewx so I'll just leave well enough alone

you *should* be able to switch from python2 to python3 simply by changing the WEEWX_PYTHON line.  however, if you were running with python2 and you have not installed the python3 dependencies, then of course weewx won't run.

m 

[Colin Larsen]

Ah ...... that will be the missing piece of the puzzle! Off to sort that now, will report back.

Thanks!

--

You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/973baa2b-1c38-45c8-a3ed-630835e0bfdd%40googlegroups.com.

[Colin Larsen]

This may be a problem? After changing to python3 ......

May  3 16:28:06 raspberrypi weewx[5437] CRITICAL __main__: Caught unrecoverable exception:

May  3 16:28:06 raspberrypi weewx[5437] CRITICAL __main__:     ****  invalid syntax (mqtt.py, line 196)

May  3 16:28:06 raspberrypi weewx[5437] CRITICAL __main__:     ****  Traceback (most recent call last):

May  3 16:28:06 raspberrypi weewx[5437] CRITICAL __main__:     ****    File "/usr/share/weewx/weewxd", line 148, in main

May  3 16:28:06 raspberrypi weewx[5437] CRITICAL __main__:     ****      engine = weewx.engine.StdEngine(config_dict)

May  3 16:28:06 raspberrypi weewx[5437] CRITICAL __main__:     ****    File "/usr/share/weewx/weewx/engine.py", line 75, in __init__

May  3 16:28:06 raspberrypi weewx[5437] CRITICAL __main__:     ****      self.loadServices(config_dict)

May  3 16:28:06 raspberrypi weewx[5437] CRITICAL __main__:     ****    File "/usr/share/weewx/weewx/engine.py", line 138, in loadServices

May  3 16:28:06 raspberrypi weewx[5437] CRITICAL __main__:     ****      obj = weeutil.weeutil.get_object(svc)(self,config_dict)

May  3 16:28:06 raspberrypi weewx[5437] CRITICAL __main__:     ****    File "/usr/share/weewx/weeutil/weeutil.py", line 1093, in get_object

May  3 16:28:06 raspberrypi weewx[5437] CRITICAL __main__:     ****      mod = __import__(module)

May  3 16:28:06 raspberrypi weewx[5437] CRITICAL __main__:     ****    File "/usr/share/weewx/user/mqtt.py", line 196

May  3 16:28:06 raspberrypi weewx[5437] CRITICAL __main__:     ****      except KeyError, e:

May  3 16:28:06 raspberrypi weewx[5437] CRITICAL __main__:     ****                     ^

May  3 16:28:06 raspberrypi weewx[5437] CRITICAL __main__:     ****  SyntaxError: invalid syntax

May  3 16:28:06 raspberrypi weewx[5437] CRITICAL __main__:     ****  Exiting.

[Jaap de Munck]

That means your MQTT extension is not Python3 proof!

Get your self an updated version.

Is it this one: https://github.com/weewx/weewx/wiki/mqtt ?

Op zondag 3 mei 2020 06:29:34 UTC+2 schreef Colin Larsen:

This may be a problem? After changing to python3 ......

May  3 16:28:06 raspberrypi weewx[5437] CRITICAL __main__: Caught unrecoverable exception:

May  3 16:28:06 raspberrypi weewx[5437] CRITICAL __main__:     ****  invalid syntax (mqtt.py, line 196)

May  3 16:28:06 raspberrypi weewx[5437] CRITICAL __main__:     ****  Traceback (most recent call last):

May  3 16:28:06 raspberrypi weewx[5437] CRITICAL __main__:     ****    File "/usr/share/weewx/weewxd", line 148, in main

May  3 16:28:06 raspberrypi weewx[5437] CRITICAL __main__:     ****      engine = weewx.engine.StdEngine(config_dict)

May  3 16:28:06 raspberrypi weewx[5437] CRITICAL __main__:     ****    File "/usr/share/weewx/weewx/engine.py", line 75, in __init__

May  3 16:28:06 raspberrypi weewx[5437] CRITICAL __main__:     ****      self.loadServices(config_dict)

May  3 16:28:06 raspberrypi weewx[5437] CRITICAL __main__:     ****    File "/usr/share/weewx/weewx/engine.py", line 138, in loadServices

May  3 16:28:06 raspberrypi weewx[5437] CRITICAL __main__:     ****      obj = weeutil.weeutil.get_object(svc)(self,config_dict)

May  3 16:28:06 raspberrypi weewx[5437] CRITICAL __main__:     ****    File "/usr/share/weewx/weeutil/weeutil.py", line 1093, in get_object

May  3 16:28:06 raspberrypi weewx[5437] CRITICAL __main__:     ****      mod = __import__(module)

May  3 16:28:06 raspberrypi weewx[5437] CRITICAL __main__:     ****    File "/usr/share/weewx/user/mqtt.py", line 196

May  3 16:28:06 raspberrypi weewx[5437] CRITICAL __main__:     ****      except KeyError, e:

May  3 16:28:06 raspberrypi weewx[5437] CRITICAL __main__:     ****                     ^

May  3 16:28:06 raspberrypi weewx[5437] CRITICAL __main__:     ****  SyntaxError: invalid syntax

May  3 16:28:06 raspberrypi weewx[5437] CRITICAL __main__:     ****  Exiting.

On Sun, 3 May 2020 at 15:45, Colin Larsen <colin...@gmail.com> wrote:

Ah ...... that will be the missing piece of the puzzle! Off to sort that now, will report back.

Thanks!

On Sun, 3 May 2020 at 15:30, mwall <mw...@users.sourceforge.net> wrote:

On Saturday, May 2, 2020 at 9:11:51 PM UTC-4, Colin Larsen wrote:

I found this /etc/default

WEEWX_PYTHON=python2

WEEWX_BINDIR=/usr/share/weewx

WEEWX_BIN=/usr/bin/weewxd

WEEWX_CFG=/etc/weewx/weewx.conf

Changing it to python3 breaks weewx so I'll just leave well enough alone

you *should* be able to switch from python2 to python3 simply by changing the WEEWX_PYTHON line.  however, if you were running with python2 and you have not installed the python3 dependencies, then of course weewx won't run.

m 

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.

To unsubscribe from this group and stop receiving emails from it, send an email to weewx...@googlegroups.com.

[Colin Larsen]

So just double checking, Matt was this the definitive answer that the version of mqtt needs updating?

Cheers

Colin

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/f3000d48-6728-47f3-9b3f-cc6dff38018d%40googlegroups.com.

[Tom Keffer]

Colin, yes, your version of mqtt.py needs to be updated. You can the link on the MQTT wiki page.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/CACjxfUt%2BN8tPjO6m6iSU9CiAYPXcSyeuwiB1wUMtPsVg_rQZqQ%40mail.gmail.com.

[Colin Larsen]

Perfect will do that tonight, thanks Tom

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/CAPq0zECic6VsOdomWgO0vdYuLniReua4XKYMPuwoYisQZA2rEw%40mail.gmail.com.

[Graham Eddy]

i have only just become interested in mqtt (to make my custom weewx gauges near realtime).

the link to the MQTT weewx extension mentioned below lists two prerequisites: paho-mqtt, python-cjson.

paho-mqtt installed fine.

python-cjson fails.

i haven't found anything in the forum relating to this failure, has someone seen it?

macos 10.13.6 with macports

pebble:weewx $ sudo python3.7 -m pip install python-cjson

...

Collecting python-cjson

  Downloading python-cjson-1.2.2.tar.gz (12 kB)

Installing collected packages: python-cjson

    Running setup.py install for python-cjson ... error

    ERROR: Command errored out with exit status 1:

...

    cjson.c:1206:9: error: non-void function 'initcjson' should return a value [-Wreturn-type]

            return;

            ^

    69 warnings and 13 errors generated.

    error: command '/usr/bin/clang' failed with exit status 1

    ----------------------------------------

ERROR: Command errored out with exit status 1: /opt/local/bin/python3.7 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/tmp/pip-install-pnhk7sf4/python-cjson/setup.py'"'"'; __file__='"'"'/private/tmp/pip-install-pnhk7sf4/python-cjson/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/tmp/pip-record-8ccz83fi/install-record.txt --single-version-externally-managed --compile --install-headers /opt/local/Library/Frameworks/Python.framework/Versions/3.7/include/python3.7m/python-cjson Check the logs for full command output.

[Graham Eddy]

this (ujson instead of python-cjson, then install mqtt extension) seems to have worked up to point that weewx.log reports its mqtt client is successfully publishing loop data

sudo python3.7 -m pip install ujson

[Colin Larsen]

Still going ......

Installed the new mqtt and now get this error 

May  6 14:16:47 raspberrypi weewx[6432] CRITICAL __main__: Caught unrecoverable exception:

May  6 14:16:47 raspberrypi weewx[6432] CRITICAL __main__:     ****  No module named 'paho'

May  6 14:16:47 raspberrypi weewx[6432] CRITICAL __main__:     ****  Traceback (most recent call last):

May  6 14:16:47 raspberrypi weewx[6432] CRITICAL __main__:     ****    File "/usr/share/weewx/weewxd", line 148, in main

May  6 14:16:47 raspberrypi weewx[6432] CRITICAL __main__:     ****      engine = weewx.engine.StdEngine(config_dict)

May  6 14:16:47 raspberrypi weewx[6432] CRITICAL __main__:     ****    File "/usr/share/weewx/weewx/engine.py", line 75, in __init__

May  6 14:16:47 raspberrypi weewx[6432] CRITICAL __main__:     ****      self.loadServices(config_dict)

May  6 14:16:47 raspberrypi weewx[6432] CRITICAL __main__:     ****    File "/usr/share/weewx/weewx/engine.py", line 138, in loadServices

May  6 14:16:47 raspberrypi weewx[6432] CRITICAL __main__:     ****      obj = weeutil.weeutil.get_object(svc)(self,config_dict)

May  6 14:16:47 raspberrypi weewx[6432] CRITICAL __main__:     ****    File "/usr/share/weewx/weeutil/weeutil.py", line 1093, in get_object

May  6 14:16:47 raspberrypi weewx[6432] CRITICAL __main__:     ****      mod = __import__(module)

May  6 14:16:47 raspberrypi weewx[6432] CRITICAL __main__:     ****    File "/usr/share/weewx/user/mqtt.py", line 94, in <module>

May  6 14:16:47 raspberrypi weewx[6432] CRITICAL __main__:     ****      import paho.mqtt.client as mqtt

May  6 14:16:47 raspberrypi weewx[6432] CRITICAL __main__:     ****  ImportError: No module named 'paho'

May  6 14:16:47 raspberrypi weewx[6432] CRITICAL __main__:     ****  Exiting.

Tried to install Paho and I get  ...

pi@raspberrypi:/etc/default $ sudo pip install paho-mqtt

Requirement already satisfied: paho-mqtt in /usr/local/lib/python2.7/dist-packages

So how do I install it for python3 please :)

Cheers

Colin

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/296D8321-100E-45B1-A47A-2657960A2942%40gmail.com.

[Colin Larsen]

Fixed that one with; sudo python3 -m pip install paho-mqtt

Now I have an error that looks like it is Open Wetaher map?

May  6 14:29:07 raspberrypi weewx[6896] CRITICAL __main__: Caught unrecoverable exception:

May  6 14:29:07 raspberrypi weewx[6896] CRITICAL __main__:     ****  invalid syntax (owm.py, line 84)

May  6 14:29:07 raspberrypi weewx[6896] CRITICAL __main__:     ****  Traceback (most recent call last):

May  6 14:29:07 raspberrypi weewx[6896] CRITICAL __main__:     ****    File "/usr/share/weewx/weewxd", line 148, in main

May  6 14:29:07 raspberrypi weewx[6896] CRITICAL __main__:     ****      engine = weewx.engine.StdEngine(config_dict)

May  6 14:29:07 raspberrypi weewx[6896] CRITICAL __main__:     ****    File "/usr/share/weewx/weewx/engine.py", line 75, in __init__

May  6 14:29:07 raspberrypi weewx[6896] CRITICAL __main__:     ****      self.loadServices(config_dict)

May  6 14:29:07 raspberrypi weewx[6896] CRITICAL __main__:     ****    File "/usr/share/weewx/weewx/engine.py", line 138, in loadServices

May  6 14:29:07 raspberrypi weewx[6896] CRITICAL __main__:     ****      obj = weeutil.weeutil.get_object(svc)(self,config_dict)

May  6 14:29:07 raspberrypi weewx[6896] CRITICAL __main__:     ****    File "/usr/share/weewx/weeutil/weeutil.py", line 1093, in get_object

May  6 14:29:07 raspberrypi weewx[6896] CRITICAL __main__:     ****      mod = __import__(module)

May  6 14:29:07 raspberrypi weewx[6896] CRITICAL __main__:     ****    File "/usr/share/weewx/user/owm.py", line 84

May  6 14:29:07 raspberrypi weewx[6896] CRITICAL __main__:     ****      except KeyError, e:

May  6 14:29:07 raspberrypi weewx[6896] CRITICAL __main__:     ****                     ^

May  6 14:29:07 raspberrypi weewx[6896] CRITICAL __main__:     ****  SyntaxError: invalid syntax

May  6 14:29:07 raspberrypi weewx[6896] CRITICAL __main__:     ****  Exiting.

[vince]

On Tuesday, May 5, 2020 at 7:31:23 PM UTC-7, Colin Larsen wrote:

May  6 14:29:07 raspberrypi weewx[6896] CRITICAL __main__:     ****      except KeyError, e:

May  6 14:29:07 raspberrypi weewx[6896] CRITICAL __main__:     ****                     ^

May  6 14:29:07 raspberrypi weewx[6896] CRITICAL __main__:     ****  SyntaxError: invalid syntax

May  6 14:29:07 raspberrypi weewx[6896] CRITICAL __main__:     ****  Exiting.

This is almost the most frequently asked question.....

Any time you see that error, you are running python2 code that has not been updated to python3 syntax.   Either update your driver or extension to the latest version, or stay with python2 until the author of the driver/extension has updated their software to python3.

[Colin Larsen]

Righto, back to Python2 it is, apologies for asking but hadn't seen this one before.

Cheers and thanks

Colin

--

You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/10c129b2-7488-418c-8380-07115b479be2%40googlegroups.com.

[Tom Keffer]

I did a quick port. Try this version of owm.py: https://raw.githubusercontent.com/tkeffer/weewx-owm/master/bin/user/owm.py

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/CACjxfUsqFccLY_T0oPPwatGRd9tJs-ceOGqJyH1T2ps3VJ6mvw%40mail.gmail.com.

[Greg from Oz]

I have run a program called 2to3 python converter program on some of the python2 scripts and it fixed most of the problems and did the conversion to the python3 syntax.

example: 2to3 -w /usr/share/weewx/user/owm.py

On Wednesday, 6 May 2020 12:31:23 UTC+10, Colin Larsen wrote:

Fixed that one with; sudo python3 -m pip install paho-mqtt

Now I have an error that looks like it is Open Wetaher map?

May  6 14:29:07 raspberrypi weewx[6896] CRITICAL __main__: Caught unrecoverable exception:

May  6 14:29:07 raspberrypi weewx[6896] CRITICAL __main__:     ****  invalid syntax (owm.py, line 84)

May  6 14:29:07 raspberrypi weewx[6896] CRITICAL __main__:     ****  Traceback (most recent call last):

May  6 14:29:07 raspberrypi weewx[6896] CRITICAL __main__:     ****    File "/usr/share/weewx/weewxd", line 148, in main

May  6 14:29:07 raspberrypi weewx[6896] CRITICAL __main__:     ****      engine = weewx.engine.StdEngine(config_dict)

May  6 14:29:07 raspberrypi weewx[6896] CRITICAL __main__:     ****    File "/usr/share/weewx/weewx/engine.py", line 75, in __init__

May  6 14:29:07 raspberrypi weewx[6896] CRITICAL __main__:     ****      self.loadServices(config_dict)

May  6 14:29:07 raspberrypi weewx[6896] CRITICAL __main__:     ****    File "/usr/share/weewx/weewx/engine.py", line 138, in loadServices

May  6 14:29:07 raspberrypi weewx[6896] CRITICAL __main__:     ****      obj = weeutil.weeutil.get_object(svc)(self,config_dict)

May  6 14:29:07 raspberrypi weewx[6896] CRITICAL __main__:     ****    File "/usr/share/weewx/weeutil/weeutil.py", line 1093, in get_object

May  6 14:29:07 raspberrypi weewx[6896] CRITICAL __main__:     ****      mod = __import__(module)

May  6 14:29:07 raspberrypi weewx[6896] CRITICAL __main__:     ****    File "/usr/share/weewx/user/owm.py", line 84

May  6 14:29:07 raspberrypi weewx[6896] CRITICAL __main__:     ****      except KeyError, e:

May  6 14:29:07 raspberrypi weewx[6896] CRITICAL __main__:     ****                     ^

May  6 14:29:07 raspberrypi weewx[6896] CRITICAL __main__:     ****  SyntaxError: invalid syntax

May  6 14:29:07 raspberrypi weewx[6896] CRITICAL __main__:     ****  Exiting.

To unsubscribe from this group and stop receiving emails from it, send an email to weewx...@googlegroups.com.

[Tom Keffer]

The problem with 2to3 is that the results are Python 3 only.

The tool python-modernize does a little better and its results will run under 2 or 3, but they will also depend on the library 'six'. 

Drivers and uploaders are especially tricky because they have to do string <--> byte conversions. Those almost always have to be hand coded.

In the end, I've found it easiest just to grind through the code manually.

-tk

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/5edba9a8-8cbb-4ae9-8b86-68b0c5a9cdf7%40googlegroups.com.

[Greg from Oz]

Thanks Tom,

I am converting to python3 and I am know nothing about python at all so going through the code manually for me was not an option. I just used the 2to3 and if it worked then it was ok if it didn't then I just had to delve deeper or contact the author.

I built a 20.04 server Ubuntu and loaded weewx and then put my configuration and all my templates etc on it to see what would I would have to do if I wanted to go wholly pyhton3.

I have converted all the addons etc I am using but I cannot get the Steel series gauges to run with python3 so I have just made the enable true to enable false in the weewx.conf

I can live without the gauges. Below is the error:

May  7 09:35:24 jed165 weewx[2390] DEBUG weewx.reportengine: Running report 'SteelSeries'

May  7 09:35:24 jed165 weewx[2390] DEBUG weewx.reportengine: Found configuration file /etc/weewx/skins/ss/skin.conf for report 'SteelSeries'

May  7 09:35:24 jed165 weewx[2390] INFO weewx.reportengine: Copied 0 files to /var/www/html/weather/ss

May  7 09:35:24 jed165 weewx[2390] DEBUG weewx.cheetahgenerator: Using search list ['weewx.cheetahgenerator.Almanac', 'weewx.cheetahgenerator.S

tation', 'weewx.cheetahgenerator.Current', 'weewx.cheetahgenerator.Stats', 'weewx.cheetahgenerator.UnitInfo', 'weewx.cheetahgenerator.Extras']

May  7 09:35:24 jed165 weewx[2390] DEBUG weewx.manager: Daily summary version is 2.0

May  7 09:35:24 jed165 weewx[2390] ERROR weewx.cheetahgenerator: Generate failed with exception '<class 'TypeError'>'

May  7 09:35:24 jed165 weewx[2390] ERROR weewx.cheetahgenerator: **** Ignoring template /etc/weewx/skins/ss/gauge-data.txt.tmpl

May  7 09:35:24 jed165 weewx[2390] ERROR weewx.cheetahgenerator: **** Reason: '>' not supported between instances of 'NoneType' and 'int'

May  7 09:35:24 jed165 weewx[2390] ERROR weewx.cheetahgenerator: ****  Traceback (most recent call last):

May  7 09:35:24 jed165 weewx[2390] ERROR weewx.cheetahgenerator: ****    File "/usr/share/weewx/weewx/cheetahgenerator.py", line 322, in genera

te

May  7 09:35:24 jed165 weewx[2390] ERROR weewx.cheetahgenerator: ****      unicode_string = compiled_template.respond()

May  7 09:35:24 jed165 weewx[2390] ERROR weewx.cheetahgenerator: ****    File "_etc_weewx_skins_ss_gauge_data_txt_tmpl.py", line 339, in respon

d

May  7 09:35:24 jed165 weewx[2390] ERROR weewx.cheetahgenerator: ****  TypeError: '>' not supported between instances of 'NoneType' and 'int'

Other than that all works so far using the simulator driver. I will try the fineoffset driver later and see what happens.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/5edba9a8-8cbb-4ae9-8b86-68b0c5a9cdf7%40googlegroups.com.

[Colin Larsen]

That seems to have fixed owm - thanks!

Getting this error on a restart - but no errors during the start itself

May  7 11:53:05 raspberrypi weewx[3923] INFO weewx.engine: Main loop exiting. Shutting engine down.

May  7 11:53:05 raspberrypi weewx[3923] INFO __main__: Terminating weewx version 4.0.0

May  7 11:53:05 raspberrypi weewx[3923] INFO __main__:     ****  Traceback (most recent call last):

May  7 11:53:05 raspberrypi weewx[3923] INFO __main__:     ****    File "/usr/share/weewx/weewxd", line 154, in main

May  7 11:53:05 raspberrypi weewx[3923] INFO __main__:     ****      engine.run()

May  7 11:53:05 raspberrypi weewx[3923] INFO __main__:     ****    File "/usr/share/weewx/weewx/engine.py", line 188, in run

May  7 11:53:05 raspberrypi weewx[3923] INFO __main__:     ****      for packet in self.console.genLoopPackets():

May  7 11:53:05 raspberrypi weewx[3923] INFO __main__:     ****    File "/usr/share/weewx/weewx/drivers/vantage.py", line 532, in genLoopPackets

May  7 11:53:05 raspberrypi weewx[3923] INFO __main__:     ****      for _loop_packet in self.genDavisLoopPackets(200):

May  7 11:53:05 raspberrypi weewx[3923] INFO __main__:     ****    File "/usr/share/weewx/weewx/drivers/vantage.py", line 559, in genDavisLoopPackets

May  7 11:53:05 raspberrypi weewx[3923] INFO __main__:     ****      loop_packet = self._get_packet()

May  7 11:53:05 raspberrypi weewx[3923] INFO __main__:     ****    File "/usr/share/weewx/weewx/drivers/vantage.py", line 572, in _get_packet

May  7 11:53:05 raspberrypi weewx[3923] INFO __main__:     ****      _buffer = self.port.read(99)

May  7 11:53:05 raspberrypi weewx[3923] INFO __main__:     ****    File "/usr/share/weewx/weewx/drivers/vantage.py", line 279, in read

May  7 11:53:05 raspberrypi weewx[3923] INFO __main__:     ****      _buffer = self.serial_port.read(chars)

May  7 11:53:05 raspberrypi weewx[3923] INFO __main__:     ****    File "/usr/lib/python3/dist-packages/serial/serialposix.py", line 472, in read

May  7 11:53:05 raspberrypi weewx[3923] INFO __main__:     ****      ready, _, _ = select.select([self.fd, self.pipe_abort_read_r], [], [], timeout.time_left())

May  7 11:53:05 raspberrypi weewx[3923] INFO __main__:     ****    File "/usr/share/weewx/weewxd", line 257, in sigTERMhandler

May  7 11:53:05 raspberrypi weewx[3923] INFO __main__:     ****      raise Terminate

May  7 11:53:05 raspberrypi weewx[3923] INFO __main__:     ****  Terminate

May  7 11:53:10 raspberrypi weewx[3964]: Stopping weewx weather system: weewx..

May  7 11:53:10 raspberrypi systemd[1]: Stopped LSB: weewx weather system.

May  7 11:53:11 raspberrypi systemd[1]: Starting LSB: weewx weather system...

May  7 11:53:11 raspberrypi weewx[4006] INFO __main__: Initializing weewx version 4.0.0

May  7 11:53:11 raspberrypi weewx[4006] INFO __main__: Using Python 3.5.3 (default, Sep 27 2018, 17:25:39) #012[GCC 6.3.0 20170516]

May  7 11:53:11 raspberrypi weewx[4006] INFO __main__: Platform Linux-4.14.71-v7+-armv7l-with-debian-9.4

May  7 11:53:11 raspberrypi weewx[4006] INFO __main__: Locale is 'en_NZ.UTF-8'

May  7 11:53:11 raspberrypi weewx[4006] INFO __main__: PID file is /var/run/weewx.pid

May  7 11:53:11 raspberrypi weewx[4010] INFO __main__: Using configuration file /etc/weewx/weewx.conf

May  7 11:53:11 raspberrypi weewx[4010] INFO weewx.engine: Loading station type Vantage (weewx.drivers.vantage)

May  7 11:53:11 raspberrypi weewx[3994]: Starting weewx weather system: weewx.

May  7 11:53:11 raspberrypi systemd[1]: Started LSB: weewx weather system.

May  7 11:53:11 raspberrypi /weewxd: filepile: Using /home/pi/AQIData.txt with the 'METRIC' unit system

May  7 11:53:11 raspberrypi /weewxd: filepile: Label map is {}

May  7 11:53:11 raspberrypi weewx[4010] INFO weewx.engine: StdConvert target unit is 0x10

May  7 11:53:11 raspberrypi weewx[4010] INFO weewx.wxservices: The following values will be calculated: pressure=prefer_hardware, barometer=prefer_hardware, altimeter=prefer_hardware, windchill=prefer_hardware, heatindex=prefer_hardware, dewpoint=prefer_hardware, inDewpoint=prefer_hardware, rainRate=prefer_hardware, appTemp=software, maxSolarRad=prefer_hardware, cloudbase=prefer_hardware, humidex=prefer_hardware, ET=prefer_hardware, windrun=prefer_hardware

May  7 11:53:11 raspberrypi weewx[4010] INFO weewx.wxservices: The following algorithms will be used for calculations: altimeter=aaASOS, maxSolarRad=RS

May  7 11:53:11 raspberrypi weewx[4010] INFO weewx.engine: Archive will use data binding wx_binding

May  7 11:53:11 raspberrypi weewx[4010] INFO weewx.engine: Record generation will be attempted in 'software'

May  7 11:53:11 raspberrypi weewx[4010] INFO weewx.engine: Using archive interval of 300 seconds (software record generation)

May  7 11:53:11 raspberrypi weewx[4010] INFO weewx.restx: StationRegistry: Station will be registered.

May  7 11:53:11 raspberrypi weewx[4010] INFO weewx.restx: Wunderground: Posting not enabled.

May  7 11:53:11 raspberrypi weewx[4010] INFO weewx.restx: PWSweather: Posting not enabled.

May  7 11:53:11 raspberrypi weewx[4010] INFO weewx.restx: CWOP: Data for station ZL2ARL will be posted

May  7 11:53:11 raspberrypi weewx[4010] INFO weewx.restx: WOW: Posting not enabled.

May  7 11:53:11 raspberrypi weewx[4010] INFO weewx.restx: AWEKAS: Posting not enabled.

May  7 11:53:12 raspberrypi weewx[4010] INFO user.mqtt: service version is 0.22

May  7 11:53:12 raspberrypi weewx[4010] INFO user.mqtt: binding to loop

May  7 11:53:12 raspberrypi weewx[4010] INFO user.mqtt: topic is weather/41south

May  7 11:53:12 raspberrypi weewx[4010] INFO user.mqtt: desired unit system is METRIC

May  7 11:53:12 raspberrypi weewx[4010] INFO user.mqtt: data will be uploaded to mqtt://192.168.20.121:1883/

May  7 11:53:12 raspberrypi /weewxd: restx: Windy: version is 0.4

May  7 11:53:12 raspberrypi /weewxd: restx: Windy: Data will be uploaded to https://stations.windy.com/pws/update

May  7 11:53:12 raspberrypi weewx[4010] INFO user.owm: service version is 0.9

May  7 11:53:12 raspberrypi weewx[4010] INFO user.owm: Data will be uploaded for 5e8d2259cca8ce0001f1aabd

May  7 11:53:12 raspberrypi weewx[4010] INFO __main__: Starting up weewx version 4.0.0

May  7 11:53:12 raspberrypi weewx[4010] INFO weewx.engine: Clock error is -1.17 seconds (positive is fast)

May  7 11:53:12 raspberrypi weewx[4010] INFO weewx.engine: Using binding 'wx_binding' to database 'weewx.sdb'

May  7 11:53:12 raspberrypi weewx[4010] INFO weewx.manager: Starting backfill of daily summaries

May  7 11:53:12 raspberrypi weewx[4010] INFO weewx.engine: Starting main packet loop.

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/7aab2ac7-b93f-4c51-893f-d8c4fc508e4f%40googlegroups.com.

[Colin Larsen]

Whoops - nope, now getting this error from owm.py

May  7 11:55:15 raspberrypi weewx[4010] ERROR weewx.restx: OWM: Unexpected exception of type <class 'TypeError'>

May  7 11:55:15 raspberrypi weewx[4010] ERROR weewx.restx: *** Traceback (most recent call last):

May  7 11:55:15 raspberrypi weewx[4010] ERROR weewx.restx: ***   File "/usr/share/weewx/weewx/restx.py", line 378, in run_loop

May  7 11:55:15 raspberrypi weewx[4010] ERROR weewx.restx: ***     self.process_record(_record, dbmanager)

May  7 11:55:15 raspberrypi weewx[4010] ERROR weewx.restx: ***   File "/usr/share/weewx/user/owm.py", line 210, in process_record

May  7 11:55:15 raspberrypi weewx[4010] ERROR weewx.restx: ***     self.post_with_retries(req)

May  7 11:55:15 raspberrypi weewx[4010] ERROR weewx.restx: ***   File "/usr/share/weewx/weewx/restx.py", line 472, in post_with_retries

May  7 11:55:15 raspberrypi weewx[4010] ERROR weewx.restx: ***     _response = self.post_request(request, data)

May  7 11:55:15 raspberrypi weewx[4010] ERROR weewx.restx: ***   File "/usr/share/weewx/weewx/restx.py", line 539, in post_request

May  7 11:55:15 raspberrypi weewx[4010] ERROR weewx.restx: ***     _response = urllib.request.urlopen(request, data=data_bytes, timeout=self.timeout)

May  7 11:55:15 raspberrypi weewx[4010] ERROR weewx.restx: ***   File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen

May  7 11:55:15 raspberrypi weewx[4010] ERROR weewx.restx: ***     return opener.open(url, data, timeout)

May  7 11:55:15 raspberrypi weewx[4010] ERROR weewx.restx: ***   File "/usr/lib/python3.5/urllib/request.py", line 464, in open

May  7 11:55:15 raspberrypi weewx[4010] ERROR weewx.restx: ***     req = meth(req)

May  7 11:55:15 raspberrypi weewx[4010] ERROR weewx.restx: ***   File "/usr/lib/python3.5/urllib/request.py", line 1183, in do_request_

May  7 11:55:15 raspberrypi weewx[4010] ERROR weewx.restx: ***     raise TypeError(msg)

May  7 11:55:15 raspberrypi weewx[4010] ERROR weewx.restx: *** TypeError: POST data should be bytes or an iterable of bytes. It cannot be of type str.

May  7 11:55:15 raspberrypi weewx[4010] CRITICAL weewx.restx: OWM: Thread terminating. Reason: POST data should be bytes or an iterable of bytes. It cannot be of type str.

[Tom Keffer]

Missed that.

Give it another try:

https://raw.githubusercontent.com/tkeffer/weewx-owm/master/bin/user/owm.py

-tk

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/CACjxfUuJP1t_wr%3Dmq8R2tyZP6K9Qo1KdZzY63k64YpuL1pXm0w%40mail.gmail.com.

[Colin Larsen]

That fixed it - thanks!!

My only error now is the one at service stop - no eros reported at start or while running

May  7 16:24:53 raspberrypi systemd[1]: Stopping LSB: weewx weather system...

May  7 16:24:53 raspberrypi weewx[10196] INFO __main__: Received signal TERM (15).

May  7 16:24:53 raspberrypi weewx[10196] INFO weewx.engine: Main loop exiting. Shutting engine down.

May  7 16:24:53 raspberrypi weewx[10196] INFO weewx.engine: Shutting down StdReport thread

May  7 16:24:53 raspberrypi weewx[10196] INFO __main__: Terminating weewx version 4.0.0

May  7 16:24:53 raspberrypi weewx[10196] INFO __main__:     ****  Traceback (most recent call last):

May  7 16:24:53 raspberrypi weewx[10196] INFO __main__:     ****    File "/usr/share/weewx/weewxd", line 154, in main

May  7 16:24:53 raspberrypi weewx[10196] INFO __main__:     ****      engine.run()

May  7 16:24:53 raspberrypi weewx[10196] INFO __main__:     ****    File "/usr/share/weewx/weewx/engine.py", line 188, in run

May  7 16:24:53 raspberrypi weewx[10196] INFO __main__:     ****      for packet in self.console.genLoopPackets():

May  7 16:24:53 raspberrypi weewx[10196] INFO __main__:     ****    File "/usr/share/weewx/weewx/drivers/vantage.py", line 532, in genLoopPackets

May  7 16:24:53 raspberrypi weewx[10196] INFO __main__:     ****      for _loop_packet in self.genDavisLoopPackets(200):

May  7 16:24:53 raspberrypi weewx[10196] INFO __main__:     ****    File "/usr/share/weewx/weewx/drivers/vantage.py", line 559, in genDavisLoopPackets

May  7 16:24:53 raspberrypi weewx[10196] INFO __main__:     ****      loop_packet = self._get_packet()

May  7 16:24:53 raspberrypi weewx[10196] INFO __main__:     ****    File "/usr/share/weewx/weewx/drivers/vantage.py", line 572, in _get_packet

May  7 16:24:53 raspberrypi weewx[10196] INFO __main__:     ****      _buffer = self.port.read(99)

May  7 16:24:53 raspberrypi weewx[10196] INFO __main__:     ****    File "/usr/share/weewx/weewx/drivers/vantage.py", line 279, in read

May  7 16:24:53 raspberrypi weewx[10196] INFO __main__:     ****      _buffer = self.serial_port.read(chars)

May  7 16:24:53 raspberrypi weewx[10196] INFO __main__:     ****    File "/usr/lib/python3/dist-packages/serial/serialposix.py", line 472, in read

May  7 16:24:53 raspberrypi weewx[10196] INFO __main__:     ****      ready, _, _ = select.select([self.fd, self.pipe_abort_read_r], [], [], timeout.time_left())

May  7 16:24:53 raspberrypi weewx[10196] INFO __main__:     ****    File "/usr/share/weewx/weewxd", line 257, in sigTERMhandler

May  7 16:24:53 raspberrypi weewx[10196] INFO __main__:     ****      raise Terminate

May  7 16:24:53 raspberrypi weewx[10372]: Stopping weewx weather system: weewx.

May  7 16:24:53 raspberrypi systemd[1]: Stopped LSB: weewx weather system.

May  7 16:24:53 raspberrypi weewx[10196] INFO __main__:     ****  Terminate

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/CAPq0zEAi7dv7g%2BRWJTPCsLvj5J9pUCxbXOfE8e5WN5%2B0eJQVFQ%40mail.gmail.com.

[gjr80]

As you are no doubt aware by now tracking down python errors that occur in a template (as distinct from python errors in a SLE or the WeeWX code base) can be a real pain as we can't reconcile the line number given in the error message with any source code. The following line tells us the error is in some in-line python code in the gauge-data.txt.tmpl template:

May  7 09:35:24 jed165 weewx[2390] ERROR weewx.cheetahgenerator: ****    File "_etc_weewx_skins_ss_gauge_data_txt_tmpl.py", line 339, in respond

If the error was in an SLE or the WeeWX code base we would have a nicely named (and known) .py (eg mySLE.py or cheetahgenerator.py) file rather than the cryptic name above. We can't use the line number from _etc_weewx_skins_ss_gauge_data_txt_tmpl.py as it is a temporary file created by Cheetah that we have no access to. The only thing we can use is the error message text, in your case:

May  7 09:35:24 jed165 weewx[2390] ERROR weewx.cheetahgenerator: ****  TypeError: '>' not supported between instances of 'NoneType' and 'int'

That message tells us a few things. We know that we have an error in an expression involving '>' (greater than). As it turns out > appears in the template only twice, at lines 69 and 277:

#if $h > 0 and #if $hour.rain.sum.raw > $hourlyrainTH

We also know the left hand side is equating to None (the cause of the error) and the right hand side of the comparison is equating to an integer. If we look at line 69 if $h was None that would throw the error we are getting (0 in considered an int). $h is calculated at the line 68 and indirectly involves lines 66 and 67:

#set $dp_K = $current.dewpoint.degree_C.raw + 273.16
#set $e = 6.11 * math.exp(5417.7530 * ((1/273.16) - 1/$dp_K))
#set $h = 0.5555 * ($e - 10.0)


We know $dp_K is not None; if it was we would get and error on line 66 where $dp_K is calculated (None + 273.16). Therefore we know $h cannot be None. So line 69 is not the source of the error.

If we look at line 277 to get the error we are seeing $hour.rain.sum.raw would need to be None and $hourlyrainTH would need to be an int. $hourlyrainTH is initialised to 0 at line 274, so $hourlyrainTH could be well be an int. Could $hour.rain.sum.raw be None, quite possible if there is no rain data (as distinct from there being data but it is 0) for a full hour sometime today. So it looks like we have found the problem line. How to fix it? Given the purpose of that piece of that piece of code is to find the hour of the highest rainfall today we can just put another conditional in there so we skip hours with no rainfall data, eg (not tested):

#for $hour in $day.hours
#if $hour.rain.sum.raw is not None and $hour.rain.sum.raw > $hourlyrainTH
#set $hourlyrainTH = $hour.rain.sum.raw

Now if $hour.rain.sum.raw is None the first conditional will cause execution to skip the rest of the if statement avoiding the problematic '>'.

Apologies this was somewhat long winded, but I thought it a perfect opportunity to highlight one of the (few) techniques for tracking down in-line python errors in templates.

Gary

[Greg from Oz]

Thanks Gary.

All this is a learning curve for me.

I will have a crack at this tomorrow and see what happens.

I worked out most of the other things that didn't work by trial and error (mostly error) 

I wasn't in a hurry to get the gauges to work and I know you would be busy looking at other more important things and I appreciate you answering the question.

I built a 20.04 server on real hardware just to test all of this and have a script to rebuild it again once I iron the bugs out.

Again Thanks

[Greg from Oz]

Hi Gary,

I add the highlighted text you suggested into line 277 of gauge-data.txt.tmpl

so it in now like this:

#if $hour.rain.sum.raw is not None and $hour.rain.sum.raw > $hourlyrainTH

and I had no errors and the gauges have now appeared.

So it works. I don't think I could have worked that one out by myself.

Hopefully this will help someone else as well. I will let you know if anything else happens but so far so good.

Thanks

On Thursday, 7 May 2020 17:03:06 UTC+10, gjr80 wrote:

[gjr80]

Good, I have submitted a PR for the change to the repo owner.

Gary

[Colin Larsen]

Hi gents

Is there anything I can do for the error I posted above that only seems to happen on Weewx service stop?

Thanks

Colin

--

You received this message because you are subscribed to the Google Groups "weewx-user" group.

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/9055c81f-e90c-4fd2-9bfa-04350f04f2f2%40googlegroups.com.

[gjr80]

There is no error. It is an informational stack trace on WeeWX exit and it cannot be turned off without modifying the code base.

Gary

[Colin Larsen]

Thanks Gary, does that mean that it's just peculiar to my setup? I haven't heard of any others reporting it. 

Good to know it's nothing serious :)

Colin

On Sun, 10 May 2020, 08:25 gjr80, <gjrod...@gmail.com> wrote:

There is no error. It is an informational stack trace on WeeWX exit and it cannot be turned off without modifying the code base.

Gary

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/34e60342-0957-4a54-83e7-c959004745b4%40googlegroups.com.

[gjr80]

A couple of things happened in WeeWX 4, a fair amount of code was moved from engine.py to weewxd, this included the code that adds the stack trace on exit, plus WeeWX logging changed from using the syslog package to using the logging package. Under 3.9.2 the stack trace on exit was sent to the log as debug level output, so it was only seen if debug >= 1. There was a subsequent change by Tom that explicitly states he was changing the log level for that stack trace to info (will appear in log irrespective of debug setting). But I had some trouble following the commits on github as the code seemed to unexplainably change again. I think it is one for Tom given the explicit commit he made on it.

Bottom line it is nothing wrong with your install or nor is it caused by anything you have done (other than shutting down or restarting WeeWX).

Gary

[Colin Larsen]

Perfect, thank you sir, case closed :)

And again many thanks for all the work that goes into this project.

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/1a88958e-02d1-456b-a37f-321c1394d960%40googlegroups.com.

[Tom Keffer]

I can't think of any reason why I changed the stack trace from INFO to DEBUG. I've changed it back again.

-tk

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/CACjxfUtx-nJGS1FJkvms9FwHmpRgKmhz%3D215kmQG2_cw3RcOqw%40mail.gmail.com.