MQTT error inconsistent units

[Tadej Javornik]

Hello, i want to use mqtt with Belchertown skin but I am having problems with the upload. When i run weewx i get following output.

Jul  3 13:07:58 raspberrypi weewx[22086]: restx: MQTT: service version is 0.19

Jul  3 13:07:58 raspberrypi weewx[22086]: restx: MQTT: binding to ['archive', 'loop']

Jul  3 13:07:58 raspberrypi weewx[22086]: restx: MQTT: topic is xxx

Jul  3 13:07:58 raspberrypi weewx[22086]: restx: MQTT: data will be uploaded to mqtt://xx...@m24.cloudmqtt.com:10417/

Jul  3 13:08:01 raspberrypi weewx[22086]: restx: MQTT: Unexpected exception of type <type 'exceptions.ValueError'>

Jul  3 13:08:01 raspberrypi weewx[22086]: restx: MQTT: Thread exiting. Reason: Inconsistent units (1 vs 1 vs 16) when querying for hourRain

What am I doing wrong?

[Pat]

This could be a unit inconsistency problem. 

What do you have set for target_unit in weewx.conf? 

And what do you have set for unit_system for [[MQTT]] in weewx.conf?

[Tadej Javornik]

weewx.conf :

target_unit = US 

[[MQTT]]

        server_url = mqtt://x...@m24.cloudmqtt.com:10417/

        topic = xxx

        binding = archive, loop

        aggregation = individual, aggregate

That is all i got in my weewx.conf.

Dne sreda, 03. julij 2019 14.12.28 UTC+2 je oseba Pat napisala:

[Pat]

I am not sure if this will help, but try setting this in your weewx.conf [[MQTT]], then restart weewx. 

I added the unit_system line. 

[[MQTT]]

    server_url = mqtt://x...@m24.cloudmqtt.com:10417/

    topic = xxx

    binding = archive, loop

    aggregation = individual, aggregate

    unit_system = US

[Tadej Javornik]

It is the same, except now it is showing that unit sistem is in US 

Jul  3 14:28:26 raspberrypi weewx[27355]: restx: MQTT: service version is 0.19

Jul  3 14:28:26 raspberrypi weewx[27355]: restx: MQTT: binding to ['archive', 'loop']

Jul  3 14:28:26 raspberrypi weewx[27355]: restx: MQTT: topic is vremetadej

Jul  3 14:28:26 raspberrypi weewx[27355]: restx: MQTT: desired unit system is US

Jul  3 14:28:26 raspberrypi weewx[27355]: restx: MQTT: data will be uploaded to mqtt:/x...@m24.cloudmqtt.com:10417/

Jul  3 14:28:31 raspberrypi weewx[27355]: restx: MQTT: Unexpected exception of type <type 'exceptions.ValueError'>

Jul  3 14:28:31 raspberrypi weewx[27355]: restx: MQTT: Thread exiting. Reason: Inconsistent units (1 vs 1 vs 16) when querying for hourRain

Dne sreda, 03. julij 2019 14.21.18 UTC+2 je oseba Pat napisala:

[Pat]

Ok, it might help to provide a full debug. Setup debug=1 in your weewx.conf, restart weewx and post the whole syslog from weewx startup to the error. Make sure to remove any passwords or sensitive information. 

[Tadej Javornik]

Jul  4 13:57:36 raspberrypi systemd[1]: Starting LSB: weewx weather system...

Jul  4 13:57:38 raspberrypi weewx[3641]: engine: Initializing weewx version 3.9.1

Jul  4 13:57:38 raspberrypi weewx[3641]: engine: Using Python 2.7.9 (default, Mar  8 2015, 00:52:26) #012[GCC 4.9.2]

Jul  4 13:57:38 raspberrypi weewx[3641]: engine: Platform Linux-4.1.13+-armv6l-with-debian-8.0

Jul  4 13:57:38 raspberrypi weewx[3641]: engine: Locale is 'en_US.UTF-8'

Jul  4 13:57:38 raspberrypi weewx[3641]: engine: pid file is /var/run/weewx.pid

Jul  4 13:57:38 raspberrypi weewx[3631]: Starting weewx weather system: weewx.

Jul  4 13:57:39 raspberrypi systemd[1]: Started LSB: weewx weather system.

Jul  4 13:57:39 raspberrypi weewx[3645]: engine: Using configuration file /home/weewx/weewx.conf

Jul  4 13:57:39 raspberrypi weewx[3645]: engine: Debug is 1

Jul  4 13:57:39 raspberrypi weewx[3645]: engine: Initializing engine

Jul  4 13:57:39 raspberrypi weewx[3645]: engine: Loading station type WS23xx (weewx.drivers.ws23xx)

Jul  4 13:57:39 raspberrypi weewx[3645]: ws23xx: driver version is 0.27

Jul  4 13:57:39 raspberrypi weewx[3645]: ws23xx: serial port is /dev/ttyUSB0

Jul  4 13:57:39 raspberrypi weewx[3645]: ws23xx: polling interval is None

Jul  4 13:57:39 raspberrypi weewx[3645]: ws23xx: create LinuxSerialPort

Jul  4 13:57:39 raspberrypi weewx[3645]: ws23xx: create Ws2300

Jul  4 13:57:39 raspberrypi weewx[3645]: engine: Loading service weewx.engine.StdTimeSynch

Jul  4 13:57:39 raspberrypi weewx[3645]: engine: Finished loading service weewx.engine.StdTimeSynch

Jul  4 13:57:39 raspberrypi weewx[3645]: engine: Loading service user.pond.PondService

Jul  4 13:57:39 raspberrypi weewx[3645]: pond: using /usb/www/test.txt

Jul  4 13:57:39 raspberrypi weewx[3645]: engine: Finished loading service user.pond.PondService

Jul  4 13:57:39 raspberrypi weewx[3645]: engine: Loading service user.strele.streleservice

Jul  4 13:57:39 raspberrypi weewx[3645]: strele: using /usb/www/strele/total_strikes10.txt

Jul  4 13:57:39 raspberrypi weewx[3645]: engine: Finished loading service user.strele.streleservice

Jul  4 13:57:39 raspberrypi weewx[3645]: engine: Loading service user.wat.watservice

Jul  4 13:57:39 raspberrypi weewx[3645]: wat: using /usb/www/index1.txt

Jul  4 13:57:39 raspberrypi weewx[3645]: engine: Finished loading service user.wat.watservice

Jul  4 13:57:39 raspberrypi weewx[3645]: engine: Loading service user.sneg.snegservice

Jul  4 13:57:39 raspberrypi weewx[3645]: pmb: using /usb/www/sneg/sneg.txt

Jul  4 13:57:39 raspberrypi weewx[3645]: engine: Finished loading service user.sneg.snegservice

Jul  4 13:57:39 raspberrypi weewx[3645]: engine: Loading service user.nula.nulaservice

Jul  4 13:57:39 raspberrypi weewx[3645]: pmb: using /usb/www/nula.txt

Jul  4 13:57:39 raspberrypi weewx[3645]: engine: Finished loading service user.nula.nulaservice

Jul  4 13:57:39 raspberrypi weewx[3645]: engine: Loading service user.pm10.pmaservice

Jul  4 13:57:39 raspberrypi weewx[3645]: pma: using /usb/www/pm/pm10.txt

Jul  4 13:57:40 raspberrypi weewx[3645]: engine: Finished loading service user.pm10.pmaservice

Jul  4 13:57:40 raspberrypi weewx[3645]: engine: Loading service user.pm25.pmbservice

Jul  4 13:57:40 raspberrypi weewx[3645]: pmb: using /usb/www/pm/pm25.txt

Jul  4 13:57:40 raspberrypi weewx[3645]: engine: Finished loading service user.pm25.pmbservice

Jul  4 13:57:40 raspberrypi weewx[3645]: engine: Loading service user.10cm.cmservice

Jul  4 13:57:40 raspberrypi weewx[3645]: pmb: using /usb/www/tla/xaa

Jul  4 13:57:40 raspberrypi weewx[3645]: engine: Finished loading service user.10cm.cmservice

Jul  4 13:57:40 raspberrypi weewx[3645]: engine: Loading service user.30cm.cmmservice

Jul  4 13:57:40 raspberrypi weewx[3645]: 30cm: using /usb/www/tla/xab

Jul  4 13:57:40 raspberrypi weewx[3645]: engine: Finished loading service user.30cm.cmmservice

Jul  4 13:57:40 raspberrypi weewx[3645]: engine: Loading service user.mqtt.MQTT

Jul  4 13:57:40 raspberrypi weewx[3645]: restx: MQTT: service version is 0.19

Jul  4 13:57:40 raspberrypi weewx[3645]: restx: MQTT: binding to ['archive', 'loop']

Jul  4 13:57:40 raspberrypi weewx[3645]: restx: MQTT: topic is vremetadej

Jul  4 13:57:40 raspberrypi weewx[3645]: restx: MQTT: desired unit system is US

Jul  4 13:57:40 raspberrypi weewx[3645]: restx: MQTT: data will be uploaded to mqtt://x...@m24.cloudmqtt.com:10417/

Jul  4 13:57:40 raspberrypi weewx[3645]: engine: Finished loading service user.mqtt.MQTT

Jul  4 13:57:40 raspberrypi weewx[3645]: engine: Loading service weewx.engine.StdConvert

Jul  4 13:57:40 raspberrypi weewx[3645]: engine: StdConvert target unit is 0x1

Jul  4 13:57:40 raspberrypi weewx[3645]: engine: Finished loading service weewx.engine.StdConvert

Jul  4 13:57:40 raspberrypi weewx[3645]: engine: Loading service weewx.engine.StdCalibrate

Jul  4 13:57:40 raspberrypi weewx[3645]: engine: Finished loading service weewx.engine.StdCalibrate

Jul  4 13:57:40 raspberrypi weewx[3645]: engine: Loading service weewx.engine.StdQC

Jul  4 13:57:40 raspberrypi weewx[3645]: engine: Finished loading service weewx.engine.StdQC

Jul  4 13:57:40 raspberrypi weewx[3645]: engine: Loading service weewx.wxservices.StdWXCalculate

Jul  4 13:57:41 raspberrypi weewx[3645]: wxcalculate: The following values will be calculated: barometer=prefer_hardware, windchill=prefer_hardware, dewpoint=prefer_hardware, appTemp=prefer_hardware, rainRate=prefer_hardware, windrun=prefer_hardware, heatindex=prefer_hardware, maxSolarRad=prefer_hardware, humidex=prefer_hardware, pressure=prefer_hardware, inDewpoint=prefer_hardware, ET=prefer_hardware, altimeter=prefer_hardware, cloudbase=prefer_hardware

Jul  4 13:57:41 raspberrypi weewx[3645]: wxcalculate: The following algorithms will be used for calculations: altimeter=aaNOAA, maxSolarRad=RS

Jul  4 13:57:41 raspberrypi weewx[3645]: engine: Finished loading service weewx.wxservices.StdWXCalculate

Jul  4 13:57:41 raspberrypi weewx[3645]: engine: Loading service weewx.engine.StdArchive

Jul  4 13:57:41 raspberrypi weewx[3645]: engine: Archive will use data binding wx_binding

Jul  4 13:57:41 raspberrypi weewx[3645]: engine: Record generation will be attempted in 'software'

Jul  4 13:57:41 raspberrypi weewx[3645]: engine: Using archive interval of 600 seconds (software record generation)

Jul  4 13:57:41 raspberrypi weewx[3645]: engine: Use LOOP data in hi/low calculations: 1

Jul  4 13:57:41 raspberrypi weewx[3645]: manager: Daily summary version is 2.0

Jul  4 13:57:41 raspberrypi weewx[3645]: manager: Daily summary version is 2.0

Jul  4 13:57:41 raspberrypi weewx[3645]: engine: Using binding 'wx_binding' to database 'weewx.sdb'

Jul  4 13:57:41 raspberrypi weewx[3645]: manager: Starting backfill of daily summaries

Jul  4 13:57:41 raspberrypi weewx[3645]: engine: Finished loading service weewx.engine.StdArchive

Jul  4 13:57:41 raspberrypi weewx[3645]: engine: Loading service weewx.restx.StdStationRegistry

Jul  4 13:57:41 raspberrypi weewx[3645]: restx: StationRegistry: Registration not requested.

Jul  4 13:57:41 raspberrypi weewx[3645]: engine: Finished loading service weewx.restx.StdStationRegistry

Jul  4 13:57:41 raspberrypi weewx[3645]: engine: Loading service weewx.restx.StdWunderground

Jul  4 13:57:41 raspberrypi weewx[3645]: restx: WU essentials: {}

Jul  4 13:57:41 raspberrypi weewx[3645]: restx: Wunderground-PWS: Data for station xxxx will be posted

Jul  4 13:57:41 raspberrypi weewx[3645]: engine: Finished loading service weewx.restx.StdWunderground

Jul  4 13:57:41 raspberrypi weewx[3645]: engine: Loading service weewx.restx.StdPWSweather

Jul  4 13:57:41 raspberrypi weewx[3645]: restx: PWSweather: Posting not enabled.

Jul  4 13:57:41 raspberrypi weewx[3645]: engine: Finished loading service weewx.restx.StdPWSweather

Jul  4 13:57:41 raspberrypi weewx[3645]: engine: Loading service weewx.restx.StdCWOP

Jul  4 13:57:41 raspberrypi weewx[3645]: restx: CWOP: Posting not enabled.

Jul  4 13:57:41 raspberrypi weewx[3645]: engine: Finished loading service weewx.restx.StdCWOP

Jul  4 13:57:41 raspberrypi weewx[3645]: engine: Loading service weewx.restx.StdWOW

Jul  4 13:57:41 raspberrypi weewx[3645]: restx: WOW: Posting not enabled.

Jul  4 13:57:41 raspberrypi weewx[3645]: engine: Finished loading service weewx.restx.StdWOW

Jul  4 13:57:41 raspberrypi weewx[3645]: engine: Loading service weewx.restx.StdAWEKAS

Jul  4 13:57:41 raspberrypi weewx[3645]: restx: AWEKAS: Posting not enabled.

Jul  4 13:57:41 raspberrypi weewx[3645]: engine: Finished loading service weewx.restx.StdAWEKAS

Jul  4 13:57:41 raspberrypi weewx[3645]: engine: Loading service weewx.engine.StdPrint

Jul  4 13:57:41 raspberrypi weewx[3645]: engine: Finished loading service weewx.engine.StdPrint

Jul  4 13:57:41 raspberrypi weewx[3645]: engine: Loading service weewx.engine.StdReport

Jul  4 13:57:41 raspberrypi weewx[3645]: engine: Finished loading service weewx.engine.StdReport

Jul  4 13:57:41 raspberrypi weewx[3645]: engine: Starting up weewx version 3.9.1

Jul  4 13:57:41 raspberrypi weewx[3645]: engine: Station does not support reading the time

Jul  4 13:57:41 raspberrypi weewx[3645]: ws23xx: gen_records: since_ts=1562241000 count=0 clock=True

Jul  4 13:57:41 raspberrypi weewx[3645]: manager: Daily summary version is 2.0

Jul  4 13:57:41 raspberrypi weewx[3645]: ws23xx: using computer clock with latest_ts of 2019-07-04 13:55:41 CEST (1562241341)

Jul  4 13:57:41 raspberrypi weewx[3645]: ws23xx: count is 0 to satisfy timestamp of 2019-07-04 13:50:00 CEST (1562241000)

Jul  4 13:57:41 raspberrypi weewx[3645]: engine: Starting main packet loop.

Jul  4 13:57:43 raspberrypi weewx[3645]: restx: MQTT: Unexpected exception of type <type 'exceptions.ValueError'>

Jul  4 13:57:43 raspberrypi weewx[3645]: *** Traceback (most recent call last):

Jul  4 13:57:43 raspberrypi weewx[3645]: ***   File "/home/weewx/bin/weewx/restx.py", line 350, in run_loop

Jul  4 13:57:43 raspberrypi weewx[3645]: ***     self.process_record(_record, dbmanager)

Jul  4 13:57:43 raspberrypi weewx[3645]: ***   File "/home/weewx/bin/user/mqtt.py", line 396, in process_record

Jul  4 13:57:43 raspberrypi weewx[3645]: ***     record = self.get_record(record, dbm)

Jul  4 13:57:43 raspberrypi weewx[3645]: ***   File "/home/weewx/bin/weewx/restx.py", line 270, in get_record

Jul  4 13:57:43 raspberrypi weewx[3645]: ***     (_result[1], _result[2], record['usUnits']))

Jul  4 13:57:43 raspberrypi weewx[3645]: *** ValueError: Inconsistent units (1 vs 1 vs 16) when querying for hourRain

Jul  4 13:57:43 raspberrypi weewx[3645]: restx: MQTT: Thread exiting. Reason: Inconsistent units (1 vs 1 vs 16) when querying for hourRain

Jul  4 13:57:43 raspberrypi weewx[3645]: ws23xx: connection changed from unknown to cable

Jul  4 13:57:43 raspberrypi weewx[3645]: ws23xx: using 8 second polling interval for cable connection

Dne sreda, 03. julij 2019 14.41.11 UTC+2 je oseba Pat napisala:

[mwall]

On Thursday, July 4, 2019 at 8:01:24 AM UTC-4, Tadej Javornik wrote:

Jul  4 13:57:43 raspberrypi weewx[3645]: restx: MQTT: Unexpected exception of type <type 'exceptions.ValueError'>

Jul  4 13:57:43 raspberrypi weewx[3645]: *** Traceback (most recent call last):

Jul  4 13:57:43 raspberrypi weewx[3645]: ***   File "/home/weewx/bin/weewx/restx.py", line 350, in run_loop

Jul  4 13:57:43 raspberrypi weewx[3645]: ***     self.process_record(_record, dbmanager)

Jul  4 13:57:43 raspberrypi weewx[3645]: ***   File "/home/weewx/bin/user/mqtt.py", line 396, in process_record

Jul  4 13:57:43 raspberrypi weewx[3645]: ***     record = self.get_record(record, dbm)

Jul  4 13:57:43 raspberrypi weewx[3645]: ***   File "/home/weewx/bin/weewx/restx.py", line 270, in get_record

Jul  4 13:57:43 raspberrypi weewx[3645]: ***     (_result[1], _result[2], record['usUnits']))

Jul  4 13:57:43 raspberrypi weewx[3645]: *** ValueError: Inconsistent units (1 vs 1 vs 16) when querying for hourRain

Jul  4 13:57:43 raspberrypi weewx[3645]: restx: MQTT: Thread exiting. Reason: Inconsistent units (1 vs 1 vs 16) when querying for hourRain

the unit system in your database is 1 (US) but the unit system in the record is 16 (METRIC)

so when weewx tries to calculate hourRain, it fails

one workaround is to disable the calculation of hourRain (and rain24 and dayRain) by setting augment_record to False:

[MQTT]

    augment_record = False

but the bigger question is how you are getting packets that have a METRIC unit system, even though you are using a database with US unit system.

i'll bet that one of those many other services that you are running is sending out packets in METRIC unit system.

so that means the root cause is units confusion between the services you have added, the mqtt uploader, and the restx base class/utilities.

let me think about a proper solution (it is probably obvious, but my head is still a bit addled by the sun and altitude)

m

[Tadej Javornik]

When i made weewx for the first time I didn change unit sistem I left it in US so my database is in US format. My webpages that are generated are all in METRIC (mm, m/s..) what is okay because i'm from EU(Slo.) I don't really know how I set it up that this is working because it was 5 years ago. All my other services are plain numbers, I used database schema and occupied those fields. At first I was having problems because for example I wanted to put in extraTemp1 my data from .txt file which was measurement from dust sensor( ex. 330) but the weewx thought that it was temperature in F and it converted it to C in web page generation so the number was different.  I then simply used another database field with no calculation example:UV.. So I made quiet a mess.. But it is working normaly  :)

But I didn't touch any rain data ..

Dne četrtek, 04. julij 2019 14.19.50 UTC+2 je oseba mwall napisala:

[mwall]

On Thursday, July 4, 2019 at 8:30:49 AM UTC-4, Tadej Javornik wrote:

When i made weewx for the first time I didn change unit sistem I left it in US so my database is in US format. My webpages that are generated are all in METRIC (mm, m/s..) what is okay because i'm from EU(Slo.) I don't really know how I set it up that this is working because it was 5 years ago. All my other services are plain numbers, I used database schema and occupied those fields. At first I was having problems because for example I wanted to put in extraTemp1 my data from .txt file which was measurement from dust sensor( ex. 330) but the weewx thought that it was temperature in F and it converted it to C in web page generation so the number was different.  I then simply used another database field with no calculation example:UV.. So I made quiet a mess.. But it is working normaly  :)

But I didn't touch any rain data ..

any service that adds observations to a packet/record must ensure that the units of each observation match the unit system of the packet/record to which it is adding

any driver that creates a packet/record must ensure that the units of each observation are consistent with the unit system of the packet

you must also ensure that the units are defined for any database fields that have units, otherwise you'll get unexpected behavior in your reports

but that is not really the issue here

if 'augment_record' is true (the default), then the mqtt uploader attempts to add some rain calculations to the packets it uploads.  it does this using the get_record method in restx, whether or not there are any rain observations in the packet, and no matter what the unit system of the packet

as a short-term fix, set augment_record=False

then restart weewx

m

[Tadej Javornik]

I tryed this but it gives me different error.. I thing i will just leave it as is and just generate pages every 10 minutes :) 

Jul  5 14:07:38 raspberrypi weewx[13829]: restx: MQTT: Unexpected exception of type <type 'exceptions.AttributeError'>

Jul  5 14:07:38 raspberrypi weewx[13829]: *** Traceback (most recent call last):

Jul  5 14:07:38 raspberrypi weewx[13829]: ***   File "/home/weewx/bin/weewx/restx.py", line 350, in run_loop

Jul  5 14:07:38 raspberrypi weewx[13829]: ***     self.process_record(_record, dbmanager)

Jul  5 14:07:38 raspberrypi weewx[13829]: ***   File "/home/weewx/bin/user/mqtt.py", line 420, in process_record

Jul  5 14:07:38 raspberrypi weewx[13829]: ***     if self.aggregation.find('aggregate') >= 0:

Jul  5 14:07:39 raspberrypi weewx[13829]: *** AttributeError: 'list' object has no attribute 'find'

Dne četrtek, 04. julij 2019 14.39.31 UTC+2 je oseba mwall napisala: