MQTTSubscribe service - not getting extraTemp1 or 2

[lloyd...@gmail.com]

Not getting extraTemp1 nor extraTemp2 that are being sent via MQTT.  Any help would be appreciated.

I know the data is getting to the mosquitto broker.

This is the setup on weewx.conf.

[MQTTSubscribeService]

# This section is for the MQTTSubscribe service.

# Turn the service on and off.

# Default is: true

# Only used by the service.

enable = true

# The MQTT server.

# Default is localhost.

host = 192.168.1.166

# The port to connect to.

# Default is 1883.

port = 1883

# Maximum period in seconds allowed between communications with the broker.

# Default is 60.

keepalive = 120

# username for broker authentication.

# Default is None.

Username = ODT

# password for broker authentication.

# Default is None.

Password = ??????

# The binding, loop or archive.

# Default is: loop

# Only used by the service.

binding = loop

# The message handler to use

[[message_callback]]

# The format of the MQTT payload.

# Currently support: individual, json, keyword

# Must be specified.

type = individual

# The topics to subscribe to.

[[topics]]

# Units for MQTT payloads without unit value.

# Valid values: US, METRIC, METRICWX

# Default is: US

unit_system = METRIC

# When true, the fieldname is set to the topic and therefore [[[[fieldname]]]] cannot be used.

# This allows the [[[[fieldname]]]] configuration to be specified directly under the [[[topic]]].

# Default is False.

use_topic_as_fieldname = True

# The first topic to subscribe to.

[[[topic/rtd_temperature]]]

# The WeeWX name.

# Default is the name from MQTT.

name = extraTemp1

# True if the incoming data should not be processed into WeeWX.

# Valid values: True, False

# Default is False

ignore = False

# True if the incoming data is cumulative.

# Valid values: True, False

# Default is False

contains_total = False

# The conversion type necessary for WeeWX compatibility

# Valid values: bool, float, int, none

# Default is float

conversion_type = float

# The units of the incoming data.

# Useful if this field's units differ from the topic's unit_system's units.

# Valid values: see, http://www.weewx.com/docs/customizing.htm#units

# Default is not set

units = degree_C

[[[topic/tmp117_temperature]]]

# The WeeWX name.

# Default is the name from MQTT.

name = extraTemp2

# True if the incoming data should not be processed into WeeWX.

# Valid values: True, False

# Default is False

ignore = False

# True if the incoming data is cumulative.

# Valid values: True, False

# Default is False

contains_total = False

# The conversion type necessary for WeeWX compatibility

# Valid values: bool, float, int, none

# Default is float

conversion_type = float

# The units of the incoming data.

# Useful if this field's units differ from the topic's unit_system's units.

# Valid values: see, http://www.weewx.com/docs/customizing.htm#units

# Default is not set

units = degree_C

When I run debug = 1 I get this but no extraTemp1 or 2

Mar 2 15:26:15 pi34 weewx[4895] DEBUG user.MQTTSubscribe: (Service) data-> final packet is 2023-03-02 15:26:16 MST (1677795976): 'dateTime': '1677795976', 'inHumidity': '22.568543063381135', 'inTemp': '17.212429480347783', 'rainRate': '0', 'txBatteryStatus': '0', 'usUnits': '17', 'windDir': '265.8379446640316', 'windSpeed': '0.0'

When I run debug=0, I get at weewx restart

Mar 2 15:31:13 pi34 weewx[7610] INFO user.MQTTSubscribe: (Service) Version is 2.2.2

Mar 2 15:31:13 pi34 weewx[7610] INFO user.MQTTSubscribe: (Service) Log level: 0

Mar 2 15:31:13 pi34 weewx[7610] INFO user.MQTTSubscribe: (Service) Log debug setting: 0

Mar 2 15:31:13 pi34 weewx[7610] INFO user.MQTTSubscribe: (Service) Log console: False

Mar 2 15:31:13 pi34 weewx[7610] INFO user.MQTTSubscribe: (Service) Log file: None

Mar 2 15:31:13 pi34 weewx[7610] INFO user.MQTTSubscribe: (Service) 'use_topic_as_fieldname' option is no longer needed and can be removed.

Mar 2 15:31:13 pi34 weewx[7610] INFO user.MQTTSubscribe: (Service) message_callback_provider_name is user.MQTTSubscribe.MessageCallbackProvider

Mar 2 15:31:13 pi34 weewx[7610] INFO user.MQTTSubscribe: (Service) clientid is MQTTSubscribe-5972

Mar 2 15:31:13 pi34 weewx[7610] INFO user.MQTTSubscribe: (Service) client_session is True

Mar 2 15:31:13 pi34 weewx[7610] INFO user.MQTTSubscribe: (Service) host is 192.168.1.166

Mar 2 15:31:13 pi34 weewx[7610] INFO user.MQTTSubscribe: (Service) port is 1883

Mar 2 15:31:13 pi34 weewx[7610] INFO user.MQTTSubscribe: (Service) keepalive is 120

Mar 2 15:31:13 pi34 weewx[7610] INFO user.MQTTSubscribe: (Service) username is ODT

Mar 2 15:31:13 pi34 weewx[7610] INFO user.MQTTSubscribe: (Service) min_delay is 1

Mar 2 15:31:13 pi34 weewx[7610] INFO user.MQTTSubscribe: (Service) max_delay is 120

Mar 2 15:31:13 pi34 weewx[7610] INFO user.MQTTSubscribe: (Service) password is set

Mar 2 15:31:13 pi34 weewx[7610] INFO user.MQTTSubscribe: (Service) Archive topic is None

Mar 2 15:31:13 pi34 weewx[7610] INFO user.MQTTSubscribe: (Service) binding is loop

Mar 2 15:31:13 pi34 weewx[7610] INFO user.MQTTSubscribe: (Service) Waiting for MQTT connection.

Mar 2 15:31:13 pi34 weewx[7610] INFO user.MQTTSubscribe: (Service) Connected with result code 0

Mar 2 15:31:13 pi34 weewx[7610] INFO user.MQTTSubscribe: (Service) Connected flags {'session present': 0}

Mar 2 15:31:13 pi34 weewx[7610] INFO user.MQTTSubscribe: (Service) Subscribing to topic/rtd_temperature has a mid 1 and rc 0

Mar 2 15:31:13 pi34 weewx[7610] INFO user.MQTTSubscribe: (Service) Subscribing to topic/tmp117_temperature has a mid 2 and rc 0

Mar 2 15:31:13 pi34 weewx[7610] INFO user.MQTTSubscribe: (Service) Subscribed to mid: 1 is size 1 has a QOS of 0

Mar 2 15:31:13 pi34 weewx[7610] INFO user.MQTTSubscribe: (Service) Subscribed to mid: 2 is size 1 has a QOS of 0

Mar 2 15:31:14 pi34 weewx[7610] INFO user.MQTTSubscribe: (Service) MQTT initialization complete.

My setup is a ESP32 reading a RTD100 and a TMP117 temperature, sending the data every 15 seconds via MQTT to the Pi 3 B+ running Bullseye and WeeWX version 4.10.2.

WeeWX is running lheijst’s github weewx-rtldavis software to read the Davis Vantage Pro2 ISS and wjcarpenter’s gitlab bme280wx.git as a service to read the BME280 for indoor temp and barometer.

mosquitto broker version 2.0.11 at the Pi. mosquitto is an MQTT v5.0/v3.1.1/v3.1 broker.

When I run

mosquitto_sub -h 192.168.1.166 -t + -v -u ??? -P ???? | xargs -d$'\n' -L1 bash -c 'date "+%Y-%m-%d %T.%3N $0"'

the ? are to not give out my user and password info and the pipe to xargs is to add date and time

I get the data sent from the ESP32

2023-03-02 13:04:20.439 rtd_temperature 20.29

2023-03-02 13:04:20.455 tmp117_temperature 18.09

[vince]

How about you just show the mosquitto_sub results without the xargs pipe please...

[lloyd...@gmail.com]

without the xargs

rtd_temperature 20.70

tmp117_temperature 18.33

[vince]

In that case should [[[topic/rtd_temperature]]] be just [[[rtd_temperature]]] perhaps ???

It's a bit hard to decipher with such a cryptic description and unfortunately google ate whatever formatting you entered on your cut+paste, so your original post is very hard to read as a result.

Give it a try.  It's already not working so you can't make anything worse.

[lloyd...@gmail.com]

Well that change helped and shows MQTTSubscribe reading the values  but now it says can't convert degree_C to percent as per the following for rtd_temperature (same for tmp117_temperature) 

Also what should I do  when copying sections of weewx.conf to make it easier to read?

Mar  2 17:56:03 pi34 weewx[10834] DEBUG user.MQTTSubscribe: (Service) MessageCallbackProvider data-> incoming topic: rtd_temperature, QOS: 0, retain: 0, payload: b'20.39'
Mar  2 17:56:03 pi34 weewx[10834] DEBUG weewx.units: Unable to convert from degree_C to percent
Mar  2 17:56:03 pi34 weewx[10834] ERROR user.MQTTSubscribe: (Service) MessageCallbackProvider on_message_individual failed with <class 'KeyError'> and reason 'percent'.
Mar  2 17:56:03 pi34 weewx[10834] ERROR user.MQTTSubscribe: (Service) **** MessageCallbackProvider Ignoring topic=rtd_temperature and payload=b'20.39'

Mar  2 17:56:03 pi34 weewx[10834] ERROR user.MQTTSubscribe: (Service) **** MessageCallbackProvider Traceback (most recent call last):#012  File "/home/weewx/bin/user/MQTTSubscribe.py", line 1527, in _on_message_individual#012    (fieldname, value) = self._update_data(fields, fields_conversion_func, key, payload_str, unit_system)#012  File "/home/weewx/bin/user/MQTTSubscribe.py", line 1264, in _update_data#012    (value, new_units, new_group) = weewx.units.convert((value, fields[orig_name]['units'], None), to_units) # match signature pylint: disable=unused-variable#012  File "/home/weewx/bin/weewx/units.py", line 1454, in convert#012    conversion_func = conversionDict[val_t[1]][target_unit]#012KeyError: 'percent'

[vince]

Sorry, dunno about formatting.  The google groups interface seems to randomly lose any formatting I try to do, but only sometime.  Frustrating.

I'm not understanding why (or where) you're trying to convert degree_C to percent. It's already a temperature reading in the correct units isn't it ?  Are you setting some custom Units someplace in skin.conf or weewx.conf perhaps ?

[jiaqi Yan]

Yan jiaqi

---

From: weewx...@googlegroups.com <weewx...@googlegroups.com> on behalf of vince <vince...@gmail.com>
Sent: Friday, March 3, 2023 9:39:13 AM
To: weewx-user <weewx...@googlegroups.com>
Subject: [weewx-user] Re: MQTTSubscribe service - not getting extraTemp1 or 2

 

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/e73d1e54-d663-4614-be5b-650e61e030e1n%40googlegroups.com.

[lloyd...@gmail.com]

The values come in as C and I wanted to convert to F.  For some unknown reason WeeWX seems to be trying to convert to %???

Interesting, I removed units = degree_C and it started to run without errors but it shows on the skin (whatever the default is)  RTD Temperature 20%  and TMP117 temperature 18% for temperatures of 20.46 and 18.09 respectively - I did change the weewx.conf to show under labels generic

                extraTemp1 = RTD Temperature
                extraTemp2 = TMP117 Temperature
                extraTemp3 = Temperature3
 

[jiaqi Yan]

Yan jiaqi

---

From: weewx...@googlegroups.com <weewx...@googlegroups.com> on behalf of lloyd...@gmail.com <lloyd...@gmail.com>
Sent: Friday, March 3, 2023 9:44:44 AM

To: weewx-user <weewx...@googlegroups.com>
Subject: [weewx-user] Re: MQTTSubscribe service - not getting extraTemp1 or 2

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/ec8c8e83-4cc8-4474-8d67-67185934cbd2n%40googlegroups.com.

[lloyd...@gmail.com]

This WeeWX is as close to stock as I can get given the SDR and MQTTSubscribe and BME280.  The only changes to weewx.conf have been the 2 services and the SDR driver plus the labels generic

[Graham Eddy]

looks like wrongly set unit label (not unit name)

⊣GE⊢

[bell...@gmail.com]

Looks like rtldavis is ‘repurposing’ the extraTemp* fields.
https://github.com/lheijst/weewx-rtldavis/blob/2f3b4b344fd70ab253aabfa837b0ffc76570c075/bin/user/rtldavis.py#L732
'extraTemp1': 'pct_good_0',  # renamed
'extraTemp2': 'pct_good_1',  # renamed
'extraTemp3': ' pct_good_2',  # renamed

Also looks like there is a way to ‘override’ the setting in weewx.conf.
https://github.com/lheijst/weewx-rtldavis/blob/2f3b4b344fd70ab253aabfa837b0ffc76570c075/bin/user/rtldavis.py#L769
if 'sensor_map' in stn_dict:
    self.sensor_map.update(stn_dict['sensor_map'])

I see two options.
1. Configure rtldavis to leave inTemp1 and inTemp2 as temperature fields.
2. Configure MQTTSubscribe to use different temperature fields.
rich

[lloyd...@gmail.com]

Rich:

Thank you very much. That would have probably taken me an eon to figure that out.

[lloyd...@gmail.com]

So I decided to change MQTTSubscribe from extraTemp1 and 2 (&3) to extraTemp4 & 5 .  This necessitated that I also change the Seasons skins to go from extraTemp1, 2, 3 to extraTemp4, 5, 6 . To do this I changed skin.conf as well as

current.inc, hilo.inc, statistics.inc and index.html.tmpl .  Note that what might look like a comment with the # in front of it (such as #if $day.extraTemp3.has_data ) is a conditional or preprocessor statement so those have to be changed.  There is probably a way to do that in weewx.conf but I took the simple way out.