user.mqtt.MQTT not found

[Paul Dunphy]

    I have mosquitto installed and communicating between two terminal sessions.  I'm running weewx 5.0.2 in a venv on a Pi 5, and it's working fine.  I followed Tom/Matthew's instructions here to set up MQTT:

https://github.com/weewx/weewx/wiki/mqtt

    It blows up because it can't find user.mqtt that I added to the weewx.conf file:

[Engine]
    [[Services]]
        restful_services = ..., user.mqtt.MQTT


    Any thoughts on where I might have gone wrong?

Paul VE1DX

Mar 20 14:00:15 violet systemd[1]: Started weewx.service - WeeWX weather system.
Mar 20 14:00:15 violet sudo[711921]: pam_unix(sudo:session): session closed for user root
Mar 20 14:00:15 violet weewxd[711925]: INFO __main__: Initializing weewxd version 5.0.2
Mar 20 14:00:15 violet weewxd[711925]: INFO __main__: Command line: /home/pdunphy/weewx-venv/lib/python3.11/site-packages/weewxd.py /home/pdunphy/weewx-data/weewx.conf
Mar 20 14:00:15 violet weewxd[711925]: INFO __main__: Using Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0]
Mar 20 14:00:15 violet weewxd[711925]: INFO __main__: Located at /home/pdunphy/weewx-venv/bin/python3
Mar 20 14:00:15 violet weewxd[711925]: INFO __main__: Platform Linux-6.6.20+rpt-rpi-2712-aarch64-with-glibc2.36
Mar 20 14:00:15 violet weewxd[711925]: INFO __main__: Locale: 'en_GB.UTF-8'
Mar 20 14:00:15 violet weewxd[711925]: INFO __main__: Entry path: /home/pdunphy/weewx-venv/lib/python3.11/site-packages/weewxd.py
Mar 20 14:00:15 violet weewxd[711925]: INFO __main__: WEEWX_ROOT: /home/pdunphy/weewx-data
Mar 20 14:00:15 violet weewxd[711925]: INFO __main__: Configuration file: /home/pdunphy/weewx-data/weewx.conf
Mar 20 14:00:15 violet weewxd[711925]: INFO __main__: User module: /home/pdunphy/weewx-data/bin/user
Mar 20 14:00:15 violet weewxd[711925]: INFO __main__: Debug: 0
Mar 20 14:00:15 violet weewxd[711925]: INFO weewx.engine: Loading station type Vantage (weewx.drivers.vantage)
Mar 20 14:00:16 violet weewxd[711925]: INFO weewx.engine: StdConvert target unit is 0x10
Mar 20 14:00:16 violet weewxd[711925]: INFO weewx.wxservices: StdWXCalculate will use data binding wx_binding
Mar 20 14:00:16 violet systemd[1]: weewx.service: Main process exited, code=exited, status=1/FAILURE
Mar 20 14:00:16 violet python3[711925]: Traceback (most recent call last):
Mar 20 14:00:16 violet python3[711925]:   File "/home/pdunphy/weewx-venv/lib/python3.11/site-packages/weewxd.py", line 265, in <module>
Mar 20 14:00:16 violet python3[711925]:     main()
Mar 20 14:00:16 violet python3[711925]:   File "/home/pdunphy/weewx-venv/lib/python3.11/site-packages/weewxd.py", line 160, in main
Mar 20 14:00:16 violet python3[711925]:     engine = weewx.engine.StdEngine(config_dict)
Mar 20 14:00:16 violet python3[711925]:              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 20 14:00:16 violet python3[711925]:   File "/home/pdunphy/weewx-venv/lib/python3.11/site-packages/weewx/engine.py", line 89, in __init__
Mar 20 14:00:16 violet python3[711925]:     self.loadServices(config_dict)
Mar 20 14:00:16 violet python3[711925]:   File "/home/pdunphy/weewx-venv/lib/python3.11/site-packages/weewx/engine.py", line 157, in loadServices
Mar 20 14:00:16 violet python3[711925]:     obj = weeutil.weeutil.get_object(svc)(self, config_dict)
Mar 20 14:00:16 violet python3[711925]:           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 20 14:00:16 violet python3[711925]:   File "/home/pdunphy/weewx-venv/lib/python3.11/site-packages/weeutil/weeutil.py", line 1404, in get_object
Mar 20 14:00:16 violet python3[711925]:     module = importlib.import_module(module_name)
Mar 20 14:00:16 violet python3[711925]:              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 20 14:00:16 violet python3[711925]:   File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module
Mar 20 14:00:16 violet python3[711925]:     return _bootstrap._gcd_import(name[level:], package, level)
Mar 20 14:00:16 violet python3[711925]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 20 14:00:16 violet python3[711925]:   File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
Mar 20 14:00:16 violet python3[711925]:   File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
Mar 20 14:00:16 violet python3[711925]:   File "<frozen importlib._bootstrap>", line 1142, in _find_and_load_unlocked
Mar 20 14:00:16 violet python3[711925]: ModuleNotFoundError: No module named 'user.mqtt'
Mar 20 14:00:16 violet weewxd[711925]: INFO weewx.engine: Archive will use data binding wx_binding
Mar 20 14:00:16 violet systemd[1]: weewx.service: Failed with result 'exit-code'.
Mar 20 14:00:16 violet weewxd[711925]: INFO weewx.engine: Record generation will be attempted in 'hardware'
Mar 20 14:00:16 violet weewxd[711925]: INFO weewx.engine: Using archive interval of 300 seconds (specified by hardware)
Mar 20 14:00:16 violet weewxd[711925]: INFO weewx.restx: StationRegistry: Station will be registered.
Mar 20 14:00:16 violet weewxd[711925]: INFO weewx.restx: Wunderground: No config info. Skipped.
Mar 20 14:00:16 violet weewxd[711925]: INFO weewx.restx: PWSweather: No config info. Skipped.
Mar 20 14:00:16 violet weewxd[711925]: INFO weewx.restx: CWOP: Data for station VE1DX will be posted
Mar 20 14:00:16 violet weewxd[711925]: INFO weewx.restx: WOW: Data for station 953456001 will be posted
Mar 20 14:00:16 violet weewxd[711925]: INFO weewx.restx: AWEKAS: Data will be uploaded for user VE1DX
Mar 20 14:00:16 violet weewxd[711925]: CRITICAL __main__: Caught unrecoverable exception:
Mar 20 14:00:16 violet weewxd[711925]: CRITICAL __main__:     ****  No module named 'user.mqtt'
Mar 20 14:00:16 violet weewxd[711925]: CRITICAL __main__:     ****  Traceback (most recent call last):
Mar 20 14:00:16 violet weewxd[711925]: CRITICAL __main__:     ****    File "/home/pdunphy/weewx-venv/lib/python3.11/site-packages/weewxd.py", line 160, in main
Mar 20 14:00:16 violet weewxd[711925]: CRITICAL __main__:     ****      engine = weewx.engine.StdEngine(config_dict)
Mar 20 14:00:16 violet weewxd[711925]: CRITICAL __main__:     ****               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 20 14:00:16 violet weewxd[711925]: CRITICAL __main__:     ****    File "/home/pdunphy/weewx-venv/lib/python3.11/site-packages/weewx/engine.py", line 89, in __init__
Mar 20 14:00:16 violet weewxd[711925]: CRITICAL __main__:     ****      self.loadServices(config_dict)
Mar 20 14:00:16 violet weewxd[711925]: CRITICAL __main__:     ****    File "/home/pdunphy/weewx-venv/lib/python3.11/site-packages/weewx/engine.py", line 157, in loadServices
Mar 20 14:00:16 violet weewxd[711925]: CRITICAL __main__:     ****      obj = weeutil.weeutil.get_object(svc)(self, config_dict)
Mar 20 14:00:16 violet weewxd[711925]: CRITICAL __main__:     ****            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 20 14:00:16 violet weewxd[711925]: CRITICAL __main__:     ****    File "/home/pdunphy/weewx-venv/lib/python3.11/site-packages/weeutil/weeutil.py", line 1404, in get_object
Mar 20 14:00:16 violet weewxd[711925]: CRITICAL __main__:     ****      module = importlib.import_module(module_name)
Mar 20 14:00:16 violet weewxd[711925]: CRITICAL __main__:     ****               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 20 14:00:16 violet weewxd[711925]: CRITICAL __main__:     ****    File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module
Mar 20 14:00:16 violet weewxd[711925]: CRITICAL __main__:     ****      return _bootstrap._gcd_import(name[level:], package, level)
Mar 20 14:00:16 violet weewxd[711925]: CRITICAL __main__:     ****             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 20 14:00:16 violet weewxd[711925]: CRITICAL __main__:     ****    File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
Mar 20 14:00:16 violet weewxd[711925]: CRITICAL __main__:     ****    File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
Mar 20 14:00:16 violet weewxd[711925]: CRITICAL __main__:     ****    File "<frozen importlib._bootstrap>", line 1142, in _find_and_load_unlocked
Mar 20 14:00:16 violet weewxd[711925]: CRITICAL __main__:     ****  ModuleNotFoundError: No module named 'user.mqtt'
Mar 20 14:00:16 violet weewxd[711925]: CRITICAL __main__:     ****  Exiting.

[Vince Skahan]

You need to install paho-mqtt 'within' your python venv.  Activate the venv then do 'pip3 install paho-mqtt'

[Paul Dunphy]

    I thought I did, but to be sure, I tried again and it's there:

(weewx-venv) pdunphy@violet:~/weewx-data $ pip3 install paho-mqtt==1.6.1
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Requirement already satisfied: paho-mqtt==1.6.1 in
/home/pdunphy/weewx-venv/lib/python3.11/site-packages (1.6.1)
(weewx-venv) pdunphy@violet:~/weewx-data $

    You have to downgrade paho-mqtt because the latest version is
incompatible with Matthew's extension (as per the wiki.)  That seems to
be OK.  It's this section of weewx.conf that blows it up, I think:

[Engine]

    # The following section specifies which services should be run and
in what order.
    [[Services]]
        prep_services = weewx.engine.StdTimeSynch
        data_services = ,
        process_services = weewx.engine.StdConvert,
weewx.engine.StdCalibrate, weewx.engine.StdQC,
weewx.wxservices.StdWXCalculate
        xtype_services = weewx.wxxtypes.StdWXXTypes,
weewx.wxxtypes.StdPressureCooker, weewx.wxxtypes.StdRainRater,
weewx.wxxtypes.StdDelta
        archive_services = weewx.engine.StdArchive
        restful_services = weewx.restx.StdStationRegistry,
weewx.restx.StdWunderground, weewx.restx.StdPWSweather,
weewx.restx.StdCWOP, weewx.restx.StdWOW, weewx.restx.StdAWEKAS,
user.mqtt.MQTT *<---- This*
        report_services = weewx.engine.StdPrint, weewx.engine.StdReport

Paul VE1DX

> --
> You received this message because you are subscribed to a topic in the
> Google Groups "weewx-development" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/weewx-development/3sQJJd6FyBo/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> weewx-developm...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/weewx-development/2ed32fc7-4348-4560-892e-43dd85e4d199n%40googlegroups.com
> <https://groups.google.com/d/msgid/weewx-development/2ed32fc7-4348-4560-892e-43dd85e4d199n%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Paul Dunphy]

    Close, Vince.  It was the extension that I installed "outside" the venv . . . you have to do it without sudo in the environment."

weectl extension install weewx-mqtt.zip
Using configuration file /home/pdunphy/weewx-data/weewx.conf
Install extension 'weewx-mqtt.zip' (y/n)? y
Extracting from zip archive weewx-mqtt.zip
Saving installer file to /home/pdunphy/weewx-data/bin/user/installer/mqtt
Saved copy of configuration as /home/pdunphy/weewx-data/weewx.conf.20240320154428

Finished installing extension mqtt from weewx-mqtt.zip

    This puts it in ~/weewx-data/bin/user where it's supposed to be.  Now, I have MQTT weather data on my LAN, which was the goal.

Paul VE1DX

[Paul Dunphy]

    I spent a day searching the documentation, the Internet, and user groups.  I even asked ChatGPT!  This likely is a dumb question, but I paid my dues B4 asking.  I have a weewx-enabled Pi spitting out aggregate archive stanzas via MQTT every 5 minutes:

{"dateTime": "1711125000.0", "usUnits": "16.0", "outTemp_C": "0.11853060499920859", "inTemp_C": "17.225573819849952", "barometer_mbar": "1053.1565009214046", "pressure_mbar": "1053.1565009214046", "windSpeed_kph": "0.002427633039238002", "windDir": "359.9421733668237", "windGust_kph": "0.004278710967144328", "windGustDir": "359.9202399679532", "outHumidity": "79.99094923258458", "inHumidity": "29.987934248540196", "radiation_Wpm2": "861.1533294284267", "UV": "12.056146611997969", "rain_cm": "0.0", "txBatteryStatus": "1.0", "windBatteryStatus": "0.0", "rainBatteryStatus": "0.7666666666666667", "outTempBatteryStatus": "0.0", "inTempBatteryStatus": "0.0", "consBatteryVoltage_volt": "12.0", "heatingVoltage_volt": "12.05558467860259", "supplyVoltage_volt": "12.071862590655277", "referenceVoltage_volt": "12.018038262199248", "rxCheckPercent": "81.54437561456504", "altimeter_mbar": "1056.4794019219778", "appTemp_C": "-2.2564686080177725", "cloudbase_meter": "396.2261771824497", "dewpoint_C": "-2.9181089732181587", "ET_cm": "0.0025520108394833227", "heatindex_C": "0.11853060499920859", "humidex_C": "0.11853060499920859", "inDewpoint_C": "-0.5139816409552426", "maxSolarRad_Wpm2": "731.1872454643874", "rainRate_cm_per_hour": "0.0", "windchill_C": "0.11853060499920859", "windrun_km": "0.0002023027532698335", "interval_minute": "5.0", "hourRain_cm": "0.0", "rain24_cm": "0.0", "dayRain_cm": "0.0"}

    They look good.  I have a newly installed weewx instance on another Pi on my LAN.  The second machine sees the stanzas with "mosquitto_sub -h 192.168.1.10 -p 1883 -t weather/# -u wee_me -P xxxxxx"

    Is there a way to set up weewx to make these 5 minute stanzas the data source for the 2nd instance of weewx?  A "repeater_mqtt" reader/driver?  Does such a thing exist?  How do others use one hardware receiver and MQTT to supply data to other systems on their LAN?  I must be missing something obvious.

Paul VE1DX

[Vince Skahan]

Probably a -user question at this point, but can you explain what you're trying to set up ?

Why would you need/want a second weewx instance ?

Why not just have system-2 subscribe to the aggregate(d) data you're already publishing to your LAN ?

FWIW my setup here is two weewx instances - one VP2 connected to the station, one Ecowitt connecting over the network to the ecowitt gateway.   The Ecowitt instance publishes to MQTT and the VP2 instance subscribes and saves as extraTemp1 etc.   The VP2 instances also publishes the aggregated data from both stations out to MQTT.  My Home Assistant and influxdb/telegraf/grafana docker containers subscribe to the MQTT data they need.  Works great.   Everything becomes MQTT that anybody who needs it can subscribe to.

[Paul Dunphy]

    I have one Vantage Vue receiver on my production weewx system. I
have 3-4 RPis on my LAN.  I have the production system spitting out
5-minutes MQTT archive stanzas on the LAN.

    I want to load weewx on a couple of the other Pis to play with the
seasons and Belchertown skins. I don't want to mess with the production
system, so I thought I could send its data to the two other guys on the
LAN with MQTT.  Send works.  How do I get the other two guys to read the
data in?

    If this is more appropriately discussed in the User group, I can
re-ask it there.

Paul VE1DX

> --
> You received this message because you are subscribed to a topic in the
> Google Groups "weewx-development" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/weewx-development/3sQJJd6FyBo/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> weewx-developm...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/weewx-development/49ef5a94-4504-4a19-a1f6-db65d3c70819n%40googlegroups.com
> <https://groups.google.com/d/msgid/weewx-development/49ef5a94-4504-4a19-a1f6-db65d3c70819n%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Vince Skahan]

Just run the simulator to play with skins on the other boxes. Subscribe to whatever mqtt data you want on those if you need to do so. The MQTT subscribe extension is discussed a lot on the -user group if you need help there.