MQTT with Belchertown skin - errors, No module named mqtt

[Colin Larsen]

Hi Pat, I think this one may be for you ...

As per the above and below, I get the no module error setting up MQTT. I've sanitised the the username and password to the MQTT service in the extract of weewx.conf below, otherwise hopefully it's all as per your Github instructions.

Aug 26 13:10:14 raspberrypi weewx[3191]: manager: Starting backfill of daily summaries

Aug 26 13:10:14 raspberrypi weewx[3191]: restx: StationRegistry: Registration not requested.

Aug 26 13:10:14 raspberrypi weewx[3191]: restx: Wunderground: Posting not enabled.

Aug 26 13:10:14 raspberrypi weewx[3191]: restx: PWSweather: Posting not enabled.

Aug 26 13:10:14 raspberrypi weewx[3191]: restx: CWOP: Posting not enabled.

Aug 26 13:10:14 raspberrypi weewx[3191]: restx: WOW: Posting not enabled.

Aug 26 13:10:14 raspberrypi weewx[3191]: restx: AWEKAS: Posting not enabled.

Aug 26 13:10:14 raspberrypi weewx[3191]: engine: Caught unrecoverable exception in engine:

Aug 26 13:10:14 raspberrypi weewx[3191]:     ****  No module named mqtt

Aug 26 13:10:14 raspberrypi weewx[3191]:     ****  Traceback (most recent call last):

Aug 26 13:10:14 raspberrypi weewx[3191]:     ****    File "/usr/share/weewx/weewx/engine.py", line 871, in main

Aug 26 13:10:14 raspberrypi weewx[3191]:     ****      engine = engine_class(config_dict)

Aug 26 13:10:14 raspberrypi weewx[3191]:     ****    File "/usr/share/weewx/weewx/engine.py", line 78, in __init__

Aug 26 13:10:14 raspberrypi weewx[3191]:     ****      self.loadServices(config_dict)

Aug 26 13:10:14 raspberrypi weewx[3191]:     ****    File "/usr/share/weewx/weewx/engine.py", line 142, in loadServices

Aug 26 13:10:14 raspberrypi weewx[3191]:     ****      self.service_obj.append(weeutil.weeutil._get_object(svc)(self, config_dict))

Aug 26 13:10:14 raspberrypi weewx[3191]:     ****    File "/usr/share/weewx/weeutil/weeutil.py", line 1132, in _get_object

Aug 26 13:10:14 raspberrypi weewx[3191]:     ****      mod = __import__(module)

Aug 26 13:10:14 raspberrypi weewx[3191]:     ****  ImportError: No module named mqtt

Aug 26 13:10:14 raspberrypi weewx[3191]:     ****  Exiting.

Aug 26 13:10:37 raspberrypi systemd[1]: Stopping LSB: weewx weather system...

Aug 26 13:10:37 raspberrypi weewx[3227]: Stopping weewx weather system: weewx not running....

Aug 26 13:10:37 raspberrypi systemd[1]: Stopped LSB: weewx weather system.

The mqtt.py file is in /usr/share/weewx/  as per below (I'll change the permissions for read-only shortly)

total 192

-rw-r--r-- 1 root root  2886 Aug 17 04:06 daemon.py

-rw-r--r-- 1 root root  2715 Aug 24 16:09 daemon.pyc

-rw-rw-rw- 1 root root 16926 Aug 26 12:14 mqtt.py

drwxr-xr-x 2 root root  4096 Aug 24 16:09 schemas

drwxr-xr-x 3 root root  4096 Aug 24 16:20 user

drwxr-xr-x 2 root root  4096 Aug 24 16:11 weecfg

-rwxr-xr-x 1 root root  5145 Aug 17 04:06 wee_config

-rwxr-xr-x 1 root root 36471 Aug 17 04:06 wee_database

drwxr-xr-x 2 root root  4096 Aug 24 16:09 weedb

-rwxr-xr-x 1 root root 16207 Aug 17 04:06 wee_debug

-rwxr-xr-x 1 root root  1681 Aug 17 04:06 wee_device

-rwxr-xr-x 1 root root  3118 Aug 17 04:06 wee_extension

drwxr-xr-x 2 root root  4096 Aug 24 16:09 weeimport

-rwxr-xr-x 1 root root 31159 Aug 17 04:06 wee_import

drwxr-xr-x 2 root root  4096 Aug 24 16:10 weeplot

-rwxr-xr-x 1 root root  2171 Aug 17 04:06 wee_reports

drwxr-xr-x 2 root root  4096 Aug 24 16:10 weeutil

drwxr-xr-x 3 root root  4096 Aug 24 16:10 weewx

-rwxr-xr-x 1 root root  2565 Aug 17 04:06 weewxd

-rwxr-xr-x 1 root root 20788 Aug 17 04:06 wunderfixer

[StdRESTful] .....

[[MQTT]]

        #This section is for the MQTT service setup

        server_url = mqtt://user:pass...@m11.cloudmqtt.com:13147/

        topic = weather/41south

        unit_system = Metric

        binding = archive, loop

        aggregation = aggregate

[Engine]

    [[Services]]

restful_services = ...... weewx.restx.StdAWEKAS, user.mqtt.MQTT

What did I miss??

Thanks

Colin

[John Smith]

On Sunday, 26 August 2018 11:18:40 UTC+10, Colin Larsen wrote:

Aug 26 13:10:14 raspberrypi weewx[3191]:     ****  ImportError: No module named mqtt

 

The mqtt.py file is in /usr/share/weewx/  as per below (I'll change the permissions for read-only shortly)

You need to put it in /usr/share/weewx/user/ 

[Colin Larsen]

Thanks John! I used the installer rather than Pat's manual method and it works great now :)

Ignore the data values it's working on Simulator only at the moment

weather/41south/loop

{"cloudbase_meter": "385.160767033", "outHumidity": "79.9998437932", "pressure_mbar": "1053.16654107", "rain_cm": "0.0", "barometer_mbar": "1053.16654107", "dewpoint_C": "-2.69897261268", "windGust_kph": "5.02780872081e-05", "heatindex_C": "0.341840163469", "dayRain_cm": "0.508", "inDewpoint_C": "-0.511415146209", "altimeter_mbar": "1053.61058559", "windchill_C": "0.341840163469", "appTemp_C": "-2.00597980905", "outTemp_C": "0.341840163469", "windGustDir": "359.999062759", "humidex_C": "0.341840163469", "rain24_cm": "0.508", "inTemp_C": "17.2222800764", "windSpeed_kph": "4.1898406007e-05", "usUnits": "16.0", "UV": "11.4155207049", "rainRate_cm_per_hour": "0.0", "dateTime": "1535247709.0", "windDir": "359.999062759", "inHumidity": "29.9997917248", "radiation_Wpm2": "815.394336067"}

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Colin Larsen]

Now I just have to get the skin.conf to read it back

These are the instance settings at CloudMQTT

Instance info

Server	m11.cloudmqtt.com
User	Restart
Password	Rotate
Port	13147
SSL Port	23147
Websockets Port (TLS only)	33147
Connection limit	5

And here is the skin.conf section .....

# MQTT Defaults

    mqtt_enabled = 1

    mqtt_host = "m11.cloudmqtt.com"

    mqtt_port = 33147

    mqtt_ssl = 0

    mqtt_topic = "weather/41south/loop"

    disconnect_live_website_visitor = 1800000

Not quite sure what is awry - Pat?

[Colin Larsen]

With a test client I've discovered that CloudMQTT require a username and password to be sent for the websocket connections (TLS only too so it's a wss connection) - that needs adding into skin.conf Pat I guess please.

Cheers

Colin

[Pat]

Yeah, I had someone else reach out to me that cloudmqtt was giving them some trouble as well. They ended up switching over to test.mosquitto.org for the time being. 

I thought you could add username:pass...@cloudmqtt.com in the hostname within options, but it didn't work. So I'll have to create an account and see what the correct syntax is. 

I also just realized the MQTT live updates are hard coded for imperial units. Those pesky units have got me again! 

I've got a bit of work to fix up on the skin. Hope to have a look at most of this in the next day or so. 

If anyone wants to contribute to the GitHub repo with a pull request in the meantime, I'm open to that as well!

[mwall]

On Sunday, August 26, 2018 at 9:45:51 AM UTC-4, Pat wrote:

I also just realized the MQTT live updates are hard coded for imperial units. Those pesky units have got me again! 

please try mqtt-weewx v0.18.  it includes support for loop and/or archive binding, and it has always supported specification of the unit system in the configuration file (US, METRIC, or METRICWX)

https://github.com/weewx/weewx/wiki/mqtt

if there are other features in the mqtt uploader that you have been using, please let me know so i can incorporate them into weewx-mqtt.

m 

[Pat]

Will do thanks! Some of the live mqtt stuff are colors for text based on temperature. That's something I'll need to determine imperial vs. metric. 

On an unrelated note, I have a pull request against your meteostick driver. Anything more I need to do with that?

[mwall]

On Sunday, August 26, 2018 at 12:49:25 PM UTC-4, Pat wrote:

On an unrelated note, I have a pull request against your meteostick driver. Anything more I need to do with that?

i see it.  i'm slowly working through a backlog of weewx-related emails and issues...

[Pat O'Brien]

Understood and no worries, thanks!

--
You received this message because you are subscribed to a topic in the Google Groups "weewx-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/weewx-user/FQfh0AAJmCA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to weewx-user+...@googlegroups.com.

[Colin Larsen]

Pat that could be my bad on the version I'm using. I went with 0.17 to get it running, I'll move to 0.18 today, that might solve the units problem?

I ended up following the wiki install as I think yours was missing a step, not 100% sure on that. Anyway let me update that. My data going up to mqtt certainly has deg C and Kph in it, it didn't look too closely at the actual data though

You received this message because you are subscribed to the Google Groups "weewx-user" group.

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

[Pat]

Yeah I need to upgrade as well. The 0.18rc1 has been working great. So I'm expecting 0.18 to work just as good. 

Your data's unit values to MQTT should be what you expect, and as you know we've been working out the kinks on the units with the non-live portion of the skin. 

The live portion processes the MQTT data a bit differently since it's live - it doesn't use weewx's StdConvert in real time - so there are some color charts that I have hard coded for Fahrenheit and I see an outTemp_F, rain inches, dew point F variables it's expecting from MQTT which probably needs to be updated. 

This weekend has been busy with (real) work, so no update on when I can get an update out quite yet

What is your weewx.conf target_unit set to?

[Colin Larsen]

That's all good Pat, I sure appreciate the time this is taking and real work and family come first always, so certainly no rush on my part.

I can send you a PM with my MQTT login if that helps for testing but if Mosquito works I'm happy to shift to that also.  In the medium to short term I'd like to get my own MQTT server running here, as much to save data being pushed everywhere as anything. Would a small server run on a pi zero?

Because I'm still using simulator and waiting for all the parts for my station to arrive (building a WeatherDuino setup including air quality monitoring) I've left it at US, however I probably should set it to Metric before any real data goes into the db. 

The WeatherDuino mimics a VP2 so I need to find out what sensors the AQ data is stored as too so I can get it out and work on displaying it ..... but that's another story down the track.

Cheers

Colin

[Colin Larsen]

Working fine with test.mosquitto.org 

weather/41south/#

Haven't looked at the data for sanity :)

[Pat]

No need for the MQTT login details, I'm able to subscribe to your data feed without a problem. This is helpful so I'll work on updating the MQTT live portion's unit labels. 

MQTT may run on a Pi Zero - I haven't tried but, and I don't see why it wouldn't work, so it's worth a shot. 

Try this to get you started:

sudo apt-get update
sudo apt-get install mosquitto mosquitto-clients

Then sudo nano /etc/mosquitto/conf.d/myconfig.conf

persistence false

allow_anonymous true

# Non-SSL MQTT
listener 1883
protocol mqtt

# Non-SSL Websockets
listener 9001
protocol websockets

Then

sudo service mosquitto stop
sudo service mosquitto start

This will start your broker on port 1883 for MQTT and 9001 for websockets (for the website). 

This won't secure your MQTT broker, and it allows anonymous users to do whatever they want. So only use this in a development capacity. 

Once data is being published to the broker, you can use the mosquitto subscribe client to subscribe to the topic and verify the data is there. 

mosquitto_sub -h localhost -p 1883 -t "your/topic"

CTRL+C will exit the program. 

I'm working on a write up on how to secure the broker, but if you want to read how to secure your broker beforehand, research mosquitto_passwd and acl_file options. If you want to use SSL, you can secure your broker and websockets using LetsEncrypt. Since this is just weather data, SSL may not be necessary, but that's at your discretion.  

[Colin Larsen]

That's excellent - thanks Pat

I'll leave the data posting to test.mosquitto.org for now so you have something to test against.

Colin

[Pat]

I should be good to go now. I've created a lab machine with the AEST timezone and METRIC and METRICWX for units. Feel free to move your data where you'd like...

Thanks for having it up for me to test against.