Belchertown Skin, MQTT, Incomplete Page with Good Data

[G Hammer]

I have the Belchertown skin installed and working well.

I then installed the weewx mqtt extension (0.18) and configured it using test.mosquitto.org

Data is sent according to the log entries and but subscribing via MQTT.fx

{"rain24_in": "0.0", "barometer_inHg": "30.34", "outHumidity": "80.0", "altimeter_inHg": "30.2521121579", "maxSolarRad": "396.841281938", "dewpoint_F": "71.4119164898", "rain_in": "0.0", "cloudbase_foot": "1945.01897958", "heatindex_F": "78.1", "inDewpoint_F": "68.9270249035", "dayRain_in": "0.0", "rain_period": "800.0", "outTemp_F": "78.1", "appTemp_F": "86.3498648255", "windchill_F": "78.1", "hourRain_in": "0.0", "humidex_F": "94.7038960786", "rainRate_inch_per_hour": "0.0", "windGust_mph": "1.1", "pressure_inHg": "29.8", "inTemp_F": "77.9", "usUnits": "1.0", "windSpeed_mph": "0.2", "UV": "0.0", "dateTime": "1536065799.0", "windDir": "106.0", "inHumidity": "74.0", "radiation_Wpm2": "66.31"}

The site shows "Connecting to weather station real time data" and stays with that status.

However, the displayed data is current if I refresh the page.

Graphs are not displayed on the main page, but are generated and shown on the graphs page.

So, to sum it up, mqtt is working, the site can get the data and displays it. Connecting to real time data never goes away and graphs on main page do not appear.

Any idea where to look?

[Pat]

Hey there, I can help a little later today - back at work today. 

Initial thoughts are there should be some debugging info in Chrome. Open the Developer Tools (F12 on Windows), and then select the Console tab. You should see status like "Connecting to MQTT" or "MQTT connect failed. Reason: xyz"

[Colin Larsen]

I used a simple mqtt client to debug mine, saved a lot of time. There are plenty of free ones about.

What are your settings in skin.conf and weewx.conf?

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[G Hammer]

Hi Colin,

I don't think the problem is with sending or retrieving data from the MQTT server, I have done both the test.mosquitto.org and then installed mosquitto on my server.

I used MQTT.fx to subscribe to both and both are getting the data.

The skin is getting data as it shows the temp, humidity, wind, but never shows a connection and never completes the main page. Graphs on the page are missing/blank but shown on the Graphs page.

You can better understand the issue by visiting the site. After you are there for 30 seconds or so, refresh the page and see that the data has changed.

My Weather Site

[Colin Larsen]

Since it continues to say 'connecting' that perhaps points to a problem with the skin.conf connection string? Once it gets a good connection it goes to 'connected waiting data'

--

[Pat]

The skin - at its core - will show the data from the previous archive. Once connected it'll update those fields from the MQTT messages. 

The MQTT will show "connected" in the status bar once it gets a timestamp from the published message. 

Check to make sure your broker allows websockets. To validate you can use this test tool. http://www.hivemq.com/demos/websocket-client/

If that test tool works, make sure your config options are correct. 

Anything in the Chrome console log?

[Pat]

On second glance, use this tool instead. http://mitsuruog.github.io/what-mqtt/ since it can support SSL brokers (even though it's not documented). Maybe I should setup my own websocket test tool? :)

If your MQTT is setup with SSL, then you'll want to connect to secure websockets with WSS. Otherwise it'll use normal websockets which is ws. 

Example: wss://your.broker.com:port for secure. Then subscribe to your topic and see if its working 

[G Hammer]

I can say that my mosquitto install is not using websockets.

With mqtt-spy on the server, I can set it up for either default or websockets.

Default works fine, websockets does not.

Though only item I changed in the mosquitto.conf is protocol websockets

So, stumped. Anyone have a working mosquitto install that they would share the conf?

[Colin Larsen]

Here's mine that work fine

Weewx.conf

[[MQTT]]

        #This section is for the MQTT service setup

        server_url = mqtt://test.mosquitto.org:1883/

        topic = weather/41south

        unit_system = METRIC

        binding = archive, loop

        aggregation = aggregate

Skin.conf

# MQTT Defaults

    mqtt_enabled = 1

    mqtt_host = "test.mosquitto.org"

    mqtt_port = 8080

    mqtt_ssl = 0

    mqtt_topic = "weather/41south/#"

    disconnect_live_website_visitor = 1800000

--

[G Hammer]

Thanks. 

I’ll swap to test tomorrow. 

It’s always something...

You received this message because you are subscribed to a topic in the Google Groups "weewx-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/weewx-user/81PjpX0Ajas/unsubscribe.
To unsubscribe from this group and all its topics, send an email to weewx-user+...@googlegroups.com.

[Colin Larsen]

You may need to change unit_system to match yours is all

[G Hammer]

Yep, I’ll just swap to the test server. 

Thanks

[Colin Larsen]

I guess as another test if needed, you could just connect to my feed and read my data into your page too

[G Hammer]

As a ‘known good’ I will try if my data fails. 

[Colin Larsen]

Just be aware that I'm only running 'simulator' so don't panic if you see awful weather here! :)

[quantum.m...@googlemail.com]

Just ‘jumping in’ to this thread and I think I can see where the problem lies.

Collins website appears to be using http

G Hammer’s website appears to be using https

The instructions that Collin gave are correct and valid for an http site, that is one not using SSL.

If you use those instructions, that is the MQTT configuration, on an SSL site, it will do exactly what you are describing, that is hang continuously stating ‘Connecting to weather station data’. This is by design, as an SSL website cannot and will not process non-SSL data.

Solutions?

1. Simply access your website using http only and it will work with the unedited config. Be careful here though, as most modern browsers will automatically default to https once they ‘have seen’ a valid https connection for a domain, so you may need to clear the cache/history of the chosen browser to make it work, if you have already visited the website via https
   
2. Simply for test, change the config details that Collin kindly provided, with two subtle changes to direct the client to use secure websockets and a secure port, thus:

Edited reproduction of Collin’s config – changes to port number and SSL = 1

[[MQTT]]
        #This section is for the MQTT service setup
        server_url = mqtt://test.mosquitto.org:1883/
        topic = weather/41south
        unit_system = METRIC
        binding = archive, loop
        aggregation = aggregate


Skin.conf


# MQTT Defaults
    mqtt_enabled = 1
    mqtt_host = "test.mosquitto.org"

    mqtt_port = 8081
    mqtt_ssl = 1

    mqtt_topic = "weather/41south/#"
    disconnect_live_website_visitor = 1800000

The Mosquitto documentation refers:

https://test.mosquitto.org/

Hope this helps - all the best,

Dave.

[quantum.m...@googlemail.com]

Just to confirm I have highlighted in Red what you need to change to make an https site work with test.mosquitto.org  I couldn't get the colouring to work properly above.......

mqtt_port = 8081 instead of 8080

mqtt_ssl = 1 (instead of 0)

Let me know how you get on - if necessary I can spin up a Pi rig for you to use as a test - but the test.mosquitto.org should work without problems - certainly works for me - I have just tested it under an https website request.

Let me know either way.

All the best, 

Dave.

Skin.conf


# MQTT Defaults
    mqtt_enabled = 1
    mqtt_host = "test.mosquitto.org"

    mqtt_port = 8081
    mqtt_ssl = 1

    mqtt_topic = "weather/41south/#"
    disconnect_live_website_visitor = 1800000

[G Hammer]

You are a genius! This worked a treat!

The security was the issue.

Thank you ever so much, I would have poked at everything but that.

[Colin Larsen]

Its my understanding that if mosquitto didn't support websockets then the skin updates wouldn't work at all, or did I read what you wrote wrong?

This is my in-house mosquitto server config file (untested)

persistence false

allow_anonymous true

# Non-SSL MQTT
listener 1883
protocol mqtt

# Non-SSL Websockets
listener 9001
protocol websockets

On Thu, Sep 6, 2018 at 2:04 AM G Hammer <gary....@gmail.com> wrote:

Here is what is in the dev console when I load the site:

Connecting to MQTT weather:416:13
jQuery.Deferred exception: An invalid or illegal string was specified Paho.MQTT</k.prototype._doConnect@https://cdnjs.cloudflare.com/ajax/libs/paho-mqtt/1.0.1/mqttws31.min.js:36:453
Paho.MQTT</k.prototype.connect@https://cdnjs.cloudflare.com/ajax/libs/paho-mqtt/1.0.1/mqttws31.min.js:31:476
I/this.connect@https://cdnjs.cloudflare.com/ajax/libs/paho-mqtt/1.0.1/mqttws31.min.js:65:80
connect@https://ghammer.net/weather/:431:13
@https://ghammer.net/weather/:90:13
j@https://ajax.googleapis.com/ajax/libs/jquery/3.2.1/jquery.min.js:2:29997
g/</k<@https://ajax.googleapis.com/ajax/libs/jquery/3.2.1/jquery.min.js:2:30313
 undefined jquery.min.js:2:31569
sendTimingEvent addSite start undefined undefined

There is more, but that seems to be a good starting point.

As my mosquitto instance is inside my LAN, it would be difficult to use web based troubleshooting on it.

But I have tested and the data is there, I'll have to investigate if it supports websockets though. A quick read says no on mosquitto, which is what I have locally.

However, it did not work (same symptoms) using test.mosquitto.org which is why I brought it inhouse.

[G Hammer]

Actually, it is the need to have a cert for the server the Mosquitto is on. 

For now, using test.mosquitto.org is working fine.  

Thanks to Dave. 

You received this message because you are subscribed to a topic in the Google Groups "weewx-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/weewx-user/81PjpX0Ajas/unsubscribe.
To unsubscribe from this group and all its topics, send an email to weewx-user+...@googlegroups.com.

[Colin Larsen]

Excellent, glad progress is being made

[G Hammer]

Did you see the cause? http (for you) vs https (for me)

[Colin Larsen]

Yes indeed, thanks, good to know in case I go that way.