Belchertown MQTT not working

[Kevin Crivelli]

locally I get

Connected. Waiting for data.

remotely I get

Connecting to weather station real time data

I followed https://obrienlabs.net/how-to-setup-your-own-mqtt-broker/ to a T but didn't do the SSL portion

my mosquitto myconfig.conf file reads

persistence false

# mqtt
listener 1883
protocol mqtt

# websockets
listener 9001
protocol websockets

allow_anonymous true

password_file /etc/mosquitto/passwd

acl_file /etc/mosquitto/acl


my weewx.conf has this for mqtt under [StdReport]

  server_url = mqtt://myusername:mypas...@192.168.1.216:1883
        topic = weather
        unit_system = US
        binding = archive, loop
        aggregation = aggregate


and this in [[[Extras]]]

#--- MQTT Websockets (for Real Time Streaming) Options ---
            mqtt_websockets_enabled = 1
            mqtt_websockets_host = 192.168.1.216
            mqtt_websockets_port = 9001
            mqtt_websockets_ssl = 0
            mqtt_websockets_topic = "weather/loop"
            disconnect_live_website_visitor = 1800000


in skin.conf I have under [Extras]

    # MQTT Websockets defaults
    mqtt_websockets_enabled = 1
    mqtt_websockets_host = 192.168.1.216
    mqtt_websockets_port = 9001
    mqtt_websockets_ssl = 0
    mqtt_websockets_topic = "weather/loop"
    mqtt_websockets_username = myusername
    mqtt_websockets_password = mypassword
    disconnect_live_website_visitor = 1800000


When I did the guide I was able to get successful data stream from mosquitto in the test so I feel like the mosquitto portion is correct.

I've completely run out of ideas. If anyone can come up with anything to help me it would be greatly appreciated. I've been at it for 2 days. Thanks in advance

[vince]

Try adding the mqtt username and password in [[[Extras]]] in weewx.conf and restart weewx and see if that helps.

[Kevin Crivelli]

so like the same way they are shown in the skin.conf file?

[Kevin Crivelli]

yeah man, no joy

[vince]

On Saturday, May 6, 2023 at 10:37:54 AM UTC-7 Kevin Crivelli wrote:

yeah man, no joy

I'd suggest turning off user/pass in MQTT at least temporarily until you get the websockets working, then reenable user/pass.  Do one thing at a time.

I spun up a clean pi without mosquitto user/pass and it worked fine the first time.  My procedure and config file entries are on github (here) if you want to compare with yours.  Note that I did a "v5 pip --user" installation so some weewx paths and commands differ, but the mosquitto stuff and weewx.conf edits should be the same.  Should be self-evident where you need to tweak my example.

Remember to restart weewx or mosquitto if you mess with their respective config files...

[Kevin Crivelli]

so I tried turning off the user/pass business and still no luck however I have noticed something interesting.

When I go through the mosquitto_sub -t mytopic -h myaddress -u myuser -P mypass

it just hangs and never finishes

wonder what could be causing that

[Kevin Crivelli]

I'm using my ip address as "myaaddress" I've also tried localhost

[Kevin Crivelli]

here's what I get when I run with -d debug

[vince]

It's not hanging.  It is not hearing anything.  Hit control-C to stop listening with mosquitto_sub.

On mine I see the loop messages every couple seconds basically immediately.  I hit control-C after the first loop packet was displayed.

$ mosquitto_sub -d -t simulator/# -h 192.168.1.232
Client null sending CONNECT
Client null received CONNACK (0)
Client null sending SUBSCRIBE (Mid: 1, Topic: simulator/#, QoS: 0, Options: 0x00)
Client null received SUBACK
Subscribed (mid: 1): 0
Client null received PUBLISH (d0, q0, r0, m0, 'simulator/loop', ... (1255 bytes))
{"dateTime": "1683403745.0", "usUnits": "1.0", "outTemp_F": "30.714330964267866", "inTemp_F": "64.4867113429638", "barometer_inHg": "31.022674842890833", "pressure_inHg": "31.022674842890833", "windSpeed_mph": "0.3866257855458439", "windDir": "346.0814717203496", "windGust_mph": "0.46395094265501235", "windGustDir": "346.0814717203496", "outHumidity": "77.68024528672494", "inHumidity": "27.026577314072398", "radiation_Wpm2": "911.9394844177033", "UV": "12.767152781847846", "rain_in": "0.0", "txBatteryStatus": "1.0", "windBatteryStatus": "0.0", "rainBatteryStatus": "0.0", "outTempBatteryStatus": "1.0", "inTempBatteryStatus": "0.0", "consBatteryVoltage_volt": "12.0", "heatingVoltage_volt": "12.876232859402455", "supplyVoltage_volt": "12.0", "referenceVoltage_volt": "11.725352060455753", "rxCheckPercent": "58.589515129677444", "altimeter_inHg": "31.812916691917174", "appTemp_F": "25.97065488238462", "cloudbase_foot": "2093.2738825982638", "dewpoint_F": "24.583925880835505", "heatindex_F": "30.714330964267866", "humidex_F": "30.714330964267866", "inDewpoint_F": "29.791543533239455", "maxSolarRad_Wpm2": "0.0", "rainRate_inch_per_hour": "0.0", "windchill_F": "30.714330964267866", "hourRain_in": "0.0", "rain24_in": "0.0", "dayRain_in": "0.0"}
Client null received PUBLISH (d0, q0, r0, m0, 'simulator/loop', ... (1236 bytes))
^CClient null sending DISCONNECT

Test your mosquitto server...

• subscribe in one window => mosquitto_sub -h x.x.x.x -t junk/testing -u mosquittoUserHere -P mosquittoPassHere -m testing123
• publish a test in another  => mosquitto_pub -h x.x.x.x -t junk/testing -u mosquittoUserHere -P mosquittoPassHere

If that doesn't work, your mosquitto setup is messed up.

It ain't rocket science unless you are running a funky os variant or you have enabled packet filters blocking incoming traffic to the MQTT or Websockets ports.

[vince]

doh - I of course got those sub/pub commands backward.  You use the -m message for publishing.  Sorry for the confusion.

• subscribe in one window => mosquitto_sub -h x.x.x.x -t junk/testing -u mosquittoUserHere -P mosquittoPassHere

• publish a test in another  => mosquitto_pub -h x.x.x.x -t junk/testing -u mosquittoUserHere -P mosquittoPassHere -m testing123

I added password protection to my pi setup and it works fine there and after restarting weewx (after editing weewx.conf).   Updated the github page to show the edits I made.

I did need to create the MQTT password file so the specified user/pass is there.  Remember it's not an account password.  It's just a user/pass combination for connecting to your MQTT server.

[Kevin Crivelli]

I was able to get the test message no problem so it seems mosquitto is set up correctly yet I am still unable to get anything more than Connected. Waiting for data using my local url and remotely I get Connecting to weather station real time

still where I left off.

[Kevin Crivelli]

ok so I seem to be getting weather when using that test from earlier

[Kevin Crivelli]

as per my last post I can use that command to see topic weather data coming in but still for some reason weewx/belchertown can connect but no data is transmitted. I gotta be missing one little thing. Messed around again this morning and still nothing. Really don't want to throw in the towel so if anyone has anymore ideas that would be awesome.

[vince]

You need (1) to be consistent among MQTT publish, Belchertown, and mosquitto re: whether you have username/password enabled or not and (2) if enabled you have to not typo the username/password in one of the places.   Other possibility is that you're going back and forth editing things and forgot to reset either weewx or mosquitto after making a change.  It happens.

[vince]

I might add that if you set debug=1 in weewx.conf it will definitely tell you if the MQTT publishing part is unable to publish successfully.

[pfams12]

For troubleshooting this sort of thing I use MQTTX; especially from the client perspective to ensure router / FW connections are working as expected.  

https://mqttx.app

It may be helpful.

[Kevin Crivelli]

Dude awesome I'm gonna try this out 

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/61cf9ee4-8308-4bed-97bd-c48d198b30a3n%40googlegroups.com.

[Kevin Crivelli]

Well, I wasn't able to get MQTTX installed. I may come back to it again. In the meantime I was able to get some information from weewx debug mode. 

Here is the result of running the command   tail -f /var/log/syslog  To watch the log as it is generated.

So I am definitely getting records published from MQTT however I still have the same problem where the local address is showing connected waiting for data and the remote instance is just connecting

no further along but if anyone can make sense of this from that log data please let me know. Thanks again for everyones help

[Graham Eddy]

try subscribe (-v) to topic ‘#’ to examine all messages (in case topic mis-named)

⊣GE⊢

[vince]

On Monday, May 8, 2023 at 4:12:03 PM UTC-7 Kevin Crivelli wrote:

Here is the result of running the command   tail -f /var/log/syslog  To watch the log as it is generated.

So I am definitely getting records published from MQTT however I still have the same problem where the local address is showing connected waiting for data and the remote instance is just connecting

Still not understanding what your setup is nor what you mean when you say 'local address' and 'remote instance'.  Also, you are not getting records published 'from' MQTT in the log snippet you showed.  You are sending packets 'to' the MQTT broker.  Unfortunately it provides no info on what exactly you published, nor what you're looking for, so we can't do much with what you've provided...

• what is the ip of your weewx system ?
  
• what is the ip of your mosquitto broker system ?
  
• what is the ip of the system running the browser displaying Belchertown ?
  
• what URL are you opening in your browser for the Belchertown skin ?
  
• what browser and version are you running ?

And you've been changing lots of stuff going back and forth trying to get to success so lets re-baseline where you are now...

• Lets see your current MQTT block from weewx.conf.   Just obfuscate the user/pass there if you still have it enabled for MQTT.
  
• Lets see your current Belchertown block from weewx.conf.  Obfuscate the user/pass there too if you still have it enabled for MQTT. 
• Lets see your current mosquitto MQTT config file as well where you set up listening on the two ports.

Most importantly, lets debug what you're publishing and to where:

• set weewx.conf debug=2 also to turn on verbose logging from the MQTT publisher extension.  Remember to restart weewx.
  

And a "tail -f" in a screen capture image is kinda hard for us to work with.   Provide the normal log snippets please.  Restart weewx and lets see all the entries through the first archive period or two.  If you have to attach that as a text attachment, that's fine.