Help with MQTT

[O S]

Hello all,

I have resisted installing and configuring MQTT for live data in case I totally mess things up, but, in a fit of positivity, I decided to have a go today, and it doesn't work.

I've used a mix of search engine (AI) advice, this post and the instructions on the Belchertown skin page.

I have documented what I did, and my settings, can someone take a look and see if anything is glaringly wrong?

For information, I am running this locally at http://192.168... and publicly through https://mydomain.co/weewx/belchertown using a cloudflared tunnel.

Live updates don't appear to be happening in either scenario though (local or via https), ultimately, I'd like them working ion the public site (if it needs to be one or the other).

Thank you,

Nick.

[michael.k...@gmx.at]

The live updates are receive by a MQTT client in the browser. You need to set up WeeWX to publish your live data to a MQTT broker, which you can connect to from:

a) WeeWX for publishing

b) The client running in the Browser for subscribing

You also need to be aware that modern browsers block connectung to insecure MQTT from secure (https) sites.

A quick glance on your documentation let's me guess you're not too far away from getting live updates when you browse your site from the same network your WeeWX machine and your MQTT broker is running.

It should be possible to browse the page from your 192.168.1.x client at htttp://192.168.1.{weewx-machine}/weewx/belchertown with live updates.

What's not going to work is https://mydomain.co/weewx/belchertown with live updates.

Check:

- WeeWX logs for MQTT related messages

- mosquitto logs if WeeWX is connecting to the broker

- mosquitto logs if the mqtt client in the browser is connecting to the broker (will only work locally)

- the browser's console for error/info messages.

Getting everything to work with https://mydomain.co/weewx/belchertown is another topic, you might want to get yourself an account for a public broker or setup the cloudflare tunnel for MQTT (I tried once but failed, so tell me what you did if you'll succeed).

[O S]

Hello Michael,

Many thanks - all a bit above my pay grade, I am afraid so excuse any silly questions.

When you say there is a client running in the browser, is this something I need to install (I haven't), or would it be there already, I am assuming the latter. Regarding your comment about the no-go with the WAN site, is this fixable with a public broker or just not at all?

I've run some tests and updated that document here.

Thanks again,

Nick.

[Vince Skahan]

If there's a comprehensive HOWTO for how to 'securely' set up a cloudflare tunnel back to a LAN-hosted weewx+belchertown that would permit realtime updates to work from both LAN and WAN, I sure have never seen one.  That would be a great thing to get written, validated, and into the wiki.  This has been coming up for 5+ years.

FWIW - I don't let 'anything' talk to my LAN, even through a tunnel.  I don't want that risk.  Too many bots.

Anyway -  the websockets connection is between your browser and the remote MQTT broker, so whatever ip address you use has to be reachable from the web browser computer.  If you use a FQDN rather than an ip address, that has to be resolvable 'and' reachable from the web browser computer.

LAN-only is not hard.  Lots of people have done so.  Many posts here and in Pat's Belchertown github page.

WAN-only is not much harder.  Set up a small VM on AWS Lightsail or the like. Set up the webserver https-only and install the MQTT broker there.  Have your LAN weewx rsync data to it and also publish MQTT to the MQTT broker.  Use 'its' FQDN in all your settings for Belchertown.  Basically connect to your Internet site for realtime updates from both LAN and WAN.

Of course that means $$$ for the VM and the time/effort to keeping 'that' up securely as it will be under bot attack instantly after it boots up.  The AWS consoles are pretty good about letting you lock that down so only https and the secure websockets ports are open.  That'll reduce your attack services.  Damn bots.  Ugh.  A minimal nginx + mosquitto VM takes almost zero maintenance if that's all it does and if you lock it down correctly.  I think I ssh into my nginx-only site about monthly to see if the auto-updates for the os require a reboot, but it's not zero sustaining labor.

[michael.k...@gmx.at]

The client in the browser usually is a piece of Javascript code that is provided by the skin. 

[O S]

Hello Vince, OK - thanks for your comments there and I do get most of it!

Well. let's help some clever soul produces a how-to for WAN.

The MQTT service appears to have broken weewx, I see:

Feb 06 19:25:04 weewx-pi systemd[1]: weewx.service: Main process exited, code=exited, status=1/FAILURE
Feb 06 19:25:04 weewx-pi systemd[1]: weewx.service: Failed with result 'exit-code'.

.... in the service status, so i have stopped it for now with:

sudo service mosquitto stop
sudo systemctl stop mosquitto.service

Thanks,

Nick.

[O S]

OK, it's broken it - how do I get rid of the MQTT thing now? I knew I shouldn't have messed with things!

[O S]

If I run a report, I get:

Traceback (most recent call last):
  File "/usr/share/weewx/weectl.py", line 75, in <module>
    main()
  File "/usr/share/weewx/weectl.py", line 67, in main
    namespace.func(namespace)
  File "/usr/share/weewx/weectllib/__init__.py", line 90, in dispatch
    namespace.action_func(config_dict, namespace)
  File "/usr/share/weewx/weectllib/report_cmd.py", line 93, in run_reports
    weectllib.report_actions.run_reports(config_dict,
  File "/usr/share/weewx/weectllib/report_actions.py", line 84, in run_reports
    engine = weewx.engine.DummyEngine(config_dict)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/share/weewx/weewx/engine.py", line 89, in __init__
    self.loadServices(config_dict)
  File "/usr/share/weewx/weewx/engine.py", line 157, in loadServices
    obj = weeutil.weeutil.get_object(svc)(self, config_dict)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/share/weewx/weeutil/weeutil.py", line 1404, in get_object
    module = importlib.import_module(module_name)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1149, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/etc/weewx/bin/user/mqtt.py", line 109, in <module>
    import paho.mqtt.client as mqtt
ModuleNotFoundError: No module named 'paho'

[O S]

OK, I think I have uninstalled everything but still getting that same error - HELP!

[Michael Serowik]

i used this guide to get mine up and running  

https://potatoforinter.net/715/updated-guide-weewx-belchertown-mqtt-ws/

Mike

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/weewx-user/42a6d1af-2fa7-4471-9271-604b0bea7c8fn%40googlegroups.com.

[O S]

Thanks, Michael - that's interesting but no time to do that now, so just need to fix this error to get things working again.

Thanks for pointing that out though, I'm sure others will be interested in that.

All the best,

Nick.

[O S]

OK, I have fixed it with:

sudo apt install python3-paho-mqtt

I have no idea how that fixed it though... I'm leaving things alone now.

[Vince Skahan]

Did you succeed for both goals ?   Live data working on your LAN ?   Live data accessible from Internet ?

If so, how did you get the latter working specifically so others can try it ?   Looking for more than "I installed this".  Looking for actual instructions others might be able to follow and also succeed....

(answering 'fixed' or 'got it but I have not idea how' does not help the next person trying to do the same thing....)

[gary....@gmail.com]

I do not access my website locally via IP address (LAN) and also from the Internet differently.

I use https://weather.ghammer.net for both.

Why? As mentioned earlier, cert mismatch and the secure/insecure mix rejection.

Quick and easy for 'live' pages using MQTT

[Vince Skahan]

Good for you.  How exactly did you set it up ?

Is it a VM on Internet ?

What's your mosquitto conf file look like ?

Which cert files are installed where and with what permissions ?

Where did you get the certs ?

Your 'about' page says you're using a wifilogger2 - is it feeding a weewx site on weather.ghammer.net ?  If so how ?

[John Smith]

I do not access my website locally via IP address (LAN) and also from the Internet differently.

The way to get around certificate issues like that is to run a local resolver such as pi-hole that can re-write DNS requests.

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/weewx-user/5fb3139d-6b3d-4b01-b682-eb44f54b356bn%40googlegroups.com.

[gary....@gmail.com]

Immaterial VM or not.

Likewise with the data source.

Certs come from lets encrypt

-rw-r--r-- 1 mosquitto mosquitto 2143 Dec 21 00:02 cert.pem
-rw-r--r-- 1 mosquitto mosquitto 1802 Dec 21 00:02 chain.pem
-rw-r--r-- 1 mosquitto mosquitto 3268 Dec 21 00:02 privkey.pem

Mosquitto conf is default except the paths and my pwfile and acl

persistence false
allow_anonymous true
password_file /etc/mosquitto/pwfile
acl_file /etc/mosquitto/acl
# Insecure mqtt to localhost only, and secure mqtt
listener 1883
listener 8883
certfile /etc/mosquitto/certs/cert.pem
cafile /etc/mosquitto/certs/chain.pem
keyfile /etc/mosquitto/certs/privkey.pem
protocol mqtt
 
# websockets
listener 8080
certfile /etc/mosquitto/certs/cert.pem
cafile /etc/mosquitto/certs/chain.pem
keyfile /etc/mosquitto/certs/privkey.pem
protocol websockets

[gary....@gmail.com]

I guess that's one way, but why?

Additional config of a separate service that many may not have.

[John Smith]

On Wed, 11 Feb 2026 at 03:12, gary....@gmail.com <gary....@gmail.com> wrote:

I guess that's one way, but why?

Because I proxy most LAN servers via Cloudflare not just weeWX output, so I can access them when away from home and I don't need to pay for a static IP from my ISP then.

While weeWX output doesn't account for much data by itself other things like Nextcloud can transfer large amounts of data that doesn't need to go via the internet and back again when I'm at home.

[O S]

Hello Vince,

I meant I fixed the error that I was getting, not fixed the setting up of MQTT.

[gary....@gmail.com]

This may be helpful. Someone may want to duplicate this as the wayback machine isn't always as helpful

https://web.archive.org/web/20240412180952/https://obrienlabs.net/how-to-setup-your-own-mqtt-broker/

[Vince Skahan]

I’m trying to rewite it for rehosting somewhere along with a full set of known working files for both mosquitto and weewx.

At this point I have two setups working on a clean box.

• no encryption, totally anonymous read/write
• no encryption, mqtt username/password and acl to control read vs. write (anon read permitted)

I have not been able to get self-signed certs for a LAN only setup to work. While I can create the CA and broker files and they pass muster with openssl (can verify the broker cert is signed by the local CA) unfortunately Belchertown doesn’t connect and logs show either SSL version, protocol, or tls version issues. More unfortunately Belchertown provides zero logging saying ‘why’ it failed to connect.  All I see is an immediate close by the client reported in the mosquitto logs, sometimes. I usually don’t even see the connect attempt.

If anybody wants to help getting self-signed certs created and working let me know and we can work that offline if you prefer.

The fork at https://github.com/uajqq/weewx-belchertown-new fortunately has a copy of Gary’s old wiki but links to the wayback machine too for Pat’s old mqtt guide.

[Greg Troxel]

Vince Skahan <vince...@gmail.com> writes:

> I have not been able to get self-signed certs for a LAN only setup to work.
> While I can create the CA and broker files and they pass muster with
> openssl (can verify the broker cert is signed by the local CA)
> unfortunately Belchertown doesn’t connect and logs show either SSL version,
> protocol, or tls version issues. More unfortunately Belchertown provides
> zero logging saying ‘why’ it failed to connect. All I see is an immediate
> close by the client reported in the mosquitto logs, sometimes. I usually
> don’t even see the connect attempt.

Not sure exactly what your setup is, but what are you using for a name
with your self-signed CA? Is it a real FQDN, that when it is resolved
via DNS, requesters get the (LAN) IP address (dns RPZ?)? Or is it
"broker.local"?

When connecting via TLS, the normal path is to do pkix validation. This
has two parts. One is checking that the name used to start the TLS
connection matches the cert, where (more or less) match means it's one
of the subject alternative names. The second is that the certificate
presented together with the set of configured trust anchors needs to
pass validation.

So assuming you are running mosquitto, the first thing would be to see
if mosquitto_sub and mosquitto_pub work from the host you are trying to
use for weewx, and from the browser host.

You didn't say what browser, and you didn't say how you configured the
browser to add the local CA as a trust anchor. Usually, browsers have
their own trust anchor config instead of using the system config. Also
some browsers are getting extra strict trying to protect users from
themselves. They may be fussier about websockets than https.


You could also use a global domain name, get a LE cert, and then use dns
response policy zone to get the right addr to the client.

[Vince Skahan]

I have little idea what you are saying, but to answer:

• mosquitto, weewx mqtt publish, belchertown websockets work unencrypted using 1883 and 9001
• so I'm trying to take the next step and get the MQTT working on 8883

Re: setup

• this is a LAN-only setup with weewx and mosquitto on the same computer, and browsers on different computers on the LAN
• for the CN -  I'm using the ip address of the weewx+mosquitto computer for now, and that is in weewx.conf in all the appropriate places

Leaving tweaking weewx.conf for later, I can't get mosquitto to use self-signed certs successfully.

If you have complete+repeatable steps for generating my own certs/keys,  copying into place, setting permissions, and updating the mosquitto .conf files, I'm all ears.   I sure can't find those steps anywhere online.

Google keeps pointing me back to all kinds of ancient/invalid stuff like a horrible "steves-internet-guide" thing that doesn't work and various Stack Overflow questions+answers that also don't work (https://stackoverflow.com/questions/70110392/mqtt-tls-certificate-verify-failed-self-signed-certificate) or are incomplete.

FWIW - I just found https://forums.raspberrypi.com/viewtopic.php?t=287326 which I'll have to try step by step.  I'll already have their Steps1+2 working, so I'd need to try their Step3 next.  Fodder for tomorrow.

[Vince Skahan]

Brief followup  - found a fellow on YouTube who appears to have the secret decoder ring for setting up secure mosquitto with self-signed certs.

• Video at https://www.youtube.com/watch?v=-bKEWqcyEBY
  
• Transcript at https://shawnhymel.com/3085/how-to-use-the-mosquitto-mqtt-broker-with-ssl-tls/
  

Ignore the beyond ridiculous cringeworthy influencer picture of the dude (yeccccch).

The video is long but actually pretty good.  It does explain things as well as how to test in docker which wasn't bad either.  A cut/paste of the steps from the transcript 'did' work for me on a debian13 vagrant vm.  Cool.

In hindsight, I can see that'll I need to get the locally-generated ca.crt trusted by the browser(s) on whatever box(es) that might want to view the weewx+belchertown output on, but I can test end-to-end on one Linux laptop I have here to try to do an end-to-end test.

[Greg Troxel]

The basic issue is that PKIX (IETF working group that published the
specifications) is very complicated. Specs at

https://datatracker.ietf.org/wg/pkix/documents/

and I bet there are updates out of working group context.

This note explains what's really happening, vs what to type, and should
help you make sense of various advice, some (most?) of which is surely
confused.

The basic certificate scheme is that there are a number of certificate
authorities, which more or less means a self-signed certificate that
people intend to have sign otheer certificates. There is a social
consensus about which ones are trustworthy, more or less (there be
dragons here). In the Free Software world, this is typically encoded in
the root set published by Mozilla. Policy is at
https://www.mozilla.org/en-US/about/governance/policies/security-group/certs/policy/
and the bits are in a hg repo. In the proprietary world, the set is
often a little bit different, but it's different entites with similar
goals and mostly it amounts to the same thing.

A root CA certificate more or less means a self-signed cert from an
entity that behaves as a CA. It's ambiguous if any particular person/sw
trusts it.

The word "trust anchor" refers to a self-signed cert, which in a
particular entity (that validates certificates) has been configured to
be an acceptable starting point for validation.

By default, operating systems come with a particular set of trust
anchors preconfigured. Some programs choose to have their own set and
configuration mechanisms instead of using the OS set.


Certificates have key usage indications and constraints. End entity
certs can't sign other certs (well, you could make a signature, but
validators will reject it). Even CA certs can have contraints; a root
CA can sign a CA cert for foo.com which constrains that sub-CA to only
sign bar.foo.com certs for vairous values of bar. This is complicated,
but the point is that when you create your own CA cert, it has to be
coded to be a CA.



When a client makes a TLS connection to a server and the server presents
certificates, both the server's certificate and zero or more
intermediate certificates ("chain" is often used for intermediates, with
"fullchain" referring to the server's cert plus the intermedidate certs;
Let's Encrypt names files this wey), the client does a validation
procedure, which involves:

- Does the name the user used to make the connection match the server
cert? The original X.509 certs were about people, not computers,
and people typically put domain names in subject alternative names.
In order to succeed here, the validating software must be ok with
matching the intended name with the offered cert.

This means that if you have a web server that serves multiple names,
the cert it uses must be valid for all of them. (There is also SNI
which I think can be used to ask the server to act as a particular
target.)

I have not tried to make certs with IP addresses. I would not be
sure that an "inet_ntoa(ipaddr)" CN would be matched by openssl or
browsers, but I would not assert that it won't be.

- Given the server cert and chain (call this 1..N), is it true that
both:

+ the cert that signed the Nth in the offered chain is a configured
trust anchor?

+ for all values of i from 1 to N, is cert i signed by cert i+1,
does the signature match, is the date within range, and all the
rest of the constraint checks are ok? (And no certs have been
revoked and/or OCSP validation is ok, depending.)

In firefox, you can click on the lock, more info, connection secure,
view certificate and then see the EE cert at left, chain certs in the
middle (0 or more) and the trust anchor cert on the right.

When you make your own CA and try to use it, besides putting the right
contents in all the files, the hard parts are:

Having a scheme for how the server will be named (what name will be
used to access it) and encoding that in the server's cert so that
validators will be content.

Configuring the CA cert as a trust anchor in all software that will
validate the cert.

For operating a local CA, I use easyrsa:

https://community.openvpn.net/Pages/EasyRSA (sorry, cloudflare clickthrough)
https://easy-rsa.readthedocs.io/en/latest/
https://github.com/OpenVPN/easy-rsa

I recommend not using IP addreses. Instead, assign a FQDN *even if it
is not valid*, run your own nameserver on your home network, and
configure a "response policy zone" to return the LAN address of your
server when a host asks for "mqtt.foo.org" (for your chosen foo). Then,
put mqtt.foo.org in the CA as the subjectAltName, so that your certs are
structurally the same as all the rest of them. That will avoid all the
issues about "Does browser X handle https://1.2.3.4/ the way I want it
to?".

Also, you can then change the IP address, which is not part of
validation. The user/software asked to connect to mqtt.foo.org and it
did, and what IP address it has today is not part of ensuring it got to
the right place.


Or, you can register a real domain and use let's encrypt and have a real
certificate, even if you don't want to expose the server. But you'll
have to set up to pass one of the challenges. This is what I mostly do
and it takes "confgure trust anchor in every validating software" off
the table.