Public webpage using MQTT from weewx

[Radek Dohnal]

Hello.

What is the safe way how to use public webpage with my meteo data (on public site) together with weex sending MQTT data on my local home network ?

I would like to use MQTT from my weewx (running on my home network). But I dont wont to open any port from my home network to the internet.

Now Im using FTP to transfer data (HTML_ROOT folder from my local RaspPI where weewx runnig) to public site where i rent a space for my webpages. But without MQTT.

[Greg Troxel]

An approach would be to run an mqtt broker on a VPS. Basically you are
using a remote webserver on a real/stable IP address and it would make
sense to do the same with MQTT.

It may be that you have a "web server" that you can't run programs on.

Note that you can (depending on broker) bridge from local to a remote
mqtt server.

[vince]

It depends on whether you want realtime MQTT-based data (ie, enabling the realtime-ish side of Belchertown).

If all you want to do is display a static page based on MQTT data, weewx can of course do that and you can rsync or ftp the data up to a public-facing website on your service provider.

If you want realtime MQTT data it gets more complicated. As Greg said, you'd have to run a MQTT broker on your public webserver.  'Definitely' set that up to be password-protected on the Internet.   Also agree with Greg that you can have your LAN MQTT broker forward to your Internet one pretty easily.

That said, I did not find a combination for Belchertown specifically that let me run on LAN and see the same nice dynamic updates on Internet, so I eventually turned that off and just run a static weewx site in both places, synced via rsync.

[Greg Troxel]

vince <vince...@gmail.com> writes:

> On Wednesday, November 20, 2019 at 9:27:02 PM UTC-8, Radek Dohnal wrote:
>
>> What is the safe way how to use public webpage with my meteo data (on
>> public site) together with weex sending MQTT data on my local home network ?
>> I would like to use MQTT from my weewx (running on my home network). But I
>> dont wont to open any port from my home network to the internet.
>> Now Im using FTP to transfer data (HTML_ROOT folder from my local RaspPI
>> where weewx runnig) to public site where i rent a space for my webpages.
>> But without MQTT.
>
> It depends on whether you want realtime MQTT-based data (ie, enabling the
> realtime-ish side of Belchertown).

Indeed, I did not get this point in the question. Adding local mqtt
does not affect the remote skin at all.

> If you want realtime MQTT data it gets more complicated. As Greg said,
> you'd have to run a MQTT broker on your public webserver. 'Definitely' set
> that up to be password-protected on the Internet. Also agree with Greg
> that you can have your LAN MQTT broker forward to your Internet one pretty
> easily.

I don't follow "password-protected" entirely. As I see it (but haven't
done it):

you have html someplace for the skin, and it may or may not have a
password. Typically these do not have access control.

The javascript from the skin has to talk to an mqtt broker that has
the data. So one has choices (assuming one is writing code):

1) let the mqtt data be accessible without a password

2) use a static username/password and put it in the skin. This is
sort of like an API key in web parlance.

3) some scheme of new username/passwords in each page view from the
skin

I suspect most people running skins that get live mqtt data are doing
option 1, that almost all readers think option 3 is nuts, and that zero
people are doing it.

[vince]

On Thursday, November 21, 2019 at 8:30:34 AM UTC-8, Greg Troxel wrote:

I don't follow "password-protected" entirely. 

oh - I meant protecting the Internet MQTT broker from nefarious denial-of-service from the script kiddies.

The LAN broker will need to forward/post to the Internet broker instance. You want to make sure it's just 'you' who can post data there, so enabling the MQTT username/password setup on the Internet broker will help stop the bad guys from messing with your data.  The LAN MQTT broker can (probably) be open for writes without username/password needed, depending on how you like to set your LAN up.

My setup at home has a bunch of pi and arduinos and sensors posting to local MQTT without any passwords needed.  When I had the Internet MQTT broker being bridged to (as MQTT uses the term) from the LAN, I had just 'that' one requiring a username/password, and also had some packet filters etc. limiting the incoming MQTT traffic to be from the pretty stable public ip address my home LAN NAT's out to Internet on via my service provider.

But no I didn't mean webserver username+pass.  Sorry for any confusion there.

[Greg Troxel]

vince <vince...@gmail.com> writes:

> On Thursday, November 21, 2019 at 8:30:34 AM UTC-8, Greg Troxel wrote:
>
>> I don't follow "password-protected" entirely.
>>
>
> oh - I meant protecting the Internet MQTT broker from nefarious
> denial-of-service from the script kiddies.
>
> The LAN broker will need to forward/post to the Internet broker instance.
> You want to make sure it's just 'you' who can post data there, so enabling
> the MQTT username/password setup on the Internet broker will help stop the
> bad guys from messing with your data. The LAN MQTT broker can (probably)
> be open for writes without username/password needed, depending on how you
> like to set your LAN up.

I understand now. It was obvious to me that writes must be
authenticated and thus I thought we were talking about allowing
unauthenticated reads. However, it is not obvious to everyone and
excellent advice to someone starting out.

> My setup at home has a bunch of pi and arduinos and sensors posting to
> local MQTT without any passwords needed. When I had the Internet MQTT
> broker being bridged to (as MQTT uses the term) from the LAN, I had just
> 'that' one requiring a username/password, and also had some packet filters
> etc. limiting the incoming MQTT traffic to be from the pretty stable public
> ip address my home LAN NAT's out to Internet on via my service provider.

Makes sense. I have set up TLS on both home and public broker and also
username/passwords and acls. All of my sensors have credentials that
allows them to write to part of the sensor subspace. Indeed, this is
much more work.

> But no I didn't mean webserver username+pass. Sorry for any confusion
> there.

No problem, and I was misunderstanding more than you -- I think it's
actually been a very useful discussion. To sum up for the OP, assuming
they want to do something like Belchertown

set up an MQTT broker on a public/stable IP address

configure acl to require user/password for writing, to avoid kiddies
writing to your topics and also storing warez fragements in various
retained topics, as happened with writable anonymous FTP. For extra
credit, set up TLS and only do password-controlled access over TLS to
prevent password sniffing.

allow anonymous reads of the data that you intend to be used by the
skin -- and only that data.

Keep in mind that because MQTT ends up being the way you connect
everything to everything, almost all data in it is sensitive with
respect to writes and some data is sensitive with respect to reads.

[Radek Dohnal]

Thanks for super explanation..

Set up MQTT broker on a public IP address - you mean to you something like this? - https://www.hivemq.com/blog/build-javascript-mqtt-web-application/

I dont want to use public MQTT (i.e. http://www.mqtt-dashboard.com/) - there is no possibility to password secure.

Dne čtvrtek 21. listopadu 2019 18:48:18 UTC+1 Greg Troxel napsal(a):

[Les Niles]

You could use a hosted MQTT service like  https://www.cloudmqtt.com/ which gives you your own broker. It’s free for a small number of connections and inexpensive for somewhat more. Or you could get a cloud compute instance on aws or your favorite cloud provider, and install an MQTT broker there. 

I just put the broker on a ~dedicated RPi on an isolated network (dmz) behind my firewall. I do have a static IP, but DDNS works pretty well with most ISPs that have dynamic IPs since the IPs rarely change, as long as they assign public IPs. 

  -Les

On Nov 22, 2019, at 3:02 AM, Radek Dohnal <dohna...@gmail.com> wrote:



--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/e4344b69-d078-413f-98e4-8dd2cc1d3d0f%40googlegroups.com.

[Greg Troxel]

Radek Dohnal <dohna...@gmail.com> writes:

> Set up MQTT broker on a public IP address - you mean to you something like
> this? - https://www.hivemq.com/blog/build-javascript-mqtt-web-application/

No, I meant

1) get a computer, perhaps virtual, that is actually on the internet,
on which you are willing to run a broker

2) install and configure mosquitto, or some other mqtt broker

but the advice about low-cost hosted mqtt brokers is interesting.

[Pat]

Hijacking this slightly; 

Vince: 

> That said, I did not find a combination for Belchertown specifically that let me run on LAN and see the same nice dynamic updates on Internet, so I eventually turned that off and just run a static weewx site in both places, synced via rsync.

Is this because of the belchertown_root_url? If so, try out the 1.1 beta skin in the development branch. That option has been removed and is replaced with root relative links. So I think this may resolve your issue you mentioned above. I'm planning on releasing 1.1 soon (as soon as life slows down a little bit!)

[Greg Senia]

Example of how I used a public mqtt endpoint to share my data on my website without opening up my mqtt broker directly..

http://weather.senia.org/

I can share further details of my mosquitto config etc.

-Greg

[Pat]

It looks like you're using mqtt.eclipse.org, so you must be bridging your private home MQTT broker to mqtt.eclipse.org? It's not a bad idea, but the concept can sometimes be confusing for new comers.

[Greg Senia]

Thats exactly what I'm doing. Here is the mqtt config:

[root@wxgen ~]# cat /etc/mosquitto/mosquitto.conf

# Place your local configuration in /etc/mosquitto/conf.d/

pid_file /var/run/mosquitto.pid

persistence true

persistence_location /var/lib/mosquitto/

log_dest file /var/log/mosquitto/mosquitto.log

include_dir /etc/mosquitto/conf.d

# Plain MQTT protocol

listener 1883

protocol mqtt

listener 1884

protocol websockets

# End of plain MQTT configuration

# Bridge to remote Broker example.com

connection bridge-to-remote

# Path to the remote Broker:

address mqtt.eclipse.org:1883

cleansession false

# client ID to be used at remote Broker:

clientid wxgen.senia.org

start_type automatic

notifications false

try_private false

# Publish everything which is published to topic /public/# on local Broker to remote Broker into topic /public/from_local.brocker/ with QOS2:

topic weather/loop out 0 "" wx.senia.org/

# Bridge to remote Broker example.com

connection replicate-to-remote

# Path to the remote Broker:

address wxgen.senia.org:1883

cleansession false

# client ID to be used at remote Broker:

clientid wxgen.senia.org_replicate

start_type automatic

notifications false

try_private false

# Publish everything which is published to topic /public/# on local Broker to remote Broker into topic /public/from_local.brocker/ with QOS2:

topic weather/loop out 0 "" wx.senia.org/

/etc/weewx/weewx.conf

    [[MQTT]]

        binding = archive, loop

        server_url = mqtt://wxgen.senia.org:1883/

        topic = weather

        post_interval = 2

[francisco javier martinez de lizarduy]

Hi, Greg.

I thank you for your ingenious contribution

I'm trying to adapt your mqtt config for my page:

                   http://kocher.es/index.html

I still haven't made it work for me. These are the main doubts I have for its operation:

- What should be the content of:

pid_file /var/run/mosquitto.pid

- I suppose I must also modify some lines of the mosquitto.conf file. 

could you help me?

Saludos desde San Sebastian, Spain

 

[Radek Dohnal]

Hi.

I Finally setup my cloud MQTT broker (flespi) and create bridge between local MosquitoMQTT and cloud Flespi. Thats work fine, topis weather/loop from Mosquito is sucesfully recieved in Flespi (I can see it in Subscriber part). But when I try to to connect Belcherol skin webpage to this FlespiMQTT, it stuck on "Connecting to weather station real time data."

My skin.conf:

    # MQTT Websockets defaults

    mqtt_websockets_enabled = 1
    mqtt_websockets_host = "ntLHZqh9S2DdCC5fYyQ6x0utCM07cSln9WUYc0VSSoWkOPpDb3zwTUQjuEfwlqKk:@mqtt.flespi.io"
    mqtt_websockets_port = 80
    mqtt_websockets_ssl = 0
    mqtt_websockets_topic = "weather/loop"
    disconnect_live_website_visitor = 1800000

Flespi has no password, but name/token generated (https://flespi.com/kb/tokens-access-keys-to-flespi-platform)

Port for noSSL websocket is 80 (https://flespi.com/mqtt-api)

Do you know where is the problem?

Dne čtvrtek 21. listopadu 2019 6:27:02 UTC+1 Radek Dohnal napsal(a):

[Michael Sanphillipo]

Did you ever get a response on this?