MQTTSubscribe and paho mqtt heads up

[bell...@gmail.com]

There is new release of paho mqtt, 2.0.0. This has a breaking change for all clients using this library. I'll let you know when MQTTSubscribe supports it.

Sigh, backwards compatibility is overrated.

rich

[gary....@gmail.com]

pip install paho-mqtt==1.6.1

Downgraded paho-mqtt in my venv, restarted weewx and all is well.

[Greg Troxel]

"bell...@gmail.com" <bell...@gmail.com> writes:

> There is new release of paho mqtt, 2.0.0. This has a breaking change for
> all clients using this library. I'll let you know when MQTTSubscribe
> supports it.

Do you expect that MQTTSubscribe can work ok if either 1.6.x or 2.0.0
is installed?

[bell...@gmail.com]

It currently will not work with paho mqtt 2.0.0 or higher.

[Greg Troxel]

"bell...@gmail.com" <bell...@gmail.com> writes:

> It currently will not work with paho mqtt 2.0.0 or higher.

Sorry, I understood that. I have looked briefly at the migration which
says "change your code because a function signature changed", and have I
have been holding off because I'll need to fully absorb it, and then
update rtl_433_mqtt_relay.pt (for which I am somewhat the maintainer)
and a program (that I haven't published yet) that polls ups-nut and
reports status over mqtt.

What I meant, is, do you expect that when you figure out how to deal
with the incompatible changes, you'll end up with a MQTTSubscribe that
can function with either, or if we'll need to match versions.

If there's a way to do it (try/catch and figure out the signature?) then
presumaly the MQTT publish extension, rtl_433_mqtt_relay and everything
else can use that technique.

[vince]

Why not stick with 1.6.1 indefinitely and the matching version of the extension and not sweat it ?

[bell...@gmail.com]

My plan is to try the 2.0 signature. I think it will throw a TypeError (but what ever it throws) and catch that and use the 1.6 signature.

Unless a Python expert has a better idea.

The alternative is use an if to check the version and call the appropriate signature.

Make sure to a read the upgrade notes. You will want to pass in the value to say to use the old signatures for other methods. Looks like a bunch changed.

I hope to start working on it soon. The biggest pain will be updating my CI/CD to (1) deal with multiple versions and (2) decide how much to test against each version… 

rich

[Greg Troxel]

vince <vince...@gmail.com> writes:

> Why not stick with 1.6.1 indefinitely and the matching version of the
> extension and not sweat it ?

Because bugs are not going to be fixed, and packaging systems are going
to have 2.0.0, and in general trying to stay in the past causes too much
trouble eventually. The same reason weewx had to cope with python
instead of just telling people to use 2.7 :)

[vince]

No reason to upgrade if it works for you already and if there are no security risks since then.

To list all versions available "pip list paho-mqtt=="

To install a specified version "pip install paho-mqtt==1.6.1"

Or for apt.......

List all versions "apt-cache policy python3-paho-mqtt"
Install one specified version "apt-get install python3-paho-mqtt=1.5.1-1"

Note that for deb12 (raspi) the latest package is 1.5.1 which is a year older than 1.6.1, so they're not upgrading either.

[michael.k...@gmx.at]

It did a fresh pip install on bookworm and ran into this for weewx-mqtt. It will be an issue people ask about in the near future, that's for sure.

[bell...@gmail.com]

Because (1) its inevitable and (2) I’ll get tired of typing the wiki url explaining this.

I just hope they wait long enough (that 1.x is gone) before removing the other signatures that have changed…

[matthew wall]

On Sunday, February 11, 2024 at 1:37:20 PM UTC-5 bell...@gmail.com wrote:

It currently will not work with paho mqtt 2.0.0 or higher.

rich, i too am interested in what you learn, as i will probably have to update weewx-mqtt as well. m

[bell...@gmail.com]

I've pretty much settled on this approach. Minimally tested

        try:
            callback_api_version = mqtt.CallbackAPIVersion.VERSION1
            self.client = mqtt.Client(callback_api_version=callback_api_version,
                                    client_id=mqtt_options['clientid'],
                                    userdata=self.userdata,
                                    clean_session=mqtt_options['clean_session'])
        except AttributeError:
            self.client = mqtt.Client(client_id=mqtt_options['clientid'],
                                    userdata=self.userdata,
                                    clean_session=mqtt_options['clean_session'])

rich

[bell...@gmail.com]

FWIW 

Successfully ran integration tests on Ubuntu 22, python 3.12 with paho-mqtt v2.0.0 and v1.6.1.

[Graham Knights]

I just migrated my install to a new machine and was pulling my hair out trying to figure this out until I saw this today, thanks @gary for figuring this out.  I've edited the weewx wiki to direct new setups to install the v1.6.1 paho-mqtt until the original mqtt extension is updated.  

[bell...@gmail.com]

Just a quick update.
I’m happy with this approach, but eventually V1 API support will be removed. Ideally this will be long after V1 clients are gone. But, I’ve been burnt once…. So, I’m implementing the approach below.
I’m going to use the factory pattern. I already had a class ‘wrapping’ paho mqtt. I will now have a base class that implements all of the logic. I will have subclasses as necessary for the different versions of paho mqtt. These subclasses will interact with paho mqtt and pass data to the base class to do the real work. I’ll have a factory to provide the appropriate class to interact with paho mqtt.
Hopefully this will insulate a bit from breaking changes, or at least make it easier to react to them.
rich

On Sunday 11 February 2024 at 16:49:25 UTC-5 bell...@gmail.com wrote:

[Graham Eddy]

i wrapped paho mqtt a few years ago, but not implementing whole interface.

i wote a bunch of apps on top of my wrapper, one being my own weewx mqtt service/driver, other apps are various mqtt producers and consumers

thanks for the heads-up on the mqtt interface change.

it prompted me to polish my wrapper a bit, with things i learnt over last few years.

one is to put json encoding/decoding at the heart of the wrapper - i expose python objects rather than mqtt messages, and the python objects are converted to json (strings) in the wrapper to/from the mqtt wire

i’ve yet to find a use for clean_session=false.

cheers

⊣GE⊢

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/8b3fba7b-dfae-4105-8138-7381e5697017n%40googlegroups.com.