Topic matching on the client side

[ma...@markallanson.net]

Hi all,

Someone has just brought to my attention a deficiency in the nMQTT topic matching on receipt of messages where I am completely ignoring the fact that topic wildcards are avaiable for use.

Has anyone happened to have written a regex for matching subscribed topics to real topics?

Ie. Some that could return true for a message recieved on Topic "finance/stock/ibm/closingprice" for subscription issued on "finance/stock/ibm/#", and all the variants.

I'd prefer to reuse something that is proven to work than reinvent the wheel!

Mark

[Roger Light]

Hi Mark,

I used to use a regex a long time ago, but it's a bit crazy. I'd
suggest using another method of matching!

https://bitbucket.org/oojah/mosquitto/src/tip/doc/historical/old-regex.txt

Cheers,

Roger

> --
> --
> To learn more about MQTT please visit http://mqtt.org
>
> To post to this group, send email to mq...@googlegroups.com
> To unsubscribe from this group, send email to
> mqtt+uns...@googlegroups.com
>
> For more options, visit this group at
> http://groups.google.com/group/mqtt
>
> ---
> You received this message because you are subscribed to the Google Groups
> "MQ Telemetry Transport" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to mqtt+uns...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

[Darren Clark]

I'd suggest taking a look at the mosquitto algorithm. I haven't dug into how other clients are doing it, but Roger's seemed pretty clean.

-Darren

[Andrea Selva]

Hi,

in moquette I use a tree where every token of hte subscription string it's a node and during the matching phase I traverse the tree, probably it's not the most performant because the tree is a bottleneck for scalability but seems to work.

 Andrea

[Roger Light]

Hi,

Thanks Darren. As you suggest, the client side "does this topic match
a subscription?" check is quite straightforward. I hadn't wanted to
suggest looking at libmosquitto right at the moment because it
normalises topics, which the oasis committee has decided is a no-no.
So I convert a/////topic into a/topic before dealing with it. I'm part
way through fixing that, it's just the broker side that needs doing -
which uses the same tree technique as Andrea describes.

Although using a////topic will work correctly in the future, I'd like
to suggest that anybody using topics like that needs their heads
examined :)

Cheers,

Roger

[ma...@markallanson.net]

Sounds like it's best to stay away from regex. I don't think I'll implement a tree because introducing a tree may require a bit too much rejigging of the existing code. I will probably end up using some simple string splitting and matching minus regex.

[ma...@markallanson.net]

All,

For reference I have finished my wildcard/appendix A support and my set of tests for topic validation (c#) can be found here:

https://github.com/markallanson/nmqtt/blob/master/nMQTTTests/SubscriptionTopicTests.cs

https://github.com/markallanson/nmqtt/blob/master/nMQTTTests/PublicationTopicTests.cs

Regards,

Mark

[Roger Light]

Hi Mark,

The tests I do are:

do_check("foo/bar", "foo/bar", False)
do_check("foo/+", "foo/bar", False)
do_check("foo/+/baz", "foo/bar/baz", False)
do_check("foo/+/#", "foo/bar/baz", False)
do_check("#", "foo/bar/baz", False)

do_check("foo/bar", "foo", True)
do_check("foo/+", "foo/bar/baz", True)
do_check("foo/+/baz", "foo/bar/bar", True)
do_check("foo/+/#", "fo2/bar/baz", True)

do_check("#", "/foo/bar", False)
do_check("/#", "/foo/bar", False)
do_check("/#", "foo/bar", True)

do_check("foo//bar", "foo//bar", False)
do_check("foo//+", "foo//bar", False)
do_check("foo/+/+/baz", "foo///baz", False)
do_check("foo/bar/+", "foo/bar/", False)

The do_check() parameters are subscription, topic and bad_result (i.e.
if my check returns bad_result it's a failure).

Cheers,

Roger

[ma...@markallanson.net]

Thanks Roger,

You've got a few there that I don't have. I'll add the missing ones to my set.

Mark