Clarification on topics - always dynamic, right?

[andyp...@gmail.com]

I believe the original intent of MQTT was that topics on a broker are
always "dynamic" and require no administrative setup.
i.e. I subscribe on topic x without knowing whether it exists; it
doesn't, so broker silently creates that topic; and I receive
publications on that topic should they be sent (which, of course, in a
pub/sub world, they may never be).

I believe this is the way that RSMB (and mosquitto?) operate, although
I've just read through the spec again... Friday night, why wouldn't I
browse a messaging spec over a pint? :-) and I can't see that
EXPLICITLY stated. We could assume that RSMB as the current
"reference" implementation of a broker is behaving to the spirit of
the unwritten spec though.

So:
- Nick/Andy/Arlen: any comment?
- do we need to clarify this for spec v.Next? if so we should lob it
on the wiki list...

(as an aside, I hope everyone is going to contribute to the lovely
shiny upgraded and reorganised wiki! http://mqtt.org/wiki)

--
Andy Piper | Farnborough, Hampshire (UK)
blog: http://andypiper.co.uk   |   skype: andypiperuk
twitter: @andypiper  |  images: http://www.flickr.com/photos/andypiper

[DanAnderson]

Hi,

Everything but $SYS is completely dynamic (as far as I know).

Related, $SYS topics should be formally defined and documented
(outside of the mosquitto man page).

It would also be nice, IMO, to add a $SYS topic tree that lists the
topics in use (and maybe the number of pubs/subs per topic (like IRC /
list).

Dan

On Jan 27, 2:35 pm, "andypipe...@gmail.com" <andypipe...@gmail.com>
wrote:

> I believe the original intent of MQTT was that topics on a broker are
> always "dynamic" and require no administrative setup.
> i.e. I subscribe on topic x without knowing whether it exists; it
> doesn't, so broker silently creates that topic; and I receive
> publications on that topic should they be sent (which, of course, in a
> pub/sub world, they may never be).
>
> I believe this is the way that RSMB (and mosquitto?) operate, although
> I've just read through the spec again... Friday night, why wouldn't I
> browse a messaging spec over a pint? :-) and I can't see that
> EXPLICITLY stated. We could assume that RSMB as the current
> "reference" implementation of a broker is behaving to the spirit of
> the unwritten spec though.
>
> So:
> - Nick/Andy/Arlen: any comment?
> - do we need to clarify this for spec v.Next? if so we should lob it
> on the wiki list...
>
> (as an aside, I hope everyone is going to contribute to the lovely

> shiny upgraded and reorganised wiki!http://mqtt.org/wiki)

[arlen....@eurotech.com]

Indeed, the original intent of MQTT was that topics were always
dynamic and never "administered". This is the dynamica nature of a
MQTT based brokering architecture. Devices might come online
publishing "stuff" that no one knew about yet. But either by wildcard
or by explicite subscription, these new topics could start to be used
by consumer applications. I think adding any level of topic
administration would hamper if not kill this very dynamic and "zero-
config" approach. You are right Andy in that this was next explicit
although we never assumed someone would read it while drinking a
pint :-). We'll definitely put this on the "To Be Defined" list of
future specs.

Cheers, Arlen

On Jan 27, 2:35 pm, "andypipe...@gmail.com" <andypipe...@gmail.com>
wrote:

> I believe the original intent of MQTT was that topics on a broker are
> always "dynamic" and require no administrative setup.
> i.e. I subscribe on topic x without knowing whether it exists; it
> doesn't, so broker silently creates that topic; and I receive
> publications on that topic should they be sent (which, of course, in a
> pub/sub world, they may never be).
>
> I believe this is the way that RSMB (and mosquitto?) operate, although
> I've just read through the spec again... Friday night, why wouldn't I
> browse a messaging spec over a pint? :-) and I can't see that
> EXPLICITLY stated. We could assume that RSMB as the current
> "reference" implementation of a broker is behaving to the spirit of
> the unwritten spec though.
>
> So:
> - Nick/Andy/Arlen: any comment?
> - do we need to clarify this for spec v.Next? if so we should lob it
> on the wiki list...
>
> (as an aside, I hope everyone is going to contribute to the lovely

> shiny upgraded and reorganised wiki!http://mqtt.org/wiki)

[Roger Light]

On Fri, Jan 27, 2012 at 9:39 PM, DanAnderson <dan-an...@cox.net> wrote:

> Related, $SYS topics should be formally defined and documented
> (outside of the mosquitto man page).

Yes, a note to say that $SYS is regarded as a separate topic space for
any information the broker wants to make available would be a good
idea. The topics that RSMB supports are listed in its help file.
Whether the $SYS hierarchy should be strictly formalised is something
I'm less sure of. I view the $SYS tree as something that is for the
most part of human rather than machine interest. The notable exception
is of course bridge connection topics.

Cheers,

Roger

[Roger Light]

On Fri, Jan 27, 2012 at 8:35 PM, andyp...@gmail.com
<andyp...@gmail.com> wrote:
> I believe the original intent of MQTT was that topics on a broker are
> always "dynamic" and require no administrative setup.
> i.e. I subscribe on topic x without knowing whether it exists; it
> doesn't, so broker silently creates that topic; and I receive
> publications on that topic should they be sent (which, of course, in a
> pub/sub world, they may never be).
>
> I believe this is the way that RSMB (and mosquitto?) operate,

Yes, with the slight caveat that where access controls are implemented
there is administrative setup. I don't think that's quite what you
mean of course - you define the ACL but don't have to do anything more
than that. It could be taken to the extreme and deny access by default
in which case you'd have admin work to configure new topics. I could
see how this might be of interest in some cases though.

On Fri, Jan 27, 2012 at 9:39 PM, DanAnderson <dan-an...@cox.net> wrote:
>

> Everything but $SYS is completely dynamic (as far as I know).

A point that occurs to me - $SYS is dynamic as well. It's effectively
populated by a client, it's just that that client is privileged and
resides inside the broker. Splitting hairs a little bit I know, but
then there's nothing particularly special about $SYS really. You could
write "Microsoft MQTT" to $SYS/broker/version as an ordinary client if
you wanted to for example, assuming no ACL prevents you doing so.

Cheers,

Roger

[Andy Piper]

On Friday, 27 January 2012 23:27:05 UTC, Roger Light wrote:

Yes, with the slight caveat that where access controls are implemented

there is administrative setup.

Peace. That's fair.

A point that occurs to me - $SYS is dynamic as well. It's effectively
populated by a client, it's just that that client is privileged and
resides inside the broker. Splitting hairs a little bit I know, but
then there's nothing particularly special about $SYS really. You could
write "Microsoft MQTT" to $SYS/broker/version as an ordinary client if
you wanted to for example, assuming no ACL prevents you doing so.

... which I think makes it a useful step to clarify for the future i.e. that $SYS is "privileged" in that sense - subscribe-only to regular clients outside the broker.

Anyway - thanks for that discussion - now posted on the wiki, with a link back here:
http://mqtt.org/wiki/doku.php?id=are_topics_dynamic
http://mqtt.org/wiki/doku.php?id=conventions

I've significantly restructured the wiki btw, and with the latest version of the dokuwiki software we have a lot more flexibility in terms of formatting and the content that we include, including e.g. downloadable code snippets and stuff. So I hope we can all dive in to it!

Andy

 

[DanAnderson]

Hi,

Yeah, the same thing occurred to me too. :)

Apparently those aren't read only for normal clients by default.
Maybe they should be.

I don't know enough about what is in $SYS to say - but maybe some
mischief could be done if clients can write to them?

What do the bridge connection topics do? I couldn't find anything
beyond the man page on mosquitto.conf. (notification)

Dan

[Roger Light]

Hi,

> Apparently those aren't read only for normal clients by default.
> Maybe they should be.
>
> I don't know enough about what is in $SYS to say - but maybe some
> mischief could be done if clients can write to them?

It's a possibility. If someone thinks it might be a problem in their
situation they can always set an ACL to restrict access. In most cases
the messages will be shortly updated anyway.

> What do the bridge connection topics do?  I couldn't find anything
> beyond the man page on mosquitto.conf. (notification)

They are a topic created by a bridge connection, both on themself and
the remote broker to indicate whether that connection is currently
active. You can actually see some on test.mosquitto.org now. This is
one case where write access to $SYS is required. Does that make more
sense?

Cheers,

Roger

[DanAnderson]

HI,

So, that tree is just about the notifications then?

I'm not sure I understand bridging well enough to understand why that
is needed.

Isn't a bridge logically just a client?

What is the effect of having this set to 1 or 0?

Thanks
Dan

[Paul Brant]

Hey Andy and all

First off Hi to everyone. We are very pleased to have added MQTT support to Nirvana 7 and are looking forward to actively contributing to the community.

The statement in the spec "These are the QoS levels at MQTT V3.1 Protocol Specification 12 of 42 which the administrators for the server have permitted the client to subscribe to a particular Topic Name." suggests an out of band administrative function for topics and the subsequent setting of permissions/ACL's of them.

The Nirvana API supports the notion of dynamic topic (and queue) creation. If we were to implement this for MQTT what should the default level of access control be for said resources ?

Regards 

Paul

[Nicholas O'Leary]

I've been mulling over this today. I whole heartedly agree that the
spirit of MQTT means you don't want to have to set up topics before
you start using them.

But I don't think it should be a hard requirement of the spec for a
broker to behave like this. There are a couple reasons for this:

1. Security. Roger touched on this already, but I think it is worth
repeating. If you're running on a public network, you will likely want
some level of control over who connects and what they can do. This
requires administration at the topic level. It may be splitting-hairs,
but I think the language in the spec would get clumsy if it were to
say "a broker must not require pre-administration of topics unless
security is being used in which case you pretty much have to
pre-administer them"

2. Integration with existing systems. We are already seeing MQTT being
added as a supported protocol to existing systems,
http://blog.my-channels.com/2012/01/27/mqtt-support-in-nirvana-7/ for
example (via AndyP). If these existing systems already use a
pre-administered topic model (as it appears, on a very brief look,
Nirvana 7 does), then I'd rather they were able to add MQTT as a
transport protocol.

I would prefer to see this added to the spec as a recommended
behaviour rather than hard requirement. (Same with the clear-text
password discussion in the other thread - a recommendation, not a
requirement)

Nick

> --
> To learn more about MQTT please visit http://mqtt.org
>
> To post to this group, send email to mq...@googlegroups.com
> To unsubscribe from this group, send email to
> mqtt+uns...@googlegroups.com
>
> For more options, visit this group at
> http://groups.google.com/group/mqtt

[DanAnderson]

Hi,

I think this is a very reasonable response and I agree with both
points.

If someone doesn't want to provide dynamic topics (and that was a very
good example) they should have that option.

And for the password one - I'd just like to see "a "security
consideration" added that suggests to broker implementers that
passwords should not be stored on the brokers in clear text." -
because a lot of the time things like this (clear text passwords) are
the result of incomplete risk analysis and a tiny reminder helps
straighten them out before they become a problem. Not that client
side storage is a _protocol_ issue. The normal advisory/CVE process
will ultimately catch any implementations that are doing this.

Dan

[andyp...@gmail.com]

See (and amend as appropriate please)

http://mqtt.org/wiki/doku.php?id=are_topics_dynamic
http://mqtt.org/wiki/doku.php?id=security_considerations

[Nicholas O'Leary]

Hi Paul,

welcome to the list and community.

> The Nirvana API supports the notion of dynamic topic (and queue) creation.
> If we were to implement this for MQTT what should the default level of
> access control be for said resources ?

I'm going to take the easy option and say that is up to you and what
is right/expected for your users.

MQTT doesn't say anything about permissions today - it leaves those
sorts of policy decisions to the specific implementations.

Whether the spec should say more about these issues is open for discussion.

Nick

[Paul Brant]

Hey Nick,

"MQTT doesn't say anything about permissions today - it leaves those sorts of policy decisions to the specific implementations."

This approach is fine when resources are administered within the broker. In the dynamic topic scenario however without the notion of standards based simple security constructs (like SUBJECTS and ACLs that can be applied on behalf of a SUBJECTS), vendor implementations will I suspect quickly become incompatible. 

"Whether the spec should say more about these issues is open for discussion"

If dynamic topic creation is a must then I feel that the spec should offer up some very clear definitions of how the business of securing topics and their content is achieved.

Regards

Paul