Refusing SUBSCRIBE requests or PUBLISH requests

[Keith Cirkel]

I've been developing an MQTT broker/bridge, which has clients it connects to and sends messages to a set of servers, but have some specific use cases around topics - that is to say certain known topics have different behaviours.

My use case, is that all clients can PUBLISH messages, but never SUBSCRIBE to any topics. If a client erroneously sends a SUBSCRIBE, however, I have to send back a SUBACK otherwise it will continually ask to SUBSCRIBE.

Similarly, my server can SUBSCRIBE to the topics the client PUBLISHes to, but cannot PUBLISH to those topics.

I can, of course, just acknowledge receipt of these from the broker but not honour them. However it seems like a bit of a minefield, especially for debug-ability, if my broker ACKS requests but doesn't act on them.

In the spec, it lists CONACK as being able to reply with a number of status codes - 0x00 being accepted, and 0x01 to 0x04 being not accepted. Could SUBACK and PUBACK/PUBREC not have a similar set of codes?

[ನಾಗೇಶ್ ಸುಬ್ರಹ್ಮಣ್ಯ (Nagesh S)]

From an authorisation perspective, the spec says, that a SUBSCRIBE failure due to authorisation will not be reported.

http://public.dhe.ibm.com/software/dw/webservices/ws-mqtt/mqtt-v3r1.html#subscribe

"Note that if a server implementation does not authorize a SUBSCRIBE request to be made by a client, it has no way of informing that client. It must therefore make a positive acknowledgement with a SUBACK, and the client will not be informed that it was not authorized to subscribe."

So, I think, you send a positive SUBACK, but, not report the authorisation failure. Bit counter-intuitive, but, that is the spec.

N

[Dave Locke]

having the ability to Nack a publish or subscribe request is a popular enhancement request for MQTT.   MQTT is going through a standardisation process at OASIS.  The charter for the first standard only allows changes to tidy up and remove ambiguities / anomalies.  As a result for the first version of the standard there is no Nack on pub and sub. The current draft version of the spec states:

If a server implementation does not authorize a PUBLISH to be performed by a client; it has no way of informing that client. It MUST either make a positive acknowledgement, according to the normal QoS rules or disconnect the TCP session.

I suspect adding Nacks will be top of the hit list for the next version


All the best
Dave Locke

Senior Inventor, Pervasive and Advanced Messaging Technologies

lo...@uk.ibm.com Dave Locke/UK/IBM@ibmgb 7-246165 (int) +44 1962816165 (ext) 37274133 (mobex) +44 7764132584 (ext)

Fringe Bluepages with fuel injection My Cattail: Share files in IBM and save your in-box

--
--
To learn more about MQTT please visit http://mqtt.org

To post to this group, send email to mq...@googlegroups.com
To unsubscribe from this group, send email to
mqtt+uns...@googlegroups.com

For more options, visit this group at
http://groups.google.com/group/mqtt

---
You received this message because you are subscribed to the Google Groups "MQ Telemetry Transport" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mqtt+uns...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

[Raphael Cohn]

Keith,

This was discussed recently at length recently as part of the standardisation of MQTT at OASIS. We've added an extra bit one can tweak for this sort of thing, but The problem is doing anything else breaks everyone else in a very non-compatible way.

We'd really like to make this much more straightforward in a future version. Suggestions welcome!

Raphael Cohn
Chief Architect, stormmq

Co-Chair, OASIS MQTT Standard

Secretary, OASIS AMQP Standard
raphae...@stormmq.com

+44 7590 675 756

UK Office:
Hamblethorpe Farm, Crag Lane, Bradley BD20 9DB, North Yorkshire, United Kingdom
Telephone: +44 845 3712 567

Registered office:
16 Anchor Street, Chelmsford, Essex, CM2 0JY, United Kingdom
StormMQ Limited is Registered in England and Wales under Company Number 07175657
StormMQ.com

--

[Keith Cirkel]

I really like the way CONNACK works, where sending a bit back as an identifier of how the message has been dealt with. It's a simple way to give more detailed confirmation.

For SUBACKs I think a similar paradigm could apply, as it already have a nice bit of space in the Payload to use[1]. The payload has the 6 reserved bits plus 2 for QoS, 4 of those bits could be used as an enumeration of return codes which could be something like:

0000 - Accepted 

0001 - Refused: General refusal

0010 - Permission denied (given credentials are insufficient to subscribe to this topic)

0011 - Refused: Topic not in list of servers allowed topics (aka topic 404)

0100 - Refused: Topic has a limit of allowed subscribers, and that limit has been reached

... (0101 - 1111 reserved for future use)

This would certainly cover my use-cases, and would (hopefully) not break existing clients/servers as existing payloads already send 0000 (accepted) as padding, and the return code is purely informational where the client isn't required to act on the return code.

For PUBACK/PUBREC I guess it becomes a different ball game as they have no payloads according to the spec. But confirming that something happened with the PUBLISH is less important, in my opinion.

[1]: http://public.dhe.ibm.com/software/dw/webservices/ws-mqtt/mqtt-v3r1.html#suback

[Raphael Cohn]

A simple coding like this is very much in the spirit of MQTT. Negative Acks (for PUBACK) are also needed, so it'd be good to try to find a way to unify them...

Raphael Cohn
Chief Architect, stormmq

Co-Chair, OASIS MQTT Standard

Secretary, OASIS AMQP Standard
raphae...@stormmq.com

+44 7590 675 756

UK Office:
Hamblethorpe Farm, Crag Lane, Bradley BD20 9DB, North Yorkshire, United Kingdom
Telephone: +44 845 3712 567

Registered office:
16 Anchor Street, Chelmsford, Essex, CM2 0JY, United Kingdom
StormMQ Limited is Registered in England and Wales under Company Number 07175657
StormMQ.com

[ನಾಗೇಶ್ ಸುಬ್ರಹ್ಮಣ್ಯ (Nagesh S)]

>>>>0011 - Refused: Topic not in list of servers allowed topics (aka topic 404)

A subscriber can make a subscription to a topic with a wild-card such as root/level1/level2/+/level4 even when not a single publication has happened yet, on say, root/level1/level2/level3a/level4. But, if/when a publication arrives on root/level1/level2/level3x/level4, then the subscriber must receive that publication. So, I am not sure about the '404' behaviour.

N

[Keith Cirkel]

I think "404" topics fit the usecase where the MQTT server/broker has a limited amount of topics to subscribe to, and can reliably determine that a topic the client is subscribing to isn't on the list of topics the server maintains. If the client subscribes to wildcards, then ambiguity is to be expected, in my opinion.

In your case of having one topic root/level2/level3/level4 is a valid topic, in the list of the servers registered topics, so if a client subscribes to root/level2/+/level4 then the clients subscription is honoured. If the client tries to subscribe to root/notlevel2/+/level4 then the server can determine that each of the levels up until the wildcard do not exist, and the server can return a "404" style ack.

This is obviously not a usecase for all servers, but I believe there is a very valid usecase for servers which can only allow subscriptions to a limited set of topics.

[Keith Cirkel]

It seems like this would be a good opportunity to codify a set of standard response codes, similar to HTTP - where a code like 403 Forbidden, means Forbidden in any context.

If all AKCS could have a single "status code" byte enumeration, but share the same list of status codes (obviously some ACKS will not use some status codes, while others will) then implementors could standardise on the set of unhappy response codes similar to how you would today with HTTP.

[Nicholas Humfrey]

Yes, please. Would rather not have a different set of response codes for CONNACK, SUBACK etc...

(although they might not all be applicable)

Sent from my phone

[Keith Cirkel]

I've had a long think about how the error status codes could be represented, and this is what I came up with...

Payloads

The payload of CONNACK, PUBACK, PUBREL and PUBACK are each 1byte long.

• For CONNACK, bits 4-7 are reserved padding (0000) and the bits 0-3 are the 4 bit enumeration of the response code (to maintain backward compatibility)
  
• For PUBACK, PUBREL, bits 4-7 are the 4 bit enumeration of the response code, and bits 0-3 are reserved.
• For SUBACK, bits 4-7 are a 4 bit enumeration of the response code, bits 2-3 are reserved, and bits 0-1 are used to indicate the granted QoS.
• If a PUBREC sends a non-0 response code, the client SHOULD destroy the message, and MUST NOT send a PUBREL back. It MAY repeat the original Publish message 

Response Codes

Certain codes are only sent via certain commands - for example, SUBACK cannot send a 0x01, 0x02 or 0x09 status. The status codes for CONNACK are all compatible with MQTT 3.1. I have attempted to clarify each of the status codes, so none are ambiguous. 0x04 has been changed from "bad username or password" to "bad payload" which for CONNACK means the same thing, but for other commands has slightly different meaning.

Enumeration HEX Meaning

0   0x0    Accepted

1   0x1    (CONNACK) Refused: unacceptable protocol version

The protocol version specified in the header is incompatible with the server's

protocol.

2   0x2    (CONNACK) Refused: identifier rejected

The idenitifer specified in the connection header does not fit the required

format, or already exists on the server.

3   0x3    (CONNACK, SUBACK, UNSUBACK, PUBACK, PUBREL) Refused: server unavailable

The server has gone down, the client should disconnect any active connections

and retry at a later time. If the server is attempting to rate limit connections,

it should use 0x06 instead. Similar to a HTTP 503 error.

4   0x4    (CONNACK, SUBACK, UNSUBACK, PUBACK, PUBREL) Refused: bad payload

The request was invalid. If coming from a CONNACK the username or password

specified is malformed or missing. This MAY indicate that the server considers

a username and or password mandatory for connection. If the username and

password are both specified, but are not registered on the server, a 0x05 should

be sent instead. If coming from a SUBACK, UNSUBACK, PUBACK, PUBREL, this MAY

indicate the topic or message are missing, or malformed and unreadable. Similar

to an HTTP 400 error.

5   0x5    (CONNACK, SUBACK, UNSUBACK, PUBACK, PUBREL) Refused: not authorized

The username or password specified does not have sufficient priveleges to

perform this action. In the case of PUBACK, PUBREL/suback the topic requires

specialised authorisation to be published/subscribed to. If the username and

password are missing, or in some way malformed and unreadable - a 0x04 should be

sent instead. Similar to an HTTP 403 error.

6   0x6    (CONNACK, SUBACK, PUBACK, PUBREL) Refused: at limit

If coming from a CONNACK, the server is at capacity and is not accepting any

more client connections. If from a SUBACK or PUBACK, PUBREL, the topic is at

capacity (already has too many subscribers, or too many publishers publishing

at once). The client MAY wait and try again at a later time. This MAY also be

used by servers to implement rate limiting - stopping clients from sending too

many requests in a given time period. Similar to an HTTP 429 error.

7   0x7    (SUBACK, UNSUBACK, PUBACK, PUBREL) Refused: Topic not registered

The server has a fixed list of registered topics, and the topic provided is not

in that list. Similar to an HTTP 404 error.

8   0x8    (SUBACK, UNSUBACK, PUBACK, PUBREL) Refused: Bad topic

The server is refusing the topic based on its own validation criteria. This MAY

be due to the length of the topic (while the spec indicates that topics can be

between 1b-64kb, the server MAY have stricter criteria), it MAY mean the number

of levels in the topic is too many for the server to handle, or the wildcard

combination means the topic is too general and will overload the server if

utilised. If the topic does not conform to the spec, a 0x04 should be sent

instead, but if the server has specific topic criteria, then this is used. This

is perhaps similar to an HTTP 413 or 414 error.

9   0x9    (PUBACK, PUBREL) Refused: Bad message

The server is refusing the message based on its own validation criteria. This

MAY be due to the length of the message (while the spec indicates messages up

to 256MB can be sent, the server MAY have stricter criteria). Alternatively it

MAY mean the server cannot read the format the message is in. If the message

does not conform to the specifications then a 0x04 should be sent, but if the

message doesn't conform to the servers own criteria, then this is used. This is

perhaps similar to an HTTP 413 or 415 error.

10 (0xA) - 15 (0xF) Reserved for future use

This proposal would keep the spec as compatible as possible, while giving all ACKS a response code. Existing response codes are maintained. A key summary of changes to the 3.1 spec are:

• 3.1 explicitly specifies that PUBACK, PUBREC and UNSUBACK have no playloads. This would change that, they'd have a 1 byte payload. This could potentially break clients if they expect these commands to not have a payload.
• 3.1 states that CONNACK Payload has 1 byte for the response code, this would change to be 4 reserved bits, and 4 bits for payload - but bits 4-7 are effectively unused in 3.1, so the actual response codes would change - byte for byte.
• 3.1 specifies SUBACK's Payload bits 4-7 as reserved. These would be changed to the response code, which could potentially break clients if they were reading the whole byte as a single integer for the granted QoS
• Existing clients that are 3.1 compliant would behave as though SUBACK, UNSUBACK, PUBACK and PUBREL are all successful responses to their counterpart messages. This would not be true for all cases, and some codes indicate permanent failures about the message, meaning subsequent retries are wasted bandwidth.

[Simon H]

Has anything changed in the 4 years since this thread?

I'm in a situation where I need to know if my publish messages are accepted by the server or not, without the server disconnecting me....  

This works for subscription now in 3.1.1 (it reports qos=128 if the subscription was not allowed, certainly from Mosca), but for publishing there is still no data attached to puback to tell us that the publish is allowed.  

I modified Mosca to allow a configuration to 'ignore' published messages which were unauthorised, so that a network disconnect did not have to occur, but I'd really like to know if a publish was refused.

Is there any discussion of a standards update which includes this kind of proposal?  I am not averse to running on the 'bleeding edge'....

simon

[Simon H]

Ahh...  I see MQTT 5.0 addresses this...

Anyone know when it will be finalised?

s

[Nick O'Leary]

MQTT 5 is currently in its second public review period: https://www.oasis-open.org/news/announcements/invitation-to-comment-on-mqtt-v5-0-from-the-mqtt-tc-ends-dec-2nd

--
To learn more about MQTT please visit http://mqtt.org

---
You received this message because you are subscribed to the Google Groups "MQTT" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mqtt+unsubscribe@googlegroups.com.
To post to this group, send email to mq...@googlegroups.com.
Visit this group at https://groups.google.com/group/mqtt.
For more options, visit https://groups.google.com/d/optout.