Re: [mqtt] QoS 1 deliveries, wait for puback from clients?

[Nicholas O'Leary]

Hi Carl,

No, the server is not required to wait for the pubacks from the subscribers - for all the reasons you identified.

The wording in the spec needs to be clarified here as you're not the first to ask this.

When the server receives a qos1 message, it must:

 1. persist it

 2. identify the subscribers and begin delivery

 3. return puback

At the point delivery is completed to all of the subscribes the message can then be deleted.

What 'begin delivery' means will depend on the implementation. That may mean the PUBLISH packet is written to the TCP socket of each subscriber or it might simply mean the message is queued up for delivery internally.

Regards,

Nick

On 20 May 2013 09:14, Carl Hörberg <carl.h...@gmail.com> wrote:

The spec explains (http://public.dhe.ibm.com/software/dw/webservices/ws-mqtt/mqtt-v3r1.html#qos-flows) that when a message is published to the server with qos 1 the server should persist it, deliver it to all subscribers, delete the message and then puback.

Does this require the server to wait for puback from all subscribers which subscribed with qos1 before sending puback to the original publisher? What about disconnected qos1 subscribers, are they exempted?

My concern is that the pub/puback takes a whole lot longer in this case, without any extra reliability benefits. Couldn't the server send puback as soon as the message is persisted?

--
--
To learn more about MQTT please visit http://mqtt.org
 
To post to this group, send email to mq...@googlegroups.com
To unsubscribe from this group, send email to
mqtt+uns...@googlegroups.com
 
For more options, visit this group at
http://groups.google.com/group/mqtt
 
---
You received this message because you are subscribed to the Google Groups "MQ Telemetry Transport" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mqtt+uns...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Matteo Collina]

Hi Nicholas,

As you said, it is a very weak point in the spec. Eventually, it will be updated.

In the meanwhile, could someone put this on the MQTT website as a FAQ or something?

Even a qos2 explanation will help.

Cheers,

Matteo

[Rahul Gupta]

Hello All - I am working as editor in MQTT OASIS TC, and I have opened a issue MQTT-14 to tighten the specification in this area. Thanks

https://tools.oasis-open.org/issues/browse/MQTT-14

Kind Regards,
Rahul Gupta

[Carl Hörberg]

Ok, thank you for the explanation!

I have further questions on QoS 2 as I fail to see the benefits of it. It seems to me that a correctly implemented server shouldn't be able send duplicates even in QoS1.

First, my view of a QoS1 flow:
1: client -> publish(qos1, msgid) -> server
2: servers stores message, puts it in internal queues for each subscriber
3: server -> puback(msgid) -> client
4: the client can now be sure that the message is received and that it will be deliviered to the subscribers at some point.
5: server starts to deliver message to subscribers
6: the server marks the msg as delivered (at least keeps the msgid) for each subscriber.

so if the client doesn't receive the puback, it resends it, the server then looks in the store for the msgid, it either discards it if it's already queued for the subscribers (or even already delivered, because we keep the msgid for a while even after sending out to the subscriber), or it processes it as normally if it haven't stored it before.

Also, what good does the DUP flag do? shouldn't the receiver just have to look at the msg id and compare it to its log, it either as it or doesn't have, should it cares if the sender had to resend it?

Now, QoS2, how does it improve the situation?

1: client -> publish(qos2, msgid) -> server
2: servers stores message, puts it in internal queues for each subscriber
3: server -> pubrec(msgid) -> client
4: client marks the msg as delivered, won't resend it msg again.
5: client -> pubrel(msgid) -> server
6: server starts to deliver message to subscribers
7: server -> pubcomp(msgid) -> client
8: marks the msg as delivered (at least keeps the msgid) for each subscriber when the qos work flow is done.

How does this improve anything? The only thing it does is to lock the messages in the internal queues of the server for a while.

I also fail to see the benefits of limiting subscriptions QoS to the QoS level of the published message. A publisher can send a message in QoS 0, but why would the broker then ignore the wish of a subscriber which want to confirm its retrieval of a message? I see it as two independent actions..

> > To post to this group, send email to mq...@googlegroups.com (mailto:mq...@googlegroups.com)

> > To unsubscribe from this group, send email to

> > mqtt+uns...@googlegroups.com (mailto:mqtt%2Bunsu...@googlegroups.com)

> >
> > For more options, visit this group at
> > http://groups.google.com/group/mqtt
> >
> > ---
> > You received this message because you are subscribed to the Google Groups "MQ Telemetry Transport" group.

> > To unsubscribe from this group and stop receiving emails from it, send an email to mqtt+uns...@googlegroups.com (mailto:mqtt%2Bunsu...@googlegroups.com).

> > For more options, visit https://groups.google.com/groups/opt_out.
> >
> >
>
>
> --
> --
> To learn more about MQTT please visit http://mqtt.org
>

> To post to this group, send email to mq...@googlegroups.com (mailto:mq...@googlegroups.com)

> To unsubscribe from this group, send email to

> mqtt+uns...@googlegroups.com (mailto:mqtt+uns...@googlegroups.com)

>
> For more options, visit this group at
> http://groups.google.com/group/mqtt
>
> ---
> You received this message because you are subscribed to the Google Groups "MQ Telemetry Transport" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to mqtt+uns...@googlegroups.com (mailto:mqtt+uns...@googlegroups.com).

[Nicholas O'Leary]

Hi Carl,

>> (or even already delivered, because we keep the msgid for a while even after sending out to the subscriber)

A server should not store Message IDs of messages it has completed handling. Once a flow completes, the message ID could legitimately be reused in a totally new flow.

>> Also, what good does the DUP flag do? shouldn't the receiver just have to look at the msg id and compare it to its log, it either as it or doesn't have, should it cares if the sender had to resend it?

Likewise, a client should not keep a log of message IDs. Keeping such a log on the client would be completely impractical on tiny devices with limited resources. The DUP flag gives the client a hint that it may have seen the publish before - in practice it isn't that useful and de-duplication is best done at the application level.

>> How does this improve anything? The only thing it does is to lock the messages in the internal queues of the server for a while.

If you assume the network connection between client and server is stable and never drops unexpectedly, then QoS 1 is sufficient to ensure once-only delivery. However that cannot be said if the network connection drops mid-flow.

In the case of QoS 1, if the connection drops after the server has sent the PUBACK but before the client receives it, the client will resend the publish when it reconnects. The Server will not have a record of the publish, so will treat it as a new flow and send out a duplicate.

The QoS 2 flow is designed to ensure that if the network drops at any point before the flow completes, the client and server can resume the flow without causing a duplicate message to be sent. All without requiring either the client or server to maintain a log of message IDs that have 'completed'.

Regards,

Nick

[Carl Hörberg]

ok, makes sense if the server isn't allowed to keep a log, which yes, is impractical with only 2^16 possible id's per client, especially in combination with a high timeout.

but what benefits does QoS 2 give on subscription?

1: server -> publish(qos2, msgid) -> client
2: client stores message
3: client -> pubrec(msgid) -> server
4: server deletes the msg
5: server -> pubrel(msgid) -> client
6: client releases the message to the application
7: client -> pubcomp(msgid) -> server
8: client deletes the message from its store
9: the application process the message


or when should pubcomp be sent from the client? when the application is done handling the message? or should the client wait to delete the message until the app has processed it?

still not sure about the reason for the QoS level downgrading thing...

[Karl Palsson]

On Thu, May 23, 2013 at 06:37:39PM +0800, Carl H�rberg wrote:
> I also fail to see the benefits of limiting subscriptions QoS to the QoS level of the published message. A publisher can send a message in QoS 0, but why would the broker then ignore the wish of a subscriber which want to confirm its retrieval of a message? I see it as two independent actions..
>

I had similar thoughts here:

https://groups.google.com/forum/?fromgroups#!searchin/mqtt/karl/mqtt/tJMUoVE_9rk/xph_x7PT8IYJ

This lead me to the real answer which is to not have mixed qos environments. I had used
qos0 on links to localhost, where if that failed, we had bigger problems. I then
_wanted_ to use qos1 to "magically" fix network interruptions. The simple way was to
just use qos1 everywhere. The data _is_ important, so using a lower qos just because I
knew it would be reliable wasn't really the right thing to do.

I'm still not entirely conviced that this is always the answer though, particularly with
multiple publishers and subscribers to a shared topic, each with their own admins, and
their own ideas of the quality, I _do_ see the point in a subscriber saying, "if it made
it to this broker who I trust, _I_ consider it important"

Cheers,
Karl P

[Nicholas O'Leary]

>>but what benefits does QoS 2 give on subscription?

I'm not sure I understand what you are getting at. The QoS 2 flow logic works the same whether a client is publishing to a server, or a server publishing to a client. The two-stage commit ensures once only delivery.

The client sending pubcomp means the original message has been dealt with. What that means will depend on the specific implementation. It basically means the client is taking full responsibility for passing the message to the application, if it hasn't already done so - and that the broker doesn't need to concern itself with the message anymore.

N 

To unsubscribe from this group and stop receiving emails from it, send an email to mqtt+uns...@googlegroups.com.

[Dave Locke]

jto add one snippet,  the message delivery flows work the same way going from client to server and server to client for all qualities of service.

>>still not sure about the reason for the QoS level downgrading thing...

in most cases a subscription is made using the max QoS of 2 which means the client will receive any messages (matching the subscription) at the QoS the message was originally published at,. One case where the ability to downgrade comes in handy is if the MQTT client library only supports a lower QoS e.g. there are some clients that only support QoS 0.

All the best
Dave

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

[Carl Hörberg]

Ok, got it with qos 2, but the diagrams in the spec could definitely need some clarification, and show the server->client work flow as well. Awesome Rahul Gupta that you're on it :)

What is the motivation of the "granted" list with qos numbers in suback if the server can give the client any qos level anyway? The client must be prepared for any qos level regardless of the suback response. Unless, which I think makes more sense, no downgrading is done. Subscribers can decide for them self if they want to receive messages in qos0/1/2 or not. As it is now, publishers are a concern for subscribers, coupled together.

Example; a publisher does only implement qos0 (as most clients today unfortunately), but a subscriber uses poho and wants to be sure that they get the messages, but with todays spec that's impossible, the subscriber is crippled because the publisher's client library doesn't support qos>0.

> > To learn more about MQTT please visit http://mqtt.org (http://mqtt.org/)
> >
> > To post to this group, send email to mq...@googlegroups.com (mailto:mq...@googlegroups.com) (mailto:mq...@googlegroups.com)

> > To unsubscribe from this group, send email to

> > mqtt+uns...@googlegroups.com (mailto:mqtt%2Bunsu...@googlegroups.com) (mailto:mqtt+uns...@googlegroups.com (mailto:mqtt%2Bunsu...@googlegroups.com))

> >
> > For more options, visit this group at
> > http://groups.google.com/group/mqtt
> >
> > ---
> > You received this message because you are subscribed to the Google Groups "MQ Telemetry Transport" group.

> > To unsubscribe from this group and stop receiving emails from it, send an email to mqtt+uns...@googlegroups.com (mailto:mqtt%2Bunsu...@googlegroups.com) (mailto:mqtt+uns...@googlegroups.com (mailto:mqtt%2Bunsu...@googlegroups.com)).

> > For more options, visit https://groups.google.com/groups/opt_out.
>
>
>
>
> --
> --

> To learn more about MQTT please visit http://mqtt.org (http://mqtt.org/)

>
> To post to this group, send email to mq...@googlegroups.com (mailto:mq...@googlegroups.com)
> To unsubscribe from this group, send email to

> mqtt+uns...@googlegroups.com (mailto:mqtt%2Bunsu...@googlegroups.com)

>
> For more options, visit this group at
> http://groups.google.com/group/mqtt
>
> ---
> You received this message because you are subscribed to the Google Groups "MQ Telemetry Transport" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to mqtt+uns...@googlegroups.com (mailto:mqtt%2Bunsu...@googlegroups.com).

> For more options, visit https://groups.google.com/groups/opt_out.
>
>
> --
> --

> To learn more about MQTT please visit http://mqtt.org (http://mqtt.org/)

>
> To post to this group, send email to mq...@googlegroups.com (mailto:mq...@googlegroups.com)
> To unsubscribe from this group, send email to
> mqtt+uns...@googlegroups.com (mailto:mqtt+uns...@googlegroups.com)
>
> For more options, visit this group at
> http://groups.google.com/group/mqtt
>
> ---
> You received this message because you are subscribed to the Google Groups "MQ Telemetry Transport" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to mqtt+uns...@googlegroups.com (mailto:mqtt+uns...@googlegroups.com).
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

[Roger Light]

Hi Carl,

> What is the motivation of the "granted" list with qos numbers in suback
> if the server can give the client any qos level anyway? The client must
> be prepared for any qos level regardless of the suback response.

The intent here is that although you may ask for QoS 2 when you
subscribe, the broker may not let you have it. Asking for QoS 0 should
not result in a subscription of greater than QoS 1 though. The client
should therefore never receive a PUBLISH message with QoS greater than
it asked for. That's the point of downgrading QoS for messages going
to subscribers. If the message was published to the broker with QoS 2
but you subscribed at QoS 1, you will receive that message with QoS 1.

> Subscribers can decide for them self if they want to receive messages
> in qos0/1/2 or not. As it is now, publishers are a concern for subscribers,
> coupled together.
>
> Example; a publisher does only implement qos0 (as most clients today
> unfortunately), but a subscriber uses poho and wants to be sure that they
> get the messages, but with todays spec that's impossible, the subscriber
> is crippled because the publisher's client library doesn't support qos>0.

But that isn't about downgrading QoS in my opinion, it is upgrading
the QoS when it goes to the subscriber. That's not to say it couldn't
work like that in the future, but it's just a different issue to
downgrading QoS for subscribers.

Cheers,

Roger

[Carl Hörberg]

The broker has the message, and can technically deliver it in any QoS level, regardless in which QoS level it got it in, right? So why this arbitrary limitation?

[Andy Stanford-Clark]

Hi Carl,
this is a great discussion - it's really bringing out the gap between what we think the spec says, and what it actually says - exactly what we need for the standardisation work :)

The "granted QoS" list in the suback response from the broker is there to allow a broker to "disappoint" a client by potentially granting a LOWER QoS subscription than the client requested.
When I put this feature in, I was thinking that a broker may, for administrative reasons, wish or need to downgrade the quality of service it offered to subscribers. Say the memory/disk was getting a bit tight, and the broker decided that any more queues of qos2 messages were going to toast it... so it downgrades all subsequent subscriptions to qos0 - setting expectations for delivery to the client.
Another possible scenario is that the broker might not (for some reason) have the capability to properly queue and deliver qos1 and 2 messages, so it only grants qos0 to its subscribers, so they have a hint about what to expect.

Your question about why the client can't UPGRADE the subscription is related to the end-to-end processing of a message. If we upgrade subscriptions to QoS2, then we set an unreasonable expectation with the client that all messages published on the topics it is subscribed-to will be delivered to that client exactly once.
However, if the publications are only at qos0 they might never get to the broker, and if at qos1 may be delivered twice. Neither of which is consistent with QoS2 assumptions at the subscribing client.

BTW, the example I always use for why the client can ask for a downgraded QoS in their subscription, is for a relatively dumb display device (e.g. one of those scrolling LED pixel displays) that is being used to give users a glimpse of what's happening in the "business transactions" flying around in the broker (which would likely be at QoS 2 - once and once only).
If on friday night the display gets turned off, and it turned back on monday morning, we wouldn't want the broker to queue up all the transactions over the weekend, and then deluge the display with them on monday morning as it faithfully delivered all the QoS 2 messages to the display.
The ability of the display to select qos 0 for its subscription means it only gets messages when it's switched on and connected, and thus doesn't chew up broker-side resource queuing messages for pointless delivery.

AndySC

.
>  
>  



-- 

[Karl P]

On 06/04/2013 05:23 PM, Andy Stanford-Clark wrote:
> Your question about why the client can't UPGRADE the subscription is related to
> the end-to-end processing of a message. If we upgrade subscriptions to QoS2,
> then we set an unreasonable expectation with the client that all messages
> published on the topics it is subscribed-to will be delivered to that client
> exactly once.
> However, if the publications are only at qos0 they might never get to the
> broker, and if at qos1 may be delivered twice. Neither of which is consistent
> with QoS2 assumptions at the subscribing client.

See, I've never read it that way. I've accepted that that's the way other
people did, and I've worked around it. But I've always considered my qos2
request to be a contract solely between one client, and one broker. _if_ the
broker gets a message that matches my subscription, I want qos2 policies applied
to how it's delivered to me. I see now reason why I should ever have to care
one iota about what the publisher thought. Messages arriving twice on qos1 are
simply two separate messages as far as the qos2 subscriber is concerned.

[Carl Hörberg]

inline

On Tuesday 4 June 2013 at 19:23, Andy Stanford-Clark wrote:
> Hi Carl,
> this is a great discussion - it's really bringing out the gap between what we think the spec says, and what it actually says - exactly what we need for the standardisation work :)
>
> The "granted QoS" list in the suback response from the broker is there to allow a broker to "disappoint" a client by potentially granting a LOWER QoS subscription than the client requested.
> When I put this feature in, I was thinking that a broker may, for administrative reasons, wish or need to downgrade the quality of service it offered to subscribers. Say the memory/disk was getting a bit tight, and the broker decided that any more queues of qos2 messages were going to toast it... so it downgrades all subsequent subscriptions to qos0 - setting expectations for delivery to the client.
> Another possible scenario is that the broker might not (for some reason) have the capability to properly queue and deliver qos1 and 2 messages, so it only grants qos0 to its subscribers, so they have a hint about what to expect.
>

eh, ok, so if memory/disk gets tight for the server the client should suddenly expect duplicate messages etc? and at any rate, why is it announced at subscription time? every message the server publishes to a client can have a arbitrary qos level. what good does the information do at subscription time? what can the client do that it can't later, when a message actually arrives?

it sounds to me that the granted list it is about as useful as the DUP flag, a "hint", which no automated application can take any action to.

>
> Your question about why the client can't UPGRADE the subscription is related to the end-to-end processing of a message. If we upgrade subscriptions to QoS2, then we set an unreasonable expectation with the client that all messages published on the topics it is subscribed-to will be delivered to that client exactly once.
> However, if the publications are only at qos0 they might never get to the broker, and if at qos1 may be delivered twice. Neither of which is consistent with QoS2 assumptions at the subscribing client.
>

ok, so lets say a client sends in qos0, maybe its client library only supports that. but a client want to subscribe with qos level 1, it wants to guarantee that it gets all the messages that at least the server has seen. Now why would we forcefully only send it in qos 0? When the client wants and supports qos1? The only thing it does is risking that the subscribing client never gets it, we're not disappoint anyone by delivering the message and retry until we get a puback back, do we?

yes, only when both the client publisher and the client subscriber supports QoS2 can we guarantee that a message is only delivered once, but the application developer knows this beforehand (just as they are aware of payload format etc). But even if publisher does not send in qos 2, I argue that the server should still deliver in the requested QoS level.

>
> BTW, the example I always use for why the client can ask for a downgraded QoS in their subscription, is for a relatively dumb display device (e.g. one of those scrolling LED pixel displays) that is being used to give users a glimpse of what's happening in the "business transactions" flying around in the broker (which would likely be at QoS 2 - once and once only).
> If on friday night the display gets turned off, and it turned back on monday morning, we wouldn't want the broker to queue up all the transactions over the weekend, and then deluge the display with them on monday morning as it faithfully delivered all the QoS 2 messages to the display.
> The ability of the display to select qos 0 for its subscription means it only gets messages when it's switched on and connected, and thus doesn't chew up broker-side resource queuing messages for pointless delivery.
>
>

i never argued against client requesting a low qos level. i argue that messages always should be delivered in the qos level they request.

everything else in the protocol mandates that the subscribing client is aware of the publishing client (content-type, content-encoding etc) so it is probably already aware of the qos level the client is publishing in too, and should set its expectations from that, not on a message to message basis.

in your example i agree that qos0 is a good delivery level for the display. but it could just as well have used qos 1 and a clean-session. that way to messages would've been stored either during the weekend.

but you bring up another question, if a client has a qos0 subscription, should the server at no point try to redeliver messages it missed when the client was away? yes, qos0, don't store to disk, but not store in memory either?

> AndySC
>
>
> On Fri, 2013-05-24 at 16:30 +0800, Carl Hörberg wrote:

> > Ok, got it with qos 2, but the diagrams in the spec could definitely need some clarification, and show the server->client work flow as well. Awesome Rahul Gupta that you're on it :) What is the motivation of the "granted" list with qos numbers in suback if the server can give the client any qos level anyway? The client must be prepared for any qos level regardless of the suback response. Unless, which I think makes more sense, no downgrading is done. Subscribers can decide for them self if they want to receive messages in qos0/1/2 or not. As it is now, publishers are a concern for subscribers, coupled together. Example; a publisher does only implement qos0 (as most clients today unfortunately), but a subscriber uses poho and wants to be sure that they get the messages, but with todays spec that's impossible, the subscriber is crippled because the publisher's client library doesn't support qos>0. On Friday 24 May 2013 at 16:13, Dave Locke wrote: > jto add one snippet, the message delivery flows work the same way going from client to server and server to client for all qualities of service. > > > > still not sure about the reason for the QoS level downgrading thing... > in most cases a subscription is made using the max QoS of 2 which means the client will receive any messages (matching the subscription) at the QoS the message was originally published at,. One case where the ability to downgrade comes in handy is if the MQTT client library only supports a lower QoS e.g. there are some clients that only support QoS 0. > > All the best > Dave > > > > > From: "Nicholas O'Leary" <nick....@gmail.com (mailto:nick....@gmail.com) (mailto:nick....@gmail.com)> > To: "mq...@googlegroups.com (mailto:mq...@googlegroups.com) (mailto:mq...@googlegroups.com)" <mq...@googlegroups.com (mailto:mq...@googlegroups.com) (mailto:mq...@googlegroups.com)>, > Date: 23/05/2013 22:17 > Subject: Re: [mqtt] QoS 1 deliveries, wait for puback from clients? > Sent by: mq...@googlegroups.com (mailto:mq...@googlegroups.com) (mailto:mq...@googlegroups.com) > > > > > > but what benefits does QoS 2 give on subscription? > > I'm not sure I understand what you are getting at. The QoS 2 flow logic works the same whether a client is publishing to a server, or a server publishing to a client. The two-stage commit ensures once only delivery. > > The client sending pubcomp means the original message has been dealt with. What that means will depend on the specific implementation. It basically means the client is taking full responsibility for passing the message to the application, if it hasn't already done so - and that the broker doesn't need to concern itself with the message anymore. > > > N > > > On 23 May 2013 13:25, Carl Hörberg <carl.h...@gmail.com (mailto:carl.h...@gmail.com) (mailto:carl.h...@gmail.com)> wrote: > ok, makes sense if the server isn't allowed to keep a log, which yes, is impractical with only 2^16 possible id's per client, especially in combination with a high timeout. > > but what benefits does QoS 2 give on subscription? > > 1: server -> publish(qos2, msgid) -> client > 2: client stores message > 3: client -> pubrec(msgid) -> server > 4: server deletes the msg > 5: server -> pubrel(msgid) -> client > 6: client releases the message to the application > 7: client -> pubcomp(msgid) -> server > 8: client deletes the message from its store > 9: the application process the message > > > or when should pubcomp be sent from the client? when the application is done handling the message? or should the client wait to delete the message until the app has processed it? > > still not sure about the reason for the QoS level downgrading thing... > > > On Thursday 23 May 2013 at 20:01, Nicholas O'Leary wrote: > > > Hi Carl, > > > > > > > > (or even already delivered, because we keep the msgid for a while even after sending out to the subscriber) > > > > A server should not store Message IDs of messages it has completed handling. Once a flow completes, the message ID could legitimately be reused in a totally new flow. > > > > > > Also, what good does the DUP flag do? shouldn't the receiver just have to look at the msg id and compare it to its log, it either as it or doesn't have, should it cares if the sender had to resend it? > > > > Likewise, a client should not keep a log of message IDs. Keeping such a log on the client would be completely impractical on tiny devices with limited resources. The DUP flag gives the client a hint that it may have seen the publish before - in practice it isn't that useful and de-duplication is best done at the application level. > > > > > > > > How does this improve anything? The only thing it does is to lock the messages in the internal queues of the server for a while. > > > > If you assume the network connection between client and server is stable and never drops unexpectedly, then QoS 1 is sufficient to ensure once-only delivery. However that cannot be said if the network connection drops mid-flow. > > > > In the case of QoS 1, if the connection drops after the server has sent the PUBACK but before the client receives it, the client will resend the publish when it reconnects. The Server will not have a record of the publish, so will treat it as a new flow and send out a duplicate. > > > > The QoS 2 flow is designed to ensure that if the network drops at any point before the flow completes, the client and server can resume the flow without causing a duplicate message to be sent. All without requiring either the client or server to maintain a log of message IDs that have 'completed'. > > > > > > Regards, > > Nick > > > > > > On 23 May 2013 11:37, Carl Hörberg <carl.h...@gmail.com (mailto:carl.h...@gmail.com) (mailto:carl.h...@gmail.com) (mailto:carl.h...@gmail.com)> wrote: > > > Ok, thank you for the explanation! > > > > > > I have further questions on QoS 2 as I fail to see the benefits of it. It seems to me that a correctly implemented server shouldn't be able send duplicates even in QoS1. > > > > > > First, my view of a QoS1 flow: > > > 1: client -> publish(qos1, msgid) -> server > > > 2: servers stores message, puts it in internal queues for each subscriber > > > 3: server -> puback(msgid) -> client > > > 4: the client can now be sure that the message is received and that it will be deliviered to the subscribers at some point. > > > 5: server starts to deliver message to subscribers > > > 6: the server marks the msg as delivered (at least keeps the msgid) for each subscriber. > > > > > > so if the client doesn't receive the puback, it resends it, the server then looks in the store for the msgid, it either discards it if it's already queued for the subscribers (or even already delivered, because we keep the msgid for a while even after sending out to the subscriber), or it processes it as normally if it haven't stored it before. > > > > > > Also, what good does the DUP flag do? shouldn't the receiver just have to look at the msg id and compare it to its log, it either as it or doesn't have, should it cares if the sender had to resend it? > > > > > > Now, QoS2, how does it improve the situation? > > > > > > 1: client -> publish(qos2, msgid) -> server > > > 2: servers stores message, puts it in internal queues for each subscriber > > > 3: server -> pubrec(msgid) -> client > > > 4: client marks the msg as delivered, won't resend it msg again. > > > 5: client -> pubrel(msgid) -> server > > > 6: server starts to deliver message to subscribers > > > 7: server -> pubcomp(msgid) -> client > > > 8: marks the msg as delivered (at least keeps the msgid) for each subscriber when the qos work flow is done. > > > > > > How does this improve anything? The only thing it does is to lock the messages in the internal queues of the server for a while. > > > > > > I also fail to see the benefits of limiting subscriptions QoS to the QoS level of the published message. A publisher can send a message in QoS 0, but why would the broker then ignore the wish of a subscriber which want to confirm its retrieval of a message? I see it as two independent actions.. > > > > > > -- > > -- > > To learn more about MQTT please visit http://mqtt.org (http://mqtt.org/) > > > > To post to this group, send email to mq...@googlegroups.com (mailto:mq...@googlegroups.com) (mailto:mq...@googlegroups.com) (mailto:mq...@googlegroups.com) > > To unsubscribe from this group, send email to > > mqtt+uns...@googlegroups.com (mailto:mqtt+uns...@googlegroups.com) (mailto:mqtt%2Bunsu...@googlegroups.com) (mailto:mqtt+uns...@googlegroups.com (mailto:mqtt%2Bunsu...@googlegroups.com)) > > > > For more options, visit this group at > > http://groups.google.com/group/mqtt > > > > --- > > You received this message because you are subscribed to the Google Groups "MQ Telemetry Transport" group. > > To unsubscribe from this group and stop receiving emails from it, send an email to mqtt+uns...@googlegroups.com (mailto:mqtt+uns...@googlegroups.com) (mailto:mqtt%2Bunsu...@googlegroups.com) (mailto:mqtt+uns...@googlegroups.com (mailto:mqtt%2Bunsu...@googlegroups.com)). > > For more options, visit https://groups.google.com/groups/opt_out. > > > > > -- > -- > To learn more about MQTT please visit http://mqtt.org (http://mqtt.org/) > > To post to this group, send email to mq...@googlegroups.com (mailto:mq...@googlegroups.com) (mailto:mq...@googlegroups.com) > To unsubscribe from this group, send email to > mqtt+uns...@googlegroups.com (mailto:mqtt+uns...@googlegroups.com) (mailto:mqtt%2Bunsu...@googlegroups.com) > > For more options, visit this group at > http://groups.google.com/group/mqtt > > --- > You received this message because you are subscribed to the Google Groups "MQ Telemetry Transport" group. > To unsubscribe from this group and stop receiving emails from it, send an email to mqtt+uns...@googlegroups.com (mailto:mqtt+uns...@googlegroups.com) (mailto:mqtt%2Bunsu...@googlegroups.com). > For more options, visit https://groups.google.com/groups/opt_out. > > > -- > -- > To learn more about MQTT please visit http://mqtt.org (http://mqtt.org/) > > To post to this group, send email to mq...@googlegroups.com (mailto:mq...@googlegroups.com) (mailto:mq...@googlegroups.com) > To unsubscribe from this group, send email to > mqtt+uns...@googlegroups.com (mailto:mqtt+uns...@googlegroups.com) (mailto:mqtt+uns...@googlegroups.com) > > For more options, visit this group at > http://groups.google.com/group/mqtt > > --- > You received this message because you are subscribed to the Google Groups "MQ Telemetry Transport" group. > To unsubscribe from this group and stop receiving emails from it, send an email to mqtt+uns...@googlegroups.com (mailto:mqtt+uns...@googlegroups.com) (mailto:mqtt+uns...@googlegroups.com). > For more options, visit https://groups.google.com/groups/opt_out. > > > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > -- > -- > To learn more about MQTT please visit http://mqtt.org > > To post to this group, send email to mq...@googlegroups.com (mailto:mq...@googlegroups.com) (mailto:mq...@googlegroups.com) > To unsubscribe from this group, send email to > mqtt+uns...@googlegroups.com (mailto:mqtt+uns...@googlegroups.com) (mailto:mqtt+uns...@googlegroups.com) > > For more options, visit this group at > http://groups.google.com/group/mqtt > > --- > You received this message because you are subscribed to the Google Groups "MQ Telemetry Transport" group. > To unsubscribe from this group and stop receiving emails from it, send an email to mqtt+uns...@googlegroups.com (mailto:mqtt+uns...@googlegroups.com) (mailto:mqtt+uns...@googlegroups.com). > For more options, visit https://groups.google.com/groups/opt_out. > > --