Unable to send continuous data through MQTT

[sur...@tantragyaansolutions.com]

Hi all, 

I have been working on bringing up MQTT Connection to send continuous data to Relayr Cloud platform.

I am using an arduino & SIM800 module to achieve the same.

I am able to connect & getting CONNACK reply from the server. 

After this, I am sending the data through PUBLISH.

But I am unable to send more than 14-15 messages to server. 

I get proper PUBACK response from the server for all the successful messages.

But after the limit, there is no response from server.the 

This is not a timing related issue, since the same pattern is seen even if I send a message once every 10 secs.

When debugged further, I found that issue is related to sending NULL character to server. 

The code has a ping function which sends the null character. 

The number of messages I can send reduces linearly as the number ping messages increases.

MQTT Standard suggests that "A UTF-8 encoded string MUST NOT include an encoding of the null character U+0000"

Since the TOPIC & PACKET IDENTIFIER fields of the Publish message are smaller, Length MSB & Packet identifier MSB byte of UTF-8 strings for these in the Publish message will be 0x00 i.e. NULL character. 

Is our issue related to Null character or are we missing anything else ?

Any support would be highly appreciated. 

Thanks in Advance. 

Regards, 

Suresha

[Vidyadhar Kothekar]

What MQTT broker are you using and which API? Are you using eclipse Paho?

Regards,
Vidyadhar

[Suresha N S]

Hi Vidyadhar,

Thanks for the response.

We are using Relayr MQTT broker. 

We are running the code on Arduino Uno board to which SIM800 Module is wired up.

The SIM800 MQTT Library  is used as a base code for connecting & publishing data to Relayr broker.

Looking forward to your support.

Regards,

Suresha 

--
To learn more about MQTT please visit http://mqtt.org
---
You received this message because you are subscribed to the Google Groups "MQTT" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mqtt+unsubscribe@googlegroups.com.

To post to this group, send email to mq...@googlegroups.com.
Visit this group at https://groups.google.com/group/mqtt.
For more options, visit https://groups.google.com/d/optout.

--

Thanks & Regards,

Suresha N S

[Gambit Communications]

The beauty of MQTT is that you can use brokers interchangeably.

You can use a broker on your intranet, eg. mosquitto, to debug problems,
ie. the broker prints error messages, you can run in the debugger, etc.
Then when everything works, you deploy on the Internet, which will have
a new set of problems.

The only way to diagnose your protocol problems is with a protocol analyzer like
wireshark.

[Paul Fremantle]

I'm not clear how you are sending any payload with a PING.

The MQTT ping has a PINGREQ that takes no payload and a PINGRESP that is the same. Both are 2-byte messages.

You are also confusing the byte messages of the protocol with UTF-8 Strings. Certain messages do include 0x00 bytes. That is different from trying to encode a U+0000 character as part of a UTF-8 String. 

Paul

--

To learn more about MQTT please visit http://mqtt.org
---
You received this message because you are subscribed to the Google Groups "MQTT" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mqtt+unsubscribe@googlegroups.com.
To post to this group, send email to mq...@googlegroups.com.
Visit this group at https://groups.google.com/group/mqtt.
For more options, visit https://groups.google.com/d/optout.

--

Paul Fremantle

Doctoral Researcher, University of Portsmouth, School of Computing

Visiting Scientist, Institute of the Architecture of Application Systems, Stuttgart

Visiting Lecturer, Software Engineering Programme, Oxford University

Co-Founder, WSO2
Apache Member and Committer

email: paul.fr...@port.ac.uk, pa...@fremantle.org

twitter: pzfreo / skype: paulfremantle / blog: http://pzf.fremantle.org

[Suresha N S]

Thanks for the response. 

Since we are using the GSM network we cannot explicitly connect to the intranet. 

But we tried publishing data to test.mosquitto.org from board & having a separate client running on the PC.

With this also, we are able to see limited number of messages delivered.

As a client our board is configured for only Publishing & not subscribed to any topics. 

Is this a issue ? 

Also, we are not monitoring any incoming messages from the MQTT server since we are only publishing. 

Is there a requirement to listen to incoming messages from Server also ?

Looking forward to your support. 

--

To learn more about MQTT please visit http://mqtt.org
---

You received this message because you are subscribed to a topic in the Google Groups "MQTT" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mqtt/HzW-BB_y7Qo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mqtt+unsubscribe@googlegroups.com.

To post to this group, send email to mq...@googlegroups.com.
Visit this group at https://groups.google.com/group/mqtt.
For more options, visit https://groups.google.com/d/optout.

--

[Suresha N S]

Sorry for the confusion. 

We are not sending any payload with ping message. But the PINGREQUEST command is followed by a byte 0x00 indicating that there is no more bytes. 

As a part of UTF-8 string, we will be sending 0x00 as Length byte MSB for TOPIC in Publish message payload also. 

Our observation is that, if we increase the number of pings , then number of messages getting delivered will be reduced. 

Thus we thought that there is a limited number of 0x00 we can send to MQTT server.

Since there is a chance that MQTT server can treat 0x00 as a NULL character, it may not be receiving any further messages from Client after a limit.

This is just one of the possible explanations we can think of , but we were not able to prove in anyway that the issue is related to this. 

Any suggestions on debugging will be really helpful .

On Thu, Aug 3, 2017 at 2:08 PM, Paul Fremantle <paul.fr...@port.ac.uk> wrote:

I'm not clear how you are sending any payload with a PING.

The MQTT ping has a PINGREQ that takes no payload and a PINGRESP that is the same. Both are 2-byte messages.

You are also confusing the byte messages of the protocol with UTF-8 Strings. Certain messages do include 0x00 bytes. That is different from trying to encode a U+0000 character as part of a UTF-8 String. 

Paul

--

[Gambit Communications]

Suresha,

your problem description is VERY vague. It could be ANYTHING. But, since it seems to be consistently
reproducible in a relatively short period of time, you are in luck that you can get to the bottom of it. To
remove inconsistencies, wrong conclusions, etc, you MUST capture protocol packets, eg. Wireshark
for a couple of iterations.

Nevertheless, from your symptoms I am going to guess that you set your "Keep Alive" parameter
to some value (maybe small), and not sending PINGREQ within that interval. It just so happens
that you happen to send PUBLISH messages in the right frequency, but when you don't, the server
disconnects.

Gambit

On Monday, August 7, 2017 at 1:55:41 AM UTC-4, Suresha wrote:

Sorry for the confusion. 

We are not sending any payload with ping message. But the PINGREQUEST command is followed by a byte 0x00 indicating that there is no more bytes. 

As a part of UTF-8 string, we will be sending 0x00 as Length byte MSB for TOPIC in Publish message payload also. 

Our observation is that, if we increase the number of pings , then number of messages getting delivered will be reduced. 

Thus we thought that there is a limited number of 0x00 we can send to MQTT server.

Since there is a chance that MQTT server can treat 0x00 as a NULL character, it may not be receiving any further messages from Client after a limit.

This is just one of the possible explanations we can think of , but we were not able to prove in anyway that the issue is related to this. 

Any suggestions on debugging will be really helpful .

On Thu, Aug 3, 2017 at 2:08 PM, Paul Fremantle <paul.fr...@port.ac.uk> wrote:

I'm not clear how you are sending any payload with a PING.

The MQTT ping has a PINGREQ that takes no payload and a PINGRESP that is the same. Both are 2-byte messages.

You are also confusing the byte messages of the protocol with UTF-8 Strings. Certain messages do include 0x00 bytes. That is different from trying to encode a U+0000 character as part of a UTF-8 String. 

Paul

On 31 July 2017 at 12:37, <sur...@tantragyaansolutions.com> wrote:

Hi all, 

I have been working on bringing up MQTT Connection to send continuous data to Relayr Cloud platform.

I am using an arduino & SIM800 module to achieve the same.

I am able to connect & getting CONNACK reply from the server. 

After this, I am sending the data through PUBLISH.

But I am unable to send more than 14-15 messages to server. 

I get proper PUBACK response from the server for all the successful messages.

But after the limit, there is no response from server.the 

This is not a timing related issue, since the same pattern is seen even if I send a message once every 10 secs.

When debugged further, I found that issue is related to sending NULL character to server. 

The code has a ping function which sends the null character. 

The number of messages I can send reduces linearly as the number ping messages increases.

MQTT Standard suggests that "A UTF-8 encoded string MUST NOT include an encoding of the null character U+0000"

Since the TOPIC & PACKET IDENTIFIER fields of the Publish message are smaller, Length MSB & Packet identifier MSB byte of UTF-8 strings for these in the Publish message will be 0x00 i.e. NULL character. 

Is our issue related to Null character or are we missing anything else ?

Any support would be highly appreciated. 

Thanks in Advance. 

Regards, 

Suresha

--
To learn more about MQTT please visit http://mqtt.org
---
You received this message because you are subscribed to the Google Groups "MQTT" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mqtt+uns...@googlegroups.com.

To post to this group, send email to mq...@googlegroups.com.
Visit this group at https://groups.google.com/group/mqtt.
For more options, visit https://groups.google.com/d/optout.

--

Paul Fremantle

Doctoral Researcher, University of Portsmouth, School of Computing

Visiting Scientist, Institute of the Architecture of Application Systems, Stuttgart

Visiting Lecturer, Software Engineering Programme, Oxford University

Co-Founder, WSO2
Apache Member and Committer

email: paul.fr...@port.ac.uk, pa...@fremantle.org

twitter: pzfreo / skype: paulfremantle / blog: http://pzf.fremantle.org

--
To learn more about MQTT please visit http://mqtt.org
---
You received this message because you are subscribed to the Google Groups "MQTT" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mqtt+uns...@googlegroups.com.

To post to this group, send email to mq...@googlegroups.com.
Visit this group at https://groups.google.com/group/mqtt.
For more options, visit https://groups.google.com/d/optout.

[Suresha N S]

Thanks for the support. 

We were able to finally resolve the issue  !! :-)

The issue was related to SIM card & GSM network, we were able to send continuous data to cloud after changing the SIM card.

Regards,

Suresha

To unsubscribe from this group and stop receiving emails from it, send an email to mqtt+unsubscribe@googlegroups.com.

To post to this group, send email to mq...@googlegroups.com.
Visit this group at https://groups.google.com/group/mqtt.
For more options, visit https://groups.google.com/d/optout.