failure to process large messages using IBM's Javascript client

[Sam Matthews]

Hello all… 

I am trying to no avail to pass the data url (below) as a payload.

I am using the PAHO javascript client (mqttws31.js). 

http://git.eclipse.org/c/paho/org.eclipse.paho.mqtt.javascript.git/

I have been going over the library for hours.  I see many references to the ‘remaining length’ field, but I cannot figure out if any action is required on my part to process large messages. 

I am thus far working with a single client that publishes and subscribes on a single topic.

 

The behavior I see is at the the messages is published, and is received on “onMessageDelivered” event, but not “onMessagedArrived” event.

I have no issue with smaller strings but beyond a certain length I stop receiving the message on “onMessageArrived”.

I am using message.payloadString to store the message.

Thanks for any help.

Sam

data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAMCAgICAgMCAgIDAwMDBAYEBAQEBAgGBgUGCQgKCgkICQkKDA8MCgsOCwkJDRENDg8QEBEQCgwSExIQEw8QEBD/2wBDAQMDAwQDBAgEBAgQCwkLEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBD/wAARCABLAGQDAREAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwD74AFIkXA5oAkRBnBGaAuShFHJoJch4QY4FBNzgPiH8cfhH8MJRZ+OPHmlaZfMFYWBl868ZWztYW8YaUqdrc7ccGk3YcYyk/dPBZP+ChPwxvrtrHRPh/41umSTa7zW1tGmwZyw2zOT0HBA4znGMVLk77GsaMpHofw3/aG+GfxeW4g8O6jPaX9o2J7DUIxBOoPRgMkMpORkE8gjikpeViJKzsdXJAPtsm4YxgD8qNGSm2zUs024OOTVX7Gl9Tat32kYJxQBpRTbehFO9gJhd8dKejHcpgkHrS9RDw3PNDJJEcj60XJb7EyycEntQS3ofLH7fH7TGsfBDwNp+geAtchtfFmvvJkIUeW3sfLkQyhSdyMZWTy2xgmOTB+Uik2ti6UHN+R+S2v6pquo3b6lqOpSzX9+TcXcs9yZGmdjuLuxLMzHqT1JPPOadup0ttbFrw7qXjOB1XTbiUxsxLNEZh1BG4lAGwMg4U9qr2fN71ioqpLS1zqvCR1TQ01PxJf3hl1a2MYjj1BmLuzb8ld3zsf3YAOf4ucZGYkkNp03Zn2j+xf+1FrfjnxNL8MfHF3LdXV0jzaVcyvvkDRqTJA7Hk4VGYE8jaQSQVNZuPs9JGFWK+OJ9twfKBzTTT1MlfqXIZscds002Ve5cSc+vtmgaZILk4q0w3F83nOP/r0CY8SAmlfsIesmeg4p3Ie9h4k+nPvQZ9D8jv2yobr4vftc+J9E0O8nubXTFht5cq2ImhhhjkjXPQeYJPqcnkYJFaLvLqd2Dp+1tE5fQv2Zrj+2Fu724W3tEKsY/KJLpzj5j3yDnHQfWuhODdz1Fl0r6aHtGhfDbRtKiSGz0uOOOMbR8o5+v1p1ZOWh6lDDU6a2PLfjn4WXQWtPEUVsWhtpR9oQjiSPI3JnB25XOGwSDgjkA1jDm1VzjzOEVBOJxHwMzpnx88DX1vcC3im8R2DrIu5XliaaPdG+35ThTliTjG5ifmG7Ouml72jPAbTi+v8AX+Z+xUU4wB0x0rBWMUyykw7mr8yiZbg54PNO6uNMcbknHX8DTaXUNehc80L1OKq4PUUSg8bqTZNrbC+ce7Gi99DOVwM/Xn6mlfoiL32Phz4w+D9D+Ev7QXinx14ml03S9G+IKWbaVO0+fNuUQ/bFcHlG3lHJPykSrg53Ksyu7WPZymrFTd90awk06W3QxNFIh5yOQfp+ldEKc9D6hSjJXMPxD418E+GyINY1q0tZWI2QmTLnjj5Rzj3xWsqcoxvJ6nPUxNNOyPPfiUND8d+GbixsLmG5ilG9HiY8N2981nTlySuzHEU44qnypnyZ4Pm1W28XWNvY20lzqtnqkTWsCQ+Y5nSQEKuAeSy4I6cmnUlzXfY+X9m7uC3P2vtrgPFGxdssoPKlTn3B6fSuC7TMEiyJzwQau/U0F+0f7X6U1LUaFFwfWkp2VmM02lPc1pcncT7RnuPTpReyJaE+0t1B6+hpb6ENlTV9Q1K10u9udJsUvr6G3kktbV5/JW4lCkpGZCCEDMAN2DjOcHpRHV6mcrpOx8r/AB/8KeKfiL8O7vxdqF5br4hF2bSDTX/fWdlaFAphMUikb3eMytIUV8SiNiyxR4qadKSfQ7sFaU3CPVM+ffDen6t4Ll1LRLJLu8tYNM+3SR28ZRIWJYDBDYTcyMBhcfkTXd7aHOo05aW81/n/AEz1aSxEU1y39Lfq1/SJ9F+FOnf8JDFq1xdNZI6+YZY7aBzNknI3vG7g9Mr8v4849HB/U5p/WnKMl2X/AAG/yPPxdPGq0sMotX69vvR1ureHbTT7a2n0dJ5mVFS5adlj8xzKcyIAi7VETJ8hDfNGSGw4VeHFQo3k6U7rs9+nlr+B24aeLioqpBX7rb1avt6O/l1Xn/wj8C3dr8fxc3GlxNpj6pYXUV1N5KJlXV5BliMlzGchcFic8kgHzpyXLbqdnJHDqtVb6dt20+3nbsfojFeA4JNcyl3PmI9ywLoMv3uKOZrVFoX7UDxup3V7DWgv2oepP0OKJWXUGkzXe5wDzWz3uDI2uM5+b/61K+tiBpuBwS9S9NiWtStf6vZaZZXGo6leQ2tpbRNNPPNIqRxRqCWZmY4VQASSTgYo5raEbH5xeL/2tYvi18arfwtoeh2Gm6DcX7tZ3TSyLcNKkMgWVxu8oySZKEbTgPhSWG891Sh7Kl+8fvO3yOzLmniIyX9aHqOmQ22n6NqV3q2oRrfXICOr/KfLB+UZP3hyT+J96wo0G23E+yjONNO/UreBrq7hMsUjQzW9vwFt2C7wRlWJIYZB4OBzgHqa71G6bno0cXsuX3INW6XV/wBUVdfkuJZzI8rfxKqLkLyc5IHU8AZ9uMZNZzikxv3Nb3Z4f+054N8eaL4M8KfEuDVJE8Katdz2QNqP+PbUImkCiXIGfMRJNmD/AMspcgcbrwlKD96T9Nn+DPncwxk6knRWyav5uxw3wi/ad+JHwlvUl0jxFNqdlk/adPvpGlil3HJ64KnPIIPHTkEg9M8BQxC5U9f6/roea6ko6s91n/4KMeLLqSxXT/AthZwLJ/psnmvezum3gRw5gCkt6yNgZ4OOeCrlc6esXzL8fuNI1k99Ge8/Br9sr4V/FlLDTV1ZNI8Q3Ue59JvCUcSZwUjlICTHuoU7ivJVcEDzZQcXzPY36nukWpRyIHTJBpRXmLQ6B7kknLde2a0e4aMia5xn5v1pXtqJ67Hyl+1R+28PgvrsfgXwBpWn6x4ghRZtQmvGZ7W0DA7Yisbq7SYwx5AVSo+YsdndhMJ7aPPJ2Xlv/wAMZSlrZH59/EP44fE/4qaoNW8feK7nVriNdsIkCpHACAp8qJAqR5CqTtUZIycnmvWg40E4042X9dTJwvqz0f8AYR+G9v8AE39o7TbfUrayu9N0TTbzVr2C6yWdfL8iNohtI81J7iCRSSu3YWB3KAcHD21XkltYdSq6MOeO9z6T8Z6frGgeIb/StX0RJ5LO4e2aUTSLyG+UAGIgoR8wY4ypBxzXNQcsNVcOp9VQ9hjqSqylb7/udrlHw+dSsrtpLe1gtYW+Zz9rdmwTyNpi/r25rqqy53Zor2Soa053K3xO8XW/h3Qb7W2dSILeR4ldgC7bTgE9snA6dTXHOOtosmrU5U5NF7xn4Our/wD4JexXviuwa41axuYPEtq6uwMc1zqjN55VcA5tb+TIIwN5OMqCPTVLlpLurHxyq82Ik3rf+v0PzwgcrKSCeRjrWVN2m7HXJaF+3vVhcAuc+q11QrW0bMnG6NnRVlu/Eej3GnX0On38d/bmO7eTygjeYCJSw6FTg7vQewrkx+HjOm5xXvLdd/l1ZpTqWfLLY/WXQ9flOmxeZJzj1r5p36M6fU9SlvEBIyOa3bsxeSPjv9s/9ri/8ByT/Cb4bXtxaeIWjU6tqaDabKKSPcsUJIz5rK6t5i42DG0lyTH24LCqb9pPbov6/Lv6GU5taI/Oue5eZmkkYsxOSW5Oa9ZSuzO1iDdk5NZ311KPsr/gltpGp3Pxx8S61bW4a0s/C8kE0pIASSS8tWRcZzysUpzjHy+4zvh4t1HPsjjxsrU1Fb3Pq79uGafwD4N0z4l2GlS3UkWpJpl1FubmCRZZlcdQCjBgOBu8zBPAxvUwlOsudP3le3n1t+ew8uzKthJ+z+w/wfc+Z9B+JVv4psnn0u8CLImDgB3HrkcY/pXHKDqP3Xc+jli5NW2PJPjH4pk1ie28IQXTyeZIkl3ubJChhgHHQZHT6mqw2G9tVUH8/T/g7fM4cbiXTpvX+v8AI/SnTfAuu6x+zCnwnITStQ1TwS2iO9xykE8lj5GWKEhgCRnbngcV6NRKpzP+tj5mnJ05rqr/AKn4nJJtdiDzjFeJGVpOx7zWgqk/exVxb+ITLUF3IpX5uV6ED8q6KdZvchxR9V/Cv9sO28L+C7Lw94y0W/1K9sMwR3NuyDfAANgfJHzAZXPcAE5OTXm4nLqlSpzU7WfcunVSVpM+vfj3+0X4J+COg3I1HVoJfE1xYy3Gk6SqNJJM4OxHkAxsi3nksykhJNm5lxXHRoSry00Xf+tzWUuXU/J/Xte1LxFrF7rmsXst3e6hPJc3M8hy0sjsWZj7kkmvXjJQVkjFRuZpbPXpS5myhM988UrsQkpI2vnp1pVHa0xrXQN8issqSsrocqwY5GPT0qZK6TBPU6TRfHPiXRZY7yz1kAuGiliA2sUVR8zcANnnHJbKnPUE7TrVKluZq/frb9fLrcqi1QvZadun/AGt4wCxSXEUEzajMWLTyOG2sQRuHHJwe/07Cu6nj6dGjyQh7zvrfr3/AOAzkq4edepzTlp2Nj/hoP4+mHyD8bvH3lAbRH/wkt7tx6Y8zGPavMdSp9pm6pU07qKODLEYz361LutWixSxGB15/KnJvRdRJXJA+Dxg9sVtGauuoncla7C4BlCcdMZqpVoQdm/z/QXLJnUfFHxBrXib4g+Jdb1/Up76+uNTnEk8zZYqp2qo9FVQFVRgKAAAAMVz0VaNl5fkXvq/63ORPDE96vqHQYvJwelTDX+vUB0XKj3FVDVahISUDy3PsKmovcYo7kcfK0Q1VhyHsSqnaSMjB9+aeyuFhg6Z7ioeiTGNJIZee9JaNDHv/SnJ2TQkhCcEEdeKHryy6ghzMcsM9M05tq/9dGJIIUVlJYAnJrOlTjNNyQTbT0P/2Q==

[Andy Piper]

Hey Sam

The MQTT Google Group is for general discussion of the protocol. You'd be better off asking this question on the paho-dev mailing list and / or in Eclipse Bugzilla, where the developers for that client hang out.

If you're able to be more specific about the size of data you're sending where things start to fail, that would probably be useful too.

Andy

--
To learn more about MQTT please visit http://mqtt.org
---
You received this message because you are subscribed to the Google Groups "MQ Telemetry Transport" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mqtt+uns...@googlegroups.com.
To post to this group, send email to mq...@googlegroups.com.
Visit this group at http://groups.google.com/group/mqtt.
For more options, visit https://groups.google.com/d/optout.

--

Andy Piper | Kingston upon Thames, London (UK)
blog: http://andypiper.co.uk   |   skype: andypiperuk
twitter: @andypiper  |  images: http://www.flickr.com/photos/andypiper

[Sam Matthews]

Thanks, I will do that.  I will also try to be more specific regarding the size of the post.

[Sam Matthews]

Hi, I wanted to get back to you on the size of the payload that is failing.. approximately 3600 chars.

I have hosted a sample of a chat program I am working on.

http://darkchat-bucksoft.rhcloud.com/chat

Currently it subscribes and publishes on the same Topic (so effectively you are chatting with yourself).

Ive been looking at Wireshark logs and incrementing the payload.  Typically I will get 2 hits back from the server for each one I send.  One is my payload and the other is .. I don't know.  In the case where my payload is > 3600 I get no payload at all.. and just the other packet.  I apologize for not understanding the protocol well enough, but I really wasn't intending on learning it intimately. 

My QOS = 1 and I have tried pointing to all the following public brokers....

broker.mqttdashboard.com

test.mosca.io

.. as well as my own broker running Mosca on OpenShift.

Thanks again,

Sam 

[Nicholas O'Leary]

Sam,

When did you grab the client code? We recently merged the develop branch to master that brings support for mqtt packets being split over multiple WS frames.

It is possible that once your message gets to a certain size it us being fragmented by the server... which wouldn't have worked with the master branch of the client until a couple weeks ago.

Nick

[Sam Matthews]

Hi Nick,

I have had it for a month or so …I will visit the git site and get the latest.  Thanks.

Sam

You received this message because you are subscribed to a topic in the Google Groups "MQ Telemetry Transport" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mqtt/ReGqXfQEoOE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mqtt+uns...@googlegroups.com.

[Matteo Collina]

Il giovedì 20 marzo 2014, Nicholas O'Leary <nick....@gmail.com> ha scritto:

When did you grab the client code? We recently merged the develop branch to master that brings support for mqtt packets being split over multiple WS frames.

GOOD news! IMHO it was the biggest problem of that library.

It is possible that once your message gets to a certain size it us being fragmented by the server... which wouldn't have worked with the master branch of the client until a couple weeks ago.

This is exactly what Mosca does. 

BTW, Mosca also have an embedded version of a MQTT over websocket client. That should work well with bigger payloads.

Matteo

[Sam Matthews]

Hi Matteo,

I would be happy to use Mosca in standalone fashion, but AFAIK I can’t deploy it to any PaaS that I know of.  This is why I opted for Mosca embedded in the node app.  When you say “this is exactly what Mosca does”.. does that mean that it also fragments the messages and therefore will not work with a large payload?  

I have successfully tested large payloads with the latest version of the Paho javascript client and “test.mosca.io” as the broker.  I still can’t get my embedded version of Mosca to process the large payloads however. 

I have just installed Docker, and your Mosca docker image.  I was able to get it running and connect my app to it… very nice stuff!!

Questions.. 

Can I expect Mosca embedded in a node app to behave the same as the standalone Mosca.  If not ….

Do you know of a way I can host Mosca standalone… much in the same way you have hosted “test.mosca.io”? 

Thanks again,

Sam 

Matteo

--
To learn more about MQTT please visit http://mqtt.org
---

You received this message because you are subscribed to a topic in the Google Groups "MQ Telemetry Transport" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mqtt/ReGqXfQEoOE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mqtt+uns...@googlegroups.com.