Vertx socket server cuts my string when receiving large message

[ga...@zoozbit.com]

I have created a simple vertx tcp server using Jenkovs tutorial and a simple tcp client to send messages to the server.

If I try to send a fairly small string, it works perfectly fine.

The problem happens when I try to send a big string. When I try to send a big string the vertx tcp server cuts my message and I get 2 separate messages instead of one, the first of which containing some part of the original string, and the second one containing the rest of the string.

If I want to send 100 fairly large messages the server starts splitting my messages and I cant distinguish between the end of a message and the start of a new one.

I want to be able to send big strings consistently without them getting cut and messed with by the server.

If thats not possible then I could use a way of distinguishing between my messages.

Also, I have worked with vertx before and had no such problem with HTTP based connections.

If you need any more information i would be happy to provide.

Thanks,

Gal 

[Julien Viet]

On 12 Aug 2018, at 14:49, ga...@zoozbit.com wrote:

I have created a simple vertx tcp server using Jenkovs tutorial and a simple tcp client to send messages to the server.

If I try to send a fairly small string, it works perfectly fine.

The problem happens when I try to send a big string. When I try to send a big string the vertx tcp server cuts my message and I get 2 separate messages instead of one, the first of which containing some part of the original string, and the second one containing the rest of the string.

If I want to send 100 fairly large messages the server starts splitting my messages and I cant distinguish between the end of a message and the start of a new one.

I want to be able to send big strings consistently without them getting cut and messed with by the server.

If thats not possible then I could use a way of distinguishing between my messages.

it's not possible unless you have a protocol for it, a very common practice is to prefix the message with its length encoded on a fixed length and write the message length before.

if you chose this practice, you can then use the vertx RecordParser on the server.

Also, I have worked with vertx before and had no such problem with HTTP based connections.

do you mean using HTTP entity chunked encoding ?

If you need any more information i would be happy to provide.

Thanks,

Gal 

--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.
Visit this group at https://groups.google.com/group/vertx.
To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/e6e2d2ab-a60f-4436-947e-94033c6e9f6b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ga...@zoozbit.com]

The problem is I dont know the strings size in advance. The user determines what will be in the string. There could be a max length if that can help solve my problem.

I have also tried to do the separation from the client, tried it many different ways including the simplest socket write/flush methods, but to no avail.

Is there really no way of sending/receiving messages separately?

Any kind of solution that gives me dynamic options (dynamic string length) would be amazing.

Thanks,

Gal

[Ronald van Raaphorst]

You could Send and end token, ie '[[@end]]' and buffer server side as long as This particular string is not received. Bear in mind that I'm not sure that the order in which separate parts are sent may not be guaranteed, so u night use a start string that gives the order in which parts are sent, like Starting with '[[001]]'

[ga...@zoozbit.com]

Hi Ronald,

thanks for your answer

I have tried to do that, but my problem is that string sizes may vary quite a lot,

one buffer could contain 6 (just an example) whole strings of different messages, and I found it very hard to distinguish between messages even with tags.

That is because the buffer could contain for example only the part between the 2 tags, for example:

if a base string would be "<start>thisIsAString<end>, and I send a lot of them(lets say 100 or so), at some point the buffer will fill up, cut my string and i might be left only with: "IsAString<end>" or "<start>ThisIsAStri" or other variations of some cut of my string

those kind of string manipulations the buffer does on my strings make it hard to differentiate between them, even with tags.

If you have an idea around that, that'll be great.

[ga...@zoozbit.com]

Hi Julien,

I tried your suggestion of using RecordParser, but the problem remains.

I did manage to distinguish between some requests using both the delimiter, and the fixed size, separately.

My problem now is that when the buffer fills up, it cuts my string. That makes it so the RecordParser cant parse correctly, and even if it does parse something, the parsed string can be unfinished/incorrect due to the buffer changing uncontrollably.

The only solution i can think of is a way to clear the buffer from either the server or the client side.

I have tried using Socket in my client for its flush function but that didnt help, and from what I understand there is no way to clear the buffer from the server side on vertx.

Is there any other workaround you can think of?

Thanks,

Gal

בתאריך יום ראשון, 12 באוגוסט 2018 בשעה 17:47:03 UTC+3, מאת Julien Viet:

[richa...@gmail.com]

Hello,

Try something like this:

def server = vertx.createNetServer()
server.connectHandler({ socket ->
  socket.handler(RecordParser.newDelimited("\n", { buffer ->

    // buffer contains your (complete) message

  }))
})

Clients will need to send messages ending with \n

one message\nAnother message\nYet another message\n and so on.

[Julien Viet]

right.

if you are using prefixed length, you need to switch the record parser length on the fly which is not really documented.

i.e you need to first tell the parser to parse 4 bytes (for 4 bytes prefixed protocol).

when you get the callback (a buffer of 4 bytes) then you decode the length and you switch the length to what you decoded.

when this new buffer is decoded you switch back to 4 bytes and so on.

HTH

Julien

-- 
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.
Visit this group at https://groups.google.com/group/vertx.

To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/1bf80028-a747-46de-ba40-745476502218%40googlegroups.com.

[ga...@zoozbit.com]

Thank you all for your answers.

The code snippet helped me understand how to handle the requests exactly as i wanted to, (im using a tag to differentiate between messages).

And i managed to make it work.

Cheers,

Gal