Handling large message through multiple web socket frames

[actorGeek]

Folks,

Just wanted to get some input. Need to handle large messages from a Java SockJS client to the Vertx eventbus.

I know we can increase the size of the web socket frame, but trying to find a solution that can split

messages into multiple frames and hence pretty much work for any frame size setting.

Assuming the default 65K frame size and lets say I want to handle a 130K message

that would require two frames. And the client sends 1 continuation frame and 1 final frame.

Appreciate some help with the following questions:

1. When sending - I know I can write the first frame as a continuation frame and then the final frame. 

But what is unclear is that does the application also have to ensure that these two frames are contiguous

in the web socket queue i.e. if there are multiple threads writing these large messages, then some kind

of synchronization is needed to ensure all frames associated with a single message are contiguous ?

2. When receiving - if these SockJS messages are destined for the event bus ..... how does Vertx

deal with the assembly ? Or is there an expectation that the SockJSBridgeHandler has to do all the assembly ?

Thanks

[Julien Viet]

On Jul 18, 2017, at 12:33 AM, actorGeek <vikul....@gmail.com> wrote:

Folks,

Just wanted to get some input. Need to handle large messages from a Java SockJS client to the Vertx eventbus.

I know we can increase the size of the web socket frame, but trying to find a solution that can split

messages into multiple frames and hence pretty much work for any frame size setting.

Assuming the default 65K frame size and lets say I want to handle a 130K message

that would require two frames. And the client sends 1 continuation frame and 1 final frame.

Appreciate some help with the following questions:

1. When sending - I know I can write the first frame as a continuation frame and then the final frame. 

But what is unclear is that does the application also have to ensure that these two frames are contiguous

in the web socket queue i.e. if there are multiple threads writing these large messages, then some kind

of synchronization is needed to ensure all frames associated with a single message are contiguous ?

if you are writing from the EventLoop thread, two consecutif writes on the websocket work.

otherwise, you can simply use Context#runOnContext to have a task that will do that.

2. When receiving - if these SockJS messages are destined for the event bus ..... how does Vertx

deal with the assembly ? Or is there an expectation that the SockJSBridgeHandler has to do all the assembly ?

I don’t think the bridge will do the assembly for you given that the bridge is not websocket specific (i.e it can handle other transports).

So you should use a Websocket server and handler probably.

Thanks

--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.
Visit this group at https://groups.google.com/group/vertx.
To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/1b1ed0a2-972b-4af5-80cf-644f0d898ea7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[actorGeek]

Hi Julien:

Thanks for your response. 

The messages are originating from a Java SockJS Client. The client is setting up a web socket

connection by connecting to the Vertx HttpServer and upgrading the connection. The HttpServer

in turn has attached a Handler<BridgeEvent>. As far as I can tell, a fairly standard setup for extending

 the event bus.

I can probably handle contiguous writes on the client side, but I don't understand how the assembly

would work over a standard Vertx HttpServer configured with a bridge handler. Since this setup is

really extending the event bus, on the server side, I don't have any visibility into the underlying

websocket upgrade that took place for the http connection.

If I whittle this down .... the question really is can an extended event bus handle multi frame messages

or is the max message size pretty much restricted to the size of a websocket frame ?

If I move to using a WebSocket Server, then I loose the ability to register handlers etc over the extended

event bus.

Appreciate a few more insights dealing with this. Thanks for your help.

[Julien Viet]

now I think about it

 you can use SockJS and restrict the transport to websockets though.

so we should dig in that direction.

To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/ca2dc90d-a035-4153-a49d-a0b567f47f56%40googlegroups.com.

[Vikul Khosla]

Thanks Julien. Can you elaborate on that ? Is that possible now or is this something you all are thinking about ?

The other option for me right now is the following ... and wanted to get your perspective on this.

I was thinking of introducing a Frame Assembly MicroService at the backend. And have a reasonable size for the

Web socket frame (say 128K). Now whenever the message size exceeds 128K, the client will automatically split

into mulitple frames of 65K each and send to this assembly service .... which will then assemble and forward to the

real recipient on the event bus. There will only be one instance of this service though. With this I probably don't even

have to worry about the contiguous writes.

Do you see any gotchas ? Once I understand your suggestion of a SockJS restricted transport, will that be preferable

over this ?

Thanks

To unsubscribe from this group and stop receiving emails from it, send an email to vertx+unsubscribe@googlegroups.com.

Visit this group at https://groups.google.com/group/vertx.
To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/ca2dc90d-a035-4153-a49d-a0b567f47f56%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "vert.x" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vertx/1_dVQla4Pv0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vertx+unsubscribe@googlegroups.com.

Visit this group at https://groups.google.com/group/vertx.

To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/A1C7F395-6936-4FA6-91E0-C77A89A73137%40julienviet.com.

[Vikul Khosla]

Sorry typo ... I meant split into 128K frames.

[Julien Viet]

Hi,

it is possible to restrict SockJS bridge to Websocket transport on the server : http://vertx.io/docs/vertx-web/java/#_the_client_side

see “disabledTransports”, so you can disable all transports but websocket.

To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/CAMasrU5OZorXL9%2BjCweZn2oVoXPJXpS2aJuiL8jzOt7ondia7Q%40mail.gmail.com.