Server Sent Events + Chunking Big Respone.

sreekanth guru

unread,

Oct 25, 2021, 2:57:39 PM10/25/21

to vert.x

Hi All,

I got into an issue with vert.x SSE implementation, appreciate any help.

Our SSE implementation with vert.x is as follows.

1. Client makes a request to server
2. Server sets the response as chunked and keep sending data (one write call for one record)
3. Client subscribes to response's handler and get server sent events(records).

The problem I am facing is some of the server write calls(records) are chunked and not appearing in the client handler() as one message, so now I am unable to identify where is each write(record) starting and ending. So SSE implementation became error prone.

I would appreciate any suggestions. Thanks in advance.

Thank you.

Julien Viet

unread,

Oct 26, 2021, 2:53:35 AM10/26/21

to vert.x

Hi,

do you mean that a single SSE event (separated by \r\n) is chunked and
the client will not be able to reassemble the chunks in a single event
?

Julien

> --
> You received this message because you are subscribed to the Google Groups "vert.x" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/bb7be939-b766-433c-94ce-01be1855776bn%40googlegroups.com.

sreekanth guru

unread,

Oct 26, 2021, 3:16:09 AM10/26/21

to vert.x

Hi Julien,

Thank you for the response.

Single SSE event is chunked randomly, there is no \r or \n in event payload. As each event is sent (write call) as a chunk, and each chunk in turn is again chunked, it is becoming hard to identify the event boundaries.

Thank you.

sreekanth guru

unread,

Oct 26, 2021, 3:19:26 AM10/26/21

to vert.x

Hi Julien,

Even without chunking we have following behavior for a request

From the documentation - https://vertx.io/docs/vertx-core/java/

To receive the body, you can use the handler on the request, this will get called every time a chunk of the request body arrives. Here’s an example:

I think the above behavior is chunking each record.

Thank you.

Julien Viet

unread,

Oct 26, 2021, 4:34:08 PM10/26/21

to vert.x

do you have a reproducer ?

> To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/a1664206-9e6a-49fd-bbc9-efa26b46d22bn%40googlegroups.com.

sreekanth guru

unread,

Nov 4, 2021, 12:34:11 PM11/4/21

to vert.x

I will create a small git project and provide the link for the same. thank you.

Message has been deleted

sreekanth guru

unread,

Nov 7, 2021, 4:20:29 PM11/7/21

to vert.x

Hi Julien,

Please find the reproducer code at https://github.com/gurusreekanth/vertx-sse-issue. I also have updated the README.txt file.

Thank you.

sreekanth guru

unread,

Nov 11, 2021, 1:22:58 AM11/11/21

to vert.x

Hi Julien,

Were you able to reproduce the issue with the reproducer code?

Thank you.

sreekanth guru

unread,

Nov 21, 2021, 11:24:15 PM11/21/21

to vert.x

Hi Julien,

Just a remainder....

Thank you.

Julien Viet

unread,

Nov 30, 2021, 4:41:27 AM11/30/21

to vert.x

Hi,

I will have a look

Julien

> --
> You received this message because you are subscribed to the Google Groups "vert.x" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.

> To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/79b18e84-87ac-4f82-b223-b07101701e1fn%40googlegroups.com.

Julien Viet

unread,

Nov 30, 2021, 5:05:42 AM11/30/21

to vert.x

Hi,

I found the issue, it happens on the client.

The client has a maxChunkSize that limits the size of the chunked
buffers, that is a protection to avoid attacks that would lead the JVM
to OOM.

The default max chunk size is 8K, so any message above 8K will be split.

You can configure the max chunk size and increase it to a value that
will above the message size you send.

Julien

> --
> You received this message because you are subscribed to the Google Groups "vert.x" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.

> To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/79b18e84-87ac-4f82-b223-b07101701e1fn%40googlegroups.com.

sreekanth guru

unread,

Nov 30, 2021, 1:03:52 PM11/30/21

to vert.x

Hi Julien,

Thank you for looking into this issue.

I have set max chunk size to 64k. So 10k messages shouldn't have been chunked.

The following output is for 7k size message write, it is still getting chunked though it is < 8k size. I tried with even 2k record size, it is also getting chunked.

Output:

Nov 30, 2021 11:30:15 PM io.netty.resolver.dns.DnsServerAddressStreamProviders <clinit>

WARNING: Can not find io.netty.resolver.dns.macos.MacOSDnsServerAddressStreamProvider in the classpath, fallback to system defaults. This may result in incorrect DNS resolutions on MacOS.

Nov 30, 2021 11:30:15 PM /vertx/message-egress

INFO: HTTP server running on port 8000

Nov 30, 2021 11:30:16 PM client.MessageConsumerClient lambda$startSessionHttpClient$1

INFO: Successfully connected to http://localhost:8000/vertx/message-egress