vert.x 3.0.0 dev preview 1 does not allow increase of RCV Buffer for UDP

[Chris Morgan]

Netty bootstrap ChannelOption.SO_RCVBUF defaults to 2048.
Messages are truncated when receiving larger messages.

DatagramSocketOptions.setReceiveBufferSize does not seem to update the RCVBUF size. Previous documentation states this is for TCP.


I cannot find a way to solve this issue. Code snippet output verifies 2048 receive buffer size.


Code Snippet -----
		DatagramSocketOptions options = new DatagramSocketOptions();
		options.setReuseAddress(true);	
		options.setReceiveBufferSize(4096);//65536);
		DatagramSocket socket = vertx.createDatagramSocket(options);
		
		socket.listen(port, host, socketResult -> {
			if (socketResult.failed()) {
				throw new RuntimeException("Streaming connection failed to be created", socketResult.cause());
			}

			socket.handler(udpPacket -> {
				String xmlPacket = null;
				try {
					Buffer buffer = udpPacket.data();
					byte[] bytes = buffer.getBytes();
					ByteBuf bBuffer = buffer.getByteBuf();
					ByteBuffer nioBBuffer = bBuffer.nioBuffer();

					System.out.println("Buffer length = " + buffer.length() + ", bytes length=" + bytes.length + "");
					System.out.println("Buffer toString length = " + buffer.toString().length());
					System.out.println("BBuffer # buffers = " + bBuffer.nioBuffers().length);
					System.out.println("NIO BBuffer size = " + nioBBuffer.array().length + ", offset=" + nioBBuffer.arrayOffset() + ", limit=" + nioBBuffer.limit() + "\n" + nioBBuffer.toString());
										
					xmlPacket = new String(udpPacket.data().getBytes(), Charset.forName("UTF-8"));
					xmlPacket = xmlPacket.trim();
					processXmlMessage(xmlPacket);					
				} catch (Exception e) {
				}
			});			
		});

[Chris Morgan]

After tracing the call stack to DatagramSocketImpl.createChannel (line 281), then stepping into DatagramChannelConfig (netty), it has a receiveBufferSize of what I set (8192) but a rcvBufAllocator of 2048.

The truncation is happening at 2048.

In Vert.x, should I be processing multiple buffers? Or, is it possible to set the rcvBufAllocator?

Any feedback or alternative solutions would be greatly appreciated.

[Tim Fox]

Looks like Vert.x is telling Netty about the desired receive buffer size, if Netty is not respecting that value... sounds like a Netty issue?

Have you tried reporting an issue against Netty? Or perhaps ping them on IRC #netty

--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Chris Morgan]

Thanks Tim for the suggestion. I posted to a Netty group here - https://groups.google.com/forum/#!topic/netty/CuqtCArwRwc. It appears that the Allocated Buffer size also needs to be set in Netty in order for larger UDP messages to be handled. This seems to be by design. Vert.x would have to provide access to manipulate the allocated buffer or automatically handle it on behalf of clients.

On Friday, January 16, 2015 at 4:26:50 PM UTC-5, Chris Morgan wrote:

[Tim Fox]

So, you're suggesting changing the code to this:

if (options.getReceiveBufferSize() != -1) {
      channel.config().setReceiveBufferSize(options.getReceiveBufferSize());
      channel.config().setRecvByteBufAllocator(new FixedRecvByteBufAllocator(options.getReceiveBufferSize()));
}

It begs the question why does Netty allow receive buffer size to be set if the value is always ignored...?

[Chris Morgan]

Unfortunately, I have not gotten much feedback in the Netty group or Stack Overflow to understand their intentions.

I am not necessarily suggesting a specific fix in Vert.x. I believe the issue needs to be reproduced by vert.x developers and a proper solution assessed (whether that involves a Netty change or not).

The bottom line for me and other vert.x users is that processing UDP packets using the vert.x DatagramSocket is problematic. It should be transparent to me on the under-workings of the libraries vert.x decides to use since they are abstracting away those interfaces for me.

On Friday, January 16, 2015 at 4:26:50 PM UTC-5, Chris Morgan wrote:

[Tim Fox]

Should be fixed now in master

[Chris Morgan]

Fabulous Tim! That works great now.

I do agree with you that it seems like more of a Netty issue. It is not intuitive in how it behaves. But in the meantime thank you much for your patch.

Do you have a time frame for when the next 3.0 tag will be done (dev preview 2, milestone 3). I like developing using those vs. master snapshot as much as possible?

On Friday, January 16, 2015 at 4:26:50 PM UTC-5, Chris Morgan wrote: