How to increase Tcp Chunk size in Akka IO using Tcp

[Laxman Vemula]

Hi,

I am using Akka-Cluster and sending large objects over network using Akka IO through Tcp. The data is being cut down into very small chunks. The size of the data ByteString received through 'Received' message is very small (around 7KB). Is there any configuration setting using which I can send and receive larger ByteStrings at a time? 

Thanks,

Laxman Vemula

[Konrad Malawski]

Hi Laxman,

I'd actually say that a 7KB message is not very small, sounds like a normal sized message (you're probably using java serialization?).

I'd rather recommend using a better serializer, like protobuf, or kryo etc in order to get the message size lower than worrying about tcp chunk sizes, until you actually do bump into performance problems related to TCP.

Is there a problem in the cluster that you have tracked back to the chunk sizes?

--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com.
To post to this group, send email to akka...@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

--

Cheers,

Konrad 'ktoso' Malawski

Akka @ Typesafe

[Konrad Malawski]

Friday evening syndrome struck me it seems:

I missed the fact that you're talking about Akka IO, and not cluster messages.

For that case 7KB indeed is not much. Are you sure the messages Write messages 

you issue should be causing larger chunks to be written?

The setting responsible for the size of chunks we write and read to the kernel is

akka.io.tcp.direct-buffer-size which defaults to 128KiB.

You can also enable akka.io.trace-logging which will show you exactly how large chunks are being written.

[Laxman Vemula]

Thank you for the response.

The payload is usually around 150MB. The network bandwidth is 1 Gbps. I am thinking that larger chunk sizes would result in a better throughput. 

I have set the value akka.io.tcp.direct-buffer-size to 64 MiB, but it didn't change anything. So does it mean that OS is not allowing larger chunks ? Should I configure anything on the OS level.

[Akka Team]

Hi,

Akka cannot control how the messages get chunked by the kernel to the wire. I think though that if you turn off the socket option TcpNoDelay (it is on by default in akka) then it enables Nagle's algorithm and the kernel might send larger chunks.

-Endre

Akka Team

Typesafe - Reactive apps on the JVM

Blog: letitcrash.com
Twitter: @akkateam