Akka.IO TCP send/receive buffer

[Carsten Saathoff]

Hi all,

I am currently implementing a stream processing component based on the new Akka IO. First of all, the new API is awesome. I have never done low-level TCP stuff before, and getting started was extremely easy.

However, under higher loads (beginning with 5k messages per second sent/received via a persistent TCP connection), the traffic suddenly "disappeared". There were no exception, no CommandFailed or other messages visible. The connection also was kept open, i.e., there were no ConnectionClosed messages, but also no data was sent or (obviously) received. After some debugging I found out that increasing the send and receive buffer sizes solves the problem, so I assume that one or both buffers overflew.

Is there some configuration option or trick to make such buffer overflows visible to consumers of the Akka IO API? At least during development that would be pretty helpful. Or are those buffers something handled by the OS and not visible from Java at all?

Thanks and best regards

Carsten

[Akka Team]

Hi Carsten,

Have you actually implemented proper flow-control? Please see http://doc.akka.io/docs/akka/2.2.0/scala/io-tcp.html#Throttling_Reads_and_Writes

If you do not handle back-pressure then you will lose data -- I am not sure that your problem is with buffer sizes.

-Endre

Carsten

--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ: http://akka.io/faq/
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com.
To post to this group, send email to akka...@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--

Akka Team

Typesafe - The software stack for applications that scale

Blog: letitcrash.com
Twitter: @akkateam

[Carsten Saathoff]

Hi Endre,

no, I haven't. Now after reading it again, it seems that I should have ;). Somehow the importance of flow-control didn't become apparent to me on the first read.

But anyway, I still think it would be useful to produce some kind of warning in the logs when packets are dropped if possible.

Thanks!

Carsten

[Akka Team]

Hi Carsten,

no, I haven't. Now after reading it again, it seems that I should have ;). Somehow the importance of flow-control didn't become apparent to me on the first read.

Yes we should be probably more "loud" about it in the docs ;)

 

But anyway, I still think it would be useful to produce some kind of warning in the logs when packets are dropped if possible.

Well, there is actually a warning. But it is in DEBUG level and only if akka.io.tcp.trace-logging is enabled :)

-Endre

[Roland Kuhn]

Akka Team <akka.o...@gmail.com> wrote:
>Hi Carsten,
>
>no, I haven't. Now after reading it again, it seems that I should have
>;).
>> Somehow the importance of flow-control didn't become apparent to me
>on the
>> first read.
>>
>
>Yes we should be probably more "loud" about it in the docs ;)
>
>
>>
>> But anyway, I still think it would be useful to produce some kind of
>> warning in the logs when packets are dropped if possible.
>>
>
>Well, there is actually a warning. But it is in DEBUG level and only if
>akka.io.tcp.trace-logging is enabled :)

Akka.io is supposed to be a network library like NIO (but more usable), and that one also does not print log statements to the console when a call fails: we only log those events which are not otherwise signaled to user code. Logging failed writes would get annoying real quick ;-)

Regards,

Roland

>
>-Endre
>
>
>>
>> Thanks!
>>
>> Carsten
>>
>> Am Freitag, 19. Juli 2013 13:53:55 UTC+2 schrieb Akka Team:
>>>
>>> Hi Carsten,
>>>
>>> Have you actually implemented proper flow-control? Please see

>>> http://doc.akka.io/docs/akka/**2.2.0/scala/io-tcp.html#**
>>>
>Throttling_Reads_and_Writes<http://doc.akka.io/docs/akka/2.2.0/scala/io-tcp.html#Throttling_Reads_and_Writes>

>>>> >>>>>>>>>> Search the archives: https://groups.google.com/**
>>>> group/akka-user <https://groups.google.com/group/akka-user>

>>>> ---
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Akka User List" group.
>>>> To unsubscribe from this group and stop receiving emails from it,
>send

>>>> an email to akka-user+...@**googlegroups.com.

>>>> To post to this group, send email to akka...@googlegroups.com.
>>>>
>>>> Visit this group at

>http://groups.google.com/**group/akka-user<http://groups.google.com/group/akka-user>
>>>> .
>>>> For more options, visit
>https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out>
>>>> .

>>>>
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Akka Team
>>> Typesafe - The software stack for applications that scale
>>> Blog: letitcrash.com
>>> Twitter: @akkateam
>>>
>> --
>> >>>>>>>>>> Read the docs: http://akka.io/docs/
>> >>>>>>>>>> Check the FAQ: http://akka.io/faq/
>> >>>>>>>>>> Search the archives:
>https://groups.google.com/group/akka-user
>> ---
>> You received this message because you are subscribed to the Google
>Groups
>> "Akka User List" group.
>> To unsubscribe from this group and stop receiving emails from it,
>send an
>> email to akka-user+...@googlegroups.com.
>> To post to this group, send email to akka...@googlegroups.com.
>> Visit this group at http://groups.google.com/group/akka-user.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>>

Regards,

Dr. Roland Kuhn
Akka Tech Lead
Typesafe – Reactive apps on the JVM
twitter: @rolandkuhn

[Carsten Saathoff]

Hi again,

just for the record: you were absolutely right. I just implemented flow control (ACK-based) and now it works without configuring the TCP.SendBuffer. Thanks for your help.

By the way, turning on the trace-logging indeed provides at least a hint what is going wrong. Maybe it would make sense to mention that option in the docs somewhere under Troubleshooting or the like.

best

Carsten

Am Freitag, 19. Juli 2013 13:53:55 UTC+2 schrieb Akka Team: