akka-http and closing connections from the client side

[Rüdiger Klaehn]

Hi all,

I am doing some testing with akka-http and chunked encoding. I want to
investigate the backpressure behavior, so I am using curl with
--limit-rate 1k on the client side. I should I see backpressure when
using akka-http to send a HttpEntity.Chunked, right?

One thing I noticed is that all hell breaks loose when the connection
is closed from the client side (e.g. by interrupting curl using
Control-C). See below. Is that going to change with 1.0, or is there a
way to silence this?

Also, is the namespace for the scala dsl going to stay scaladsl**2**?
That would be weird.

cheers,

Rüdiger

---

FINE: 2014/10/28 10:12:16:484:
satmonjavaservices.akka.EventListener$$anonfun$receive$1: Closing
connection due to IO error java.io.IOException: Connection reset by
peer, source: akka://Server/system/IO-TCP/selectors/$a/39class: class
akka.io.TcpIncomingConnection
WARNING: 2014/10/28 10:12:16:484:
satmonjavaservices.server.SatmonJavaServices$$anonfun$1$$anon$4$$anonfun$receive$1:
DeadLetter(ChannelReadable,Actor[akka://Server/deadLetters],Actor[akka://Server/system/IO-TCP/selectors/$a/39#-1098197434])
SEVERE: 2014/10/28 10:12:16:484:
satmonjavaservices.akka.EventListener$$anonfun$receive$1: failure
during processing, source:
akka://Server/user/IO-HTTP/$a/flow-552-1-rootParserclass: class
akka.stream.impl.TransformProcessorImpl:
akka.stream.io.TcpStreamActor$TcpStreamException: The connection
closed with error Connection reset by peer

WARNING: 2014/10/28 10:12:16:484:
satmonjavaservices.server.SatmonJavaServices$$anonfun$1$$anon$4$$anonfun$receive$1:
DeadLetter(Abort,Actor[akka://Server/system/IO-TCP-STREAM/server-1-%2F0.0.0.0%3A8888/$M#801209365],Actor[akka://Server/system/IO-TCP/selectors/$a/39#-1098197434])
SEVERE: 2014/10/28 10:12:16:484:
satmonjavaservices.akka.EventListener$$anonfun$receive$1: failure
during processing, source:
akka://Server/user/IO-HTTP/$a/flow-552-2-splitWhenclass: class
akka.stream.impl2.SplitWhenProcessorImpl:
akka.stream.io.TcpStreamActor$TcpStreamException: The connection
closed with error Connection reset by peer

SEVERE: 2014/10/28 10:12:16:484:
satmonjavaservices.akka.EventListener$$anonfun$receive$1: failure
during processing, source:
akka://Server/user/IO-HTTP/$a/flow-555-1-prefixAndTailclass: class
akka.stream.impl2.PrefixAndTailImpl:
akka.stream.io.TcpStreamActor$TcpStreamException: The connection
closed with error Connection reset by peer

WARNING: 2014/10/28 10:12:16:484:
satmonjavaservices.server.SatmonJavaServices$$anonfun$1$$anon$4$$anonfun$receive$1:
DeadLetter(Terminate(),Actor[akka://Server/user/IO-HTTP/$a/flow-560-0-iterator#435212900],Actor[akka://Server/user/IO-HTTP/$a/flow-560-0-iterator#435212900])
SEVERE: 2014/10/28 10:12:16:484:
satmonjavaservices.akka.EventListener$$anonfun$receive$1: failure
during processing, source:
akka://Server/user/IO-HTTP/$a/flow-552-3-mapclass: class
akka.stream.impl.TransformProcessorImpl:
akka.stream.io.TcpStreamActor$TcpStreamException: The connection
closed with error Connection reset by peer

WARNING: 2014/10/28 10:12:16:484:
satmonjavaservices.server.SatmonJavaServices$$anonfun$1$$anon$4$$anonfun$receive$1:
DeadLetter(Cancel(akka.stream.impl.ActorSubscription@4a9cf35),Actor[akka://Server/deadLetters],Actor[akka://Server/user/IO-HTTP/$a/flow-552-4-concatFlatten#1535190045])
SEVERE: 2014/10/28 10:12:16:484:
satmonjavaservices.akka.EventListener$$anonfun$receive$1: failure
during processing, source:
akka://Server/user/IO-HTTP/$a/flow-552-4-concatFlattenclass: class
akka.stream.impl2.ConcatAllImpl:
akka.stream.io.TcpStreamActor$TcpStreamException: The connection
closed with error Connection reset by peer

SEVERE: 2014/10/28 10:12:16:484:
satmonjavaservices.akka.EventListener$$anonfun$receive$1: failure
during processing, source:
akka://Server/user/IO-HTTP/$a/flow-547-broadcastclass: class
akka.stream.impl2.Broadcast:
akka.stream.io.TcpStreamActor$TcpStreamException: The connection
closed with error Connection reset by peer

SEVERE: 2014/10/28 10:12:16:484:
satmonjavaservices.akka.EventListener$$anonfun$receive$1: failure
during processing, source:
akka://Server/user/IO-HTTP/$a/flow-548-1-collectclass: class
akka.stream.impl.TransformProcessorImpl:
akka.stream.io.TcpStreamActor$TcpStreamException: The connection
closed with error Connection reset by peer

SEVERE: 2014/10/28 10:12:16:484:
satmonjavaservices.akka.EventListener$$anonfun$receive$1: failure
during processing, source:
akka://Server/user/$b/flow-554-1-mapAsyncUnorderedclass: class
akka.stream.impl2.MapAsyncUnorderedProcessorImpl:
akka.stream.io.TcpStreamActor$TcpStreamException: The connection
closed with error Connection reset by peer

[Akka Team]

Hi Rüdiger,

I am doing some testing with akka-http and chunked encoding. I want to
investigate the backpressure behavior, so I am using curl with
--limit-rate 1k on the client side. I should I see backpressure when
using akka-http to send a HttpEntity.Chunked, right?

Yes, you should.

 

One thing I noticed is that all hell breaks loose when the connection
is closed from the client side (e.g. by interrupting curl using
Control-C). See below. Is that going to change with 1.0, or is there a
way to silence this?

It is just the onError signal caused by "connection reset by peer" propagating downwards, and all elements log it (which looks scary and is completely useless). We will remove this logging completely, there is at least one ticket for that: https://github.com/akka/akka/issues/15349

 

Also, is the namespace for the scala dsl going to stay scaladsl**2**?
That would be weird.

No: https://github.com/akka/akka/pull/16154/

-Endre

 

--
>>>>>>>>>>      Read the docs: http://akka.io/docs/
>>>>>>>>>>      Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>>      Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com.
To post to this group, send email to akka...@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

--

Akka Team

Typesafe - The software stack for applications that scale

Blog: letitcrash.com
Twitter: @akkateam

[Rüdiger Klaehn]

On Tue, Oct 28, 2014 at 10:40 AM, Akka Team <akka.o...@gmail.com> wrote:
> Hi Rüdiger,
>
>> I am doing some testing with akka-http and chunked encoding. I want to
>> investigate the backpressure behavior, so I am using curl with
>> --limit-rate 1k on the client side. I should I see backpressure when
>> using akka-http to send a HttpEntity.Chunked, right?
>
>
> Yes, you should.
>

Yes, my mistake. It seems that there are some buffers that are being
filled before the backpressure kicks in, so you have to make sure to
produce enough data to see the effect.

>>
>>
>> One thing I noticed is that all hell breaks loose when the connection
>> is closed from the client side (e.g. by interrupting curl using
>> Control-C). See below. Is that going to change with 1.0, or is there a
>> way to silence this?
>
>
> It is just the onError signal caused by "connection reset by peer"
> propagating downwards, and all elements log it (which looks scary and is
> completely useless). We will remove this logging completely, there is at
> least one ticket for that: https://github.com/akka/akka/issues/15349
>

OK. That's good to know.

>>
>>
>> Also, is the namespace for the scala dsl going to stay scaladsl**2**?
>> That would be weird.
>
>
> No: https://github.com/akka/akka/pull/16154/
>

Great. Thanks. By the way, is http client functionality going to be
included in 1.0, or is that planned for a later release?

Cheers,

Rüdiger

[Akka Team]

> Yes, you should.
>
Yes, my mistake. It seems that there are some buffers that are being
filled before the backpressure kicks in, so you have to make sure to
produce enough data to see the effect.

That is a likely explanation. Nevertheless if it still does not work then we have a bug.

 

> No: https://github.com/akka/akka/pull/16154/
>
Great. Thanks. By the way, is http client functionality going to be
included in 1.0, or is that planned for a later release?

Yes it is planned for 1.0

-Endre

 

Cheers,

Rüdiger

--
>>>>>>>>>>      Read the docs: http://akka.io/docs/
>>>>>>>>>>      Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>>      Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com.
To post to this group, send email to akka...@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[dan costelloe]

Hi Rüdiger,

On Tuesday, 28 October 2014 09:22:07 UTC, rklaehn wrote:

Hi all,

I am doing some testing with akka-http and chunked encoding. I want to
investigate the backpressure behavior, so I am using curl with
--limit-rate 1k on the client side. I should I see backpressure when
using akka-http to send a HttpEntity.Chunked, right?

For the purposes of educating a newbie, would you be so kind as to share your code and curl settings for this test?  

 

Much obliged!

dan

[Rüdiger Klaehn]

Here is a gist of a small file server that uses chunked encoding. You
should be able to see the backpressure work with this.

https://gist.github.com/rklaehn/3f26c3f80e5870831f52#file-file-server-example

If you run it, it opens a server on localhost, port 8080, that serves
all files on your system. So just put some large text file somewhere
in your home directory, start the server, and get it with curl like
this:

curl --limit-rate 1k http://127.0.0.1:8080/Users/rklaehn/tmp/somelargetext.txt

You can see on the server side that new chunks are produced at the
rate that curl consumes them => backpressure.

Warning: I wrote this before I was really familiar with akka-streams.
So the code might be not exactly idiomatic.

Hth,

Rüdiger

[dan costelloe]

Many thanks for posting this. 

[Roland Kuhn]

Hi Rüdiger,

thanks for sharing! Since you ask, there is one thing that might be improved (though not problematic in your example): when constructing a Source from an Iterator the Iterator should be created by the factory that is provided, so that the Source may be reused (remember our mantra: everything in the DSL shall serve as reusable blueprints wherever possible). This would mean:

val chunks = Source(() => new ByteBufferIterator(mappedByteBuffer, 4096)).map(ChunkStreamPart)

Regards,

Roland

Dr. Roland Kuhn
Akka Tech Lead
Typesafe – Reactive apps on the JVM.
twitter: @rolandkuhn

[dan costelloe]

Actually I was going to ask a follow-up question on developing this code -- and now that the Akka tech-lead has got involved, maybe there's no better time ;-) -- I'd like to try and adapt it to receive files via chunked POST. 

I'm wondering if it is possible to dynamically adjust the upload rate of a single request using a flow that employs an ActorSubscriber[ByteString]. Before I start wandering in circles -- is this sort of thing even possible using akka-http?

Thanks again for the foundation.

[dan costelloe]

I've expanded on this question in a new thread.. https://groups.google.com/forum/#!topic/akka-user/iw_fOzWvDR4