AsyncHttpClient.close() hasn't been invoked, which may produce file descriptor leaks

[Timothy Perrett]

Hey guys,

I'm seeing this notice when my application exits: 

AsyncHttpClient.close() hasn't been invoked, which may produce file descriptor leaks

If I execute in the SBT shell, it continues to spit out things like: 

[DEBUG] Entry count for : http://myurl.com : 1

[DEBUG] Adding Candidate Idle Channel [id: 0x42c505f7, /<someip>:59334 => <myurl.com>/<myip>:<myport>]

[DEBUG] Closing Idle Channel [id: 0x42c505f7, /<someip>:59334 => <myurl.com>/<myip>:<myport>]

[DEBUG] Channel Closed: [id: 0x42c505f7, /<someip>:59334 :> <myurl.com>/<myip>:<myport>] with attachment com.ning.http.client.providers.netty.NettyAsyncHttpProvider$DiscardEvent@7fcaa2ba

[DEBUG] Entry count for : http://myurl.com : 0

[DEBUG] Entry count for : http://myurl.com : 0

[DEBUG] Entry count for : http://myurl.com : 0

And it just continues to spit out those logs if I leave the shell running. I can press enter at any time and continue working, but it indicates there are still some threads running in the background. I've tried to clean up explicitly using dispatch.Http.shutdown() but that actually seems to make matters worse, as the application then blocks and wont let me hit enter to continue working. 

I cant think how else I can shut this down? Looking at the async-http-client source it seems dispatch already calls its close method internally...

Thoughts?

Cheers, Tim

[Timothy Perrett]

And oh, this *only* happens when I use a proxy with the following configuration:

  import com.ning.http.client.{Response,ProxyServer,AsyncHttpClientConfig}, 

    AsyncHttpClientConfig.Builder

  private def withProxy(b: Builder): Builder =

    (for {

      host <- Option(System.getenv("proxy_host"))

      port <- Option(System.getenv("proxy_port")).map(_.toInt)

    } yield b.setProxyServer(new ProxyServer(host, port))).getOrElse(b)

// later

Http.configure(withProxy)(url(resource) OK handler).either

[Nathan Hamblen]

Hi Tim,

As you probably know it is a struggle to make background threads
cooperate with sbt's interactive mode, especially when working with
underlying java libraries that do not have this concern. In Dispatch
0.9.3 [1] I weeded out every background thread I could find and made it
daemon, which did the trick for typcial operation.

I would guess that this class is spinning off a thread:
com.ning.http.client.ProxyServer

[1]: http://notes.implicit.ly/post/34495822045/dispatch-0-9-3

Nathan

On 01/29/2013 06:17 PM, Timothy Perrett wrote:
> And oh, this *only* happens when I use a proxy with the following
> configuration:
>
> import
> com.ning.http.client.{Response,ProxyServer,AsyncHttpClientConfig},
> AsyncHttpClientConfig.Builder
>
> private def withProxy(b: Builder): Builder =
> (for {
> host <- Option(System.getenv("proxy_host"))
> port <- Option(System.getenv("proxy_port")).map(_.toInt)
> } yield b.setProxyServer(new ProxyServer(host, port))).getOrElse(b)
>
> // later
> Http.configure(withProxy)(url(resource) OK handler).either
>
> On Tuesday, 29 January 2013 15:14:23 UTC-8, Timothy Perrett wrote:
>
> Hey guys,
>
> I'm seeing this notice when my application exits:
> AsyncHttpClient.close() hasn't been invoked, which may produce file
> descriptor leaks
>
> If I execute in the SBT shell, it continues to spit out things like:
>
> [DEBUG] Entry count for : http://myurl.com : 1
> [DEBUG] Adding Candidate Idle Channel [id: 0x42c505f7,

> /<someip>:59334 => <myurl.com <http://myurl.com>>/<myip>:<myport>]

> [DEBUG] Closing Idle Channel [id: 0x42c505f7, /<someip>:59334

> => <myurl.com <http://myurl.com>>/<myip>:<myport>]

> [DEBUG] Channel Closed: [id: 0x42c505f7, /<someip>:59334

> :> <myurl.com <http://myurl.com>>/<myip>:<myport>] with attachment

> com.ning.http.client.providers.netty.NettyAsyncHttpProvider$DiscardEvent@7fcaa2ba
> [DEBUG] Entry count for : http://myurl.com : 0
> [DEBUG] Entry count for : http://myurl.com : 0
> [DEBUG] Entry count for : http://myurl.com : 0
>
> And it just continues to spit out those logs if I leave the shell
> running. I can press enter at any time and continue working, but it
> indicates there are still some threads running in the background.
> I've tried to clean up explicitly using dispatch.Http.shutdown() but
> that actually seems to make matters worse, as the application then
> blocks and wont let me hit enter to continue working.
>
> I cant think how else I can shut this down? Looking at the
> async-http-client source it seems dispatch already calls its close
> method internally...
>
> Thoughts?
>
> Cheers, Tim
>
>

> --
> You received this message because you are subscribed to the Google
> Groups "Dispatch" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to dispatch-scal...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Tim Perrett]

Hey Nathan!

That was originally my thought, but I trawled the source code and ProxyServer is a simple data class that doesn't appear to spawn anything. Moreover, in all the places its used it doesn't appear to do much either... It's odd.

I need to look into it more, but if I call configure in the way I outlined above, it should still use the default configuration supplied by dispatch, right? 

I was wondering if I might be doing something that is somehow screwing the default config?

Cheers

an email to dispatch-scala+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "Dispatch" group.

To unsubscribe from this group and stop receiving emails from it, send an email to dispatch-scala+unsubscribe@googlegroups.com.

[Nathan Hamblen]

On 01/30/2013 01:06 AM, Tim Perrett wrote:

I need to look into it more, but if I call configure in the way I outlined above, it should still use the default configuration supplied by dispatch, right?

Well... I don't see how it would leak threads, but it isn't exactly what I had in mind with `configure`. Instead I would keep a reference to the HttpExecutor instance returned by `configure`, and use that for all subsequent calls. (But if that's the one and only call, it would make no difference.)

Nathan

an email to dispatch-scal...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "Dispatch" group.

To unsubscribe from this group and stop receiving emails from it, send an email to dispatch-scal...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "Dispatch" group.

To unsubscribe from this group and stop receiving emails from it, send an email to dispatch-scal...@googlegroups.com.

[Timothy Perrett]

Huh, good to know!

So one should really use like:

val h = Http.configure(...).threads(…)

// later

h(url(…) OK as.String)

Something like that? Rather than using Http.apply at every call site?

Cheers

[Nathan Hamblen]

That's right, assuming you do need to configure it.

Nathan

[Benjamin Maus]

Luckily I found this thread. I was reconfiguring for every request I made depending on a couple of circumstances (eg. stream vs. call and response). At some points the active threads were running away, ending up crashing the JVM. 

Now I configure two times only, but only once each. Everything is fine and dandy that way.

Should be mentioned somewhere in the docs...

[Helen Williamson]

Same thing happened to me - was wondering why I was seeing so many entries for timers from NettyConnectionsPool.

Would be really useful to have this clearly explained in the documentation as to why you should use the pattern above.

Helen

[Nathan Hamblen]

Pull requests for documentation are heartily encouraged. :-)

[Sawyer]

I think I am getting the exact problem as Tim said in the title of the post:

AsyncHttpClient.close() hasn't been invoked, which may produce file descriptor leaks

At first I thought it was AKKA. but now I think it's dispatch.
My server has open files limits of 1024, I when I started Dispatch, it quickly used up all the file descriptors. At first I thought it would be solved after increasing the open file limits. After changes to 6000, it used up in a few hours.

I am only sending several http requests, and from the output of lsof -p java-pid, I saw huge amount of output like this:

java    17978  2375r  0000               0,11        0   229587 eventpoll
java    17978  2376r  FIFO                0,6            229588  pipe

I actually don't need connection pool and not even need to send the http request asynchronously, so I use i like:

def getConfigBuilder = new AsyncHttpClientConfig.Builder().setAllowPoolingConnection(false)

val http = Http.configure(_ => getConfigBuilder)
http(url(u) OK as.String).apply()

Is there a way to send a request and after it's returned just close it in a clean way?

Thanks!

[Nathan Hamblen]

On 06/12/2013 11:36 PM, Sawyer wrote:
> def getConfigBuilder = new
> AsyncHttpClientConfig.Builder().setAllowPoolingConnection(false)
>
> val http = Http.configure(_ => getConfigBuilder)
> http(url(u) OK as.String).apply()
>
> Is there a way to send a request and after it's returned just close it
> in a clean way?

It's unclear if your `http` val is shared, but in general, it should be.
dispatch.Http and specifically its underlying async-http-client are not
designed to be instantiated and discarded many times. Aside from
connection-pooling there are several executors that manage the async
i/o. It's just not efficient to create and destroy them over and over,
and worse, I can't say that *n* instances will behave well for all
values of n, even if you call close() on each one.

Which is a long way of saying, please use lazy val on a singleton object
or some other facility to for initting one Http executor. Or as many
executors as you have unique configurations.

Nathan