Graceful Timeout and defaults

[Daniel Natic]

We are seeing a lot of weird errors as our client's work load ends.  This leads our team to believe that we are disconnecting the client prior to all responses returning.  We just found the gracefulTimeout setting and will begin to experiment with it.  Is there a default or recommended value for this?  We are a very high throughput use case.

An off-topic question:  Is there a maximum bound to the number of pending requests in the HTTP/2 multiplex pipeline?

Thanks for your help!

Dan Natic

The Weather Channel

[Daniel Natic]

I should have been more specific.  We are seeing a lot of timeout exceptions.

2016-04-28 03:58:20,390 INFO c.w.d.d.a.ApnsHttp2PostActor [ActorManager-akka.actor.default-dispatcher-24385] <- DC [Sent] : (Chan=iphone-free, Prod=lightning, UserIds=UZg_PsiShsYOz, Content={"aps":{"alert":{"body":"Lightning )

2016-04-28 03:58:20,407 INFO c.r.p.a.ApnsClient [nioEventLoopGroup-77-1] Connected to api.push.apple.com/17.143.164.78:2197.

2016-04-28 03:58:20,498 INFO c.w.d.d.a.ApnsHttp2PostActor [ActorManager-akka.actor.default-dispatcher-24320] <- DC [Sent] : (Chan=iphone-free, Prod=lightning, UserIds=QIpHHRprq62Usqz0OqlI0d, Content={"aps":{"alert":{"body":"Lightning )

2016-04-28 03:58:21,260 INFO c.w.d.d.WnsPushActor [ActorManager-akka.actor.default-dispatcher-24380] <- DC [Sent] : (Chan=wns-desktop, Prod=lightning, UserIds=95Vkat9unyJ5c, Content=<toast launch="{&quot;type&quot;:&q)

2016-04-28 03:58:21,260 WARN c.w.d.ActorManager$ [ActorManager-akka.actor.default-dispatcher-24380] Event Client [IGNORED]: Events disabled!  See distrib.events.enabled config key.

2016-04-28 03:58:29,727 ERROR c.w.d.d.a.ApnsHttp2PostActor [ActorManager-akka.actor.default-dispatcher-24379] <* DC [Failed w/excpt] : Timed out after 10000 milliseconds (Chan=iphone-free, Prod=lightning, UserIds=AYCEdmZZfgcYy)

java.util.concurrent.TimeoutException: Timed out after 10000 milliseconds

at scalaz.concurrent.Future$$anonfun$unsafePerformSyncAttemptFor$2.apply(Future.scala:225) ~[Distribution-Server-2.3.8-RELEASE-jar-with-dependencies.jar:?]

at scalaz.concurrent.Future$$anonfun$unsafePerformSyncAttemptFor$2.apply(Future.scala:223) ~[Distribution-Server-2.3.8-RELEASE-jar-with-dependencies.jar:?]

at scala.Option.getOrElse(Option.scala:120) ~[Distribution-Server-2.3.8-RELEASE-jar-with-dependencies.jar:?]

at scalaz.concurrent.Future.unsafePerformSyncAttemptFor(Future.scala:223) ~[Distribution-Server-2.3.8-RELEASE-jar-with-dependencies.jar:?]

at scalaz.concurrent.Task.unsafePerformSyncAttemptFor(Task.scala:202) ~[Distribution-Server-2.3.8-RELEASE-jar-with-dependencies.jar:?]

at com.weather.distribution.distributor.apns.ApnsHttp2PostActor.sendDC(ApnsHttp2PostActor.scala:144) [Distribution-Server-2.3.8-RELEASE-jar-with-dependencies.jar:?]

at com.weather.distribution.distributor.apns.ApnsHttp2PostActor$$anonfun$receive$1.applyOrElse(ApnsHttp2PostActor.scala:180) [Distribution-Server-2.3.8-RELEASE-jar-with-dependencies.jar:?]

at akka.actor.Actor$class.aroundReceive(Actor.scala:467) [Distribution-Server-2.3.8-RELEASE-jar-with-dependencies.jar:?]

at com.weather.distribution.distributor.apns.ApnsHttp2PostActor.aroundReceive(ApnsHttp2PostActor.scala:38) [Distribution-Server-2.3.8-RELEASE-jar-with-dependencies.jar:?]

at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516) [Distribution-Server-2.3.8-RELEASE-jar-with-dependencies.jar:?]

at akka.actor.ActorCell.invoke(ActorCell.scala:487) [Distribution-Server-2.3.8-RELEASE-jar-with-dependencies.jar:?]

at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:254) [Distribution-Server-2.3.8-RELEASE-jar-with-dependencies.jar:?]

at akka.dispatch.Mailbox.run(Mailbox.scala:221) [Distribution-Server-2.3.8-RELEASE-jar-with-dependencies.jar:?]

at akka.dispatch.Mailbox.exec(Mailbox.scala:231) [Distribution-Server-2.3.8-RELEASE-jar-with-dependencies.jar:?]

at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [Distribution-Server-2.3.8-RELEASE-jar-with-dependencies.jar:?]

at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [Distribution-Server-2.3.8-RELEASE-jar-with-dependencies.jar:?]

at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [Distribution-Server-2.3.8-RELEASE-jar-with-dependencies.jar:?]

at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [Distribution-Server-2.3.8-RELEASE-jar-with-dependencies.jar:?]

2016-04-28 03:58:29,825 INFO c.r.p.a.ApnsClient [nioEventLoopGroup-13-1] Failed to connect.

javax.net.ssl.SSLException: handshake timed out

2016-04-28 03:58:29,825 WARN i.n.h.s.ApplicationProtocolNegotiationHandler [nioEventLoopGroup-13-1] [id: 0x724c1e59, L:/10.224.193.4:50722 - R:api.push.apple.com/17.143.164.78:2197] TLS handshake failed:

javax.net.ssl.SSLException: handshake timed out

2016-04-28 03:58:30,059 INFO c.r.p.a.ApnsClient [ActorManager-akka.actor.default-dispatcher-24370] Disconnecting.

2016-04-28 03:58:30,059 INFO c.r.p.a.ApnsClient [ActorManager-akka.actor.default-dispatcher-24343] Disconnecting.

2016-04-28 03:58:30,059 INFO c.r.p.a.ApnsClient [ActorManager-akka.actor.default-dispatcher-24320] Disconnecting.

2016-04-28 03:58:30,069 INFO c.r.p.a.ApnsClient [ActorManager-akka.actor.default-dispatcher-24370] Disconnecting.

2016-04-28 03:58:30,079 INFO c.r.p.a.ApnsClient [ActorManager-akka.actor.default-dispatcher-24370] Disconnecting.

2016-04-28 03:58:30,089 INFO c.r.p.a.ApnsClient [ActorManager-akka.actor.default-dispatcher-24370] Disconnecting.

2016-04-28 03:58:30,109 INFO c.r.p.a.ApnsClient [ActorManager-akka.actor.default-dispatcher-24370] Disconnecting.

2016-04-28 03:58:30,109 INFO c.r.p.a.ApnsClient [ActorManager-akka.actor.default-dispatcher-24370] Disconnecting.

2016-04-28 03:58:30,139 INFO c.r.p.a.ApnsClient [ActorManager-akka.actor.default-dispatcher-24370] Disconnecting.

2016-04-28 03:58:30,379 INFO c.r.p.a.ApnsClient [ActorManager-akka.actor.default-dispatcher-24370] Disconnecting.

2016-04-28 03:58:30,389 INFO c.r.p.a.ApnsClient [ActorManager-akka.actor.default-dispatcher-24370] Disconnecting.

2016-04-28 03:58:30,399 INFO c.r.p.a.ApnsClient [ActorManager-akka.actor.default-dispatcher-24370] Disconnecting.

2016-04-28 03:58:30,399 INFO c.r.p.a.ApnsClient [ActorManager-akka.actor.default-dispatcher-24370] Disconnecting.

2016-04-28 03:58:30,409 INFO c.r.p.a.ApnsClient [ActorManager-akka.actor.default-dispatcher-24370] Disconnecting.

2016-04-28 03:58:30,509 INFO c.r.p.a.ApnsClient [ActorManager-akka.actor.default-dispatcher-24370] Disconnecting.

[j...@turo.com]

I'm not entirely sure what we're looking at here (just by virtue of not knowing much about your application!), but I'm very curious about the TLS handshake timeout. Could the two be related? I'm wondering if there's a scenario where you're creating a client and attempting to send notifications before the handshake completes, and it's the handshake timeout that's causing notifications to time out. On a related note, are you sure that it's a notification send attempt that's timing out?

To your other questions, the default graceful shutdown timeout is quite long—30 seconds, in fact. I'd expect that to be plenty long for mere mortals. There isn't a limit (beyond available memory) to the number of push notifications that can be sitting in an ApnsClient's queue, though; depending on your use case, it might be worth keeping track of how many notifications you have "in flight" to see if there's an enormous quantity still in play when you start your shutdown process.

Does that make sense?

-Jon

[Daniel Natic]

Ever since we moved up to the latest version of pushy with 30 seconds graceful, we've seen significantly less exceptions for timeouts.  However, we're still experiencing really high rates of connection failures.  Most of them are "IllegalStateException".  Some are "SSLHandshakeException".  We force the HTTP/2 connection to close after 10 seconds of being idle.  Is there a recommended time to disconnect?  I was going to move it up to one or two hours and see what happens.  If I can stop so many connection exceptions from occurring, I bet I will see much better numbers than below.

  Apns/Internal/ConnectionAttemptFailed: 8502

  Apns/Internal/ConnectionAttemptStarted: 56645

  Apns/Internal/ConnectionAttemptSucceeded: 48143

  Apns/Internal/NotificationAccepted: 1592272

  Apns/Internal/NotificationRejected: 19091

  Apns/Internal/NotificationSent: 1611363

Thanks!

Daniel Natic

Managing Engineer

The Weather Channel

2016-05-04 01:59:46,338 INFO c.r.p.a.ApnsClient [nioEventLoopGroup-21-1] Failed to connect.

java.lang.IllegalStateException: Channel closed before HTTP/2 preface completed.

at com.relayrides.pushy.apns.ApnsClient$2.operationComplete(ApnsClient.java:708) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at com.relayrides.pushy.apns.ApnsClient$2.operationComplete(ApnsClient.java:700) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:683) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:568) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:407) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:82) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.channel.AbstractChannel$CloseFuture.setClosed(AbstractChannel.java:1064) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.channel.AbstractChannel$AbstractUnsafe.doClose0(AbstractChannel.java:689) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.channel.AbstractChannel$AbstractUnsafe.close(AbstractChannel.java:667) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.channel.AbstractChannel$AbstractUnsafe.close(AbstractChannel.java:611) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.channel.DefaultChannelPipeline$HeadContext.close(DefaultChannelPipeline.java:1416) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.channel.ChannelHandlerInvokerUtil.invokeCloseNow(ChannelHandlerInvokerUtil.java:133) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.channel.DefaultChannelHandlerInvoker.invokeClose(DefaultChannelHandlerInvoker.java:297) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.channel.AbstractChannelHandlerContext.invokeClose(AbstractChannelHandlerContext.java:379) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.channel.AbstractChannelHandlerContext.close(AbstractChannelHandlerContext.java:226) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.channel.ChannelOutboundHandlerAdapter.close(ChannelOutboundHandlerAdapter.java:71) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.channel.ChannelHandlerInvokerUtil.invokeCloseNow(ChannelHandlerInvokerUtil.java:133) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.channel.DefaultChannelHandlerInvoker.invokeClose(DefaultChannelHandlerInvoker.java:297) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.channel.AbstractChannelHandlerContext.invokeClose(AbstractChannelHandlerContext.java:379) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.channel.AbstractChannelHandlerContext.close(AbstractChannelHandlerContext.java:226) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.handler.ssl.SslHandler.closeOutboundAndChannel(SslHandler.java:1250) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.handler.ssl.SslHandler.close(SslHandler.java:459) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.channel.ChannelHandlerInvokerUtil.invokeCloseNow(ChannelHandlerInvokerUtil.java:133) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.channel.DefaultChannelHandlerInvoker.invokeClose(DefaultChannelHandlerInvoker.java:297) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.channel.AbstractChannelHandlerContext.invokeClose(AbstractChannelHandlerContext.java:379) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.channel.AbstractChannelHandlerContext.close(AbstractChannelHandlerContext.java:226) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.channel.AbstractChannelHandlerContext.close(AbstractChannelHandlerContext.java:185) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.channel.DefaultChannelPipeline.close(DefaultChannelPipeline.java:1119) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.channel.AbstractChannel.close(AbstractChannel.java:224) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.channel.ChannelFutureListener$2.operationComplete(ChannelFutureListener.java:56) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.channel.ChannelFutureListener$2.operationComplete(ChannelFutureListener.java:52) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:683) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:604) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:564) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:425) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe$1.run(AbstractNioChannel.java:286) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:120) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:339) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:373) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:742) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:145) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at java.lang.Thread.run(Thread.java:722) [?:1.7.0_17]

2016-05-04 01:54:11,305 WARN i.n.c.DefaultChannelPipeline [nioEventLoopGroup-159-1] An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.

io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: error:14094416:SSL routines:ssl3_read_bytes:sslv3 alert certificate unknown

at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:418) ~[Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:245) ~[Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.channel.ChannelHandlerInvokerUtil.invokeChannelReadNow(ChannelHandlerInvokerUtil.java:83) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.channel.DefaultChannelHandlerInvoker.invokeChannelRead(DefaultChannelHandlerInvoker.java:154) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:354) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:145) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:1078) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:117) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:527) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:484) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:398) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:370) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:742) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:145) [Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at java.lang.Thread.run(Thread.java:722) [?:1.7.0_17]

Caused by: javax.net.ssl.SSLHandshakeException: error:14094416:SSL routines:ssl3_read_bytes:sslv3 alert certificate unknown

at io.netty.handler.ssl.OpenSslEngine.shutdownWithError(OpenSslEngine.java:575) ~[Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.handler.ssl.OpenSslEngine.sslReadErrorResult(OpenSslEngine.java:778) ~[Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.handler.ssl.OpenSslEngine.unwrap(OpenSslEngine.java:733) ~[Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.handler.ssl.OpenSslEngine.unwrap(OpenSslEngine.java:810) ~[Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.handler.ssl.OpenSslEngine.unwrap(OpenSslEngine.java:853) ~[Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1098) ~[Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:970) ~[Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:904) ~[Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:387) ~[Distribution-Server-2.3.11-RELEASE-jar-with-dependencies.jar:?]

... 14 more

[Jon Chambers]

I thiiiiiiiiink I might be a little confused about the situation. I hate to do a point-by-point response, but I'm hoping that doing so will help us figure out where the disconnect is.

> Ever since we moved up to the latest version of pushy with 30 seconds graceful, we've seen significantly less exceptions for timeouts.

To be clear, the graceful timeout hasn't changed, and has always been 30 seconds. What version are you using now?

> However, we're still experiencing really high rates of connection failures.  Most of them are "IllegalStateException".  Some are "SSLHandshakeException".

That sounds like the server is just rejecting connections, which is surprising. There was some discussion of a related problem on the forums (see https://forums.developer.apple.com/thread/44677). I wonder if that could still be in effect here.

> We force the HTTP/2 connection to close after 10 seconds of being idle.  Is there a recommended time to disconnect?

Now that the APNs protocol is based on HTTP/2 and provides a mechanism for checking connection health when idle (via PING frames), our recommendation is actually to close connections only when you're 100% done with them (i.e. you're shutting down your whole application). It should be safe to leave connections open indefinitely.

Back before the HTTP/2 days (herein referred to as "The Dark Times"), there was no way to check connection health, and also no way to tell the difference between "I sent a notification and everything is fine" and "I sent a notification but it didn't go anywhere." Under those circumstances, we recommended closing connections frequently because that was the only way to reduce the probability of a quietly-dead connection. That's no longer necessary, though, and we recommend against it.

Hope that helps!

-Jon

--
Pushy is an open-source Java library for sending APNs (iOS and OS X) push notifications. Pushy is brought to you by the engineers at RelayRides.
---
You received this message because you are subscribed to the Google Groups "pushy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pushy-apns+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Daniel Natic]

We are going to move up to 0.7.2 on our release this week.  We are going to take out all of the disconnection logic and let you know how it goes.  We expect it will pretty much mitigate all of our issues when we do so.

[j...@turo.com]

Great! Looking forward to hearing how it goes.

-Jon

[Daniel Natic]

That certainly did it!  Not one message failed or timed out for the entire day versus 1-3% prior to the changes you recommended.  Thank you for your help.

[Jon Chambers]

Awesome. Glad to hear it!

-Jon