Problem with persisting a DDP client connection to to a meteor project deployed to a .meteor.com

[Peter Kutrumbos]

I have an application that involves a java DDP client sending updates to a Meteor application deployed using 'meteor deploy'.  If I run the applications long enough (8-12 hours), the Meteor app will stop responding to the DDP client method calls however I do not get any notification of an onError or onClose event over the web socket connection in my DDP client and if I create a new DDP client and make a new connection to the Meteor server it works fine.  

If I run the meteor project locally (i.e. not deployed to meteor.com), I never see this behavior and the connection will last as long as I leave the processes running.  I believe that this behavior relates to how the Meteor projects are deployed, my only need is some type of event/notification that the DDP client connection has a problem so my applications can be alerted and recover properly.

I realize this may be an issue with my underlying web-socket code used in the DDP client, however using that same client I do get the appropriate onError and onClose messages during other scenarios such as when I redeploy the meteor server or if my DDP client loses internet connection.

Has anyone had a similar experience or ideas on how to fix it or test this behavior better?

I've been able to consistently reproduce this behavior using the two projects below by running the TestDdpClient meteor project both locally and deployed (note it takes 8+ hours before it happens typically).

https://github.com/kutrumbo/java-ddp-client

https://github.com/kutrumbo/TestDdpClient

Thanks,

Pete

[Avital Oliver]

When deployed on meteor.com, apps are turned off after an hour of not being accessed. Unfortunately open DDP connections over websockets don't count as accessed for this purpose. The easiest workaround at the moment is to just curl your app once every 30m (anything less than an hour).

Going forward, our hosting solution will have a more explicit SLA.

--
You received this message because you are subscribed to the Google Groups "meteor-talk" group.
To view this discussion on the web visit https://groups.google.com/d/msg/meteor-talk/-/IBU5NWzKP8cJ.
To post to this group, send email to meteo...@googlegroups.com.
To unsubscribe from this group, send email to meteor-talk...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/meteor-talk?hl=en.

[Iain Shigeoka]

On Nov 30, 2012, at 8:51 AM, Peter Kutrumbos <peter.k...@gmail.com> wrote:

> I realize this may be an issue with my underlying web-socket code used in the DDP client, however using that same client I do get the appropriate onError and onClose messages during other scenarios such as when I redeploy the meteor server or if my DDP client loses internet connection.
>
> Has anyone had a similar experience or ideas on how to fix it or test this behavior better?

In general you can't rely on connection termination events/notification from the tcp/ip layer. Timeouts and other network setting beyond your control may prevent you from ever receiving a socket termination notice when communicating over the Internet. Only in tightly controlled environments (like using a loopback connection locally) will you get reliable socket close events.

If you need to properly detect and tear down socket connections you must do so at the application layer. Obviously this is a good idea to do in general as zombie connections will become memory leaks that will eventually take down your server. The easies technique is to add a "heartbeat" to your protocol - in meteor just create a meteor method that you can call every so often. - if your client doesn't receive a response after so many seconds assume the connection is no longer open and clean it up and do whatever connection recovery you can.

I would assume ddp itself contains a heartbeat in the underlying protocol for the same reasons so hopefully you can support the ddp heartbeat in your java library instead of creating your own. I don't know there is a built in ddp heartbeat for sure though so you'll have to research.

A heartbeat has the added bonus that you detect not only the socket being connected but also that the server is still responsive. If the server is hung up, blocked, resource starved, etc you would want your client to also detect that and respond appropriately.

-iain

[David Glasser]

Just to be clear: this issue will affect you if you're writing a
custom program using websockets to connect to DDP, but not if you're
just using normal Meteor apps.

By default, Meteor apps don't use websockets to connect to the server.
They use various emulation strategies from SockJS, and all of them
involve some level of periodic connections that have a result of
keeping the app alive. In fact, this limitation of the meteor deploy
servers is part of why Meteor apps don't use raw WebSockets yet

[Peter Kutrumbos]

^^ Thanks for the advice/info guys, it was all helpful