CBL stops receiving Sync Gateway changes after 5 minutes of inactivity

[Eric Levine]

We are using the .Net version of CBL, and it stops receiving changes from the Sync Gateway if there is 5 minutes or more of inactivity. When I look at the Sync Gateway logs, I see that the Pull Replicator sets the heartbeat query parameter to 5 minutes when it requests the changes feed.  The status of the pull replicator doesn't change from idle, and it doesn't fire a changed event at that time.  It seems to still have the HTTP connection open, but we are going to look at that closer.  

Is this a known issue?  What can we do to fix it?

[Jens Alfke]

On Apr 2, 2015, at 8:14 AM, Eric Levine <levin...@gmail.com> wrote:

We are using the .Net version of CBL, and it stops receiving changes from the Sync Gateway if there is 5 minutes or more of inactivity. When I look at the Sync Gateway logs, I see that the Pull Replicator sets the heartbeat query parameter to 5 minutes when it requests the changes feed.  The status of the pull replicator doesn't change from idle, and it doesn't fire a changed event at that time.

Well, it wouldn’t fire any event if there’s inactivity. Are you talking about a situation where, after >5 minutes of inactivity, the gateway does have a change but the client doesn’t receive it? Can you describe the sequence of events (and what gets logged on either side) in more detail?

Also, please describe the network topology between SG and CBL — in particular any proxies/gateways/load-balancers (especially ones from cloud services like AWS, which are known for terminating ‘idle’ sockets.)

—Jens

[Eric Levine]

We are creating a bunch of documents through the Sync Gateway's REST API.  If we wait more than 5 minutes, then create more documents through the REST API, the changes aren't synced to the client.  In the Sync Gateway logs, we see the documents being created, and the client making the request for the changes feed, but we do not see the follow up requests for the client to retrieve the documents.  

Here is our setup:

• The server is a physical machine in our office, and has pretty good specs (at least quad core and 8GB memory).  It is running Windows Server 2012, IIS, Couchbase, and the Sync Gateway.  IIS is acting as a reverse proxy to the Sync Gateway, using URL rewrite rules.  We are running the Sync Gateway as a Windows Service that was setup through NSSM,
• The client is connecting to our office WiFi

I'll make a follow up post shortly with a more detailed log capture.

--

Eric Levine

[Eric Levine]

Jens,

Thank you for bringing up the question about network topology, it helped us to narrow down the problem.  We switched over to a direct Sync Gateway connection, and no longer observe this issue.  The culprit is likely a setting in the IIS Application Request Routing module that is causing the issue.  I'll post an update once that is figured out.

--

Eric Levine

[Eric Levine]

We may have found the fix, which was to set the Application Request Routing's "Response buffer threshold" to 0. It was set to something like 256KB by default, and our theory is that IIS was waiting for that buffer to fill instead of sending the heartbeats to the client. Since we made the change, everything appears to be working as expected.

[Zachary Gramana]

Thanks for the follow-up. BTW, which ARR load balancing algorithm are you using?

[Eric Levine]

As far as I know, we don't have any load balancing enabled in this configuration.

--

Eric