Outbound connections to android.googleapis.com and www.googleapis.com timing out

[Dean Harding]

For the last day or so, my application has been getting connection timeouts trying to connect to various Google APIs (Google Cloud Messaging API and the OAuth2 endpoints in particular). If I SSH into the server and just manually try 'telnet www.googleapis.com 443' then connection succeeds and indeed most connections due go through in short order, but occassionaly the connections will just time out for no particular reason.

I've checked the usual suspects:

$ sysctl net.netfilter.nf_conntrack_count

net.netfilter.nf_conntrack_count = 1426

$ sysctl net.nf_conntrack_max

net.nf_conntrack_max = 65536

Nothing in any logs (except my application's logs, for the connection timeouts, but I don't think those are very helpful).

I run munin for performance monitoring, and it doesn't seem to have changed dramatically in the last month:

I'm not really a sysadmin so at this point, I'm not really sure what else to look for -- any pointers would be much appreciated!

Dean.

[Dean Harding]

Some more information:

$ traceroute www.googleapis.com

traceroute to www.googleapis.com (74.125.192.95), 30 hops max, 60 byte packets

 1  ib-in-f95.1e100.net (74.125.192.95)  5.471 ms  5.454 ms  5.419 ms

$ traceroute android.googleapis.com

traceroute to android.googleapis.com (74.125.193.95), 30 hops max, 60 byte packets

 1  ig-in-f95.1e100.net (74.125.193.95)  5.763 ms  5.742 ms  5.697 ms

$ route

Kernel IP routing table

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface

default         10.240.0.1      0.0.0.0         UG    100    0        0 eth0

10.240.0.1      *               255.255.255.255 UH    0      0        0 eth0

My GCE project ID is codeka.com.au:warwworldsmmo