Connectivity Problems To NYC

Chris Bardos

unread,

Feb 10, 2016, 12:01:51 PM2/10/16

to gce-discussion

Hi Guys,

I'm having connectivity problems to my NYC servers hosted with Linode - GCE simply cannot connect. I cannot ping, traceroute or MTR.

I'm trying to connect from: 104.197.18.164 to: 45.79.137.102

This is what an MTR looks like:

|------------------------------------------------|------|------|------|------|------|------|

| No response from host - 100 | 1 | 0 | 0 | 0 | 0 | 0 |

|________________________________________________|______|______|______|______|______|______|

I've submitted a ticket to Linode, and they said that the connectivity issue is not on their network. I can successfully connect in the reverse (from Linode to Google).

Could someone point me in the right direction?

Faizan (Google Cloud Support)

unread,

Feb 11, 2016, 3:09:41 PM2/11/16

to gce-discussion

Hello Chris,

I am looking into this issue I'll post on this thread once I have update to share.

Thanks

Faizan

Chris Bardos

unread,

Feb 11, 2016, 4:12:34 PM2/11/16

to gce-discussion

Great, Thanks!

It is still occurring - so I hope you find out what the issue is :)

Raymond Hawkins

unread,

Feb 11, 2016, 5:05:11 PM2/11/16

to gce-discussion

We're experiencing this issue also. I posted over on ServerFault here about it yesterday. Contacted Linode and they said it seems to be on GCP's end. Can't connect to or from any of the following IP addresses, which is preventing our websites from communicating with our software on GCP.

45.33.85.78

45.79.136.107

45.79.178.156

45.79.184.254

45.79.163.197

Chris Bardos

unread,

Feb 11, 2016, 5:11:18 PM2/11/16

to gce-discussion

Hey Raymond,

I'm curious - how did you figure out Google had blacklisted Linode's IPs?

Raymond Hawkins

unread,

Feb 11, 2016, 5:23:12 PM2/11/16

to gce-discussion

I guess I don't know that for sure, but it's the only conclusion I've been able to come up with. Several of the IP addresses above were already blocked when they were assigned to us, so I'm pretty sure it's not anything we were doing that caused this. After going through a few though, changing the IP address on my server without changing anything else fixed the problem and other linode instances with identical setups are still able to connect just fine. They also stopped working without me making any changes to my setup on GCP or Linode, which along with everything else makes me convinced it's not a setup issue on my end.

In addition, when running mtr from linode to one of my compute engine instances, it bounces around a bit then ends on a google IP with no response as follows:

root@li1277-156:~# mtr -rw --address 45.33.85.78 104.196.39.229

Start: Thu Feb 11 12:31:01 2016

HOST: li1277-156.members.linode.com Loss% Snt Last Avg Best Wrst StDev

1.|-- router1-nac.linode.com 0.0% 10 0.8 1.2 0.7 5.3 1.3

2.|-- 173.255.239.0 0.0% 10 0.9 0.8 0.7 1.0 0.0

3.|-- de-cix.nyc.net.google.com 0.0% 10 2.0 2.5 1.6 9.4 2.4

4.|-- 209.85.248.178 0.0% 10 1.7 1.7 1.6 1.9 0.0

5.|-- 72.14.236.208 0.0% 10 1.8 2.1 1.6 4.9 0.9

6.|-- 216.239.40.170 0.0% 10 7.8 8.8 7.8 16.9 2.8

7.|-- 74.125.37.172 0.0% 10 18.9 18.8 18.4 18.9 0.0

8.|-- 74.125.37.181 0.0% 10 18.7 18.6 18.3 19.4 0.0

9.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0

Same thing on multiple linode instances. But when I try to go from a google compute instance, it just dies instantly as follows:

root@brightsideweb1:~# mtr -rw 45.33.85.78

Start: Thu Feb 11 12:23:23 2016

HOST: brightsideweb1 Loss% Snt Last Avg Best Wrst StDev

(no output. Same thing from 2 other servers)

Chris Bardos

unread,

Feb 11, 2016, 5:25:20 PM2/11/16

to gce-discussion

Ah - pretty good theory. It hadn't occurred to me that Google might be blocking Linode IPs. I assumed it was some sort of transit link problem - but I think I like your theory more.

Terrance

unread,

Feb 11, 2016, 7:21:06 PM2/11/16

to gce-discussion

Hi All,

These IPs are currently blocked for being geo-located to a T6 region. I understand that these IP address may belong to Linode which is a US company but we have blocked access to them. We are legally not allowed to provide certain services to users in a T6 region. We have submitted a request to have this reviewed. The team responsible for this information may choose to change the Geo Location or they may not. If the team chooses to revert the block it may take up to 2 weeks for the changes to propagate through the system. In order to prevent future occurrences of this problem we recommend you discuss this problem with Linode. Due to T6 restrictions and the sensitivity of our IP GEO database we will not provide more information.

Best,

Terrance

Technical Solutions Engineer

Google Cloud Platform Support

Chris Bardos

unread,

Feb 11, 2016, 7:48:50 PM2/11/16

to gce-discussion

I guess I'm sticking with AWS then.

Chris Bardos

unread,

Feb 11, 2016, 7:54:19 PM2/11/16

to gce-discussion

This is what Linode has to say:

I'm sorry you're being effected by this. We have no idea why Google would think that our addresses are in embargoed countries and this is a bit of a mystery to us. The most wildly wrong IP Geolocation database I've seen for one of our IPs shows it being in the Netherlands which still isn't a T6 region so it seems like google has dropped the ball somewhere in regards to this. Unfortunately this is on Google to sort out but in the meantime we could try swapping your IP address for a new one if you'd like.

On Thursday, February 11, 2016 at 4:21:06 PM UTC-8, Terrance wrote:

Shri GeoClicks

unread,

Feb 14, 2016, 6:55:22 PM2/14/16

to gce-discussion

Did you ever resolve this?

One of our vendor's licensing server is on Linode and we're unable to use their software.

At this point the only remedy I can think of is setting up an instance on a third party (AWS?) and proxying requests to Linode through this instance.

Mike McBrien

unread,

Feb 15, 2016, 2:35:36 PM2/15/16

to gce-discussion

We are also having similar issues - the response I got from linode was...

We're really sorry about this! Where did you find information on what Google has to say about this? We're trying to collect as much information as we can in order to investigate this further.

As far as we know, they are only blocking our Newark datacenter at this time so migrating to Atlanta may be a good solution for now until we get this sorted out. We also can swap your IPs to see if you get better results. From what it seems, they've only blocked some of our IPs and not the entire block for Newark (thank goodness). I know this is really inconvenient for you, as it is for us because we didn't seem to be aware of it until customers started coming to us about it, but please be patient with us while we get this solved.

A good portion of me is hoping google will unblock the IP's or at least conditionally whitelist IPS that appear safe, its a ton of work for me to move the site i have hosted in this datacenter :(

@Google can you help us?

-Mike

Terrance

unread,

Feb 15, 2016, 4:38:39 PM2/15/16

to gce-discussion

Hi All,

We have are currently in the process of rolling out the lift on this block. We expect partial resolution in the next few hours and full resolution in the next 24. If that is not the case I will post again before 24 hours is up.

Terrance

unread,

Feb 15, 2016, 4:38:42 PM2/15/16

to gce-discussion

Hi All,

We have are currently in the process of rolling out the lift on this block. We expect partial resolution in the next few hours and full resolution in the next 24. If that is not the case I will post again before 24 hours is up.

Best,

Chris Bardos

unread,

Feb 15, 2016, 5:53:41 PM2/15/16

to gce-discussion, Terrance

Huzzah!

Hi All,

We have are currently in the process of rolling out the lift on this block. We expect partial resolution in the next few hours and full resolution in the next 24. If that is not the case I will post again before 24 hours is up.

Best,
Terrance
Technical Solutions Engineer
Google Cloud Platform Support

-- 
© 2014 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043

Email preferences: You received this email because you signed up for the Google Compute Engine Discussion Google Group (gce-dis...@googlegroups.com) to participate in discussions with other members of the Google Compute Engine community and the Google Compute Engine Team.
--- 
You received this message because you are subscribed to a topic in the Google Groups "gce-discussion" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/gce-discussion/Z5zc06Zo2Zo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to gce-discussio...@googlegroups.com.
To post to this group, send email to gce-dis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gce-discussion/12c71a3d-2e23-4f12-8f93-a32b20c263c5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Steven Dai

unread,

Feb 16, 2016, 12:37:05 PM2/16/16

to gce-discussion, t...@google.com

Hi all,

The rollout of the lift on this block is now fully completed. If you are still having any issues with connections to Linode from GCE, please let us know in this thread.

Best regards,

Steven

Technical Solutions Engineer

Google Cloud Platform Support

Raymond Hawkins

unread,

Feb 16, 2016, 12:45:36 PM2/16/16

to gce-discussion, t...@google.com

Thank you!

Mike McBrien

unread,

Feb 16, 2016, 7:08:07 PM2/16/16

to gce-discussion, t...@google.com

I had 3 IP's that were blocked, two have been fixed but I am still having issues with one of my IP's... 45.33.70.89

Thanks

On Tuesday, February 16, 2016 at 12:37:05 PM UTC-5, Steven Dai wrote:

Shri GeoClicks

unread,

Feb 16, 2016, 7:16:35 PM2/16/16

to gce-discussion, t...@google.com

Can confirm that we're able to make requests into the 45.79.163.0 range.

Thanks.

Terrance

unread,

Feb 16, 2016, 7:24:17 PM2/16/16

to gce-discussion, t...@google.com

HI Mike,

Thanks for the update. I have reported the error. I will post an update when I have an ETA.

Best,
Terrance

Technical Solutions Engineer

Google Cloud Platform Support

Mike McBrien

unread,

Feb 16, 2016, 7:37:08 PM2/16/16

to gce-discussion, t...@google.com

Thanks!

-Mike

Mike McBrien

unread,

Feb 18, 2016, 9:40:41 AM2/18/16

to gce-discussion, t...@google.com

Any updates on this?

Thanks.

Terrance

unread,

Feb 18, 2016, 6:55:41 PM2/18/16

to gce-discussion, t...@google.com

Hi Mike,

We are a rolling out a new version with a few more blocks lifted. This is one should include the IP address you mentioned. I do not have a time the roll out will be complete yet but I will provide one when I can.

Best,

Terrance

Technical Solutions Engineer

Google Cloud Platform

Steven Dai

unread,

Feb 19, 2016, 1:26:11 PM2/19/16

to gce-discussion, t...@google.com

Hi Mike,

The rollout finished around 22:48 PST last night. The changes may take up to 24 hours to fully propagate. Please let us know if you see any other issues after 22:48 PST tonight.