Latency of GCE created in southamerica-east1 region ( São Paulo / Brazil )

[Rogerio Amaral]

I created a GCE (Compute Engine) in the new region southamerica-east1 (São Paulo - Brazil), I imagined that it would use a reserved IP for Brazil, even using a static IP was reserved for the USA, this is not a problem, but latency still remains very high, the same latency of GCE created in the us-central1 or us-east1 regions. Should not the southamerica-east1 region be able to reduce the access latency in South America?

[Karthick (Cloud Platform Support)]

Hello Rogerio,

How are you measuring the latency? 

Can you please run traceroutes to the Brazil instance from different instances and show me the result, in a private message? 

For the moment, all external IP address ranges used by the Google Cloud Platform are from Mountain View, California. The data and the hardware for your instance are located in data centers from the region you selected and Google Public DNS uses anycast routing to direct all packets to the closest DNS server. 

[Rogerio Amaral]

I'm in Brazil, I tested using PING, observe the image, the time compared between servers.

Rogerio Amaral

[Karthick (Cloud Platform Support)]

Rogerio,

While ping is a common way to measure server reachability, ICMP ping does not give a good indication of end-user latency. Have you attempted a traceroute from multiple sources?  This will identify latencies on every hop. Capturing the MTR will let you know if there are packets being lost on one particular node.

Also you might want to check to see if any devices in the path have a significant usage of CPU/RAM.

[Renan Alves Fonseca]

Hello,
I agree with Rogerio that latency we observe in Brazil for GCE based in southamerica-east1 is too high. It is easy to check from any machine in Brazil using www.gcping.com
For me, I have 168 ms for southamerica-east1 and 170 ms for us-east1.
For comparison, www.cloudping.info from AWS gives me 37ms for South America (São Paulo) and US-East (Virginia) 176 ms.
Furthermore, running mtr, we can observe exactly where that latency happens (I did it).
That said, I would appreciate to have that 37ms latency on GCE. Is it possible?

[Julio Carvalho]

Hello guys,

It looks like an issue related to the ISP, I'm also in Brazil and for me latency is around 14 ms, logs below.

I had a similar issue before with a CDN provider, although they have a POP in Brazil, my ISP was routing my requests to a POP in the US.

Not sure if is the same problem here, but looks much like the same.

Pinging 35.199.111.157 with 32 bytes of data:

Reply from 35.199.111.157: bytes=32 time=15ms TTL=58

Reply from 35.199.111.157: bytes=32 time=15ms TTL=58

Reply from 35.199.111.157: bytes=32 time=14ms TTL=58

Reply from 35.199.111.157: bytes=32 time=14ms TTL=58

Ping statistics for 35.199.111.157:

    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),

Approximate round trip times in milli-seconds:

    Minimum = 14ms, Maximum = 15ms, Average = 14ms

C:\Users\Julio>tracert 35.199.111.157

Tracing route to 157.111.199.35.bc.googleusercontent.com [35.199.111.157]

over a maximum of 30 hops:

  1    <1 ms     1 ms    <1 ms  192.168.0.201

  2     1 ms    <1 ms    <1 ms  192.168.0.1

  3     9 ms     8 ms     8 ms  10.21.0.1

  4     9 ms     8 ms     9 ms  201.76.16.11

  5    12 ms    10 ms    10 ms  embratel-T0-1-0-5-uacc04.cas.embratel.net.br [200.213.44.25]

  6    19 ms    15 ms    15 ms  ebt-H0-7-0-3-tcore01.cas.embratel.net.br [200.244.213.103]

  7    16 ms    14 ms    15 ms  ebt-B1191-tcore01.spoph.embratel.net.br [200.230.252.14]

  8    13 ms    15 ms    15 ms  ebt-B1081-tcore01.spomb.embratel.net.br [200.244.212.26]

  9    13 ms    13 ms    16 ms  ebt-B11711-puacc01.spolp.embratel.net.br [200.244.213.176]

 10    14 ms    13 ms    13 ms  209.85.173.92

 11    15 ms    13 ms    14 ms  108.170.245.193

 12    13 ms    13 ms    13 ms  209.85.241.76

 13     *        *        *     Request timed out.

 14    15 ms    15 ms    15 ms  157.111.199.35.bc.googleusercontent.com [35.199.111.157]

Trace complete.

[Renan Alves Fonseca]

Hello Julio,
this is indeed a strange thing for me.
If I target at the same IP you did, after reaching ebt-B11711-puacc01.spolp.embratel.net.br, I observe a huge gap as if the traffic was going to US. The point is that with the same computer and same ISP, I do not observe this gap when I target at known brazilian servers or at AWS SP based servers...

[Karthick (Cloud Platform Support)]

Renan,

Failing on the hop ebt-B11711-puacc01.spolp.embratel.net.br indicates a potential problem with your ISP. Please contact your provider for further assistance. 

[Renan Alves Fonseca]

Hello,

here follows the traceroute from mtr (linux, ubuntu)

                                                               Packets               Pings
 Host                                                           Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. 192.168.0.1                                                  0.0%    28    1.6   2.3   1.4   5.9   1.2
 2. 10.21.0.1                                                    0.0%    28   10.7  30.2   9.5 188.6  46.2
 3. flnctrrtd01.fln.virtua.com.br                                0.0%    27   14.7  16.4  10.4  69.3  11.5
 4. flndtcrtd04.fln.virtua.com.br                                0.0%    27   14.4  13.9   9.9  29.7   3.4
 5. embratel-T0-4-0-4-uacc03.ctamc.embratel.net.br               3.7%    27   14.3  17.3  14.3  22.2   2.1
    embratel-T0-2-0-1-uacc01.ctamr.embratel.net.br
 6. ebt-H0-0-0-2-tcore01.ctamr.embratel.net.br                   0.0%    27   26.5  30.9  23.5  41.7   4.8
    ebt-H0-14-0-3-tcore01.ctamc.embratel.net.br
 7. ebt-B1421-tcore01.spoph.embratel.net.br                      0.0%    27   23.2  27.4  22.8  31.9   2.3
    ebt-B1451-tcore01.spo.embratel.net.br
 8. ebt-B1081-tcore01.spomb.embratel.net.br                      0.0%    27   28.7  27.6  23.2  41.5   3.7
    ebt-B10-tcore01.spolp.embratel.net.br
 9. ebt-B11511-puacc01.spolp.embratel.net.br                     0.0%    27   38.9  26.9  22.9  38.9   3.4
    ebt-B11711-puacc01.spolp.embratel.net.br
10. 209.85.173.92                                                0.0%    27  165.7 166.0 162.8 179.5   3.1
11. 108.170.245.161                                              0.0%    27  134.1 135.7 132.7 146.2   2.8
12. 108.170.232.118                                              0.0%    27  150.1 150.3 147.6 155.0   1.9
13. ???
14. 35.199.93.91                                                 0.0%    27  165.1 165.1 162.0 176.6   2.7

The strange thing is that we reach the same server ( line 10) with very different latencies...

--
© 2017 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043
 
Email preferences: You received this email because you signed up for the Google Compute Engine Discussion Google Group (gce-discussion@googlegroups.com) to participate in discussions with other members of the Google Compute Engine community and the Google Compute Engine Team.
---
You received this message because you are subscribed to a topic in the Google Groups "gce-discussion" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/gce-discussion/hlKY_9cHD38/unsubscribe.
To unsubscribe from this group and all its topics, send an email to gce-discussion+unsubscribe@googlegroups.com.
To post to this group, send email to gce-discussion@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gce-discussion/b54f9261-1de8-40e0-9d4e-43ec63d6bcae%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Karthick (Cloud Platform Support)]

Hello Renan,

Thank you for sharing the MTR. I am currently discussing with the internal team and unfortunately I am not able to provide an E.T.A on the update. I will get back to you as soon as I have any information. 

[Rogerio Amaral]

Hello Karthick,

You can check my problem, see below the tracert for the server and the latency is very high, it is at 152 for the region São Paulo / Brazil, see the attached image.

Tks

Rogério Amaral

--------------------------------

tracert 35.198.58.226

Tracking the route to 226.58.198.35.bc.googleusercontent.com [35.198.58.226]

with a maximum of 30 jumps:

  1     1 ms    <1 ms    <1 ms  192.168.0.1

  2    20 ms    13 ms    27 ms  bd3d8c01.virtua.com.br [189.61.140.1]

  3    16 ms    14 ms    14 ms  c91180fe.virtua.com.br [201.17.128.254]

  4    15 ms    14 ms    13 ms  c91180fd.virtua.com.br [201.17.128.253]

  5    16 ms    17 ms    18 ms  embratel-T0-1-2-0-tacc01.bhe.embratel.net.br [200.244.95.33]

  6    25 ms    29 ms    29 ms  ebt-H0-10-0-0-tcore01.bhe.embratel.net.br [200.244.213.123]

  7    33 ms    29 ms    30 ms  ebt-B10-tcore02.bhe.embratel.net.br [200.244.160.230]

  8    29 ms    38 ms    31 ms  ebt-B1312-tcore01.rjoen.embratel.net.br [200.244.212.128]

  9    26 ms    29 ms    29 ms  ebt-B10-tcore01.rjo.embratel.net.br [200.230.252.158]

 10    27 ms    30 ms    27 ms  ebt-B1121-tcore01.spo.embratel.net.br [200.230.251.2]

 11    27 ms    29 ms    29 ms  ebt-B12-tcore01.spoph.embratel.net.br [200.230.158.29]

 12    34 ms    30 ms    29 ms  ebt-B1081-tcore01.spomb.embratel.net.br [200.244.212.26]

 13    27 ms    29 ms    27 ms  ebt-B11711-puacc01.spolp.embratel.net.br [200.244.213.176]

 14   133 ms   132 ms   135 ms  209.85.173.92

 15   139 ms   143 ms   141 ms  108.170.245.193

 16   144 ms   151 ms   140 ms  209.85.143.182

 17     *        *        *     Out of order timeout.

 18   146 ms   143 ms   143 ms  226.58.198.35.bc.googleusercontent.com [35.198.58.226]

Tracking completed.

---------------------------------------------------------

 

[Karthick (Cloud Platform Support)]

Hello,

Thank you for your patience in this issue.

I verified with our bakckend team that this is a problem between your ISP and Embratel, who needs to advertise prefixes that would cover your network. This asymmetric routing was due to Embratel is not advertising the prefix in the BGP table that covers your  network and due to that, return path goes to MIA in North America, where it goes out via Verizon peering where they advertise that prefix to us. The return path is the reason why you see a 100ms increase.