Site to site VPN, cant see box on other side?
296 views
Skip to first unread message
Chris Betts
Aug 17, 2017, 9:14:45 AM8/17/17
to gce-discussion
We've setup a site to site vpn from gcloud to another site so we could VPN into a specific box (They didnt want to do client/server VPN, only site2site)
VPN shows up, Box sees traffic moving, I believe we are setup correctly... I cannot, however see that box they say we should be able to view.
Multi NIC on our box thats being used to VPN. all showing traffic, CentOS shows traffic moving on the proper IP's, doesnt show interfaces, shows a virbr.
Thanks!
VPN shows up, Box sees traffic moving, I believe we are setup correctly... I cannot, however see that box they say we should be able to view.
Multi NIC on our box thats being used to VPN. all showing traffic, CentOS shows traffic moving on the proper IP's, doesnt show interfaces, shows a virbr.
Thanks!
Carlos (Cloud Platform Support)
Aug 17, 2017, 4:38:01 PM8/17/17
to gce-discussion
Hi Chris,
From your description it is not clear if you are using Cloud VPN (which actually only supports Site to Site connections) or instead you have installed a software in a VM on GCP to handle the VPN.
Have you setup a Centos VM in your Google Cloud project using multiple cards as a gateway? If so checking the internal logs of the application you are using might help. Tracing the routes on both directions might also be useful to discard a routing problem.
If you are using Cloud VPN you might want to check your console to be sure the VPN is up. I would also suggest having a look to Cloud VPN logs.
As a side note, this article contain some common setups that can be used while peering Cloud VPN with popular 3rd party appliances.
From your description it is not clear if you are using Cloud VPN (which actually only supports Site to Site connections) or instead you have installed a software in a VM on GCP to handle the VPN.
Have you setup a Centos VM in your Google Cloud project using multiple cards as a gateway? If so checking the internal logs of the application you are using might help. Tracing the routes on both directions might also be useful to discard a routing problem.
If you are using Cloud VPN you might want to check your console to be sure the VPN is up. I would also suggest having a look to Cloud VPN logs.
As a side note, this article contain some common setups that can be used while peering Cloud VPN with popular 3rd party appliances.
Chris Betts
Aug 17, 2017, 6:01:55 PM8/17/17
to gce-discussion
Hey Carlos!
Cloud VPN is what I am using.
Basically, trying to ensure traffic is being pushed through the VPN interface... How do I enforce that?
Cloud VPN is what I am using.
Basically, trying to ensure traffic is being pushed through the VPN interface... How do I enforce that?
Carlos (Cloud Platform Support)
Aug 18, 2017, 3:01:56 PM8/18/17
to gce-discussion
Hi Chris,
For example setting a VPN between two of my projects in the following configuration (Apologize for the diagram):
192.168.3.2 - VM_P1 -10.128.0.2 --- Project 1 Gateway ->> << Project 2 Gateway ----172.16.0.2 -VM_P2
I defined the instance “VM_P1” which uses two different interfaces in subnets 192.168.3.0/24 and 10.128.0.0/8. By login in the VM I can see the following routing table being used by the Linux kernel
sudo route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
default 10.128.0.1 0.0.0.0 UG 0 0 0 eth0
10.128.0.0 10.128.0.1 255.255.240.0 UG 0 0 0 eth0
10.128.0.1 0.0.0.0 255.255.255.255 UH 0 0 0 eth0
192.168.3.0 192.168.3.1 255.255.255.0 UG 0 0 0 eth1
192.168.3.1 0.0.0.0 255.255.255.255 UH 0 0 0 eth1
The table defines which traffic must be handled by each interface. Although the subnet 172.16.0.0/16 is not defined in the VM per se, it will be handled by the network routing. Similarly the instance “VM_P2” in the second project, which was configured with only one interface will use the following routing table:
As previously mentioned, when you create Cloud VPN some other routes are created in your project network. You can verify them by going to Console->VPC Network or by the use of Cloud SDK. In my case, Project 2 routing was defined like:
gcloud compute routes list | grep proy2net
default-route-0baf85a9d7c70e00 proy2net 172.16.0.0/16 1000
default-route-2cff31f529e3d213 proy2net 0.0.0.0/0 default-internet-gateway 1000
vpn-2-proy2-tunnel-1-route-1 proy2net 10.0.0.0/8 us-central1/vpnTunnels/vpn-2-proy2-tunnel-1 1000
Which basically tell you that traffic going to the subnet 10.0.0.0/8 must be sent via the tunnel.
One additional thing I did not mentioned in my previous message but I believe you should check are your firewall rules. If you have configured a firewall in the VM, you must be sure it is also allowing the appropriate traffic.
The enforcement is automatically done by routes that are deployed in your Project and also in the VMs themselves. You can certainly verify that those routes has been correctly created.
For example setting a VPN between two of my projects in the following configuration (Apologize for the diagram):
192.168.3.2 - VM_P1 -10.128.0.2 --- Project 1 Gateway ->> << Project 2 Gateway ----172.16.0.2 -VM_P2
I defined the instance “VM_P1” which uses two different interfaces in subnets 192.168.3.0/24 and 10.128.0.0/8. By login in the VM I can see the following routing table being used by the Linux kernel
sudo route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
default 10.128.0.1 0.0.0.0 UG 0 0 0 eth0
10.128.0.0 10.128.0.1 255.255.240.0 UG 0 0 0 eth0
10.128.0.1 0.0.0.0 255.255.255.255 UH 0 0 0 eth0
192.168.3.0 192.168.3.1 255.255.255.0 UG 0 0 0 eth1
192.168.3.1 0.0.0.0 255.255.255.255 UH 0 0 0 eth1
The table defines which traffic must be handled by each interface. Although the subnet 172.16.0.0/16 is not defined in the VM per se, it will be handled by the network routing. Similarly the instance “VM_P2” in the second project, which was configured with only one interface will use the following routing table:
sudo route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
default 172.16.0.1 0.0.0.0 UG 0 0 0 eth0
172.16.0.1 0.0.0.0 255.255.255.255 UH 0 0 0 eth0
gcloud compute routes list | grep proy2net
default-route-0baf85a9d7c70e00 proy2net 172.16.0.0/16 1000
default-route-2cff31f529e3d213 proy2net 0.0.0.0/0 default-internet-gateway 1000
vpn-2-proy2-tunnel-1-route-1 proy2net 10.0.0.0/8 us-central1/vpnTunnels/vpn-2-proy2-tunnel-1 1000
Which basically tell you that traffic going to the subnet 10.0.0.0/8 must be sent via the tunnel.
One additional thing I did not mentioned in my previous message but I believe you should check are your firewall rules. If you have configured a firewall in the VM, you must be sure it is also allowing the appropriate traffic.
Chris Betts
Aug 18, 2017, 3:40:49 PM8/18/17
to gce-discussion
Just double checking, eth0 is what is supposed to be tunneled, but the 127 IP is showing 510 MB of transfer and loop back? Normal? I Just have a feeling its looping back into itself instead of pushing through the VPN.
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1460
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1460
inet 192.168.0.2 netmask 255.255.255.255 broadcast 192.168.0.2
ether 42:01:c0:a8:00:02 txqueuelen 1000 (Ethernet)
RX packets 7225 bytes 856572 (836.4 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 11672 bytes 8902185 (8.4 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1430
inet 10.138.0.2 netmask 255.255.255.255 broadcast 10.138.0.2
ether 42:01:0a:8a:00:02 txqueuelen 1000 (Ethernet)
RX packets 29985 bytes 1814908 (1.7 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 36519 bytes 26570546 (25.3 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1 (Local Loopback)
RX packets 105895 bytes 535343634 (510.5 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 105895 bytes 535343634 (510.5 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
virbr0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
inet 192.168.122.1 netmask 255.255.255.0 broadcast 192.168.122.255
ether 52:54:00:f6:27:65 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
Chris Betts
Aug 18, 2017, 5:29:51 PM8/18/17
to gce-discussion
I RDP into the eth1 interface, via XRDP.
I want to be sure remoting into that, wont ruin the VPN tunnel and traffic flow supposed ot be going to eth0. Is there a way to ensure this?
I want to be sure remoting into that, wont ruin the VPN tunnel and traffic flow supposed ot be going to eth0. Is there a way to ensure this?
Carlos (Cloud Platform Support)
Aug 21, 2017, 4:24:21 PM8/21/17
to gce-discussion
Hi Chris,
Yes that looks suspicious. Now when I spin new VMs my setup will look like this one:
Ethernet adapter Ethernet:
Connection-specific DNS Suffix . : c.project.internal
IPv4 Address. . . . . . . . . . . : 192.168.3.6
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Default Gateway . . . . . . . . . : 192.168.3.1
Ethernet adapter Ethernet 2:
Connection-specific DNS Suffix . : c.project.internal
IPv4 Address. . . . . . . . . . . : 10.128.0.6
Subnet Mask . . . . . . . . . . . : 255.255.240.0
Default Gateway . . . . . . . . . :
I did some additional testing using now Windows OS. I used the same diagram described previously. There is something I actually overlooked.
If I deploy the VM, setting up the first adapter in the VPN tunnel (10.128.0.0/8) and the second on 192.168.0.0/24 subnet, I reach the other side of the tunnel with no problem.
In the reverse situation where my first adapter was 192.168.0.0/24, I was not able to reach the subnet 172.16.0.0/16. The reason was that the default gateway in the VM was defined as 192.168.3.1 and not 10.128.0.1. The network defined in my project holding 192.168.0.0/16 does not know how to route packets to 172.16.0.0/16. Therefore packets did arrived to this network but they got lost.
The routing on my 192.168.0.0 subnet looks like
There are couple of workarounds that I can think of, but what I did is to add a static route in the routing table of the Windows VM
C:\Program Files (x86)\Google\Cloud SDK>route print
===========================================================================
Interface List
4...42 01 c0 a8 03 06 ......Red Hat VirtIO Ethernet Adapter
3...42 01 0a 80 00 06 ......Red Hat VirtIO Ethernet Adapter #2
1...........................Software Loopback Interface 1
===========================================================================
IPv4 Route Table
===========================================================================
Active Routes:
Network Destination Netmask Gateway Interface Metric
0.0.0.0 0.0.0.0 192.168.3.1 192.168.3.6 15
10.128.0.0 255.255.240.0 10.128.0.1 10.128.0.6 16
10.128.0.1 255.255.255.255 On-link 10.128.0.6 16
10.128.0.6 255.255.255.255 On-link 10.128.0.6 271
127.0.0.0 255.0.0.0 On-link 127.0.0.1 331
127.0.0.1 255.255.255.255 On-link 127.0.0.1 331
127.255.255.255 255.255.255.255 On-link 127.0.0.1 331
169.254.169.254 255.255.255.255 On-link 192.168.3.6 16
192.168.3.0 255.255.255.0 192.168.3.1 192.168.3.6 16
192.168.3.1 255.255.255.255 On-link 192.168.3.6 16
192.168.3.6 255.255.255.255 On-link 192.168.3.6 271
224.0.0.0 240.0.0.0 On-link 127.0.0.1 331
224.0.0.0 240.0.0.0 On-link 192.168.3.6 271
224.0.0.0 240.0.0.0 On-link 10.128.0.6 271
255.255.255.255 255.255.255.255 On-link 127.0.0.1 331
255.255.255.255 255.255.255.255 On-link 192.168.3.6 271
255.255.255.255 255.255.255.255 On-link 10.128.0.6 271
===========================================================================
Persistent Routes:
Network Address Netmask Gateway Address Metric
169.254.169.254 255.255.255.255 On-link 1
===========================================================================
C:\Program Files (x86)\Google\Cloud SDK>ping 172.16.0.2
Pinging 172.16.0.2 with 32 bytes of data:
Request timed out.
Request timed out.
Ping statistics for 172.16.0.2:
Packets: Sent = 2, Received = 0, Lost = 2 (100% loss),
Control-C
^C
C:\Program Files (x86)\Google\Cloud SDK>route add 172.16.0.0 MASK 255.255.0.0 10.128.0.1
OK!
C:\Program Files (x86)\Google\Cloud SDK>ping 172.16.0.2
Pinging 172.16.0.2 with 32 bytes of data:
Reply from 172.16.0.2: bytes=32 time=3ms TTL=64
Reply from 172.16.0.2: bytes=32 time=1ms TTL=64
RDP in the VM should not be a problem as long as the routing and the firewall rules are correctly setup. Some ideas to test is to try a fresh VM or a VM using only one interface. That will help you to identify if the problem is in fact having two interfaces.
Yes that looks suspicious. Now when I spin new VMs my setup will look like this one:
Ethernet adapter Ethernet:
Connection-specific DNS Suffix . : c.project.internal
IPv4 Address. . . . . . . . . . . : 192.168.3.6
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Default Gateway . . . . . . . . . : 192.168.3.1
Ethernet adapter Ethernet 2:
Connection-specific DNS Suffix . : c.project.internal
IPv4 Address. . . . . . . . . . . : 10.128.0.6
Subnet Mask . . . . . . . . . . . : 255.255.240.0
Default Gateway . . . . . . . . . :
I did some additional testing using now Windows OS. I used the same diagram described previously. There is something I actually overlooked.
If I deploy the VM, setting up the first adapter in the VPN tunnel (10.128.0.0/8) and the second on 192.168.0.0/24 subnet, I reach the other side of the tunnel with no problem.
In the reverse situation where my first adapter was 192.168.0.0/24, I was not able to reach the subnet 172.16.0.0/16. The reason was that the default gateway in the VM was defined as 192.168.3.1 and not 10.128.0.1. The network defined in my project holding 192.168.0.0/16 does not know how to route packets to 172.16.0.0/16. Therefore packets did arrived to this network but they got lost.
The routing on my 192.168.0.0 subnet looks like
Name | Destination IP ranges | Priority | Instance tags | Next hop | |
1000 | None | Default internet gateway | |||
1000 | None | Virtual network |
There are couple of workarounds that I can think of, but what I did is to add a static route in the routing table of the Windows VM
C:\Program Files (x86)\Google\Cloud SDK>route print
===========================================================================
Interface List
4...42 01 c0 a8 03 06 ......Red Hat VirtIO Ethernet Adapter
3...42 01 0a 80 00 06 ......Red Hat VirtIO Ethernet Adapter #2
1...........................Software Loopback Interface 1
===========================================================================
IPv4 Route Table
===========================================================================
Active Routes:
Network Destination Netmask Gateway Interface Metric
0.0.0.0 0.0.0.0 192.168.3.1 192.168.3.6 15
10.128.0.0 255.255.240.0 10.128.0.1 10.128.0.6 16
10.128.0.1 255.255.255.255 On-link 10.128.0.6 16
10.128.0.6 255.255.255.255 On-link 10.128.0.6 271
127.0.0.0 255.0.0.0 On-link 127.0.0.1 331
127.0.0.1 255.255.255.255 On-link 127.0.0.1 331
127.255.255.255 255.255.255.255 On-link 127.0.0.1 331
169.254.169.254 255.255.255.255 On-link 192.168.3.6 16
192.168.3.0 255.255.255.0 192.168.3.1 192.168.3.6 16
192.168.3.1 255.255.255.255 On-link 192.168.3.6 16
192.168.3.6 255.255.255.255 On-link 192.168.3.6 271
224.0.0.0 240.0.0.0 On-link 127.0.0.1 331
224.0.0.0 240.0.0.0 On-link 192.168.3.6 271
224.0.0.0 240.0.0.0 On-link 10.128.0.6 271
255.255.255.255 255.255.255.255 On-link 127.0.0.1 331
255.255.255.255 255.255.255.255 On-link 192.168.3.6 271
255.255.255.255 255.255.255.255 On-link 10.128.0.6 271
===========================================================================
Persistent Routes:
Network Address Netmask Gateway Address Metric
169.254.169.254 255.255.255.255 On-link 1
===========================================================================
C:\Program Files (x86)\Google\Cloud SDK>ping 172.16.0.2
Pinging 172.16.0.2 with 32 bytes of data:
Request timed out.
Request timed out.
Ping statistics for 172.16.0.2:
Packets: Sent = 2, Received = 0, Lost = 2 (100% loss),
Control-C
^C
C:\Program Files (x86)\Google\Cloud SDK>route add 172.16.0.0 MASK 255.255.0.0 10.128.0.1
OK!
C:\Program Files (x86)\Google\Cloud SDK>ping 172.16.0.2
Pinging 172.16.0.2 with 32 bytes of data:
Reply from 172.16.0.2: bytes=32 time=3ms TTL=64
Reply from 172.16.0.2: bytes=32 time=1ms TTL=64
RDP in the VM should not be a problem as long as the routing and the firewall rules are correctly setup. Some ideas to test is to try a fresh VM or a VM using only one interface. That will help you to identify if the problem is in fact having two interfaces.
Chris Betts
Aug 21, 2017, 4:36:24 PM8/21/17
to gce-discussion
I got it sorted out now! Thanks!
Is there a way to force the VPN Connection initiation? it dies out after an hour, if theres a command I can run to retry the connection, to reconnect, that would be great! What I am doing now, is I have to add 0.0.0.0/0 and remove it, then it retries, and connects.
Is there a way to force the VPN Connection initiation? it dies out after an hour, if theres a command I can run to retry the connection, to reconnect, that would be great! What I am doing now, is I have to add 0.0.0.0/0 and remove it, then it retries, and connects.
Carlos (Cloud Platform Support)
Aug 22, 2017, 12:35:50 PM8/22/17
to gce-discussion
I am glad you figure it out.
In that sense I would think the issue could be related instead to your server. A parameter you might want to watch is the TCP keep-alive which is setup by default to 10 minutes.
Reply all
Reply to author
Forward
0 new messages