One VM, Two Interfaces

[William Sharlow]

About a year ago, I started this discussion:

Multiple External / Static IP Addresses on One Instance

I FINALLY got around to testing this and while it took a bit to understand certain limitation, I was able to create a single VM that has two vpc network interfaces and a working firewall. The primary interface nic0 / eth0 is functioning fine. In fact, it's been seamlessly running a webserver all day long.

The problem I ran into has to do with the second interface nic1 / eth1. When I tried to ping 8.8.8.8 from 10.140.0.2 (its primary internal IP), I couldn't reach 8.8.8.8.  Fortunately, I found this in the documentation:

Configuring Policy Routing
For Google supported images, when you need a secondary network interface (an interface other than nic0) to communicate with any IP address not local to the primary subnet range of that secondary interface's associated subnet, you need to configure policy routing to ensure that egress packets will leave through the correct interface. In such cases, you must configure a separate routing table for each network interface using policy routing.

They provide an example that I've applied to my instance:

sudo ifconfig eth1 10.140.0.2 netmask 255.255.255.255 broadcast 10.140.0.2 mtu 1430
sudo echo "1 rt1" | sudo tee -a /etc/iproute2/rt_tables # (sudo su - first if permission denied)
sudo ip route add 10.140.0.1 src 10.140.0.2 dev eth1
sudo ip route add default via 10.140.0.1 dev eth1 table rt1
sudo ip rule add from 10.140.0.2/20 table rt1
sudo ip rule add to 10.140.0.2/20 table rt1

After entering these commands, I was able to successful ping 8.8.8.8 from 10.140.0.2.  Being ever vigilant (and a Linux noob), I rebooted the vm to make sure the configuration was permanent. The experts here know it's not.. and of course I could not successfully ping 8.8.8.8 from 10.140.0.2.

What files / values do I need to edit to make this configuration permanent?

Bill

[Navi Aujla (Google Cloud Support)]

Hello Bill, 

IP routes and IP rules needs to be persistent routes to avoid the routes being lost after VM reboot or network services restart. Depending upon the operating system configuration files required to make the routes persistent can be different. Here is a stackexchange thread for CentOS 7, mentioning files: "/etc/sysconfig/network-scripts/route-ethX" and "/etc/sysconfig/network/scripts/rule-ethX" to keep the IP route and rule peristent. Here is the CentOS documentation for the persistent static routes. 

Furthermore, I recommend posting your question for the static persistent routes, on stackoverflow.com where you have access to a large community of enthusiasts and Linux/Unix experts to share ideas with and get support from. Please make sure to include all relevant details and any action taken to resolve it.

[William Sharlow]

Thanks Navi - I will take a look at the Centos information since the server is Centos 7. Thanks also for the stackoverflow lead too.  Admittedly, I did post a question there, explaining that I had already created a script that ran on a reboot and asked if a more permanent solution is possible. The answer I got was something like "Your solution looks pretty permanent to me." I had this feeling a better solution was possible.