Jail unable to access some network devices

[Dean E. Weimer]

I am trying to setup some jails on a server utilizing multiple routing
fib tables I have a few jails already running some of them in different
fibs, one in the same fib as the host. But this last one I added has
caused some problems. Its the first one I am trying to build in this
fib. Below is sections from the fib routing setup and the jail.

## Set up the DMZ[1] routing table
# Interface route(s)
setfib 1 route add -net 10.50.0.0/16 -iface lagg0.4
setfib 1 route add -net 10.52.0.0/16 -iface lagg0.4
setfib 1 route add -net 172.17.0.0/25 -iface lagg0.900
# Default route
setfib 1 route add default 10.50.110.9

# Default Jail Options
allow.mount.devfs;
mount.devfs;
devfs_ruleset = '4';
allow.sysvipc;
allow.dying;
exec.start = "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown";
exec.consolelog = "/jails/${name}/console.log";

proxy2 {
exec.fib=1;
allow.raw_sockets = 1;
ip4.addr = 'lagg0.4|10.50.20.42/16';
ip4.addr += 'lagg0.4|10.52.20.42/16';
ip4.addr += 'lagg0.900|172.17.0.5/25';
}

The jail can talk to every host on the 172.17.0.0/25 subnet, and can
talk to every hos tin the 10.52.0.0/16 subnet. I can't talk to some
things in the 10.50.0.0/16 subnet though. The 3 devices I can't talk to
also have 10.52.0.0/16 addresses on the same adapter as the 10.50.0.0/16
I can talk to their 10.52 address but receive an invalid argument error
when attempting to connect via 10.50 address.

Below the 10.50.20.4 and 10.52.20.4 are the same device, an older
FreeBSD server that this jail is intending to replace.

root@fbsdhost2:/jails/proxy2 # proxy2
root@proxy2-temp:/ # ping 10.52.20.4
PING 10.52.20.4 (10.52.20.4): 56 data bytes
64 bytes from 10.52.20.4: icmp_seq=0 ttl=64 time=0.749 ms
64 bytes from 10.52.20.4: icmp_seq=1 ttl=64 time=0.311 ms
^C
--- 10.52.20.4 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.311/0.530/0.749/0.219 ms
root@proxy2-temp:/ # ping 10.50.20.4
PING 10.50.20.4 (10.50.20.4): 56 data bytes
ping: sendto: Invalid argument
ping: sendto: Invalid argument
^C
--- 10.50.20.4 ping statistics ---
2 packets transmitted, 0 packets received, 100.0% packet loss
root@proxy2-temp:/ # ping 10.50.110.9
PING 10.50.110.9 (10.50.110.9): 56 data bytes
64 bytes from 10.50.110.9: icmp_seq=0 ttl=255 time=0.599 ms
64 bytes from 10.50.110.9: icmp_seq=1 ttl=255 time=0.253 ms
^C
--- 10.50.110.9 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.253/0.426/0.599/0.173 ms

root@fbsdhost2:/jails/proxy2 # setfib 1 netstat -rnf inet
Routing tables (fib: 1)

Internet:
Destination Gateway Flags Netif Expire
default 10.50.110.9 UGS lagg0.4
10.50.0.0/16 84:2b:2b:4d:6e:08 US lagg0.4
10.52.0.0/16 84:2b:2b:4d:6e:08 US lagg0.4
127.0.0.1 lo0 UHS lo0
172.17.0.0/25 84:2b:2b:4d:6e:08 US lagg0.90

I know it has to be something on the jail configuration, as it doesn't
even attempt to send the traffic to the interface, watched on host
adapter. If I ping this jail address from the other system the echo
requests do show up, on that adapter. Can anyone see anything in the
above configuration that I have missed, or got wrong?

--
Thanks,
Dean E. Weimer
http://www.dweimer.net/
_______________________________________________
freebsd-...@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questi...@freebsd.org"

[Dean E. Weimer]

I noticed these in the nightly emails over the weekend.

+arpresolve: can't allocate llinfo for 10.50.20.4 on lagg0.4

Didn't think to look at the dmesg output for errors last Friday. After
searching on that I tried a reboot. The system is now talking to
everything. Perhaps something to do with the fact that the main host is
getting NTP currently from the devices I was having difficulty
connecting to for a while before the other fib routing table was used.

Looks like I have a chicken before the egg issue, the host had an arp
entry for these devices already assigned to its other interface through
a route. when the lagg0.4 interface had the new IP address added for the
jail (previously no IP was defined on that interface) the arp entry no
longer matched the correct routing interface.

Short term defining an IP on the host in that subnet on the adapter will
keep it from occurring long term though I may need to find a more secure
solution. As this creates a path around the external firewall that is
controlling the traffic going from the DMZ to the LAN where the host is
located.