Networking in Zones
spacegoose
address.
I have 3 sparse zones with the IPs 192.168.0.10, 192.168.0.20, and
192.168.0.30.
I would like to use NAT and IPF to, for example, forward port 2222
requests from the public IP to 22 on 192.168.0.10 which is bound as
elxl:1; and 3333 to 192.168.0.20, bound to elxl0:2, etc.
I have pfil and ipfilter services online and enabled, permissive
ipf.conf. and ipnat.conf and pfil.ap files configured.
I cannot ping the zones from the global zones, but the zones can ping
eachother - they are all in the same net/subnet.
Also, when logged into the zones, they can't see the internet - they
have /etc/resolv.conf and nsswitch.conf configured (but not
defaultrouter).
Do I need to add routes from the global zone to the sparse zones, and
from the sparse zones to the global zone? If so, how is this done?
More details below:
I have enabled ipv4 forwarding on the interface with routeadm:
# netstat -rn
Routing Table: IPv4
Destination Gateway Flags Ref Use Interface
default 12.12.60.129 UG 1 2
12.12.60.128 12.12.60.121 U 1 2 elxl0
224.0.0.0 12.12.60.121 U 1 0 elxl0
127.0.0.1 127.0.0.1 UH 6 111 lo0
# more /etc/ipf/pfil.ap
elxl -1 0 pfil
# more /etc/ipf/ipf.conf
#
# ipf.conf
#
# IP Filter rules to be loaded during startup
#
# See ipf(4) manpage for more information on
# IP Filter rules syntax.
pass in quick all
pass out quick all
# more /etc/ipf/ipnat.conf
# forward from a public port to a private zone port
rdr elxl0 12.12.60.121/32 port 2222 -> 192.168.0.10 port 22
# force outbound zone traffic thru a certain ip address
# required for mail servers because of reverse lookup
map elxl0 192.168.0.10/32 -> 12.12.60.121/32 proxy port ftp ftp/tcp
map elxl0 192.168.0.10/32 -> 12.12.60.121/32 portmap tcp/udp auto
map elxl0 192.168.0.10/32 -> 12.12.60.121
# allow any 192.168.0.x zone to use the internet
map elxl0 192.168.0.0/24 -> 0/32 proxy port ftp ftp/tcp
map elxl0 192.168.0.0/24 -> 0/32 portmap tcp/udp auto
map elxl0 192.168.0.0/24 -> 0/32
# ipfstat
bad packets: in 0 out 0
IPv6 packets: in 0 out 0
input packets: blocked 0 passed 1013 nomatch 0 counted 0 short 0
output packets: blocked 0 passed 494 nomatch 0 counted 0 short 0
input packets logged: blocked 0 passed 0
output packets logged: blocked 0 passed 0
packets logged: input 0 output 0
log failures: input 0 output 0
fragment state(in): kept 0 lost 0 not fragmented 0
fragment state(out): kept 0 lost 0 not fragmented 0
packet state(in): kept 0 lost 0
packet state(out): kept 0 lost 0
ICMP replies: 0 TCP RSTs sent: 0
Invalid source(in): 0
Result cache hits(in): 136 (out): 462
IN Pullups succeeded: 0 failed: 0
# ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu
8232 index 1
inet 127.0.0.1 netmask ff000000
lo0:1: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL>
mtu 8232 index 1
zone redzone
inet 127.0.0.1 netmask ff000000
lo0:2: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL>
mtu 8232 index 1
zone bluezone
inet 127.0.0.1 netmask ff000000
lo0:3: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL>
mtu 8232 index 1
zone greenzone
inet 127.0.0.1 netmask ff000000
elxl0: flags=1100843<UP,BROADCAST,RUNNING,MULTICAST,ROUTER,IPv4> mtu
1500 index 2
inet 12.12.60.121 netmask ffffff80 broadcast 12.12.60.255
ether 0:6:5b:87:78:11
elxl0:1: flags=1100843<UP,BROADCAST,RUNNING,MULTICAST,ROUTER,IPv4> mtu
1500 index 2
zone redzone
inet 192.168.0.10 netmask ffffff00 broadcast 192.168.0.255
elxl0:2: flags=1100843<UP,BROADCAST,RUNNING,MULTICAST,ROUTER,IPv4> mtu
1500 index 2
zone bluezone
inet 192.168.0.30 netmask ffffff00 broadcast 192.168.0.255
elxl0:3: flags=1100843<UP,BROADCAST,RUNNING,MULTICAST,ROUTER,IPv4> mtu
1500 index 2
zone greenzone
inet 192.168.0.20 netmask ffffff00 broadcast 192.168.0.255
#
OUT Pullups succeeded: 0 failed: 0
Fastroute successes: 0 failures: 0
TCP cksum fails(in): 0 (out): 0
IPF Ticks: 2112
Packet log flags set: (0)
none
Thanks for any help,
sg
RJ
817-1592-15 (SAG: Solaris Containers-Resource Managament and Solaris
Zones). I think that will answer the question(s).
Daniel Rock
> I have an 08/07 Sol 10 x86 machine with 1 physical NIC and1 IP
> address.
>
> I have 3 sparse zones with the IPs 192.168.0.10, 192.168.0.20, and
> 192.168.0.30.
> I would like to use NAT and IPF to, for example, forward port 2222
> requests from the public IP to 22 on 192.168.0.10 which is bound as
> elxl:1; and 3333 to 192.168.0.20, bound to elxl0:2, etc.
1. The global zone must also have an address in your private subnet.
2. You have to create a fake default route out of your private subnet
so the zones can access the internet
I use names for the following examples:
fakerouter dummy IP address of a fake default router in
your subnet
global IP address of global zone in your subnet
zone1 IP address of one non-global zone
They could look line
172.16.1.254 fakerouter
172.16.1.1 global
172.16.1.101 zone1
The network interface in the following examples will be "bge0". Set
netmask as required in /etc/inet/netmasks
(1) setup the global zone. The script should start before starting zones.
You can either define a service with proper dependencies or put the
script in /etc/rc2.d (zones will start as milestone:multi-user-server
which is equivalent to run level 3)
#!/sbin/sh
iface="bge0"
globalname="global"
fakerouter="fakerouter"
routerip=`netstat -rn | awk '/default/ { print $2 }'`
ping $routerip >/dev/null 2>&1
routermac=`netstat -pn | nawk '$2 == "'$routerip'" { print $NF }'`
ifconfig $iface addif $globalname deprecated netmask + broadcast + up
route add default $fakerouter
arp -s $fakerouter $routermac
/usr/sbin/ipf -y
This script will get the MAC address of the default router, create a static
arp entry with the MAC address of the router to "fakerouter" IP address
and sets a default router from the private net to this router. Then it
will reload ipfilter rules.
Change variable names as required.
You will see two default routes in your global zones, but don't worry - that's
Ok.
(2) setup ipfilter/ipnat
ipf.conf should contain at least the following rules so private addresses
won't leak out:
block out quick on bge0 from any to 10.0.0.0/8
block out quick on bge0 from any to 172.16.0.0/12
block out quick on bge0 from any to 192.168.0.0/16
block in quick on bge0 from 10.0.0.0/8 to any
block in quick on bge0 from 172.16.0.0/12 to any
block in quick on bge0 from 192.168.0.0/16 to any
In ipnat.conf you can use the usual redirect rules. Say, zone1 contains a
mail server:
rdr bge0 0/0 port smtp -> zone1 port smtp tcp age 120
rdr bge0 0/0 port submission -> zone1 port submission tcp age 120
And for outgoing traffic
map bge0 172.16.0.0/12 -> 0 age 120
("age 120" is (was?) needed because of a ipfilter bug introduced in S10U4)
--
Daniel
spacegoose
Thanks for the hint:
I'd found this blog:
http://blogs.sun.com/droux/entry/private_virtual_networks_for_solaris
which talks about OpenSolaris's Crossbow, which seems exactly what I
need, and seems really cool.
and:
http://docs.sun.com/app/docs/doc/817-1592/6mhahuooo?l=en&a=view#geprv
according to your suggestion (as I'm running 08/07), which seems
related to what Crossbow does using dladm (though with not as many
features), hopefully I'll be able to figure this out and report back.
Essentially, I'm trying to duplicate what an off-the-shelf DSL/Cable
router does with NAT/port forwarding etc, though instead of forwarding
to machines on a LAN, forwarding to local zones, and thus not needing
a physical switch.
Thanks again!
sg
spacegoose
Wow - this seems to work! I already had an interface in the global
zone on the same subnet as the zones (so I could ping them). I halted
a zone, ran the script* and was able to SSH into the redzone on port
2222 from a completely different machine.
By the way in your ipnat you refer to zone1 will the name instead of
IP address, does this work? I did not modify my ipf or ipnat .confs
for now, as they were already in place, I will consider rules to
prevent IP address leaking though.
*I did receive an error on running the script, but enough of it seems
to have worked (perhaps all I needed was the additional gateway?):
ifconfig: setifflags: SIOCSLIFFLAGS: elxl0:2: Cannot assign requested
address ???
# zoneadm -z redzone halt
# ./setup
Setting netmask of elxl0 to 255.255.255.128
Created new logical interface elxl0:2
Setting netmask of elxl0:2 to 255.255.255.128
ifconfig: setifflags: SIOCSLIFFLAGS: elxl0:2: Cannot assign requested
address
add net default: gateway 192.168.0.50
filter sync'd
# zoneadm -z redzone boot
I did see two default gateways as you said I would. And redzone could
access the internet, and be accessed via my ipnat and ipf rules (e.g.
port 2222).
Do you know if this is a workaround to what the Exclusive-IP zones do
- with dladm, etc.?
Is this as effective? Any drawbacks to this?
Thanks,
sg
Daniel Rock
> By the way in your ipnat you refer to zone1 will the name instead of
> IP address, does this work? I did not modify my ipf or ipnat .confs
> for now, as they were already in place, I will consider rules to
> prevent IP address leaking though.
ipfilter does the usual name resolving like any other utility. Because
it is started early in the boot process, external naming services might
not be available.
If prefer names in ipf.conf/ipnat.conf for readability, but don't forget
to add them to your /etc/hosts. Same goes for port names (/etc/services)
> ifconfig: setifflags: SIOCSLIFFLAGS: elxl0:2: Cannot assign requested
> address ???
Hmm, don't know. Shouldn't happen. Try running the ifconfig line parameter
for parameter and see when the error happens:
ifconfig elxl0 addif <name>
(now replace elxl0:2 with the right index from the output above)
ifconfig elxl0:2 deprecated
ifconfig elxl0:2 netmask +
ifconfig elxl0:2 broadcast +
ifconfig elxl0:2 up
> Is this as effective? Any drawbacks to this?
Just one minor inconvenience: during "ifconfig" the interface sends
out 3 arp requests with its own (in this case private) IP address.
--
Daniel