sppp0 ipfilter/ipnat zones

[James]

Hello,
I have Solaris 10 with zones. The zones all have internal addresses on
192.168.1.0/24 connected to the bge1 interface
global 192.168.1.100
mailhost 192.168.1.101
webserver 192.168.1.102
etc...
...and everything works internally.

My ISP provides a /29 address block giving 6 usable addresses.
Previously everything was plumbed to another interface connected to an
external router. I assigned the addresses:
global 111.111.111.161
mailhost 111.111.111.162
webserver 111.111.111.163
router 111.111.111.166
The zones knew about 111.111.111.160/29 and had the global default
route. Everything worked as needed. Zones responded on both internal
and external IP addresses and could access the outside with no NAT
needed. All worked.


I am now connected to the internet using pppoe via a new DSL modem. This
creates the interface sppp0 and a default route. After connection:

global# ifconfig sppp0
sppp0:
flags=10011008d1<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST,ROUTER,IPv4,FIXEDMTU>
mtu 1492 index 30
inet 111.111.111.166 --> 1.2.3.4 netmask ff000000
ether 0:0:0:0:0:0

global# netstat -rn

Routing Table: IPv4
Destination Gateway Flags Ref Use Interface
--------------- -------------------- ----- ----- ---------- --------
default 1.2.3.4 UG 1 990
1.2.3.4 111.111.111.166 UH 1 158 sppp0
192.168.1.0 192.168.1.100 U 1 857592 bge1
224.0.0.0 192.168.1.100 U 1 0 bge1
127.0.0.1 127.0.0.1 UH 8 1587568 lo0

...where the following values are obtained from the modem on connection,
(real values edited):
111.111.111.166 is my address assigned by the ISP.
1.2.3.4 is the ISP.


The outside thinks I'm .166. Services in the global zone on .166 work
directly. ipfilter works. ipnat works. I can access a service on any
of my IP addresses in the global zone by using rdr, eg, to make DNS work
on .165 not .166 I put in ipnat.conf:
rdr sppp0 111.111.111.165/32 port 53 -> 111.111.111.166 port 53 tcp
rdr sppp0 111.111.111.165/32 port 53 -> 111.111.111.166 port 53 udp
...it works, tested from outside. The global zone works.




*** The problem ***

With the external address/route on sppp0:
+ Zones cannot access the outside.
+ Zones are not accessible from the outside.




*** Failed method 1 ***

When pppd runs it create the default route in the global zone. Nothing
changes in the zones because there is no zone interface with the
external address nor does a default routes exist in the zones.

I want to do something like:
global# ifconfig sppp0 addif 111.111.111.162/29 zone mailhost
global# ifconfig sppp0 addif 111.111.111.163/29 zone webserver

...which is close to working, ifconfig -a gives:
sppp0:
flags=10011008d1<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST,ROUTER,IPv4,FIXEDMTU>
mtu 1492 index 30
inet 111.111.111.166 --> 1.2.3.4 netmask ff000000
ether 0:0:0:0:0:0
sppp0:1: flags=11008d0<POINTOPOINT,RUNNING,NOARP,MULTICAST,ROUTER,IPv4>
mtu 1492 index 30
zone mailhost
inet 111.111.111.162 --> 0.0.0.0 netmask fffffff8
sppp0:2: flags=11008d0<POINTOPOINT,RUNNING,NOARP,MULTICAST,ROUTER,IPv4>
mtu 1492 index 30
zone webserver
inet 111.111.111.163 --> 0.0.0.0 netmask fffffff8
...but the interfaces do not appear in the zones and are not "UP",
unless there are some options this is not a solution.





*** Failed method 2 ***

I'd prefer if all the zones had local addresses (allowing for more zones
than external address and flexibility). I try to use ipfilter rdr and
map. Using the example of SMTP to the mailhost, ipnat.conf has:

# redirect
rdr sppp0 111.111.111.162/32 port 25 -> 192.168.1.101 port 25 tcp
map sppp0 from any to 192.168.1.101 port = 25 -> 111.111.111.162/32

...which is half working because I can see the RDR in both "ipnat -l"
and the ipfilter log:
May 29 10:23:26 sessia ipmon[11787]: [ID 702911 local0.notice]
10:23:26.392533 sppp0 @0:23 p 123.123.123.123,60068 -> 192.168.1.101,25
PR tcp len 20 44 -S K-S IN NAT
...but the requests get lost. Do they reach the zone? Does the zone
know where to send them back? Is the map correct? I'm logging all so I
don't think ipfilter is blocking.


Also in ipnat.conf:
# outgoing NAT:
map sppp0 192.168.0.0/16 -> 0/32 proxy port ftp ftp/tcp
map sppp0 192.168.0.0/16 -> 0/32 portmap tcp/udp auto
map sppp0 192.168.0.0/16 -> 0/32 tcp/udp
...which I hope says make anything internal going out on sppp0 look like
it came from the address of sppp0, ie, 111.111.111.166.


My attempt to let the zones know the default route:
global# route add -net 1.2.3.4/32 192.168.1.101 -interface
global# route add -net 1.2.3.4/32 192.168.1.102 -interface
...that is add the global default's gateway to the zones. Result in zone:

mailhost$ netstat -rn

Routing Table: IPv4
Destination Gateway Flags Ref Use Interface
--------------- -------------------- ----- ----- ---------- ---------
default 1.2.3.4 UG 1 2956
1.2.3.4 192.168.1.101 UH 1 2649 bge1:2
192.168.1.0 192.168.1.101 U 1 6110 bge1:2
224.0.0.0 192.168.1.101 U 1 0 bge1:2
127.0.0.1 127.0.0.1 UH 5 9139772 lo0:2

...which I think says anything from the zone for outside send to 1.2.3.4
via 192.168.1.101. But this doesn't work either. Zones can now reach
sppp0 but not the outside.

mailhost$ ping 111.111.111.166
111.111.111.166 is alive
mailhost$ ping 1.2.3.4
no answer from 1.2.3.4




*** Other information ***

global# routeadm | egrep '(enabled|online)'
IPv4 routing enabled enabled
IPv4 forwarding enabled enabled
online svc:/network/routing/route:default

global# grep intercept_loopback /etc/ipf/ipf.conf
set intercept_loopback true;




So how do I get zones to route in and out via sppp0? This should be a
trivial ipfilter/ipnat routing problem but I'm stumped.



James.

[John D Groenveld]

In article <5387160c$0$1419$5b6a...@news.zen.co.uk>,

James <li...@xdrv.co.uk> wrote:
>I have Solaris 10 with zones. The zones all have internal addresses on

Can you upgrade your global zone to S11?
Crossbow makes your preferred configuration and lots of others possible.

John
groe...@acm.org

[James]

On 29/05/2014 12:16, John D Groenveld wrote:

>> I have Solaris 10 with zones. The zones all have internal addresses on
>
> Can you upgrade your global zone to S11?

Not without a lot of effort, I accept S11 might make this easy but am I
to understand this can never work on S10?


James.

[John D Groenveld]

In article <538718aa$0$1413$5b6a...@news.zen.co.uk>,

James <li...@xdrv.co.uk> wrote:
>Not without a lot of effort, I accept S11 might make this easy but am I
>to understand this can never work on S10?

I vaguely recall a discussion on the OpenSolaris Crossbow m/l
about why the S10 logical interfaces used by the S10 zone
shared network interfaces can't be made to do what you want,
but I forget the details.

I think the ugly hack was to add physical interfaces and a
physical switch.

John
groe...@acm.org

[Andrew Gabriel]

In article <5387160c$0$1419$5b6a...@news.zen.co.uk>,

James <li...@xdrv.co.uk> writes:
>
> *** Failed method 1 ***
>
> When pppd runs it create the default route in the global zone. Nothing
> changes in the zones because there is no zone interface with the
> external address nor does a default routes exist in the zones.
>
> I want to do something like:
> global# ifconfig sppp0 addif 111.111.111.162/29 zone mailhost
> global# ifconfig sppp0 addif 111.111.111.163/29 zone webserver
>
> ...which is close to working, ifconfig -a gives:
> sppp0:
> flags=10011008d1<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST,ROUTER,IPv4,FIXEDMTU>
> mtu 1492 index 30
> inet 111.111.111.166 --> 1.2.3.4 netmask ff000000
> ether 0:0:0:0:0:0
> sppp0:1: flags=11008d0<POINTOPOINT,RUNNING,NOARP,MULTICAST,ROUTER,IPv4>
> mtu 1492 index 30
> zone mailhost
> inet 111.111.111.162 --> 0.0.0.0 netmask fffffff8
> sppp0:2: flags=11008d0<POINTOPOINT,RUNNING,NOARP,MULTICAST,ROUTER,IPv4>
> mtu 1492 index 30
> zone webserver
> inet 111.111.111.163 --> 0.0.0.0 netmask fffffff8
> ...but the interfaces do not appear in the zones and are not "UP",
> unless there are some options this is not a solution.

try:
ifconfig sppp0 addif 111.111.111.162/29 zone mailhost up

(or better, do it with zonecfg which knows all the other things which
need doing)

--
Andrew Gabriel
[email address is not usable -- followup in the newsgroup]

[James]

On 29/05/2014 12:53, John D Groenveld wrote:

> I vaguely recall a discussion on the OpenSolaris Crossbow m/l
> about why the S10 logical interfaces used by the S10 zone
> shared network interfaces can't be made to do what you want,
> but I forget the details.
>
> I think the ugly hack was to add physical interfaces and a
> physical switch.

The local net is already physical as it connects to many other machines:
laptop, TV, a music player but thankfully not the lawn mower and fridge
(yet).

This would involve sending packets on a journey to force them to "cross"
the ipfilter interface. I thought that's what "set intercept_loopback
true" did. I'll play with this idea.

Adding another device and hop is less clean than using a router in the
input chain. I want to avoid adding (yet another) router. It costs
money, uses electricity, is one more thing feed from UPS, trip over and
can fail. Previous routers, I have 6 obsolete ones, only sent
everything to my server for filtering and distribution. All a router
does for me is the PPPoE chap/pap authentication which Solaris can do.

Thank you for your interest.


James Lee.

[James]

On 29/05/2014 16:40, Andrew Gabriel wrote:

>> sppp0:2: flags=11008d0<POINTOPOINT,RUNNING,NOARP,MULTICAST,ROUTER,IPv4>
>> mtu 1492 index 30
>> zone webserver
>> inet 111.111.111.163 --> 0.0.0.0 netmask fffffff8
>> ...but the interfaces do not appear in the zones and are not "UP",
>> unless there are some options this is not a solution.
>
> try:
> ifconfig sppp0 addif 111.111.111.162/29 zone mailhost up

# ifconfig sppp0 addif 111.111.111.162/29 zone mailhost up
Created new logical interface sppp0:1
ifconfig: setifflags: SIOCSLIFFLAGS: sppp0:1: Invalid argument

...and it's not up, the addition of the "up" triggered the "Invalid
argument". It might help if I knew the command that pppd used to create
sppp0.

Sorry I didn't make that clear the first time but if I listed everything
I have tried that didn't work you wouldn't have read to the end!

> (or better, do it with zonecfg which knows all the other things which
> need doing)

"Have you tried turning it off and on". I ought to be able to do this
without rebooting when the connection drops and sppp0 disappears and
remakes. I don't think exporting sppp0 to the zones is the correct
direction but it follows what I do when the global zone has the external
network on a normal Ethernet connection. It's a (failed) example of
trying to get an interface with the external network / default route
known to the zones.


Note: as a temporary workaround I've set up proxies in global for SMTP,
IMAP and HTTP to the zones.

BT once told me: "It's not possible to connect a Unix computer to the
Internet". We know that's not true but it sometimes feels like I'm the
only person doing it.

Thank you Andrew.


James Lee.

[Andrew Gabriel]

In article <53886546$0$1125$5b6a...@news.zen.co.uk>,

James <li...@xdrv.co.uk> writes:
> On 29/05/2014 16:40, Andrew Gabriel wrote:
>
>>> sppp0:2: flags=11008d0<POINTOPOINT,RUNNING,NOARP,MULTICAST,ROUTER,IPv4>
>>> mtu 1492 index 30
>>> zone webserver
>>> inet 111.111.111.163 --> 0.0.0.0 netmask fffffff8
>>> ...but the interfaces do not appear in the zones and are not "UP",
>>> unless there are some options this is not a solution.
>>
>> try:
>> ifconfig sppp0 addif 111.111.111.162/29 zone mailhost up
>
>
> # ifconfig sppp0 addif 111.111.111.162/29 zone mailhost up
> Created new logical interface sppp0:1
> ifconfig: setifflags: SIOCSLIFFLAGS: sppp0:1: Invalid argument
>
> ...and it's not up, the addition of the "up" triggered the "Invalid
> argument". It might help if I knew the command that pppd used to create
> sppp0.

Check you aren't already using 111.111.111.162.
You can have the same address on multiple interfaces,
but only one of them can be UP.

[James]

On 30/05/2014 12:35, Andrew Gabriel wrote:

>> # ifconfig sppp0 addif 111.111.111.162/29 zone mailhost up
>> Created new logical interface sppp0:1
>> ifconfig: setifflags: SIOCSLIFFLAGS: sppp0:1: Invalid argument
>>
>> ...and it's not up, the addition of the "up" triggered the "Invalid
>> argument". It might help if I knew the command that pppd used to create
>> sppp0.
>
> Check you aren't already using 111.111.111.162.
> You can have the same address on multiple interfaces,
> but only one of them can be UP.

No existing entry. Adding one twice gives:
ifconfig: setifflags: SIOCSLIFFLAGS: nge0:3: Cannot assign requested address

However... Adding a destination gets it to appear in the zone.

global# ifconfig sppp0 addif 111.111.111.162/29 zone mailhost
destination 1.2.3.4


mailhost$ ifconfig sppp0:1
sppp0:1:
flags=11008d1<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST,ROUTER,IPv4> mtu
1492 index 34
inet 111.111.111.162 --> 1.2.3.4 netmask fffffff8

mailhost$ netstat -rn

Routing Table: IPv4
Destination Gateway Flags Ref Use Interface

-------------------- -------------------- ----- ----- ---------- ---------
default 1.2.3.4 UG 1 2494
1.2.3.4 111.111.111.162 UH 1 1 sppp0:1
...


mailhost$ ping bbc.co.uk
bbc.co.uk is alive

mailhost$ curl -I bbc.co.uk
HTTP/1.1 301 Moved Permanently
Content-Type: text/html
Date: Fri, 30 May 2014 12:08:33 GMT
Location: http://www.bbc.co.uk/
Connection: Keep-Alive
Content-Length: 0


Woohoooo. Let me test this some more and I'll report back again.



James.

[Ian Collins]

James wrote:
> On 30/05/2014 12:35, Andrew Gabriel wrote:
>
>>> # ifconfig sppp0 addif 111.111.111.162/29 zone mailhost up
>>> Created new logical interface sppp0:1
>>> ifconfig: setifflags: SIOCSLIFFLAGS: sppp0:1: Invalid argument
>>>
>>> ...and it's not up, the addition of the "up" triggered the "Invalid
>>> argument". It might help if I knew the command that pppd used to create
>>> sppp0.
>>
>> Check you aren't already using 111.111.111.162.
>> You can have the same address on multiple interfaces,
>> but only one of them can be UP.
>
> No existing entry. Adding one twice gives:
> ifconfig: setifflags: SIOCSLIFFLAGS: nge0:3: Cannot assign requested address
>
> However... Adding a destination gets it to appear in the zone.
>
> global# ifconfig sppp0 addif 111.111.111.162/29 zone mailhost
> destination 1.2.3.4

John's original suggestion would have saved you more pain than it would
have cost!

Migrating from Solaris 10 to 11 can be relatively painless if you are
using zones for most tasks.

By the way, did you try adding a non-exclusive network interface to the
zone, I think that is what Andrew Gabriel was suggesting.

--
Ian Collins