Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.

Dismiss

freebsd-pf Digest, Vol 277, Issue 1

11 views

Skip to first unread message

freebsd-p...@freebsd.org

unread,

Feb 22, 2010, 6:09:02 AM2/22/10

to freeb...@freebsd.org

Send freebsd-pf mailing list submissions to
freeb...@freebsd.org

To subscribe or unsubscribe via the World Wide Web, visit
http://lists.freebsd.org/mailman/listinfo/freebsd-pf
or, via email, send a message with subject or body 'help' to
freebsd-p...@freebsd.org

You can reach the person managing the list at
freebsd-...@freebsd.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of freebsd-pf digest..."

Today's Topics:

1. Re: using pf to NAT with only one NIC (jhell)
2. Current problem reports assigned to freeb...@FreeBSD.org
(FreeBSD bugmaster)
3. Re: using pf to NAT with only one NIC (Maurice)
4. Re: How make the route-to working ? (geoffroy desvernay)
5. Re: How make the route-to working ? (Albert Shih)
6. Re: How make the route-to working ? (geoffroy desvernay)
7. Re: How make the route-to working ? (Sam Fourman Jr.)
8. pf: nat works, ip blocking and logging do not (Alex Teslik)
9. Re: pf: nat works, ip blocking and logging do not
(Morgan Wesstr?m)
10. Re: How make the route-to working ? (geoffroy desvernay)
11. Re: pf: nat works, ip blocking and logging do not (David DeSimone)
12. Server unresponsive when using transparent bridging w/ pf +
pfsync (kevin)
13. Re: How make the route-to working ? (Albert Shih)
14. Current problem reports assigned to freeb...@FreeBSD.org
(FreeBSD bugmaster)
15. Re: How make the route-to working ? (geoffroy desvernay)
16. Re: How make the route-to working ? (Albert Shih)
17. Possible bug in TSO or in pf on bce (Albert Shih)
18. Re: Possible bug in TSO or in pf on bce (Pyun YongHyeon)
19. Re: Possible bug in TSO or in pf on bce (Julian Elischer)
20. Re: Network simulation using jails & vimage (Bjoern A. Zeeb)
21. Re: Network simulation using jails & vimage (Julian Elischer)
22. Re: Network simulation using jails & vimage (Jim Sifferle)
23. Re: Network simulation using jails & vimage (Jim Sifferle)
24. Re: Network simulation using jails & vimage (Julian Elischer)
25. Current problem reports assigned to freeb...@FreeBSD.org
(FreeBSD bugmaster)

----------------------------------------------------------------------

Message: 1
Date: Sat, 6 Feb 2010 00:09:46 -0500
From: jhell <jh...@DataIX.net>
Subject: Re: using pf to NAT with only one NIC
To: Peter Maxwell <pe...@allicient.co.uk>
Cc: freeb...@freebsd.org
Message-ID: <alpine.BSF.2.00.1...@pragry.qngnvk.ybpny>
Content-Type: TEXT/PLAIN; charset=ISO-8859-1; format=flowed

On Fri, 5 Feb 2010 19:47, peter@ wrote:
> Hi Maurice,
>
> Yes, you can do it without much difficulty and I've got my server
> setup in that manner: there's about twenty separate jails that can
> access the internet via specific NAT rules and incoming services
> handled via RDR rules. Note: you won't be able to ping from a jail,
> unless you want to allow your jailed processes to create raw sockets
> (you don't) :-)
>
> There's probably many ways it can be done, but what I did was something like:
>
>
> i) create a second loopback interface, lo1 (c.f. cloned interfaces)
> and assign appropriate alias netblocks for your jails on that
> interface;
>
>
> ii) create your pf.conf, set skip on lo0 but not the external or lo1 interface;
>
>
> iii) I'd set "set state-policy if-bound" so you know what's going on;
>
>
> iv) don't use the antispoof keyword, it will make a mess in this situation;
>
>
> v) setting up bind to handle local dns resolution is a good idea -
> point your jails towards this and you'll need to add in an appropriate
> rule(s) later on;
>
>
> vi) setup outgoing nat rules, e.g.
>
> nat on $ext_if inet from $int_ip_smtp to ! $int_lo1_if:network port
> smtp -> $ext_ip
>
>
> vii) setup incoming services, e.g.
>
> rdr on $ext_if proto tcp from any to $ext_ip port smtp -> $int_ip_mail port smtp
>
>
> viii) put in pass rules to allow nat out and rdr in; remember NAT is
> done first, so your outgoing packets ALL have source IP of the
> external IP now and not the jail IP
>
> pass out log on $ext_if proto tcp from $ext_ip to any port smtp flags
> S/SA modulate state
> pass in log on $ext_if proto tcp from any to $int_ip_mail port smtp
> flags S/SA modulate state
>
>
> ix) allow jail implicit access to itself
>
> pass log on $int_lo1_if proto { udp, tcp } from $int_ip_mail to
> $int_ip_mail flags S/SA keep state
>
>
> x) add in rules to allow any interjail communication as needed
> (remember the incoming/outgoing packets appear the other way round
> here - use tcpdump to check if in doubt)
>
>
> If you have any problems, run tcpdump in a serarate terminal window to
> determine what's going on.
>
>
> Peter
>
>
>
>
>
>
> On 5 February 2010 22:53, Maurice <mau...@gmail.com> wrote:
>> Hi,
>>
>> I have been looking for a couple days now, with no luck, for some direction
>> as to whether I can successfully configure my freebsd to NAT with only one
>> NIC. �This is because I am setting up my system to jail my webserver, and I
>> don't think I can get it to work without NATting it. If you have an
>> alternate solution that would be great too. This is what my pf.conf looks
>> like right now:
>>
>>
>> # � � � $FreeBSD: src/share/examples/pf/pf.conf,v 1.1.2.1.6.1 2009/04/15
>> 03:14:26 kensmith Exp $
>> # � � � $OpenBSD: pf.conf,v 1.34 2007/02/24 19:30:59 millert Exp $
>> #
>> # See pf.conf(5) and /usr/share/examples/pf for syntax and examples.
>> # Remember to set net.inet.ip.forwarding=1 and/or net.inet6.ip6.forwarding=1
>> # in /etc/sysctl.conf if packets are to be forwarded between interfaces.
>>
>> block in all
>> block out all
>>
>> ext_if="fxp0"
>> #int_if="int0"
>> all_if="{fxp0, lo0}"
>>
>> #Internal network subnet
>> int_net="10.0.0.0/32"
>>
>> #name and IP of webserver
>> APACHE="10.0.0.1"
>>
>> #table <spamd-white> persist
>>
>> set skip on lo
>>
>> scrub in
>>
>> #nat-anchor "ftp-proxy/*"
>> #rdr-anchor "ftp-proxy/*"
>> #nat on $ext_if from !($ext_if) -> ($ext_if:0)
>> #rdr pass on $int_if proto tcp to port ftp -> 127.0.0.1 port 8021
>> #no rdr on $ext_if proto tcp from <spamd-white> to any port smtp
>> #rdr pass on $ext_if proto tcp from any to any port smtp \
>> # � � � -> 127.0.0.1 port spamd
>>
>> #anchor "ftp-proxy/*"
>> #pass out
>>
>> #pass quick on $int_if no state
>> #antispoof quick for { lo $int_if }
>> block in quick from urpf-failed
>>
>> pass in on $ext_if proto tcp to ($ext_if) port ssh synproxy state
>> rdr on $all_if proto tcp from any to fxp0 port 80 -> $APACHE port 80
>> nat on $ext_if from $APACHE to any -> fxp0
>>

Your placement of nat and redirect rules are a little bit worrisome.
pf.conf as stated by its manual page is ordered (see following)

# [Macros] i.e. variable=lo1
# [Options] i.e. set etc.. etc..
# [Normalization] i.e. scrub
# [Queuing] i.e. ALTQ
# [Translation] i.e. NAT RDR etc...
# [Filtering] i.e. pass & block rules

Beware that there is quite the change for rule-sets ahead if the newer
version of pf that is in the works for OpenBSD ever makes it downstream to
FreeBSD.

I Personally do not know if the way you have your rule-set configured would
cause any havoc with NAT since you have it mingled between filtering rules
but it would be good practice to stick to whats already drawn in the
manual page.

Best of luck.

>> #pass in log on $ext_if proto tcp to ($ext_if) port smtp
>> #pass out log on $ext_if proto tcp from ($ext_if) to port smtp
>>
>> That doesn't seem to be doing the trick, since I can't ping and DNS won't
>> resolve anything from within the jail (APACHE). I am going off some examples
>> I found that would seem to suggest it is possible with only one NIC, but I
>> can't seem to get it to work. Any help/advice would be greatly appreciated.
>>
>> thanks,
>>
>> Maurice
>

jhell

------------------------------

Message: 2
Date: Mon, 8 Feb 2010 11:07:01 GMT
From: FreeBSD bugmaster <bugm...@FreeBSD.org>
Subject: Current problem reports assigned to freeb...@FreeBSD.org
To: freeb...@FreeBSD.org
Message-ID: <201002081107....@freefall.freebsd.org>

Note: to view an individual PR, use:
http://www.freebsd.org/cgi/query-pr.cgi?pr=(number).

The following is a listing of current problems submitted by FreeBSD users.
These represent problem reports covering all versions including
experimental development code and obsolete releases.

S Tracker Resp. Description
--------------------------------------------------------------------------------
o kern/143543 pf [pf] [panic] PF route-to causes kernel panic
o bin/143504 pf [patch] outgoing states are not killed by authpf(8)
o conf/142961 pf [pf] No way to adjust pidfile in pflogd
o conf/142817 pf [patch] etc/rc.d/pf: silence pfctl
o kern/141905 pf [pf] [panic] pf kernel panic on 7.2-RELEASE with empty
o kern/140697 pf [pf] pf behaviour changes - must be documented
o kern/137982 pf [pf] when pf can hit state limits, random IP failures
o kern/136781 pf [pf] Packets appear to drop with pf scrub and if_bridg
o kern/135948 pf [pf] [gre] pf not natting gre protocol
o kern/135162 pf [pfsync] pfsync(4) not usable with GENERIC kernel
o kern/134996 pf [pf] Anchor tables not included when pfctl(8) is run w
o kern/133732 pf [pf] max-src-conn issue
o kern/132769 pf [pf] [lor] 2 LOR's with pf task mtx / ifnet and rtent
f kern/132176 pf [pf] pf stalls connection when using route-to [regress
o conf/130381 pf [rc.d] [pf] [ip6] ipv6 not fully configured when pf st
o kern/129861 pf [pf] [patch] Argument names reversed in pf_table.c:_co
o kern/127920 pf [pf] ipv6 and synproxy don't play well together
o conf/127814 pf [pf] The flush in pf_reload in /etc/rc.d/pf does not w
o kern/127439 pf [pf] deadlock in pf
f kern/127345 pf [pf] Problem with PF on FreeBSD7.0 [regression]
o kern/127121 pf [pf] [patch] pf incorrect log priority
o kern/127042 pf [pf] [patch] pf recursion panic if interface group is
o kern/125467 pf [pf] pf keep state bug while handling sessions between
s kern/124933 pf [pf] [ip6] pf does not support (drops) IPv6 fragmented
o kern/124364 pf [pf] [panic] Kernel panic with pf + bridge
o kern/122773 pf [pf] pf doesn't log uid or pid when configured to
o kern/122014 pf [pf] [panic] FreeBSD 6.2 panic in pf
o kern/121704 pf [pf] PF mangles loopback packets
o kern/120281 pf [pf] [request] lost returning packets to PF for a rdr
o kern/120057 pf [pf] [patch] Allow proper settings of ALTQ_HFSC. The c
o bin/118355 pf [pf] [patch] pfctl(8) help message options order false
o kern/114567 pf [pf] [lor] pf_ioctl.c + if.c
o kern/114095 pf [carp] carp+pf delay with high state limit
o kern/111220 pf [pf] repeatable hangs while manipulating pf tables
s conf/110838 pf [pf] tagged parameter on nat not working on FreeBSD 5.
o kern/103283 pf pfsync fails to sucessfully transfer some sessions
o kern/103281 pf pfsync reports bulk update failures
o kern/93825 pf [pf] pf reply-to doesn't work
o sparc/93530 pf [pf] Incorrect checksums when using pf's route-to on s
o kern/92949 pf [pf] PF + ALTQ problems with latency
o bin/86635 pf [patch] pfctl(8): allow new page character (^L) in pf.
o kern/82271 pf [pf] cbq scheduler cause bad latency

42 problems total.

------------------------------

Message: 3
Date: Mon, 8 Feb 2010 11:15:24 -0700
From: Maurice <mau...@gmail.com>
Subject: Re: using pf to NAT with only one NIC
To: jhell <jh...@dataix.net>
Cc: freeb...@freebsd.org
Message-ID:
<d3e0b6a01002081015i82...@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1

On Fri, Feb 5, 2010 at 10:09 PM, jhell <jh...@dataix.net> wrote:

>
> On Fri, 5 Feb 2010 19:47, peter@ wrote:
>
>> Hi Maurice,
>>
>> Yes, you can do it without much difficulty and I've got my server
>> setup in that manner: there's about twenty separate jails that can
>> access the internet via specific NAT rules and incoming services
>> handled via RDR rules. Note: you won't be able to ping from a jail,
>> unless you want to allow your jailed processes to create raw sockets
>> (you don't) :-)
>>
>> There's probably many ways it can be done, but what I did was something
>> like:
>>
>>
>> i) create a second loopback interface, lo1 (c.f. cloned interfaces)
>> and assign appropriate alias netblocks for your jails on that
>> interface;
>>
>>
>> ii) create your pf.conf, set skip on lo0 but not the external or lo1
>> interface;
>>
>>
>> iii) I'd set "set state-policy if-bound" so you know what's going on;
>>
>>
>> iv) don't use the antispoof keyword, it will make a mess in this
>> situation;
>>
>>
>> v) setting up bind to handle local dns resolution is a good idea -
>> point your jails towards this and you'll need to add in an appropriate
>> rule(s) later on;
>>
>>
>> vi) setup outgoing nat rules, e.g.
>>
>> nat on $ext_if inet from $int_ip_smtp to ! $int_lo1_if:network port
>> smtp -> $ext_ip
>>
>>
>> vii) setup incoming services, e.g.
>>
>> rdr on $ext_if proto tcp from any to $ext_ip port smtp -> $int_ip_mail
>> port smtp
>>
>>
>> viii) put in pass rules to allow nat out and rdr in; remember NAT is
>> done first, so your outgoing packets ALL have source IP of the
>> external IP now and not the jail IP
>>
>> pass out log on $ext_if proto tcp from $ext_ip to any port smtp flags
>> S/SA modulate state
>> pass in log on $ext_if proto tcp from any to $int_ip_mail port smtp
>> flags S/SA modulate state
>>
>>
>> ix) allow jail implicit access to itself
>>
>> pass log on $int_lo1_if proto { udp, tcp } from $int_ip_mail to
>> $int_ip_mail flags S/SA keep state
>>
>>
>> x) add in rules to allow any interjail communication as needed
>> (remember the incoming/outgoing packets appear the other way round
>> here - use tcpdump to check if in doubt)
>>
>>
>> If you have any problems, run tcpdump in a serarate terminal window to
>> determine what's going on.
>>
>>
>> Peter
>>
>>
>>
>>
>>
>>
>> On 5 February 2010 22:53, Maurice <mau...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I have been looking for a couple days now, with no luck, for some
>>> direction
>>> as to whether I can successfully configure my freebsd to NAT with only
>>> one
>>> NIC. This is because I am setting up my system to jail my webserver, and
>>> I
>>> don't think I can get it to work without NATting it. If you have an
>>> alternate solution that would be great too. This is what my pf.conf looks
>>> like right now:
>>>
>>>
>>> # $FreeBSD: src/share/examples/pf/pf.conf,v 1.1.2.1.6.1 2009/04/15
>>> 03:14:26 kensmith Exp $
>>> # $OpenBSD: pf.conf,v 1.34 2007/02/24 19:30:59 millert Exp $
>>> #
>>> # See pf.conf(5) and /usr/share/examples/pf for syntax and examples.
>>> # Remember to set net.inet.ip.forwarding=1 and/or
>>> net.inet6.ip6.forwarding=1
>>> # in /etc/sysctl.conf if packets are to be forwarded between interfaces.
>>>
>>> block in all
>>> block out all
>>>
>>> ext_if="fxp0"
>>> #int_if="int0"
>>> all_if="{fxp0, lo0}"
>>>
>>> #Internal network subnet
>>> int_net="10.0.0.0/32"
>>>
>>> #name and IP of webserver
>>> APACHE="10.0.0.1"
>>>
>>> #table <spamd-white> persist
>>>
>>> set skip on lo
>>>
>>> scrub in
>>>
>>> #nat-anchor "ftp-proxy/*"
>>> #rdr-anchor "ftp-proxy/*"
>>> #nat on $ext_if from !($ext_if) -> ($ext_if:0)
>>> #rdr pass on $int_if proto tcp to port ftp -> 127.0.0.1 port 8021
>>> #no rdr on $ext_if proto tcp from <spamd-white> to any port smtp
>>> #rdr pass on $ext_if proto tcp from any to any port smtp \
>>> # -> 127.0.0.1 port spamd
>>>
>>> #anchor "ftp-proxy/*"
>>> #pass out
>>>
>>> #pass quick on $int_if no state
>>> #antispoof quick for { lo $int_if }
>>> block in quick from urpf-failed
>>>
>>> pass in on $ext_if proto tcp to ($ext_if) port ssh synproxy state
>>> rdr on $all_if proto tcp from any to fxp0 port 80 -> $APACHE port 80
>>> nat on $ext_if from $APACHE to any -> fxp0
>>>
>>>
> Your placement of nat and redirect rules are a little bit worrisome.
> pf.conf as stated by its manual page is ordered (see following)
>
> # [Macros] i.e. variable=lo1 # [Options] i.e. set etc.. etc..
> # [Normalization] i.e. scrub
> # [Queuing] i.e. ALTQ
> # [Translation] i.e. NAT RDR etc...
> # [Filtering] i.e. pass & block rules
>
> Beware that there is quite the change for rule-sets ahead if the newer
> version of pf that is in the works for OpenBSD ever makes it downstream to
> FreeBSD.
>
> I Personally do not know if the way you have your rule-set configured would
> cause any havoc with NAT since you have it mingled between filtering rules
> but it would be good practice to stick to whats already drawn in the manual
> page.
>
> Best of luck.
>
>
> #pass in log on $ext_if proto tcp to ($ext_if) port smtp
>>> #pass out log on $ext_if proto tcp from ($ext_if) to port smtp
>>>
>>> That doesn't seem to be doing the trick, since I can't ping and DNS won't
>>> resolve anything from within the jail (APACHE). I am going off some
>>> examples
>>> I found that would seem to suggest it is possible with only one NIC, but
>>> I
>>> can't seem to get it to work. Any help/advice would be greatly
>>> appreciated.
>>>
>>> thanks,
>>>
>>> Maurice
>>>
>>
>>
> --
>
> jhell
>

Thank you for your instructions, gentlemen. I will do my best to follow
them. This is my first stab at setting up a UNIX box, so thank you for your
patience as well!

regards,

------------------------------

Message: 4
Date: Thu, 11 Feb 2010 23:38:56 +0100
From: geoffroy desvernay <dg...@centrale-marseille.fr>
Subject: Re: How make the route-to working ?
To: Albert Shih <Alber...@obspm.fr>
Cc: freeb...@freebsd.org
Message-ID: <4B74870...@centrale-marseille.fr>
Content-Type: text/plain; charset="iso-8859-1"

Albert Shih a �crit :
> Hi all,
>
> I've a problem with route-to.
>
> I've a server with 2 interfaces, and I'm running jail on this server. Each
> interface have is own public IP address.
>
> eth0 -- IP0 eth1 -- IP1
>
> and I've a default route (for example in IP0 subnet).
>
> So if the jail is in the IP0 subnet no problem everything work.
>
> Now if I put a jail in IP1 subnet, and some client try to connect to this
> jail the answer come out through eth0 because of the default route (suppose
> the client is not on my subnet).
>
> I don't want that. I want the answer come out through the eth1
>
> I'm trying to use pf to do that and put in my pf.conf something like
>
> pass in all
> pass out all
> pass out on eth0 route-to {(eth0 IP0_Gateway)} from <IP0> to ! IP0_subnet
> pass out on eth1 route-to {(eth1 IP1_Gateway)} from <IP1> to ! IP1_subnet
>
> but it's not working, if I run a tcpdump on the host I can see the
> incoming packet come in from eth1 and the outgoing come out on eth0.
>
> And if I try do remove default route the outgoing packet don't come out....
>
> Any help ?
>
> Regards.
>
>
Hi,

I'm using that for the same case:

You just have to catch packets on the interface they would go normally:

pass out on *eth0* route-to {(eth1 IP1_Gateway)} from <IP1> to !eth1:network

The other rule is not needed in this case

You may also try instead a 'reply-to' rule on eth1's inbound, as David
DeSimone suggested.

A third and cleaner solution would be to use multiple routing-tables -
see setfib(1) and 'options ROUTETABLES' of the kernel...

HTH
--
*Geoffroy Desvernay*
C.R.I - Administration syst�mes et r�seaux
Ecole Centrale de Marseille

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 490 bytes
Desc: OpenPGP digital signature
Url : http://lists.freebsd.org/pipermail/freebsd-pf/attachments/20100211/5ab224c6/signature-0001.pgp

------------------------------

Message: 5
Date: Fri, 12 Feb 2010 17:44:54 +0100
From: Albert Shih <Alber...@obspm.fr>
Subject: Re: How make the route-to working ?
To: geoffroy desvernay <dg...@centrale-marseille.fr>
Cc: freeb...@freebsd.org
Message-ID: <20100212164...@obspm.fr>
Content-Type: text/plain; charset=iso-8859-1

Le 11/02/2010 � 23:38:56+0100, geoffroy desvernay a �crit
> Albert Shih a �crit :
> > Hi all,
> >
> > I've a problem with route-to.
> >
> > I've a server with 2 interfaces, and I'm running jail on this server. Each
> > interface have is own public IP address.
> >
> > eth0 -- IP0 eth1 -- IP1
> >
> > and I've a default route (for example in IP0 subnet).
> >
> > So if the jail is in the IP0 subnet no problem everything work.
> >
> > Now if I put a jail in IP1 subnet, and some client try to connect to this
> > jail the answer come out through eth0 because of the default route (suppose
> > the client is not on my subnet).
> >
> > I don't want that. I want the answer come out through the eth1
> >
> > I'm trying to use pf to do that and put in my pf.conf something like
> >
> > pass in all
> > pass out all
> > pass out on eth0 route-to {(eth0 IP0_Gateway)} from <IP0> to ! IP0_subnet
> > pass out on eth1 route-to {(eth1 IP1_Gateway)} from <IP1> to ! IP1_subnet
> >
> > but it's not working, if I run a tcpdump on the host I can see the
> > incoming packet come in from eth1 and the outgoing come out on eth0.
> >
> > And if I try do remove default route the outgoing packet don't come out....
> >
> > Any help ?
> >
> > Regards.
> >
Lots of thanks for your answer.

>
> You just have to catch packets on the interface they would go normally:
>
> pass out on *eth0* route-to {(eth1 IP1_Gateway)} from <IP1> to !eth1:network
>
> The other rule is not needed in this case
>
> You may also try instead a 'reply-to' rule on eth1's inbound, as David
> DeSimone suggested.

OK now it's working. But I have some big trouble about the bandwith.

Now when I try to do something like a scp, or ftp or wget from inside a
jail to outside, everything work fine. The traffic go to right interface,
the answer too.

But when I try to do some network connection (ssh, scp etc..) from outside
to a jail the bandwith is catastrophic (~40kB/s on 1Gbit/s).

And for you ?

>
> A third and cleaner solution would be to use multiple routing-tables -
> see setfib(1) and 'options ROUTETABLES' of the kernel...

I already try this, I don't known how to make it work. I'm going to try
again.

Regards.

Thanks again.

--
Albert SHIH
SIO batiment 15
Observatoire de Paris Meudon
5 Place Jules Janssen
92195 Meudon Cedex
T�l�phone : 01 45 07 76 26/06 86 69 95 71
Heure local/Local time:
Ven 12 f�v 2010 17:41:22 CET

------------------------------

Message: 6
Date: Sat, 13 Feb 2010 09:11:24 +0100
From: geoffroy desvernay <dg...@centrale-marseille.fr>
Subject: Re: How make the route-to working ?
To: Albert Shih <Alber...@obspm.fr>
Cc: freeb...@freebsd.org
Message-ID: <4B765EAC...@centrale-marseille.fr>
Content-Type: text/plain; charset="iso-8859-1"

Albert Shih a �crit :
> Le 11/02/2010 � 23:38:56+0100, geoffroy desvernay a �crit
>> Albert Shih a �crit :
>>> Hi all,
>>>
>>> I've a problem with route-to.
>>>
>>> I've a server with 2 interfaces, and I'm running jail on this server. Each
>>> interface have is own public IP address.
>>>
>>> eth0 -- IP0 eth1 -- IP1
>>>
>>> and I've a default route (for example in IP0 subnet).
>>>
>>> So if the jail is in the IP0 subnet no problem everything work.
>>>
>>> Now if I put a jail in IP1 subnet, and some client try to connect to this
>>> jail the answer come out through eth0 because of the default route (suppose
>>> the client is not on my subnet).
>>>
>>> I don't want that. I want the answer come out through the eth1
>>>
>>> I'm trying to use pf to do that and put in my pf.conf something like
>>>
>>> pass in all
>>> pass out all
>>> pass out on eth0 route-to {(eth0 IP0_Gateway)} from <IP0> to ! IP0_subnet
>>> pass out on eth1 route-to {(eth1 IP1_Gateway)} from <IP1> to ! IP1_subnet
>>>
>>> but it's not working, if I run a tcpdump on the host I can see the
>>> incoming packet come in from eth1 and the outgoing come out on eth0.
>>>
>>> And if I try do remove default route the outgoing packet don't come out....
>>>
>>> Any help ?
>>>
>>> Regards.
>>>
> Lots of thanks for your answer.
>
>> You just have to catch packets on the interface they would go normally:
>>
>> pass out on *eth0* route-to {(eth1 IP1_Gateway)} from <IP1> to !eth1:network
>>
>> The other rule is not needed in this case
>>
>> You may also try instead a 'reply-to' rule on eth1's inbound, as David
>> DeSimone suggested.
>
> OK now it's working. But I have some big trouble about the bandwith.
>
> Now when I try to do something like a scp, or ftp or wget from inside a
> jail to outside, everything work fine. The traffic go to right interface,
> the answer too.
>
> But when I try to do some network connection (ssh, scp etc..) from outside
> to a jail the bandwith is catastrophic (~40kB/s on 1Gbit/s).
>
> And for you ?
>
Using this kind of setup since at least two years for ~500 real users
without complains... (three different 'ssh jails' on the same machine
with many vlans and three "default" gateways)

>> A third and cleaner solution would be to use multiple routing-tables -
>> see setfib(1) and 'options ROUTETABLES' of the kernel...
>
> I already try this, I don't known how to make it work. I'm going to try
> again.
>
I'm also planning to test this... since more than a year :-|

--
*Geoffroy Desvernay*
C.R.I - Administration syst�mes et r�seaux
Ecole Centrale de Marseille

------------------------------

Message: 7
Date: Sat, 13 Feb 2010 05:19:20 -0600
From: "Sam Fourman Jr." <sfou...@gmail.com>
Subject: Re: How make the route-to working ?
To: geoffroy desvernay <dg...@centrale-marseille.fr>
Cc: Albert Shih <Alber...@obspm.fr>, freeb...@freebsd.org
Message-ID:
<11167f521002130319h42e...@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1

On Thu, Feb 11, 2010 at 4:38 PM, geoffroy desvernay
<dg...@centrale-marseille.fr> wrote:
> Albert Shih a �crit :
>> Hi all,
>>
>> I've a problem with route-to.
>>
>> I've a server with 2 interfaces, and I'm running jail on this server. Each
>> interface have is own public IP address.
>>
>> � � � eth0 -- IP0 � � � � � � eth1 -- IP1
>>
>> and I've a default route (for example in IP0 subnet).
>>
>> So if the jail is in the IP0 subnet no problem everything work.
>>
>> Now if I put a jail in IP1 subnet, and some client try to connect to this
>> jail the answer come out through eth0 because of the default route (suppose
>> the client is not on my subnet).
>>
>> I don't want that. I want the answer come out through the eth1
>>
>> I'm trying to use pf to do that and put in my pf.conf something like
>>
>> pass in all
>> pass out all
>> pass out on eth0 route-to {(eth0 IP0_Gateway)} from <IP0> to ! IP0_subnet
>> pass out on eth1 route-to {(eth1 IP1_Gateway)} from <IP1> to ! IP1_subnet
>>
>> but it's not working, if I run a tcpdump on the host I can see the
>> incoming packet come in from eth1 and the outgoing come out on eth0.
>>
>> And if I try do remove default route the outgoing packet don't come out....
>>
>> Any help ?
>>
>> Regards.
>>
>>
> Hi,
>
> I'm using that for the same case:
>
> You just have to catch packets on the interface they would go normally:
>
> pass out on *eth0* route-to {(eth1 IP1_Gateway)} from <IP1> to !eth1:network
>
> The other rule is not needed in this case
>
> You may also try instead a 'reply-to' rule on eth1's inbound, as David
> DeSimone suggested.
>
> A third and cleaner solution would be to use multiple routing-tables -
> see setfib(1) and 'options ROUTETABLES' of the kernel...

I have searched the net high and low and I can not find any good
examples on how to use multiple routing tables.
I agree that it would be cleaner do you have a example of how to do this?
if anyone has links to examples for Multiple routing tables examples
post them please.

Sam Fourman Jr.

------------------------------

Message: 8
Date: Sat, 13 Feb 2010 10:05:22 -0800
From: Alex Teslik <wherei...@gmail.com>
Subject: pf: nat works, ip blocking and logging do not
To: freeb...@freebsd.org
Message-ID:
<d24a9c161002131005h50...@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1

I setup pf with nat on my Freebsd 7 system:

[gouda:root]/root# pfctl -sa -vvvv
No ALTQ support in kernel
ALTQ related functions disabled
TRANSLATION RULES:
@0 nat on em0 inet from 192.168.4.0/24 to any -> (em0) round-robin
[ Evaluations: 29986 Packets: 67086 Bytes: 54746182 States:
21 ]
[ Inserted: uid 0 pid 66358 ]

FILTER RULES:
@0 scrub in all fragment reassemble
[ Evaluations: 1030123 Packets: 539441 Bytes: 76737270 States:
0 ]
[ Inserted: uid 0 pid 66358 ]
@0 block drop in log (all) quick on em0 inet from 11.11.11.111 to any
[ Evaluations: 75127 Packets: 0 Bytes: 0 States:
0 ]
[ Inserted: uid 0 pid 66358 ]
@1 block drop in log (all) quick on em0 inet from 22.22.22.222 to any
[ Evaluations: 32476 Packets: 0 Bytes: 0 States:
0 ]
[ Inserted: uid 0 pid 66358 ]
@2 block drop out log (all) quick on em0 inet from 11.11.11.111 to any
[ Evaluations: 56044 Packets: 0 Bytes: 0 States:
0 ]
[ Inserted: uid 0 pid 66358 ]
@3 block drop out log (all) quick on em0 inet from 22.22.22.222 to any
[ Evaluations: 23568 Packets: 0 Bytes: 0 States:
0 ]
[ Inserted: uid 0 pid 66358 ]
@4 pass in all flags S/SA keep state
[ Evaluations: 75130 Packets: 311544 Bytes: 126402695 States:
62 ]
[ Inserted: uid 0 pid 66358 ]
@5 pass out all flags S/SA keep state
[ Evaluations: 75130 Packets: 239954 Bytes: 97798568 States:
55 ]
[ Inserted: uid 0 pid 66358 ]

nat works great.

Unfortunately, I can still go to 11.11.11.111 or 22.22.22.222 with no
blocking and no logging on /var/log/pflog.

When I tcpdump listen to pflog0 there are no entries when I go to those ips.
What am I doing wrong here that is preventing logging and blocking from
working?

[gouda:root]/root# tcpdump -vvveni pflog0
tcpdump: WARNING: pflog0: no IPv4 address assigned
tcpdump: listening on pflog0, link-type PFLOG (OpenBSD pflog file), capture
size 96 bytes

[gouda:root]/root# cat /etc/pf.conf
ext_if="em0"
int_if="sk0"
set skip on lo0
scrub in
nat on $ext_if from $int_if:network to any -> ($ext_if)
block drop in log (all) quick on $ext_if from { 11.11.11.111, 22.22.22.222 }
to any
block drop out log (all) quick on $ext_if from { 11.11.11.111, 22.22.22.222
} to any
pass in all
pass out all

Thank you for your thoughts.

------------------------------

Message: 9
Date: Sat, 13 Feb 2010 21:36:52 +0100
From: Morgan Wesstr?m <freeb...@pp.dyndns.biz>
Subject: Re: pf: nat works, ip blocking and logging do not
To: Alex Teslik <wherei...@gmail.com>
Cc: freeb...@freebsd.org
Message-ID: <4B770D6...@pp.dyndns.biz>
Content-Type: text/plain; charset=ISO-8859-1

> nat works great.
>
> Unfortunately, I can still go to 11.11.11.111 or 22.22.22.222 with no
> blocking and no logging on /var/log/pflog.
>
> When I tcpdump listen to pflog0 there are no entries when I go to those ips.
> What am I doing wrong here that is preventing logging and blocking from
> working?
>
> [gouda:root]/root# tcpdump -vvveni pflog0
> tcpdump: WARNING: pflog0: no IPv4 address assigned
> tcpdump: listening on pflog0, link-type PFLOG (OpenBSD pflog file), capture
> size 96 bytes
>
> [gouda:root]/root# cat /etc/pf.conf
> ext_if="em0"
> int_if="sk0"
> set skip on lo0
> scrub in
> nat on $ext_if from $int_if:network to any -> ($ext_if)
> block drop in log (all) quick on $ext_if from { 11.11.11.111, 22.22.22.222 }
> to any
> block drop out log (all) quick on $ext_if from { 11.11.11.111, 22.22.22.222
> } to any
> pass in all
> pass out all
>

You have to reverse the order of the source and destination in your
outgoing rule. It should be:

block drop out log (all) quick on $ext_if from any to { 11.11.11.111,
22.22.22.222 }

/Morgan

------------------------------

Message: 10
Date: Sun, 14 Feb 2010 00:36:07 +0100
From: geoffroy desvernay <dg...@centrale-marseille.fr>
Subject: Re: How make the route-to working ?
To: "Sam Fourman Jr." <sfou...@gmail.com>
Cc: freeb...@freebsd.org
Message-ID: <4B773767...@centrale-marseille.fr>
Content-Type: text/plain; charset="iso-8859-1"

Sam Fourman Jr. a �crit :
[...]
> I have searched the net high and low and I can not find any good
> examples on how to use multiple routing tables.
> I agree that it would be cleaner do you have a example of how to do this?
> if anyone has links to examples for Multiple routing tables examples
> post them please.
>
I don't have any skills on that theme, nor any (usable) experience, but
it seems that you have to:
1. recompile kernel with (for 4 tables):
option ROUTETABLES=4
2. modify the '1' table for example (default one is 0), prefixing
'route' commands with 'setfib 1'
eg: # setfib 1 route delete default
# setfib 1 route add default 10.1.2.3
3. start a jail with 'jail_xxx_fib="1"' in rc.conf

This should do the trick (if I understood it correctly)

One more time I did not test it, just reading freebsd-jail@ and googling
;) I'll test it myself when i'll have time for it.

HTH
--
Geoffroy Desvernay

------------------------------

Message: 11
Date: Sat, 13 Feb 2010 18:30:55 -0600
From: "David DeSimone" <f...@verio.net>
Subject: Re: pf: nat works, ip blocking and logging do not
To: <freeb...@freebsd.org>
Message-ID: <2010021400...@verio.net>
Content-Type: text/plain; charset="us-ascii"

Alex Teslik <wherei...@gmail.com> wrote:
>
> Unfortunately, I can still go to 11.11.11.111 or 22.22.22.222 with no
> blocking and no logging on /var/log/pflog.

Your outbound traffic first matches this rule:

> pass out all [keep state]

The [keep state] I added because it is implied; you would have to
explicitly add "no state" in order to avoid it, and you probably
actually do want state-tracking on most outbound connections.

This is important to your question, though, because if you open an
outbound connection to 11.11.11.111 or 22.22.22.222, it will store a
state entry, and state entries are checked BEFORE any rule-checking, for
reply traffic coming back from those hosts. So, these rules:

> block drop in log (all) quick on $ext_if from { 11.11.11.111, 22.22.22.222 } to any
> block drop out log (all) quick on $ext_if from { 11.11.11.111, 22.22.22.222 } to any

will never be checked, because they match the previously built state.

In order to prevent communications with these hosts, you must also add
"block out [quick]" rules which prevent you from initiating the
connection to them and thus building state entries.

--
David DeSimone == Network Admin == f...@verio.net
"I don't like spinach, and I'm glad I don't, because if I
liked it I'd eat it, and I just hate it." -- Clarence Darrow

This email message is intended for the use of the person to whom it has been sent, and may contain information that is confidential or legally protected. If you are not the intended recipient or have received this message in error, you are not authorized to copy, distribute, or otherwise use this message or its attachments. Please notify the sender immediately by return e-mail and permanently delete this message and any attachments. Verio, Inc. makes no warranty that this email is error or virus free. Thank you.

------------------------------

Message: 12
Date: Mon, 15 Feb 2010 02:05:56 -0500
From: "kevin" <k...@kevinkevin.com>
Subject: Server unresponsive when using transparent bridging w/ pf +
pfsync
To: <freeb...@freebsd.org>
Message-ID: <006401caae0d$530b5560$f9220020$@com>
Content-Type: text/plain; charset="us-ascii"

Hello,

I'd like to get thoughts / input to the following application of 2x FreeBSD
redundant firewalls.

I have two firewalls with transparent bridges of the inside/outside
interfaces (2 interfaces each firewall). A third interface is used for
PFSYNC state synchronization. Synchronization of states is fine. However, if
the bridge0 interface is enabled on the second firewall, the first firewall
completely halts and is unresponsive ,with no message on the console or in
/var/log/messages. Unfortunately debug flags are disabled on both machines.

One is 7.1-PRERELEASE and the other is 7.2-STABLE. I am in the process of
upgrading the first one to 7.2-STABLE as well.

Is this a viable scenario in the first place? Forgive my inexperience ,
any suggestions are welcome.

Many Thanks.

------------------------------

Message: 13
Date: Mon, 15 Feb 2010 11:56:29 +0100
From: Albert Shih <Alber...@obspm.fr>
Subject: Re: How make the route-to working ?
To: geoffroy desvernay <dg...@centrale-marseille.fr>
Cc: freeb...@freebsd.org
Message-ID: <20100215105...@obspm.fr>
Content-Type: text/plain; charset=iso-8859-1

Le 13/02/2010 � 09:11:24+0100, geoffroy desvernay a �crit
> Albert Shih a �crit :
> > OK now it's working. But I have some big trouble about the bandwith.
> >
> > Now when I try to do something like a scp, or ftp or wget from inside a
> > jail to outside, everything work fine. The traffic go to right interface,
> > the answer too.
> >
> > But when I try to do some network connection (ssh, scp etc..) from outside
> > to a jail the bandwith is catastrophic (~40kB/s on 1Gbit/s).
> >
> > And for you ?
> >
> Using this kind of setup since at least two years for ~500 real users
> without complains... (three different 'ssh jails' on the same machine
> with many vlans and three "default" gateways)
>

OK I find the problem. It's come from TSO.

If I disable TSO by sysctl everything become �normal� and works fine.

Thanks again for your help.

Regards.

JAS
--
Albert SHIH
SIO batiment 15
Observatoire de Paris Meudon
5 Place Jules Janssen
92195 Meudon Cedex
T�l�phone : 01 45 07 76 26/06 86 69 95 71
Heure local/Local time:
Lun 15 f�v 2010 11:54:01 CET

------------------------------

Message: 14
Date: Mon, 15 Feb 2010 11:07:06 GMT
From: FreeBSD bugmaster <bugm...@FreeBSD.org>
Subject: Current problem reports assigned to freeb...@FreeBSD.org
To: freeb...@FreeBSD.org
Message-ID: <201002151107....@freefall.freebsd.org>

Note: to view an individual PR, use:
http://www.freebsd.org/cgi/query-pr.cgi?pr=(number).

The following is a listing of current problems submitted by FreeBSD users.
These represent problem reports covering all versions including
experimental development code and obsolete releases.

42 problems total.

------------------------------

Message: 15
Date: Mon, 15 Feb 2010 14:53:27 +0100
From: geoffroy desvernay <dg...@centrale-marseille.fr>
Subject: Re: How make the route-to working ?
To: Albert Shih <Alber...@obspm.fr>
Cc: freeb...@freebsd.org
Message-ID: <4B7951D7...@centrale-marseille.fr>
Content-Type: text/plain; charset=windows-1252

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 15.02.2010 11:56, Albert Shih wrote:
> Le 13/02/2010 � 09:11:24+0100, geoffroy desvernay a �crit
>> Albert Shih a �crit :
>>> OK now it's working. But I have some big trouble about the bandwith.
>>>
>>> Now when I try to do something like a scp, or ftp or wget from inside a
>>> jail to outside, everything work fine. The traffic go to right interface,
>>> the answer too.
>>>
>>> But when I try to do some network connection (ssh, scp etc..) from outside
>>> to a jail the bandwith is catastrophic (~40kB/s on 1Gbit/s).
>>>
>>> And for you ?
>>>
>> Using this kind of setup since at least two years for ~500 real users
>> without complains... (three different 'ssh jails' on the same machine
>> with many vlans and three "default" gateways)
>>
>
> OK I find the problem. It's come from TSO.
>
> If I disable TSO by sysctl everything become �normal� and works fine.
>
> Thanks again for your help.
>
You're welcome...

Just to be sure to get it: you do use an 'fxp' ethernet card and
7.2-RELEASE or not ?

(In other words, is it the fxp bug described here
http://www.freebsd.org/releases/7.2R/errata.html ?)

On the server I use, I've
server# sysctl -a|grep tso
net.inet.tcp.tso: 1

� without any (known) problem, on a:
FreeBSD 7.2-STABLE #3: Thu Sep 24 21:32:33 CEST 2009

with two bge(4) interfaces...

- --
*geoffroy desvernay*
C.R.I - Administration syst�mes et r�seaux
Ecole Centrale de Marseille

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.14 (FreeBSD)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkt5UdcACgkQGbFYzwF8gKrAVACfV9Kuq6jef7nQHzzRREvllCUg
7a8AoI4BhBP4WciZgrPSw1/E2TPkcflo
=spTC
-----END PGP SIGNATURE-----

------------------------------

Message: 16
Date: Mon, 15 Feb 2010 21:47:16 +0100
From: Albert Shih <Alber...@obspm.fr>
Subject: Re: How make the route-to working ?
To: geoffroy desvernay <dg...@centrale-marseille.fr>
Cc: freeb...@freebsd.org
Message-ID: <20100215204...@obspm.fr>
Content-Type: text/plain; charset=iso-8859-1

Le 15/02/2010 � 14:53:27+0100, geoffroy desvernay a �crit
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> >
> > Thanks again for your help.
> >
> You're welcome...
>
> Just to be sure to get it: you do use an 'fxp' ethernet card and
> 7.2-RELEASE or not ?

No it's bce (the server is Dell PowerEdge 2900).

I'm running 7.2-p6.

>
> On the server I use, I've
> server# sysctl -a|grep tso
> net.inet.tcp.tso: 1
>
> ? without any (known) problem, on a:
> FreeBSD 7.2-STABLE #3: Thu Sep 24 21:32:33 CEST 2009
>
> with two bge(4) interfaces...
>
I'm going to send a email to freebsd-net. I don't knwon if it's a bug or
not.

Regards.

Thanks for your help (I insist ;-) ).

------------------------------

Message: 17
Date: Mon, 15 Feb 2010 22:11:41 +0100
From: Albert Shih <Alber...@obspm.fr>
Subject: Possible bug in TSO or in pf on bce
To: freeb...@freebsd.org, freeb...@freebsd.org
Message-ID: <20100215211...@obspm.fr>
Content-Type: text/plain; charset=iso-8859-1

Hi all,

I'm not a tcp/ip guru, so I don't known if it's a bug or not.

The situation is little complexe, so I'm going to explain that.

I've one server with tree interfaces two bce and one bge. All test is on
two bce.

This server running FreeBSD-7.2-p6 and have lot of jail (but the problem is
the same for one jail, so I assume I've just one jail). The bce0 and bce1
are in different vlan.

The jail is on bce1 (meaning the jail IP is on the bce1 subnet).

The default gateway is on bce0

So to make all traffic of the jail pass only throught bce1 and not using
bce0 I'm using pf with something like

pass out route-to (bce1 bce1_subnet_gw) from jail_IP to ! bce1_subnet keep state
pass in on bce1 reply-to (bce1 bce1_subnet_gw) from ! bce1_subnet to jail_IP keep state

if I do that all traffic pass through the right interface (bce1), but...the
bandwith drop to ~60kb/s (on gigabit interface).

So I find the problem is with TSO, if I deactivated the TSO the bandwith is
return to normal.

I don't knwon if it's a bug in PF (the problem is same if I use scrub or
not) or in the TSO support of bce.

I can run some few tests if someone like to debug, but because the server
is in production I cannot make lot of test.

Regards.

JAS

--
Albert SHIH
SIO batiment 15
Observatoire de Paris Meudon
5 Place Jules Janssen
92195 Meudon Cedex
T�l�phone : 01 45 07 76 26/06 86 69 95 71
Heure local/Local time:
Lun 15 f�v 2010 22:10:52 CET

------------------------------

Message: 18
Date: Tue, 16 Feb 2010 10:23:06 -0800
From: Pyun YongHyeon <pyu...@gmail.com>
Subject: Re: Possible bug in TSO or in pf on bce
To: Albert Shih <Alber...@obspm.fr>
Cc: freeb...@freebsd.org, freeb...@freebsd.org
Message-ID: <2010021618...@michelle.cdnetworks.com>
Content-Type: text/plain; charset="us-ascii"

On Mon, Feb 15, 2010 at 10:11:41PM +0100, Albert Shih wrote:
> Hi all,
>
> I'm not a tcp/ip guru, so I don't known if it's a bug or not.
>
> The situation is little complexe, so I'm going to explain that.
>
> I've one server with tree interfaces two bce and one bge. All test is on
> two bce.
>
> This server running FreeBSD-7.2-p6 and have lot of jail (but the problem is
> the same for one jail, so I assume I've just one jail). The bce0 and bce1
> are in different vlan.
>
> The jail is on bce1 (meaning the jail IP is on the bce1 subnet).
>
> The default gateway is on bce0
>
> So to make all traffic of the jail pass only throught bce1 and not using
> bce0 I'm using pf with something like
>
> pass out route-to (bce1 bce1_subnet_gw) from jail_IP to ! bce1_subnet keep state
> pass in on bce1 reply-to (bce1 bce1_subnet_gw) from ! bce1_subnet to jail_IP keep state
>
> if I do that all traffic pass through the right interface (bce1), but...the
> bandwith drop to ~60kb/s (on gigabit interface).
>
> So I find the problem is with TSO, if I deactivated the TSO the bandwith is
> return to normal.
>
> I don't knwon if it's a bug in PF (the problem is same if I use scrub or
> not) or in the TSO support of bce.
>

At first I thought you hit one of edge case of TSO on bce(4). But
it seems the issue comes from pf's route handling. When I ported pf
from OpenBSD, there was no TSO capability in FreeBSD at that time
so the pf_route() had no special handling code for TSO. Since it
was long time ago I'm not sure whether it's correct or not but try
attached patch.

Apart from TSO FreeBSD got several new features like fib,
flow-table and vnet. We may need to check whether these new
features are still working with pf(4).
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pf.routeto.patch
Type: text/x-diff
Size: 790 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-pf/attachments/20100216/f4cdad9f/pf.routeto-0001.bin

------------------------------

Message: 19
Date: Tue, 16 Feb 2010 10:47:32 -0800
From: Julian Elischer <jul...@elischer.org>
Subject: Re: Possible bug in TSO or in pf on bce
To: pyu...@gmail.com
Cc: Albert Shih <Alber...@obspm.fr>, freeb...@freebsd.org,
freeb...@freebsd.org
Message-ID: <4B7AE844...@elischer.org>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed

Pyun YongHyeon wrote:
> On Mon, Feb 15, 2010 at 10:11:41PM +0100, Albert Shih wrote:
>> Hi all,
>>
>> I'm not a tcp/ip guru, so I don't known if it's a bug or not.
>>
>> The situation is little complexe, so I'm going to explain that.
>>
>> I've one server with tree interfaces two bce and one bge. All test is on
>> two bce.
>>
>> This server running FreeBSD-7.2-p6 and have lot of jail (but the problem is
>> the same for one jail, so I assume I've just one jail). The bce0 and bce1
>> are in different vlan.
>>
>> The jail is on bce1 (meaning the jail IP is on the bce1 subnet).
>>
>> The default gateway is on bce0
>>
>> So to make all traffic of the jail pass only throught bce1 and not using
>> bce0 I'm using pf with something like
>>
>> pass out route-to (bce1 bce1_subnet_gw) from jail_IP to ! bce1_subnet keep state
>> pass in on bce1 reply-to (bce1 bce1_subnet_gw) from ! bce1_subnet to jail_IP keep state
>>
>> if I do that all traffic pass through the right interface (bce1), but...the
>> bandwith drop to ~60kb/s (on gigabit interface).
>>
>> So I find the problem is with TSO, if I deactivated the TSO the bandwith is
>> return to normal.
>>
>> I don't knwon if it's a bug in PF (the problem is same if I use scrub or
>> not) or in the TSO support of bce.
>>
>
> At first I thought you hit one of edge case of TSO on bce(4). But
> it seems the issue comes from pf's route handling. When I ported pf
> from OpenBSD, there was no TSO capability in FreeBSD at that time
> so the pf_route() had no special handling code for TSO. Since it
> was long time ago I'm not sure whether it's correct or not but try
> attached patch.
>
> Apart from TSO FreeBSD got several new features like fib,
> flow-table and vnet. We may need to check whether these new
> features are still working with pf(4).

yes, in 8.0 you have options ot do what you want to do in several
ways. From a quick look, both multi-FIBs and vnet may be directly
applicable to you.

As for pf, it works with multi fibs but the patch for vnet depends
upon teh installation of a newer revision of pf and that may not
be possible in 8.x.

(ceri may be able so shed more light on that, I'm assuming he's seeing
this.)

>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> freeb...@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net...@freebsd.org"

------------------------------

Message: 20
Date: Sun, 21 Feb 2010 08:45:22 +0000 (UTC)
From: "Bjoern A. Zeeb" <bzeeb...@lists.zabbadoz.net>
Subject: Re: Network simulation using jails & vimage
To: Julian Elischer <jul...@elischer.org>
Cc: p...@freebsd.org, FreeBSD virtualization mailing list
<freebsd-vir...@freebsd.org>, Jim Sifferle <j...@sifferle.net>
Message-ID: <2010022108...@maildrop.int.zabbadoz.net>
Content-Type: text/plain; charset="iso-8859-1"

On Sun, 21 Feb 2010, Julian Elischer wrote:

Hi,

> Jim Sifferle wrote:
>> Hi,
>>
>> I've used ipfw and Dummynet as well as ipfw + DSCP recognition patch and
>> pf/altq to simulate Internet and MPLS WAN environments for several
>> years. All of my setups have run under VMWare, which for many reasons
>> isn't ideal. I would like to collapse all of these VMs into one FreeBSD
>> box using jails and vimages.
>>
>> Does any FreeBSD branch / vimage release combination support separate pf
>> AND ipfw configurations per jail? I need ipfw+pf/altq for HFSC queuing
>> to simulate the queueing effects of MPLS provider edge and core
>> routers.
>
> -current (9) should be close, with patches for pf supplied by ceri.

s,ceri,eri, (Ermal Lu�i)

> 8 can do separate ipfw but pf is not changed.
> 9 has bugs fixed. but I'm not sure if the changes for pf went in..
> they do exis tif they are not in already.

No, pf hasn't gone in yet; it lives in user/eri/pf45/ in svn and I am
not sure what the plans are.

Apart from the latest changes 8 and 9 should be pretty much in sync
wrt to VIMAGE I think.

>> I'm hoping the latest 7.2-STABLE-201001 snapshot will work. The DSCP
>> recognition patch for ipfw that I rely on doesn't seem to work with
>> 8.0.
>> If 7.2 won't work for my needs, but 8 or 9-CURRENT will, is anyone aware
>> of an updated ipfw DSCP patch? I haven't seen anything on Google or the
>> freebsd-ipfw mailing list.
>
> what is DSCP?

I guess Differentiated Services CodePoint (if talking MPLS).

/bz

--
Bjoern A. Zeeb It will not break if you know what you are doing.

------------------------------

Message: 21
Date: Sun, 21 Feb 2010 00:36:06 -0800
From: Julian Elischer <jul...@elischer.org>
Subject: Re: Network simulation using jails & vimage
To: Jim Sifferle <j...@sifferle.net>
Cc: p...@freebsd.org, freebsd-vir...@freebsd.org
Message-ID: <4B80F076...@elischer.org>
Content-Type: text/plain; charset=UTF-8; format=flowed

Jim Sifferle wrote:
> Hi,
>
> I've used ipfw and Dummynet as well as ipfw + DSCP recognition patch and
> pf/altq to simulate Internet and MPLS WAN environments for several
> years. All of my setups have run under VMWare, which for many reasons
> isn't ideal. I would like to collapse all of these VMs into one FreeBSD
> box using jails and vimages.
>
> Does any FreeBSD branch / vimage release combination support separate pf
> AND ipfw configurations per jail? I need ipfw+pf/altq for HFSC queuing
> to simulate the queueing effects of MPLS provider edge and core
> routers.

-current (9) should be close, with patches for pf supplied by ceri.

8 can do separate ipfw but pf is not changed.
9 has bugs fixed. but I'm not sure if the changes for pf went in..
they do exis tif they are not in already.

>
> I'm hoping the latest 7.2-STABLE-201001 snapshot will work. The DSCP
> recognition patch for ipfw that I rely on doesn't seem to work with
> 8.0.
>
> If 7.2 won't work for my needs, but 8 or 9-CURRENT will, is anyone aware
> of an updated ipfw DSCP patch? I haven't seen anything on Google or the
> freebsd-ipfw mailing list.

what is DSCP?

>
> Thanks for your time,
>
> Jim
>
> _______________________________________________
> freebsd-vir...@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
> To unsubscribe, send any mail to "freebsd-virtualiz...@freebsd.org"

------------------------------

Message: 22
Date: Sun, 21 Feb 2010 01:24:34 -0800
From: Jim Sifferle <j...@sifferle.net>
Subject: Re: Network simulation using jails & vimage
To: Julian Elischer <jul...@elischer.org>
Cc: p...@freebsd.org, freebsd-vir...@freebsd.org
Message-ID: <1266744274.3871.26.camel@localhost>
Content-Type: text/plain; charset="UTF-8"

On Sun, 2010-02-21 at 01:14 -0800, Jim Sifferle wrote:

> > what is DSCP?
>
> DSCP stands for Differentiated Services Code Point, a six byte field in

I should have proofread better... the DSCP field is six bits, not
bytes. :)

Jim

------------------------------

Message: 23
Date: Sun, 21 Feb 2010 01:14:13 -0800
From: Jim Sifferle <j...@sifferle.net>
Subject: Re: Network simulation using jails & vimage
To: Julian Elischer <jul...@elischer.org>
Cc: p...@freebsd.org, freebsd-vir...@freebsd.org
Message-ID: <1266743653.3871.24.camel@localhost>
Content-Type: text/plain; charset="utf-8"

On Sun, 2010-02-21 at 00:36 -0800, Julian Elischer wrote:
> Jim Sifferle wrote:
> > Hi,
> >
> > I've used ipfw and Dummynet as well as ipfw + DSCP recognition patch and
> > pf/altq to simulate Internet and MPLS WAN environments for several
> > years. All of my setups have run under VMWare, which for many reasons
> > isn't ideal. I would like to collapse all of these VMs into one FreeBSD
> > box using jails and vimages.
> >
> > Does any FreeBSD branch / vimage release combination support separate pf
> > AND ipfw configurations per jail? I need ipfw+pf/altq for HFSC queuing
> > to simulate the queueing effects of MPLS provider edge and core
> > routers.
>
> -current (9) should be close, with patches for pf supplied by ceri.
>
> 8 can do separate ipfw but pf is not changed.
> 9 has bugs fixed. but I'm not sure if the changes for pf went in..
> they do exis tif they are not in already.

Hmmm... I think I need separate pf instances. I apply pf/altq QoS
queues to both interfaces of the VM that simulates the MPLS provider
edge router. The customer facing interface is a VLAN, and the QoS
queues for this interface could be applied using the system-wide pf
instance. The provider facing interface would be an eiface attached to
the vimage and I don't believe available to pf at boot time.

I will have to look around to see if the changes to support multiple pf
instances have made it into 9-CURRENT. Where in the source tree should
I look, or which mailing list would be best to ask this question on?

> > I'm hoping the latest 7.2-STABLE-201001 snapshot will work. The DSCP
> > recognition patch for ipfw that I rely on doesn't seem to work with
> > 8.0.
> >
> > If 7.2 won't work for my needs, but 8 or 9-CURRENT will, is anyone aware
> > of an updated ipfw DSCP patch? I haven't seen anything on Google or the
> > freebsd-ipfw mailing list.
>
> what is DSCP?

DSCP stands for Differentiated Services Code Point, a six byte field in
the IP header used to differentiate between Classes of Service, and
commonly used for CoS/QoS provisioning on MPLS WAN networks. The DSCP
IP header field superceded the IP TOS field as of RFC 2474. Here's a
pretty lightweight overview of how DSCP can be used:

http://www.cisco.com/en/US/tech/tk543/tk757/technologies_tech_note09186a00800949f2.shtml

For those who are interested, I've attached a simple ASCII diagram of
the vimage layout I envision (and currently have using VMWare).

Thanks for your help...

Jim
-------------- next part --------------

---------------------------------------------------------------------------------------------------------------------------------------------
| Cisco 2950 48 Port Switch: Site 1: Vlan 100: LAN, Vlan 101: Internet, Vlan 102: MPLS |
| Site 2: Vlan 200: LAN, Vlan 201: Internet, Vlan 202: MPLS |
| Site 3: Vlan 300: LAN, Vlan 301: Internet, Vlan 302: MPLS |
| Site 4: Vlan 400: LAN, Vlan 401: Internet, Vlan 402: MPLS |
| |
| Trunk Trunk Trunk Trunk |
--------------------|-------------------------------|-------------------------------|-------------------------------|------------------------
| | | |
/ \ / \ / \ / \
/ \ / \ / \ / \
/ \ / \ / \ / \
---------------------------------------------------------------------------------------------------------------------------------------------
| | | | | | | | | |
| | | | | | | | | |
| | ---------------------- | ---------------------- | ---------------------- | ---------------------- |
| | | | | | | | | | | | | |
| | | MPLS PE RTR Vimage | | | MPLS PE RTR Vimage | | | MPLS PE RTR Vimage | | | MPLS PE RTR Vimage | |
| | | x2 Interfaces | | | x2 Interfaces | | | x2 Interfaces | | | x2 Interfaces | |
| | | (Vlan 102, eiface) | | | (Vlan 202, eiface) | | | (Vlan 302, eiface) | | | (Vlan 302, eiface) | |
| | | IPFW+DSCP/ALTQ | | | IPFW+DSCP/ALTQ | | | IPFW+DSCP/ALTQ | | | IPFW+DSCP/ALTQ | |
| | | Quagga bgpd | | | Quagga bgpd | | | Quagga bgpd | | | Quagga bgpd | |
| | ---------------------- | ---------------------- | ---------------------- | ---------------------- |
| | | | | | | | | |
| | | | | | | | | |
| | ---------------------------------------------------------------------------------------------------------------------- |
| | | : : : | |
| | | : : : | |
| | | MPLS Cloud RTR Vimage: x4 Interfaces (Via MPLS PE Router Vimages); IPFW + Dummynet Pipes / Quagga bgpd | |
| | | : : : | |
| | | : : : | |
| | ---------------------------------------------------------------------------------------------------------------------- |
| | | | | |
| | | | | |
| | | | | |
| -------------------------------------------------------------------------------------------------------------------------- |
| | | |
| | | |
| | Internet Cloud RTR Vimage: x4 Interfaces (Vlan 101, 201, 301, 401); IPFW + Dummynet Pipes; Static routing | |
| | | |
| | | |
| -------------------------------------------------------------------------------------------------------------------------- |
| |
|--------------------------------------------------------------------------------------------------------------------------------------------

------------------------------

Message: 24
Date: Sun, 21 Feb 2010 09:14:19 -0800
From: Julian Elischer <jul...@elischer.org>
Subject: Re: Network simulation using jails & vimage
To: "Bjoern A. Zeeb" <bzeeb...@lists.zabbadoz.net>
Cc: p...@freebsd.org, FreeBSD virtualization mailing list
<freebsd-vir...@freebsd.org>, Jim Sifferle <j...@sifferle.net>
Message-ID: <4B8169EB...@elischer.org>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed

Bjoern A. Zeeb wrote:
> On Sun, 21 Feb 2010, Julian Elischer wrote:
>
> Hi,
>
>> Jim Sifferle wrote:
>>> Hi,
>>>
>>> I've used ipfw and Dummynet as well as ipfw + DSCP recognition patch and
>>> pf/altq to simulate Internet and MPLS WAN environments for several
>>> years. All of my setups have run under VMWare, which for many reasons
>>> isn't ideal. I would like to collapse all of these VMs into one FreeBSD
>>> box using jails and vimages.
>>>
>>> Does any FreeBSD branch / vimage release combination support separate pf
>>> AND ipfw configurations per jail? I need ipfw+pf/altq for HFSC queuing
>>> to simulate the queueing effects of MPLS provider edge and core
>>> routers.
>>
>> -current (9) should be close, with patches for pf supplied by ceri.
>
> s,ceri,eri, (Ermal Lu�i)

err yeah..

it'd be nice if itcould get committed

Ermal, is it ready?

>
>
>> 8 can do separate ipfw but pf is not changed.
>> 9 has bugs fixed. but I'm not sure if the changes for pf went in..
>> they do exis tif they are not in already.
>
> No, pf hasn't gone in yet; it lives in user/eri/pf45/ in svn and I am
> not sure what the plans are.
>
> Apart from the latest changes 8 and 9 should be pretty much in sync
> wrt to VIMAGE I think.
>
>
>>> I'm hoping the latest 7.2-STABLE-201001 snapshot will work. The DSCP
>>> recognition patch for ipfw that I rely on doesn't seem to work with
>>> 8.0. If 7.2 won't work for my needs, but 8 or 9-CURRENT will, is
>>> anyone aware
>>> of an updated ipfw DSCP patch? I haven't seen anything on Google or the
>>> freebsd-ipfw mailing list.
>>
>> what is DSCP?
>
> I guess Differentiated Services CodePoint (if talking MPLS).
>
>
> /bz
>

------------------------------

Message: 25
Date: Mon, 22 Feb 2010 11:07:03 GMT
From: FreeBSD bugmaster <bugm...@FreeBSD.org>
Subject: Current problem reports assigned to freeb...@FreeBSD.org
To: freeb...@FreeBSD.org
Message-ID: <201002221107....@freefall.freebsd.org>

Note: to view an individual PR, use:
http://www.freebsd.org/cgi/query-pr.cgi?pr=(number).

The following is a listing of current problems submitted by FreeBSD users.
These represent problem reports covering all versions including
experimental development code and obsolete releases.

42 problems total.

------------------------------

End of freebsd-pf Digest, Vol 277, Issue 1
******************************************

0 new messages