last call for L2/L3 rewrite code review

[Qing Li]

As you know the L2+L3 rewrite project (aka arp-v2) for
both ARP and ND6 has been active for quite some time now.
In a nutshell, the work removes the L2 tables (ARP and ND6)
from the L3 routing table. This redesign simplifies the
routing code and completely eliminates the route cloning
concept.

I discussed about this work at BSDCan 2007 and again at
the recent devsummit at Google. I have requested code
review and feedback since last May.

It's becoming increasingly difficult to maintain a separate
branch (p4 or svn) due to the high volume of new features'
related commits. The integration and unit testing efforts
increase in complexity by the week.

We (Julian, Sam, Kip, Robert and I) discussed about a possible
commit timeframe during the devsummit. And it seems now is
as good as any because we all have overcommitted ourselves,
and frankly speaking unless the feature is fully committed,
getting the thorough code review and broader testing is simply
unrealistic.

I have been running the new arp-v2 kernel as a regular production
machine on a daily basis, however, I am certain my testing is
insufficient, which is another good reason to get the feature
into the official -current tree.

Putting a closure on this work will allow me to focus on
developing additional networking features and completing my
other backlog tasks for the project.

Although the code will continue to evolve, on the other hand,
the code is stable enough that I believe it is ready for general
(-CURRENT) consumption. I would like to commit the code within
a week if not sooner.

Again, I would like to ask for your review and feedback.
Flaming is fine, which I prefer getting the flames now rather
than later.

Quite a few developers have contributed to this project in the
past: Glebius Smirnoff, Luigi Rizzo, Alessandro Cerri, and
Andre Oppermann. And most recently:

- Kip Macy revised the locking code completely, thus completing
the last piece of the puzzle, Kip has also been conducting
active functional testing
- Sam Leffler has helped me improving/refactoring the code, and
provided valuable reviews
- Julian Elischer setup the perforce tree for me and has helped
me maintaining that branch

The latest patch that is generated out of Perforce without
Kip Macy's locking modification is at

http://people.freebsd.org/~qingli/arp-v2-p4-diff

The complete P4 project tree is located at

//depot/projects/arp-v2

The full project sits in the svn branch that is located at

svn+ssh://svn.freebsd.org/base/projects/arpv2_merge_1

Once this code is fully committed into -CURRENT, I will be on
standby to fix any arp/nd6 related bugs. Kip Macy will be on
standby to fix any locking related issues. Kip will also be
acting as my backup when I'd be out partying.

Thanks,

-- Qing
mailto: qin...@freebsd.org

_______________________________________________
freeb...@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net...@freebsd.org"

[Bjoern A. Zeeb]

On Wed, 10 Dec 2008, Qing Li wrote:

Hi,

> It's becoming increasingly difficult to maintain a separate
> branch (p4 or svn) due to the high volume of new features'
> related commits. The integration and unit testing efforts
> increase in complexity by the week.

[..]

> I would like to commit the code within
> a week if not sooner.

I think waiting another few days would be good, so end of the weekend
or something seems realistic, but not so before.
The reason I am asking is that people are still seeing panics from
the rnh locking and aren't even able to boot machines. Mixng route
locking bugs with this rewrite will be painful. As I hope that the
rnh bugs will be solved within 2-3 days things would be good for
getting this monster in:)

> The complete P4 project tree is located at
>
> //depot/projects/arp-v2
>
> The full project sits in the svn branch that is located at
>
> svn+ssh://svn.freebsd.org/base/projects/arpv2_merge_1

Which of the two is the one to track the next days or are you going to
keep them in sync?

/bz

--
Bjoern A. Zeeb The greatest risk is not taking one.

[Kip Macy]

> The reason I am asking is that people are still seeing panics from
> the rnh locking and aren't even able to boot machines.

I have not seen this. Please tell me where this occurs.

> Mixng route
> locking bugs with this rewrite will be painful. As I hope that the
> rnh bugs will be solved within 2-3 days things would be good for
> getting this monster in:)

To the best of my knowledge, the few bugs that have occurred have been
fixed within a day of them being reported.

>> svn+ssh://svn.freebsd.org/base/projects/arpv2_merge_1
>
> Which of the two is the one to track the next days or are you going to
> keep them in sync?
>

The svn branch will be the one that is kept up to date with all fixes.
I will be IFC'ing in to it once a day.

Thanks,
Kip

[Marko Zec]

On Wednesday 10 December 2008 10:30:35 Kip Macy wrote:
> > The reason I am asking is that people are still seeing panics from
> > the rnh locking and aren't even able to boot machines.
>
> I have not seen this. Please tell me where this occurs.

I had a machine with defaultrouter from /etc/rc.conf pointing to a
gateway which wasn't directly reachable, so the machine would panic
before going multiuser. Nevertheless I can confirm that your change
185849 just resolved this issue, thanks a lot for a quick fix!

Marko

[Bruce M. Simpson]

Hi,

Just skimming this I notice it uses the if_afdata[AF_INET] pointer
purely for lltbl purposes; this clashes with the IGMPv3 code drop.

Please look in the bms_netdev branch, where I introduce a 'struct
ip_ifinfo' to make more general use of that slot. IGMPv3 needs to store
per-interface state for AF_INET, so this slot really needs to be shared
with other AF_INET stuff.

Looks like it needs to be updated for VIMAGE also, hopefully others more
familiar with this can help -- I am busy enough with non-programming
activity as it is to get up to speed on this, although I have at least
managed to print Julian's write-up...

Other than that, it looks like a much needed improvement and we are all
very grateful for our work on this.

thanks
BMS

[Kip Macy]

I think that you're request to not monopolize the AF_INET slot is
reasonable. Do you intend to merge bms_netdev before 8 branches (I'm
guessing this coming summer)?

Cheers,
Kip

--
If we desire respect for the law, we must first make the law respectable.
- Louis D. Brandeis