Re: Some more info on wifi routing
Jeremy McDermond
I'm Jeremy, NH6Z, and I run the T2OREGON APRS gateway and digipeater.
I'm also a systems and network engineer who works for a small service
provider in Corvallis. Tom and I have been exchanging some e-mails
regarding you folks proposed network, and I agreed to interject some
commentary regarding routing 44.0.0.0/8 addresses and Internet
architecture in general.
On Sep 18, 2009, at 4:07 PM, Tom Hayward wrote:
> Here's some info/ideas regarding mesh routing in the network.
>
>
> ---------- Forwarded message ----------
> From: Jim Kusznir <kus...@eecs.wsu.edu>
> Date: Fri, Sep 18, 2009 at 9:51 AM
> Subject: Some more info on wifi routing
> To: Tom Hayward <esa...@gmail.com>
>
>
> Hi Tom:
>
> I've been working/thinking more and more on the wireless network
> problem, and things are becoming clear now :)
>
> We definitely want to run batman. Specifically, I think we want to
> run
> batman-adv. I've been talking on IRC with the batman devs, and
> getting
> a much better design idea on how to set up the network.
>
> You might want to check out the website and some videos of talks given
> by some of the devs:
>
> http://downloads.open-mesh.net/batman/misc/
>
> Main website:
> http://downloads.open-mesh.net
I'll have to check out the batman stuff a little more in detail. I'll
be the first to admit that I'm not really briefed in on wireless mesh
technology.
> summary version: If we run batman-adv, we'll create one big layer2
> network. This also works well with your IP routing from the internet
> idea. We'd have a dhcp server (with failover, of course) running on
> the
> network, and if you join the network, it will go grab an IP for
> you. It
> will do this because batman-adv will cover routing through multiple
> hops
> transparently.
>
> You can't see my house from your house. Lets say we also didn't
> have a
> node on kamiak, but several other hams in the area did have nodes up.
> Specifically, there's another ham that can see you, but not me, and
> there's another ham that can see me, and this first other ham, but not
> you. If you wanted to communicate to me, the system would need to
> know
> that it could get to me via these other two hams, and route the
> traffic
> that way transparently to you. That's what you need batman for.
>
> The basic idea behind batman is that you're flattening the network.
> You
> no longer have wifi access points and clients (which is a star
> topology), you have a peer to peer topology: any node can help get to
> any other node. This is basically how the netrom backbone works.
The only thing you're going to want to be careful of us broadcast
control of some sort. The 1990's was all about flattening wired
networks and moving from a routed to a switched topology. The problem
that people rapidly ran into is that broadcast storms were doing nasty
things to the switched networks, and it was found that there was a
limit to how much you could flatten your network. You'll just want to
make sure that the traffic isn't enough that you're going to run into
broadcast issues. As I recall, as well, 802.11 doesn't handle
broadcasts well anyhow.
> As to the ID problem: we could run mDNS: a distributed DNS system.
> Each
> user would set their hostname with their callsign included in it.
> mDNS
> would be set to broadcast that every ~10 minutes (or maybe more), and
> thus meet their ID requirements while simultaneously providing useful
> information to the network.
Have you asked the Batman developers about multicast IP on their
software? It appears as though support for multicast isn't complete
in batman-adv from the cursory searches I've done. Remember that
multicast is going to have different IP addresses and MAC addresses to
deal with.
> As I see it, there are two pieces still to be flushed out:
> 1) I don't like the single point of failure for gateways; we'd need to
> work on that
> a) there's a soon-to-be-released patch for batman-adv that will
> manage gateway host assignment. We'd loose the direct 44.x
> Internet-routable IP, but at least we can get to the Internet someway
> (this may be solvable through other means, but this is actually a
> major
> problem in the Internet at large, and is not easy to fix).
Yes, you need to figure out the problem of where the Internet is going
to send traffic to you at. In general, the Internet isn't going to
have a lot of granularity on how this happens. Nobody on the net
accepts routes longer than /24. That's not such a huge deal when you
have L3 routing to your networks because you can have an "area" type
of thing going on, but you're definitely not going to be able to get
detailed topology information out to the Internet at large. You could
have redundant gateways to try to solve this, but you're still going
to need to funnel things to someplace that knows more detailed
topology for the RF-side network.
> 2) wide-area routing dynamicism needs to be worked on
> a) do we run the layer3 batman as well, and have it manage routes
> between networks? batman people say "not a good idea", and have some
> proposals for fixes, but its not clear yet how to proceed.
If you're just running point-to-point links between disparate areas,
IP already has protocols that do this. OSPF or IS-IS is what folks
use as an IGP typically. There's no reason why these can't be run
over RF networks. We ran them on microwave links back in the day.
> I sent an e-mail to Jeremiah from Spokane, who is also in on the high
> speed working group, and he also works for a WISP, so I asked if he
> had
> the connections to do routing as well.
The big issue isn't necessarily getting someone who will do it, it's
getting the owner of 44.0.0.0/8 (Brian Kantor) to give you permission
to do so. Nobody is going to advertise your addresses without his
consent. As I mentioned to Tom, you might want to consider using
regular Internet addresses assigned by an ISP for your network.
There's no requirement that you use 44.0.0.0/8 on RF, it's just
convenient to do so because it's a large amount of space that Brian
got long ago.
> In short, though, it looks like OpenWRT + batman is the way to go.
> BTW:
> robin is just a distribution that includes BATMAN ready to run :)
I would think that it depends on what you're trying to do. If you're
trying to hook up an ad-hoc network where things can be connected off
and on quickly, then I think you're 100% right. If you're just trying
to connect a bunch of mountaintops and a few "base" stations in the
lowlands, I'm not sure what the added win of the additional complexity
is. Something like OSPF will handle this well. The other
consideration you might have to deal with if you're doing an "ad-hoc"
on high-power 2.4GHz 802.11g is going to be how you're going to keep
the unwashed masses out of your network. In theory, if you're running
an omni antenna on top of a tall mountain, my little < 1W Apple
Airport is going to hear you fairly well. If your mountaintop happens
to be able to hear me, it might be difficult to keep me out. You
can't use WEP or WPA because that's encryption on ham radio. The only
thing you might be able to do is restrict to MAC addresses. Please
forgive me if you guys figured this out long ago.
>
> Oh, please take a look at the batman web site and docs, and let me
> know
> of any holes in the docs. I told the batman people that we'd check
> their docs out and advise of areas that need more material / are
> missing
> stuff.
>
> --Jim
>