Adrian Robinson reckons that I might have some useful input here.
I'm operating in Spain, and have built a 65 node net on dd-wrt, custom
linux on wrap boards, and now mikrotik too (for 5ghz).
The way i did the dd-wrt stuff was to set them all to ad-hoc mode and
manage the routing though the use of different ip ranges on each
wireless interface.
The authentication is done at the MAC level, and each dd-wrt 'node' is
controlled using simple shell scripts - the nodes fetch commands from
a web server on the local net every few minutes.
As the kit is so cheap, i use two routers back-to back in the same box
to shift channels for the next 'hop'. 14db directional panels work
well for me out here.
The upshot of having them in ad-hoc mode is that they can talk to each
other without wds, so no bandwidth reservation issue either.
QoS and bandwidth control are also scripted, so if anyone could use
the scripts, i can post them.
Overall the dd-wrt (i use v23 on bare buffalo boards) has been
brilliant. No powerup issues, solid performance. The only problem i
get is that they can revert to the default settings if you cycle the
power repeatedly for about 10 minutes, but a ups or battery backup
sorts that one out.
Another great thing about these cheapo boards is that you can do a
really simple 2-wire mod to make them passive PoE too, and the power
chip handles anything between 4.5v and 24v. There are 4 RJ45's as
well, which makes a back-to-back setup very simple.
A *vital* command to put in the startup script is
wl -i eth2 interference off
without that, the nodes can automatically switch channels if they
think there's interference about.
Spooky to see that happen.
> The authentication is done at the MAC level, and each dd-wrt 'node' is
> controlled using simple shell scripts - the nodes fetch commands from
> a web server on the local net every few minutes.
where in the system does the MAC authentication occur ?
if you're in ad-hoc mode I presume it can't be using the wrt-radauth
method to deny wireless connections based on RADIUS checking.
Phil
As below, where it drops anything from the client's ip range, or
redirects it to 192.168.100.200 if you prefer :-
# mark everything with 0x01
iptables -t mangle -A PREROUTING -s 192.168.13.0/24 -j MARK --set-mark
1
# choose whether to drop or redirect.
#iptables -t nat -A PREROUTING -m mark --mark 1/1 -j DNAT --to
192.168.100.200
iptables -A FORWARD -m mark --mark 1/1 -j DROP
then :-
iptables -A PREROUTING -t mangle -m mac --mac-source 00:14:78:79:1a:d6
-j MARK --set-mark 2
so that MAC address 00:14:78:79:1a:d6 works, but nothing else does.
The scripts that they get add all of the expected MACs to the mangle
table in the same way.
This basically what all the various auth systems do anyway, but this
was easier for me to understand.
If you use Radius, then there's just another thing to learn about, and
it still ends up doing the above - more or less.
Incidentally, i did a couple of scripts for the control server to
catch POP and SMTP too, so unauthenticated users can connect, but get
my splash page if they try the web, or if they try to download email,
they repeatedly get one saying 'pay up', or if they try to send email
they get an SMTP 'no way hose' error message.
Regards,
Richard B.
Basic requirements were these :-
1. cheap kit.
2. max throughput
3. simple setup ('absolute zero' config is impossible, as someone has
to fit the kit, i.e. me).
4. all control in my hands - not dependant on others
So, with dd-wrt it turns out that you can use cheap kit, and you can
run custom scripts under 'Administration->Commands'
dd-wrt is good in that it has a web interface, so basic setup is easy.
The commands wget, tc, ip, and iptables are available, so *all*
control is possible.
I run a script from the startup script that continuously gets commands
(simple shell scripts) from the control server, which is basically a
web server with php and mysql.
here's the startup script :-
IP=33
GW=192.168.33.194
UPLINK=192.168.34.2
ULMASK=255.255.255.0
CONTROL=192.168.100.200/wifi/a.php
CF=/tmp/ctrl
ifconfig eth1 down
ifconfig br0 192.168.$IP.1 netmask 255.255.255.192 mtu 1450
ifconfig br0:1 192.168.$IP.65 netmask 255.255.255.192 mtu 1450
ifconfig eth2 192.168.$IP.129 netmask 255.255.255.192 mtu 1450
ifconfig eth2:1 192.168.$IP.193 netmask 255.255.255.192 mtu 1450
#ifconfig eth2:2 $UPLINK netmask $ULMASK mtu 1450
iptables -F -t nat
echo 1 > /proc/sys/net/ipv4/ip_forward
wl -i eth2 interference 0
route del default gw 192.168.2.1
route add default gw $GW
echo "interface=br0">/tmp/dnsmasq.conf
echo "interface=eth2">>/tmp/dnsmasq.conf
echo "resolv-file=/tmp/resolv.dnsmasq">>/tmp/dnsmasq.conf
echo "dhcp-leasefile=/tmp/dnsmasq.leases">>/tmp/dnsmasq.conf
echo "dhcp-lease-max=60">>/tmp/dnsmasq.conf
echo "dhcp-authoritative">>/tmp/dnsmasq.conf
echo "dhcp-range=e,192.168.$IP.2,192.168.$IP.
62,255.255.255.192,1440m">>/tmp/dnsmasq.conf
echo "dhcp-option=e,3,192.168.$IP.1">>/tmp/dnsmasq.conf
echo "dhcp-option=e,6,10.0.2.1">>/tmp/dnsmasq.conf
echo "dhcp-range=w,192.168.$IP.130,192.168.$IP.
190,255.255.255.192,1440m">>/tmp/dnsmasq.conf
echo "dhcp-option=w,3,192.168.$IP.129">>/tmp/dnsmasq.conf
echo "dhcp-option=w,6,10.0.2.1">>/tmp/dnsmasq.conf
killall dnsmasq
dnsmasq --conf-file /tmp/dnsmasq.conf
echo "#!/bin/sh">$CF
echo "echo 30 >$CF.t">$CF
echo "while [ 1 = 1 ]; do">>$CF
echo "if wget -O $CF.c \"http://$CONTROL?s=\`cat $CF.s\`&n=`hostname`
\";then chmod 755 $CF.c;. $CF.c $CF;fi">>$CF
echo "sleep \`cat $CF.t\`">>$CF
echo "done">>$CF
echo 0 > $CF.s
chmod 755 $CF
. $CF &
Two subnets are for customers - one for wifi, one for ethernet, and
the other two are to allow downstream routers to connect by ethernet
or wifi.
You set this router's ip range by changing the top line :-
> IP=33
The uplink - providing that this isn't a gateway, is set here :-
> GW=192.168.33.194
This bit is the IP you set as your address on the node you're
connecting to :-
> UPLINK=192.168.34.2
> ULMASK=255.255.255.0
Most of the rest is to create the addresses on the interfaces, and to
set up dnsmasq.conf so dhcp works on ethernet and wifi.
This bit kicks off the long-running script to go and get more command
from the control server.
for DEV in br0 eth2
do
/usr/sbin/tc qdisc del dev $DEV root 2> /dev/null > /dev/null
/usr/sbin/tc qdisc del dev $DEV ingress 2> /dev/null > /dev/null
/usr/sbin/tc qdisc add dev ${DEV} root handle 1: prio bands 2 priomap
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
/usr/sbin/tc qdisc add dev ${DEV} parent 1:1 handle 11: pfifo
/usr/sbin/tc qdisc add dev ${DEV} parent 1:2 handle 12: htb r2q 10
/usr/sbin/tc class add dev ${DEV} parent 12: classid 12:1 htb rate
16mbit burst 2k
C=0
for RATE in 16mbit 8mbit 4mbit 2mbit 1mbit 512kbit 256kbit 128kbit
64kbit 32kbit
do
/usr/sbin/tc class add dev ${DEV} parent 12:1 classid 12:1${C} htb
rate ${RATE} ceil ${RATE} burst 2k prio 1
/usr/sbin/tc qdisc add dev ${DEV} parent 12:1${C} handle 121${C}: sfq
perturb 10
C=`expr $C + 1`
done
echo 0 > /proc/sys/net/ipv4/conf/${DEV}/send_redirects
done
iptables -A FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-
mss-to-pmtu
iptables -A INPUT -p tcp --syn -m limit --limit 5/s -j ACCEPT
echo 1 > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts
echo 1 > /proc/sys/net/ipv4/ip_forward
echo 1 > /proc/sys/net/ipv4/tcp_sack
echo 1 > $1.s
echo 1 > $1.t
This bit is really important, and you must set an MTU of 1450 on your
interfaces too :-
iptables -A FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-
mss-to-pmtu
It seems the internet is broken, and you can't do proper path-mtu-
discovery, so unless you get your packet sizes lower than the
wire ...... The effect is usually that Google works, but no other
website will if you forget this bit.
iptables -F -t nat
iptables -F -t mangle
iptables -P INPUT ACCEPT
iptables -F INPUT
iptables -P OUTPUT ACCEPT
iptables -F OUTPUT
iptables -F FORWARD
iptables -P FORWARD ACCEPT
# Prevent windows browsing
for I in 136 137 139
do
iptables -A INPUT -p tcp --dport $I -j DROP
iptables -A INPUT -p udp --dport $I -j DROP
done
echo 3 > $1.s
echo 1 > $1.t
iptables -t nat -A PREROUTING -m mark --mark 1/1 -j DNAT --to
192.168.100.200
iptables -t mangle -A PREROUTING -s 192.168.13.0/26 -j MARK --set-mark
1
iptables -t mangle -A PREROUTING -s 192.168.13.128/26 -j MARK --set-
mark 1
echo 5 > $1.s
echo 1 > $1.t
iptables -A PREROUTING -t mangle -m mac --mac-source 00:02:8a:b8:c6:04
-j MARK --set-mark 2
iptables -A PREROUTING -t mangle -m mac --mac-source 00:15:00:23:C0:83
-j MARK --set-mark 2
iptables -A PREROUTING -t mangle -m mac --mac-source 00:14:78:79:1a:de
-j MARK --set-mark 2
iptables -A PREROUTING -t mangle -m mac --mac-source 00:40:63:D8:E0:24
-j MARK --set-mark 2
echo 5 > $1.s
echo 1 > $1.t
BTW. those last two lines tell the router how long (seconds) to sleep
until it gets the next script, and what it's next 'step' is.
That lets me remotely adjust the frquency that each node checks in. On
startup it's 1 second, so it loads fast, but after that it's once
every 10 mins. The 'ctrl.s' is the step counter, so i can remotely
tell it to re-start (not reboot) the script sequence. I only ever do
that when i'm messing, but it seemed useful.
I had the idea of 'every node a hotspot' but i live in the middle of
no-where, and goats tend not to have laptops or PDAs ...
Some people have criticised my approach, saying that client data can
be seen by other clients. This is basically possible, as both could
know the keys, and both would have some measure of network access.
My response to that is that if someone holds a gun (or even holds a
trowel in a threatening manner) to my head, then i'll probably give
them full access to everything.
Also, anything critical is SSL, e.g. banking, is as un-crackable as we
got anyway.
Simple really ;)
.
<snip>
> I had the idea of 'every node a hotspot' but i live in the middle of
> no-where, and goats tend not to have laptops or PDAs ...
>
<snip>
I guess there is a market/application for tracking animals though
perhaps not goats :)
How about cattle, horses or indeed wild animals - I'm thinking of
tracking and/or determining whether each beast is within range of a
given node (or has just been nicked for example)
G
Here's the 'nodes' table schema :-
CREATE TABLE `nodes` (
`id` int(10) unsigned NOT NULL auto_increment,
`auth_seq` int(10) unsigned default NULL,
`name` varchar(45) NOT NULL default '',
`power` text,
`devs` varchar(45) default NULL,
`gateway` int(10) unsigned default NULL,
`ip` varchar(45) default NULL,
`poll_interval` int(10) unsigned default NULL,
`town` varchar(45) default NULL,
`country` varchar(45) default NULL,
`region` varchar(45) default NULL,
`notes` text,
`last_contact` datetime NOT NULL default '0000-00-00 00:00:00',
`restart` int(10) unsigned NOT NULL default '0',
`last_seq` int(10) unsigned NOT NULL default '0',
`reboot` int(10) unsigned NOT NULL default '0',
PRIMARY KEY (`id`,`name`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
<?
$username = "mysqlusername";
$password = "mysqlpassword";
$hostname = "localhost";
$database = "wifi";
$conn = mysql_connect($hostname, $username, $password)
or die();
mysql_select_db($database, $conn)
or die();
# Save last contact time
if(empty($s)) $s=0;
$q="update nodes set last_contact=now(), last_seq=$s where name='$n'";
mysql_query($q, $conn);
# Get node info
$q="select * from nodes where name='$n'";
$res=mysql_query($q, $conn);
if(mysql_numrows($res)==0) // eek. node id doesn;t exist
{ $poll_interval=300;
$gateway=0;
$restart=0;
$reboot=0;
}
else
{ $row=mysql_fetch_object($res);
$poll_interval=$row->poll_interval;
$gateway=$row->gateway;
$restart=$row->restart;
$reboot=$row->reboot;
}
# See if a reload has been flagged (just re-do the scripts)
if($restart==1)
{ $q="update nodes set restart=0 where name='$n'";
mysql_query($q, $conn);
$s=0;
}
# See if a reboot has been flagged
if($reboot==1)
{ $q="update nodes set reboot=0 where name='$n'";
mysql_query($q, $conn);
echo "reboot";
die();
}
# Dunno what went wrong, so restart the scrit sequence
if(empty($s)||!is_numeric($s)) $s=0;
switch($s)
{ case 0: // restarted - create filters
$SEQ=$s+1;
$poll_interval=1;
include "q.php";
break;
case 1: // assign VoIP to PRIO
$SEQ=$s+1;
$poll_interval=1;
include "f1.php";
break;
case 2:
$SEQ=$s+1;
$poll_interval=1;
include "f2.php";
break;
case 3: // set dns server and remove dd-wrt login prompt
$SEQ=$s+1;
$poll_interval=1;
echo "echo \"\" > /tmp/loginprompt\n";
echo "echo \"nameserver 192.168.10.1\" > /etc/resolv.conf\n";
break;
case 4:
$SEQ=$s+1;
$poll_interval=1;
include "auth-x.php";
break;
case 5:
$SEQ=$s;
$poll_interval=1;
include "auth-x2.php";
break;
case 6:
$SEQ=$s+1;
$poll_interval=1;
if(!empty($row->power)) echo str_replace("\r","",$row->power)."\n";
break;
case 7: // Blacklist
$SEQ=$s+1;
$poll_interval=1;
include "b.php";
break;
case 8: // set gateways to NAT all
$SEQ=$s+1;
$poll_interval=1;
if($gateway==1) include "g.php";
break;
case 98:
$SEQ=$s;
include "m1.php"; // Test - get associated macs etc
break;
default:
$SEQ=$s;
break;
}
?>
echo <? echo $SEQ ?> > $1.s
echo <? echo $poll_interval ?> > $1.t
<?
$TC="/usr/sbin/tc";
$DEVS=$row->devs;
// 0 1 2 3 4 5 6 7 8
9
$RATES="16mbit 8mbit 4mbit 2mbit 1mbit 512kbit 256kbit 128kbit 64kbit
32kbit";
$MAXRATE="16mbit";
echo "
for DEV in $DEVS
do
$TC qdisc del dev \$DEV root 2> /dev/null > /dev/null
$TC qdisc del dev \$DEV ingress 2> /dev/null > /dev/null
$TC qdisc add dev \${DEV} root handle 1: prio bands 2 priomap 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1
$TC qdisc add dev \${DEV} parent 1:1 handle 11: pfifo
$TC qdisc add dev \${DEV} parent 1:2 handle 12: htb r2q 10
$TC class add dev \${DEV} parent 12: classid 12:1 htb rate $MAXRATE
burst 2k
C=0
for RATE in $RATES
do
$TC class add dev \${DEV} parent 12:1 classid 12:1\${C} htb rate \$
{RATE} ceil \${RATE} burst 2k prio 1
$TC qdisc add dev \${DEV} parent 12:1\${C} handle 121\${C}: sfq
perturb 10
C=`expr \$C + 1`
done
echo 0 > /proc/sys/net/ipv4/conf/\${DEV}/send_redirects
done
iptables -A FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-
mss-to-pmtu
iptables -A INPUT -p tcp --syn -m limit --limit 5/s -j ACCEPT
echo 1 > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts
echo 1 > /proc/sys/net/ipv4/ip_forward
echo 1 > /proc/sys/net/ipv4/tcp_sack
"
?>
<?
$IP=$row->ip;
$LOCAL="192.168.$IP.128/26 192.168.$IP.0/26";
$SIPSERVERS="217.14.138.177 217.14.132.183 217.14.132.162
194.120.0.198";
$TC="/usr/sbin/tc";
$DEVS=$row->devs;
$q="select * from products where node_default=1";
$pres=mysql_query($q, $conn);
if(mysql_numrows($pres)==0)
{ $CIN=5;
$COUT=9;
}
else
{ $prow=mysql_fetch_object($pres);
$CIN=$prow->down;
$COUT=$prow->up;
}
echo "
for DEV in $DEVS
do
for S in $SIPSERVERS
do
$TC filter add dev \$DEV parent 1: prio 1 protocol ip u32 match ip
src \${S} flowid 1:1
$TC filter add dev \$DEV parent 1: prio 1 protocol ip u32 match ip
dst \${S} flowid 1:1
done
$TC filter add dev \$DEV parent 1: protocol ip prio 3 u32 match ip src
0.0.0.0/0 flowid 1:2
$TC filter add dev \$DEV parent 1: protocol ip prio 3 u32 match ip dst
0.0.0.0/0 flowid 1:2
for L in $LOCAL
do
$TC filter add dev \$DEV parent 12: protocol ip prio 3 u32 match ip
src \${L} flowid 12:1$COUT
$TC filter add dev \$DEV parent 12: protocol ip prio 3 u32 match ip
dst \${L} flowid 12:1$CIN
done
$TC filter add dev \$DEV parent 12: protocol ip prio 3 u32 match ip
src 0.0.0.0/0 flowid 12:10
$TC filter add dev \$DEV parent 12: protocol ip prio 3 u32 match ip
dst 0.0.0.0/0 flowid 12:10
done
"
?>
<?
// set up the basic table rules
echo "iptables -F -t nat
iptables -F -t mangle
iptables -P INPUT ACCEPT
iptables -F INPUT
iptables -P OUTPUT ACCEPT
iptables -F OUTPUT
iptables -F FORWARD
iptables -P FORWARD ACCEPT
iptables -A FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-
mss-to-pmtu
iptables -A INPUT -p tcp --syn -m limit --limit 5/s -j ACCEPT
# Prevent windows browsing
for I in 136 137 139
do
iptables -A INPUT -p tcp --dport \$I -j DROP
iptables -A INPUT -p udp --dport \$I -j DROP
done
"
?>
<?
global $SEQ;
global $row;
global $n;
$CONTROL="192.168.100.200";
$IP=$row->ip;
echo "
#iptables -t nat -A PREROUTING -m mark --mark 1/1 -j DNAT --to
$CONTROL
iptables -A FORWARD -m mark --mark 1/1 -j DROP
iptables -t mangle -A PREROUTING -s 192.168.$IP.0/26 -j MARK --set-
mark 1
iptables -t mangle -A PREROUTING -s 192.168.$IP.128/26 -j MARK --set-
mark 1
";
$q="update nodes set auth_seq=0 where nodes.id=".$row->$id;
mysql_query($q, $conn);
$F="/tmp/nodes/$n.macs";
$fp=fopen($F,"w");
$q="select trim(roaming_macs) as macs from sales where
length(trim(roaming_macs))>0";
$res=mysql_query($q, $conn);
$rows=mysql_num_rows($res);
for($i=0;$i<$rows;$i++)
{ $srow=mysql_fetch_object($res);
fputs($fp, $srow->macs."\n");
}
$q="select trim(static_macs) as macs from sales, nodes where
sales.home_node=nodes.id and length(trim(static_macs))>0 and
nodes.id=".$row->id; $res=mysql_query($q, $conn);
$rows=mysql_num_rows($res);
for($i=0;$i<$rows;$i++)
{ $srow=mysql_fetch_object($res);
fputs($fp, $srow->macs."\n");
}
fclose($fp);
?>
<?
global $SEQ;
global $row;
global $n;
$limit=5;
$F="/tmp/nodes/$n.macs";
$fp=fopen($F,"r");
$q="select auth_seq from nodes where id=".$row->id;
$res=mysql_query($q, $conn);
$srow=mysql_fetch_object($res);
$auth_seq=$row->auth_seq;
// skip to next block of <limit> macs.
for($i=0;$i<$auth_seq&&!feof($fp);$i++) fgets($fp);
if(!feof($fp))
{ for($i=0;$i<$limit&&!feof($fp);$i++)
{ $m=trim(fgets($fp));
$m=str_replace("\m","",$m);
if(!feof($fp)) echo "iptables -A PREROUTING -t mangle -m mac --
mac-source $m -j MARK --set-mark 2 \n";
}
if(feof($fp))
{ $auth_seq=0;
$SEQ=$s+1;
}
else $auth_seq+=$limit;
}
else //feof hit in skipping
{ $auth_seq=0;
$SEQ=$s+1;
}
$q="update nodes set auth_seq=$auth_seq where id=".$row->id;
$res=mysql_query($q, $conn);
fclose($fp);
?>
The WiFi bit is relatively easy (lol).
route, tc, and iptables is where all the magic is at.
route - for setting up your routing. I do static routing, so after a
re-start, every nodes knows exactly where the internet is.
tc - (traffic control) for creating traffic queues to do QoS and speed
limiting
iptables - mark packets so tc can shove them in the right queues,
allow/disallow MACs, IPs etc
tc is quite hard to get to know, but it is certainly worth it.
Whatever auth or commercial control system you buy, if it's linux,
then it'll use tc and iptables at the end of the day.
DD-WRT also has Layer-7 filtering, so you can catch protocols as well
- e.g. edonkey
i.e. to kill off emule/edonkey traffic you just do this :-
iptables -t mangle -A POSTROUTING -m layer7 --l7proto edonkey -j DROP
Telnet/ssh into your dd-wrt box and type this to see all the available
protocols you can detect/drop/shape :-
ls /etc/l7-protocols/protocols
CREATE TABLE `products` (
`id` int(10) unsigned NOT NULL auto_increment,
`name` varchar(45) NOT NULL default '',
`node_default` int(10) unsigned default NULL,
`nominal` varchar(45) default NULL,
`price` float default NULL,
`up` int(10) unsigned default NULL,
`down` int(10) unsigned default NULL,
`notes` text,
PRIMARY KEY (`id`,`name`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
CREATE TABLE `sales` (
`id` int(10) unsigned NOT NULL auto_increment,
`roaming_macs` text,
`static_macs` text,
`home_node` int(10) unsigned NOT NULL default '0',
`name` varchar(45) NOT NULL default '',
`address` text,
`postcode` varchar(45) default NULL,
`tel` varchar(45) default NULL,
`mobile` varchar(45) default NULL,
`priceoverride` float NOT NULL default '0',
`product` int(10) unsigned NOT NULL default '0',
`email` varchar(45) default NULL,
`town` varchar(45) default NULL,
`region` varchar(45) default NULL,
`country` varchar(45) default NULL,
`notes` text,
`started` date NOT NULL default '0000-00-00',
Actually, I got silence! Perhaps everyone else was trying it at the
same time. Or is it a Spanish number?
And speaking personally I thank you for all you have shared about how
you are doing things.