Afrikaburns report back

54 views
Skip to first unread message

David Carman

unread,
Apr 28, 2010, 9:28:10 AM4/28/10
to village-...@googlegroups.com, Afrikaburn Mesh Telco, Monique Schiess - AfrikaBurn
Before I set up a todo list a mile long and start to catch up on the
day job, I must write down my Afrikaburns experience while it is still
fresh.

http://www.afrikaburns.com is modelled of the Burning Man festival in
the US. The basic idea is to set up a temporary town in the desert
(Tankwa Town, named after the nearby national park), offer services to
your neighbours, whether it be food, music or telecommunications, and
have a thoroughly good time doing it.

For me, it all started when Antoine invited me to join friends of his
from Johannesburg who wanted to provide a mesh telephone network at
Tankwa, based on the MP01 beta units from Atcom. Living in a region
that suffers long power outages and rising electricity prices, I was
also keen to explore 12VDC as a power source with solar input. I had
already bought some batteries, chargers and regulators, so this was a
marvellous opportunity to kickstart my solar home office.

So we assembled a team consisting of myself, Antoine, Bretton and
Diaan. However we had to make phone booths to hold the telecoms gear
and none of us can bang a nail in straight. So Bruce joined us. Not
only can Bruce bang a nail in straight, but he can do so while telling
how many other nails he's banged in straight and simultaneously
pointing out all the other nails in the vicinity that he can bang in
straight. He also removed the worry of the escalating cost of
deployment by applying to the Shuttleworth Foundation for funding, the
deal being that we return the 5 phone booths to Shuttleworth
Foundation for deployment at trade/tech expos for Village Telco.

One of the principal aims of Afrikaburns is to remove commercial
pressures from the experimental town. There is no trading, bartering,
advertising or marketing of any commercial product. The community
relies on "gifting" - an economy with the basic premise of abundance,
not sparsity.

However, if we were going to provide a WiFi network, we required a
recognisable SSID and it had to be displayed physically on the booths,
on ourselves and at our location on the town map. In other words, we
required a brand name, not out of a drive to establish ownership and
protect copyright, but to provide a recognisable link between the
phones, the WiFi network, the base camp and ourselves. We settled on
"Isigidimi" for a number of reasons:
1. It means "important message" or "messenger" in Xhosa, a rough
equivalent to the English "clarion call".
2. It was the name of the first magazine written exclusively in an
African language: http://www.britannica.com/blackhistory/article-57059
3. It has 5 equally-spaced "i"s in it that look like WiFi antennae and
coincide with the graphical representation in Village Telco logo.

Bruce sourced frames and sleeves for the booths, and drove to Maseru
to commission 5 custom 1m diameter Basotho hats for the roofs. Diaan
designed the Isigidimi sleeves and T-shirt logos. Bretton sourced EL
wire to decorate base camp and booths. Antoine prepared the firmware
and I focussed on power source and delivery.

During early testing with solar, I learned a few lessons:
1. Solar power is pricey.
2. If you are going to go sustainable solar, you are going to have to
minimize your consumption. You are going to be counting current
provision and load in milliamps.
3. You can only experminent live with power provision and load if you
have enough lead-acid battery capacity as reserve.
4. There is an abundance of 12VDC technology and devices available,
mostly designed for cars. Like the gamers driving the top-end of
desktop graphics, so Knightrider-esque car pimps drive the top end of
12VDC technology - see http://www.mp3car.com/
5. Place 15-20A fuses throughout the load circuitry so that short
circuits don't damage devices or drain the battery.

I sourced 5 x 10W flexible solar panels and regulators for the booth
roofs. The regulators were 6A Solsum 6.6Fs. The load was connected
through the regulator to the battery, so that the battery could only
discharge to 50% before the regulator cut the load, saving the battery
from a damaging deep cycle. We were not able to afford solar panels
for base camp (this time), so decided to have big storage capacity and
Bretton brought a backup petrol generator for charging.

I decided to do all wiring, both network and power, using solid-core
UTP (Ethernet):
1. Ethernet runs 13V and can handle 577mA per wire safely, giving
about 13W per wire pair (52W for all 4 pairs).
2. The copper quality is good and wire pairs are twisted, reducing attenuation.
3. Wall sockets and RJ45 connectors can be used as plugs and switches.
4. Cables can combine power and network.
5. Only one kind of cable and its corresponding tools had to be stocked.

Our batteries were all sealed lead-acid type. We had 6 x 102AH
"tractor" batteries for base camp, 5 x 12AH batteries for the booths
and 10 x 4.5AH batteries for spares.

Steve lent us 10 MPs and we had Antoine's and my 4 testing units. They
were flashed with the latest version of Village Telco firmware.
Antoine configured 2 WRTs, an ION and an IP04 for base camp. Bruce
lent us 3 netbooks as fanless consoles to survive the dust. I bought
some regulators for the netbooks, which were the only non-12VDC
devices that we used.

Diaan was not able to make it due to work circumstances, but was able
to deliver the T-shirts to me in Pretoria. I arrived back from
Pretoria the day before we left for Tankwa Town. Bretton arrived in
Cape Town the day before. Bruce had just got back from Maseru. So on
the 21st April Antoine, Bretton, Bruce and I (and my two children)
packed one camper van, two pickups and a trailer with all the bits and
pieces, tools, devices, 100m of CAT5 cable, a large tent, several
smaller tents, bicycles, lots of water, food and clothes and headed
into the desert the next morning.

I arrived last at sunset on Thursday. The desert can get really cold
at night and our first night was the coldest and windiest. So the next
morning, we set up a sturdy base camp and wired up all our batteries
and devices. We did not have an inverter, so had to use the petrol
generator to power the soldering iron for end wetting and battery
lugs. We then spent the Friday afternoon and the whole of Saturday
assembling and wiring the booths. A couple of lessons from the
installation:

1. Solid-core UTP may be up to spec to deliver power, but cutting,
stripping, crimping and connecting takes time and wears the fingers,
which were already wondering what had happened to the atmospheric
humidity. Do as much of the wiring in the comfort of your own home,
not at the installation site.
2. 12VDC is a pleasure to work with. One can wire live systems as long
as you don't short the circuit. The high current from lead-acid
batteries can heat up a short and burn you.

By Saturday evening there were 5 Isigidimi telephone booths up and
running, lit up by a 1.5W LED and 4 x 1W EL wires. We had written the
other booth numbers on the inside of the sleeves and left a marker pen
at each booth to write down other numbers that they discovered. Our
base camp was 999 and we gave a phone to the medical tent and put them
on 911. Signal strength was superb in the wide open, high-altitude,
dry climate. Steve had come up for the night and we shared phone duty
amongst the five of us. The phone rang incessantly and young and old
were making all sorts of calls. A few lessons from the first night:
1. Mark each phone with its own number clearly.
2. When deploying a VT network, ensure that users know what networks
are available on the system. Failing that, provide a catch-all IVRS at
the end of the Asterisk dialplan to explain to callers what networks
are available.
3. If your handsets have a pulse/tone dialling option, glue it to the
tone-dialling setting.
4. Busy WiFi Asterisk networks need changes to the default sip.conf:
a) Jitter buffering on either SIP or IAX is essential.
b) Qualify=5000 - we had a lot of calls hanging up after the first
ring. Once the call is established, we could talk forever with the
nutter on the other end of the line.
c) Autokill=5000 for iax.conf - otherwise the call will terminate with
no ACK within 2 seconds.
5. In the case of Afrikaburns, use a ringtone that doesn't sound like
anyone's office tone and set the public booth ring volume low.

On Sunday morning we set out to install the last of the nodes. We
mounted one in a car, one at a multihop point at a far camp, one at
the "post office" and one with the organisers. The organisers were
using VHF transceivers but could not communicate with the gate 5km
away. We had a Nanostation for each of the outer booths for
infrastructure Wifi access, hoping to attract a few Android phones and
laptops, but time constraints had left us focussing on telephony. We
flashed them quickly with a stable OpenWRT image, but by the time I
got to the gate on Sunday afternoon, they were closing the ticket
office and didn't require comms. The organisers were impressed with
the network and suggested using it next year instead of the VHF
radios. The advantages cited were:
1. We could reach the gate with LoS Nanostations.
2. They could make person-to-person calls without broadcast.
3. They didn't have to use so many batteries.
They did however request a broadcast option - a dialplan to work on
for next year - and wanted to know about mobile devices - enter the
Android phone.

The medics were not impressed with all the 911 bogus calls reaching
them. When I offered to field all 911 calls and patch through dinkum
requests for help, they preferred to answer the phone themselves -
seems like they were having fun too.

The "post office" at Tankwa Town has been returning every year. They
run a telegram service through the town, deliver post and have a few
old pulse dialler telephones linked with a 9V batteries. The
postmaster says that he knows very little about electronics, but would
like to collaborate more closely next year. He walked to each booth to
write his number in them and received a lot of calls. So we'll be
looking into configuring pulse dialling on chan_mp for the
"retro-potato".

By Monday morning, we had discovered the major miscalculation in booth
power consumption. The regulator had cut the load at 50% battery
capacity on most of the booths. From a 10W solar panel, one can draw
about 3W sustainably. We had beefed the battery up to 12AH to make up
for the deficit, and were expecting to deliver a sustainable 5W for
the duration of the event. However the EL wire was 1W per 2m length
and the booths had 4 each. So, with the constant phone activity, we
were drawing 3W for the MP continuously, 1.5W for the LED and 4W for
the EL wire at night. We replaced the batteries on the booths with the
4.5AH backups, but they didn't keep the booths up for long.

So for future booth deployments, we'll provide just the 1.5W LED for
the booth. The sleeves glowed brightly in their light.

We packed up early on Tuesday morning for the drive home. The booths
were a lot more easily dismantled and packed than deployed, but we
have a lot of ways to speed up deployment next time. We shall be
modifying the design slightly and trunking the wiring inside the
frames. We'll also be having Village Telco sleeves printed and the
booths will be available from Steve for use by the Village Telco
community.

We'll be looking into the Afrikaburns organisers and regular camps
buying their own hardware for next year, and during the year will be
delving into Android phone SIP interfaces and the exciting possibility
of a virtual infrastructure-mode access point embedded into the VT
firmware - both for SIP client hookup and intranet web.

Part of the excitement of Afrikaburns is that there is no contact
between you and the outside world. However that excitement can turn to
misery when you have left something essential behind, need to check
whether your friends that didn't arrive are fine, or even need to call
an ambulance. So we'll be looking into establishing either a high-gain
GSM or VSAT link for a basic interface to the outside. While the
organisers would have unlimited use of such a connection, perhaps the
"post office" could send and receive SMS "telegrams".

We'll be arriving at Tankwa Town a whole lot earlier next year.
Perhaps a week of deployment, testing and tweaking beforehand will
allow us to relax over the weekend. We'll also be teaming up with
another camp next year to pool tent space and power supply. We'll
hopefully have a few solar panels by then and they are keen to use the
wind. However the 102AH batteries will stay at the core of the base
camp power supply - without plenty of storage, solar and wind are
unsustainable power sources.

Thanks to Steve and Shuttleworth Foundation for all their help. Thanks
to the Village-Telco community for a reliable low-power device with
such diverse application. Thanks to Afrikaburns for a wonderful party.
Thanks to the Isigidimi team for a spectacular coordinated deployment
of complex gear, when few of us had even met each other the day before
the event.

Time to remember what I did for a living.

David Carman

--
You received this message because you are subscribed to the Google Groups "village-telco-dev" group.
To post to this group, send email to village-...@googlegroups.com.
To unsubscribe from this group, send email to village-telco-...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/village-telco-dev?hl=en.

antoine setting up.jpg
antoine on 999.jpg
base camp power.jpg
base station.jpg
booth at night.jpg
booth construction.jpg
booth in use.jpg
field workbench.jpg
steve deploying.jpg

Steve Song

unread,
Apr 28, 2010, 9:59:22 AM4/28/10
to village-...@googlegroups.com
David,

You, Antoine, Bruce, Bretton, and Diaan produced something quite
amazing at AfrikaBurn. I'm over the moon happy to have been a small
part of it.

I too can see many possible opportunities for next year. The usage of
the phones reminded me of http://chatroulette.com. Because most of
the booths were in the open and not at a particular camp, it was hard
to call someone in particular. This led to a lot of random calling
which mostly turned out to be a lot of fun and source of social
connection at the camp. Next year perhaps a "random extension" and a
way of setting your phone 'open' to randomness. And perhaps voicemail
for each camp. Alberto has been working with a very handy little
USB-GSM dongle that could be used with a Yagi antenna to gateway SMSes
to the outside world. The possibilities.... :-)

Thanks again.... Steve
--
Steve Song
Telecommunications Fellow, Shuttleworth Foundation

email: steve...@shuttleworthfoundation.org
work: +27 21 970 1220
mobile: +27 83 482 2088
skype: steve_l_song
blog: http://manypossibilities.net
twitter: stevesong

Vickram Crishna

unread,
Apr 28, 2010, 2:20:28 PM4/28/10
to village-...@googlegroups.com
Steve

I entirely agree about the 'possibilities'. Much of our (traditional
telephone users) thinking on telephony is coloured and downright
moulded by the kind of telephony implicit in a centralised switching
design.

Your reminder about chatroulette is timely. Not that we may have much
of a 'use' for it today, but unless the possibility exists, it is
arrogant to imagine that it has no role to play.

On the other hand, figuring out an entirely new way to call 'someone'
is exciting. Is there some way that users can broadcast their
individual availability at a particular MP? Can it be GPS based, a
conversation between personal GPS devices and fixed MPs? Perhaps a
personal dongle, to plug into a future version of the MP, that acts as
a personal identifier, annunciator etc. Whatever makes sense, it will
be a hugely new way to be available - or not, depending.
--
Vickram
http://communicall.wordpress.com

David Rowe

unread,
Apr 28, 2010, 5:56:04 PM4/28/10
to village-...@googlegroups.com
Hi David (and Antoine, Bretton, Diaan, Bruce and Steve),

Congratulations on a very cool deployment, and thank you for a detailed
report.

Couple of questions/comments:

1/ How far apart were the MPs? How high were the MP antennas?

2/ What jitter buffer settings did you use?

3/ Good to see IAX2 is working for you, it was locking up for me last I
tried it. Hmmm, could be a conf file setting. In general IAX2 is
heavier on the CPU than SIP/RTP.

4/ I deleted the qualify= entry entirely from sip.conf as it was causing
my SIP Links to go down due to ARP time outs, this meant 50% of calls
wouldn't go through. Setting a big value like you did is another
solution.

Cheers,

David

Paul Gardner-Stephen

unread,
Apr 28, 2010, 7:45:01 PM4/28/10
to village-telco-dev
Hi All,

This issue of how to find someone in particular on a temporary phone
network is one that I have been thinking about as well. Serval wants
to be able to setup temporary telephony in disaster zones, which
actually has a great many similarities to the AfrikaBurn / Burning Man
temporary communities in that there is this issue of how to enable
people to find one another, and more so, how to enable secure/private
communications between people. Voice mail would seem to have a role
to play here, but how do you know the VM extension for the other
person?

My current thinking is to implement some kind of "voice directory"
feature. Maybe using a persons initials as a filter so that one can
more rapidly go through the recorded voice signatures of other people
who have registered. The voice signatures could be played in order of
proximity, recency of registration or some other scheme.

Another complementary scheme is to let people enter their regular
telephone number as well. Then searching could also be based on
telephone number, using voice signatures to pick out the right person
if (through malice or accident) multiple people enter the same
telephone number.

I'd welcome any thoughts on these or other approaches for discovering
people on a temporary telephone network.

Paul.

On Apr 28, 10:59 pm, Steve Song
<steve.s...@shuttleworthfoundation.org> wrote:
> David,
>
> You, Antoine, Bruce, Bretton, and Diaan produced something quite
> amazing at AfrikaBurn.  I'm over the moon happy to have been a small
> part of it.
>
> I too can see many possible opportunities for next year.  The usage of
> the phones reminded me ofhttp://chatroulette.com.  Because most of
> the booths were in the open and not at a particular camp, it was hard
> to call someone in particular.  This led to a lot of random calling
> which mostly turned out to be a lot of fun and source of social
> connection at the camp.  Next year perhaps a "random extension" and a
> way of setting your phone 'open' to randomness.  And perhaps voicemail
> for each camp.  Alberto has been working with a very handy little
> USB-GSM dongle that could be used with a Yagi antenna to gateway SMSes
> to the outside world.   The possibilities.... :-)
>
> Thanks again.... Steve
>
> On 28 April 2010 15:28, David Carman <tidg...@gmail.com> wrote:
>
>
>
> > Before I set up a todo list a mile long and start to catch up on the
> > day job, I must write down my Afrikaburns experience while it is still
> > fresh.
>
> >http://www.afrikaburns.comis modelled of the Burning Man festival in
> > 12VDC technology - seehttp://www.mp3car.com/
> ...
>
> read more »

David Carman

unread,
Apr 29, 2010, 3:43:18 AM4/29/10
to village-...@googlegroups.com
Hi David

On Wed, Apr 28, 2010 at 11:56 PM, David Rowe <da...@rowetel.com> wrote:
> 1/ How far apart were the MPs?  How high were the MP antennas?

All 12 MPs were within 2km of each other. Booth MPs were 2m above the
ground inside the sleeve. Others were on car dashboards or sitting on
a chair/table inside tents or caravans.

> 2/ What jitter buffer settings did you use?

I didn't configure jitterbuffer - just noticed that it was absent.
Back home I set jitterbuffer=yes and let the default stack settings do
the job.

> 3/ Good to see IAX2 is working for you, it was locking up for me last I
> tried it.  Hmmm, could be a conf file setting.  In general IAX2 is
> heavier on the CPU than SIP/RTP.

We installed vanilla village-telco images, so were running SIP. I've
only been using IAX for testing back home. While I still consider IAX2
a superior protocol for WiFi VoIP, there are more SIP-enabled pocket
devices out there that can be recruited into a MP-based network. So
SIP it is - I promise.

I expect that a lean SIP installation may consume less CPU than IAX2,
but once you add the jitterbuffer FIFO stack to SIP (which was
originally written into IAX and then wrapped into an extra layer for
SIP), SIP will consume more CPU time. We are running into CPU time as
the limiting factor - and that is what needs addressing.

The FXS module was always going to chew the CPU, but I'm now convinced
that it is what makes the MP more useful than other VoIP-WiFi clients.
Had we installed USB phones in the booths, fewer people would have
used them. The familiarity of the POTS hardware is what makes the MP
so friendly in a non-tech environment.

So the one major aspect of MP firmware that we can improve is
Asterisk. Asterisk barely fits on any router and we are never going to
see Asterisk-lite. So it's time to move to FreeSWITCH. Asterisk should
be reserved for lumpy gateways with a2billing and mysql - they will be
able to communicate happily with FreeSwitch SIP.

FreeSWITCH is more stable and can handle 10x more calls than Asterisk
on a given CPU - see http://www.anders.com/cms/266

FreeSWITCH uses OpenZAP to communicate with FXS. Both FreeSwitch and
its OpenZAP module are packaged for OpenWRT Kamikaze and up. So the
only task required to integrate all functions of the MP into the
OpenWRT trunk is to write OpenZAP support for the MP FXS. This will
not only improve individual MP performance, but will integrate full MP
function into OpenWRT.

So then we'll all work off the OpenWRT trunk and compile images at
will from the revision of our choice. We can then drop images for
different applications onto a server for the VT-dev community to use.

Does this sound right, David, Elektra?

David

David Rowe

unread,
Apr 29, 2010, 5:11:27 AM4/29/10
to village-...@googlegroups.com
Hi David C,

I don't know much about Freeswitch but many other people really like it.
Several people have recommended it for the MP but no one (so far) has
come forward to do the actual work. It would be great to see it running
on the MP.

My understanding is that it is better engineered that Asterisk.

However with my management hat on I'm not (yet) convinced Freeswitch
offers any real advantages over Asterisk apart from technical appeal.

Couple of questions/comments:

1/ How did you load test the MP and determine Asterisk performance was
the limiting factor? Elektra and I have tested a MP making a call (e.g.
Asterisk fully active) while simultaneously relaying 15 calls over the
mesh. This met our performance target so there is no huge need to
replace Asterisk from a performance point of view.

A jitter buffer shouldn't be adding huge amounts of CPU load, it's just
a glorified FIFO. The reasons for excess CPU load would be worth
looking into.

Of course if Freeswitch really is 10x faster for exactly the same
functionality that would be cool, and free up more CPU for mesh
networking. However we just need to make one phone call with
Asterisk/Freeswitch from any one MP.

2/ What is the run-time RAM and Flash footprint of Freeswitch compared
to Asterisk on say a WRT54G or NS2 for comparison? Last I checked
Freeswitch wouldn't fit on a router like ours. IIRC it needs the Apache
Run Time library. But this sort of development moves fast, maybe it's
been thinned out.

Asterisk has been happily running on small boxes WRT54Gs for 6
years.....

3/ Is anyone running OpenZap under OpenWRT on a CPU with router class
(e.g. 200MHz) performance and Zaptel type hardware? For example a USB
handset.

I decided not to port Zaptel to the MP as its big, and needs to execute
lots of code at interrupt time. On a small processor with not much
cache Zaptel may have serious performance issues as the interrupt rate
of Zaptel (1KHz) is very fast (by design, so hard to change).

There is a real risk (Open)Zaptel would choke a MP CPU. But a custom,
non-zaptel Freeswitch channel driver could be written, like chan_mp that
was written for the MP.

Cheers,

David R

David Carman

unread,
Apr 29, 2010, 7:43:53 AM4/29/10
to village-...@googlegroups.com
Hi David

On Thu, Apr 29, 2010 at 11:11 AM, David Rowe <da...@rowetel.com> wrote:
> I don't know much about Freeswitch but many other people really like it.
> Several people have recommended it for the MP but no one (so far) has
> come forward to do the actual work.  It would be great to see it running
> on the MP.
>
> My understanding is that it is better engineered that Asterisk.
>
> However with my management hat on I'm not (yet) convinced Freeswitch
> offers any real advantages over Asterisk apart from technical appeal.

If software is better engineered, then it consumes less CPU time and
memory to accomplish the same task. If we are in agreement that
FreeSwitch is better engineered than Asterisk, then we are in
agreement that FreeSwitch uses less CPU and RAM than Asterisk.

> 1/ How did you load test the MP and determine Asterisk performance was
> the limiting factor?  Elektra and I have tested a MP making a call (e.g.
> Asterisk fully active) while simultaneously relaying 15 calls over the
> mesh.  This met our performance target so there is no huge need to
> replace Asterisk from a performance point of view.

I ran top on a MP that was running openwrt/OLSR/Asterisk/chan_mp on
the SWUG mesh. CPU load increased from 40-50% to 90+% when making a
call. So I expect that half the CPU load is from Asterisk/chan_mp.

> 2/ What is the run-time RAM and Flash footprint of Freeswitch compared
> to Asterisk on say a WRT54G or NS2 for comparison?  Last I checked
> Freeswitch wouldn't fit on a router like ours.  IIRC it needs the Apache
> Run Time library. But this sort of development moves fast, maybe it's
> been thinned out.

Both FreeSwitch and >> its OpenZAP module are packaged for OpenWRT
Kamikaze and up. FreeSwitch doesn't need Apache. It appears from
cursory inspection that a FreeSwitch install is half the size of an
Asterisk install on OpenWRT.

> Asterisk has been happily running on small boxes WRT54Gs for 6
> years.....

Yes, but without the burden of a FXS module.

> 3/ Is anyone running OpenZap under OpenWRT on a CPU with router class
> (e.g. 200MHz) performance and Zaptel type hardware?  For example a USB
> handset.

The task of comparing FreeSwitch and Asterisk in standby on a MP is
trivial but time-consuming. It should be done, even if we can't test
the FXS load. I'll do it before I do anything else on an Asterisk MP -
but that's only going to be in a few weeks.

> I decided not to port Zaptel to the MP as its big, and needs to execute
> lots of code at interrupt time.  On a small processor with not much
> cache Zaptel may have serious performance issues as the interrupt rate
> of Zaptel (1KHz) is very fast (by design, so hard to change).
>
> There is a real risk (Open)Zaptel would choke a MP CPU.  But a custom,
> non-zaptel Freeswitch channel driver could be written, like chan_mp that
> was written for the MP.

Yes, my mistake. The whole reason why we can put a FXS channel on a
200MHz device is because of your chan_mp, derived from your work on
the IP04. OpenZap is simpler than Zaptel, but will most likely still
require the 1KHz interrupts.

So the FreeSwitch solution would involve the writing of
freeswitch-chan-mp. I think it is best to make this decision as early
as possible, based on practical results. I don't think anyone wants to
work on Asterisk for the MP until we have the facts and the matter is
resolved.

David Carman

unread,
Apr 29, 2010, 9:53:47 AM4/29/10
to village-...@googlegroups.com
On Thu, Apr 29, 2010 at 1:43 PM, David Carman <tid...@gmail.com> wrote:
> Both FreeSwitch and >> its OpenZAP module are packaged for OpenWRT
> Kamikaze and up. FreeSwitch doesn't need Apache. It appears from
> cursory inspection that a FreeSwitch install is half the size of an
> Asterisk install on OpenWRT.

Attempting to install Asterisk and FreeSwitch on a full Nano gives
helpful errors:

* verify_pkg_installable: Only have 668kb available on filesystem /,
pkg asterisk14 needs 3476

* verify_pkg_installable: Only have 668kb available on filesystem /,
pkg freeswitch needs 1658

So FreeSwitch takes up less than half the space of Asterisk.

It's a raw extrapolation to suggest that FreeSwitch would take up half
the RAM and CPU load of Asterisk, but I think that is enough evidence
to suggest that Asterisk is bloating the MP.

Paul Gardner-Stephen

unread,
Apr 30, 2010, 1:40:08 AM4/30/10
to village-telco-dev
Hi All,

As many of my ideas for immediate action revolve around the PBX
software, I have been thinking about the proposed Asterisk vs
Freeswitch transition.

First, I agree that it makes sense to make a firm decision one way or
the other in the very near future, so that we have the certainty to
act.

Second, I think we should move from Asterisk only if there is a
compelling reason, as it does set us back a little to switch PBX
software. As they say, better the devil you know than the one you
don't. Also, I think that there is probably a slightly larger
asterisk savvy user base than freeswitch at this time.

Third, I started having a look at where the bloat in asterisk is, and
whether anything can be done about it. Here is what I have
discovered:

asterisk binary = 990KB
asterisk sounds = 1234KB
asterisk modules = 1965KB

(note this adds up to 4191KB, not the 3476KB suggested by the
installer. Is there more than one asterisk package involved?)

The binary and sounds seem to be non-negotiable, however the modules
may be a different story.

There are 118 modules, but only 52 are loaded on my MP (and that
includes the addition of the ReadFile and other modules I used to
implement 19123).
These modules use only 1216KB, suggesting that we can save 749KB.
Looking at the top 10 biggest modules loaded:

26 format_wav_gsm.so
27 res_musiconhold.so
29 chan_mp.so
45 app_dial.so
45 pbx_config.so
49 app_followme.so
51 res_features.so
56 codec_gsm.so
207 chan_iax2.so
321 chan_sip.so

I see that both the IAX2 and SIP channel modules are loaded and are by
far the biggest modules.
Do we actually need both IAX2 *and* SIP channel modules? My
understanding is that these are alternate protocals for doing much the
same thing, but I am happy to be corrected.

In any case, I see that almost a MB of Asterisk can be readily removed
because it is not actually used. This naturally translates into a
reduced memory footprint for Asterisk now (since we aren't using the
749KB of modules). Indeed some examination of an asterisk process
reveals:

VmHWM: 4080 kB
VmRSS: 4072 kB
VmData: 17544 kB
VmStk: 372 kB
VmExe: 936 kB
VmLib: 2496 kB

That is, that asterisk on an MP uses about 4MB of RAM at any given
point (VmRSS), and that most of that is in fact the shared executable
(VmExe) and modules (VmLib), with a bit for stack (VmStk).

So I guess my question is, does saving the best part of a MB (if not a
whole MB if we can ditch either the IAX2 or SIP module) remove our
motivation to switch from Asterisk?

I think that the final answer to this requires an actual installation
of freeswitch on an MP01 (without channel driver would be fine), so
that we can do a rational comparison.
I'm still in the process of getting my MP build environment setup, but
if I get a chance I will try to do this, though I will be happy if
someone beats me too it :)

Paul.

On Apr 29, 10:53 pm, David Carman <tidg...@gmail.com> wrote:

elektra

unread,
Apr 30, 2010, 4:29:33 AM4/30/10
to village-...@googlegroups.com
Hello Paul -

very good contribution, indeed.

> asterisk binary = 990KB
> asterisk sounds = 1234KB
> asterisk modules = 1965KB

As we can get rid of some Asterisk modules we can also get rid of abdominable
asterisk sound files.

Cheers,
Elektra

Paul Gardner-Stephen

unread,
Apr 30, 2010, 6:42:38 AM4/30/10
to village-telco-dev
Hi Elektra,

On Apr 30, 5:29 pm, elektra <onelek...@gmx.net> wrote:
> Hello Paul -
>
> very good contribution, indeed.

You're welcome.

> > asterisk binary = 990KB
> > asterisk sounds = 1234KB
> > asterisk modules = 1965KB
>
> As we can get rid of some Asterisk modules we can also get rid of abdominable
> asterisk sound files.

True. I had figured that we will need some sound files, anyway.

But I am certainly happy to replace the asterisk bundled ones with VT/
MP specific ones. Maybe we should go for New Zealand accent, since it
is half-way between South Australian and South African accents? ;)

I guess the sound files should be excluded for the freeswitch
comparison as well. In that case Asterisk's MP size is currently
990+1216 = 2106KB.

This is what Freeswitch has to beat. I'm still bludgeoning my build
environment together (installing missing dependencies on my debian VM
on my mac), but all going well I will try to build freeswitch for
openwrt and see how big it is on disk and in RAM.

Paul.

Marc Abrams

unread,
Apr 30, 2010, 10:57:59 AM4/30/10
to village-...@googlegroups.com
Paul:

Be careful in the comparison. The Freeswitch default or mini builds that we've contributed and maintain for OpenWRT contain a lot of things not included in the default Asterisk builds. In addition, one of the many advantages that Freeswitch has in comparison to Asterisk is support for a lot of different codecs and encoding schemes, which result in literally tons of sounds and MOH files that will have to be removed for a base, apples to apples comparison. There are also a bunch of really nice sound-dependent modules for dial by name and voice recognition that can really take up space. We haven't done this yet, but our plan is to carve these up so they are more sane for a small device such as our board or the MP.

Personally, having done just this comparison for our project, we believe that, apples to apples, the footprint is very similar but with Freeswitch, we gain a more modern codebase and feature set. In addition, while you can certainly program the feature set with config files, there is a very complete API that allows you to control Freeswitch in real-time. In fact, the fs_cli command shell is a stand-alone user space application that uses the API to control Freeswitch. It has a Lua interface module already, so it would be easier to make work more like OpenWRT or MP apps.

Also, though I know David has written a few Asterisk channel drivers including one for the MP, the endpoint module for Freeswitch is a much simpler, cleaner affair. And there is an endpoint module called OpenZap that will allow you to quickly adapt the existing MP channel driver, if not use it completely without changes.

marc.

David Rowe

unread,
Apr 30, 2010, 6:09:28 PM4/30/10
to village-...@googlegroups.com
Hi Marc,

Hi Marc,

> Also, though I know David has written a few Asterisk channel drivers
> including one for the MP, the endpoint module for Freeswitch is a much
> simpler, cleaner affair. And there is an endpoint module called
> OpenZap that will allow you to quickly adapt the existing MP channel
> driver, if not use it completely without changes.

Actually writing the asterisk channel driver for the MP was not a
pleasant experience - I still feel uncomfortable with the code as I
never really understood the Asterisk threading model completely. It
took weeks to get stable due to weird threading issues. So I say bring
on a cleaner channel driver architecture.....

The MP01 asterisk channel driver chan_mp has no relation to Zaptel, so
OpenZap won't help us. For reasons I posted earlier it is best to steer
away from Zaptel on small cache machines. Instead I would recommend
porting chan_mp to Freeswitch. This won't be so hard, the lower (non
asterisk) layers of chan_mp are straight forward and we wouldn't need to
change the kernel mode code.

BTW much of the CPU load is due to signal processing inside chan_mp -
changing to Freeswitch won't suddenly decimate CPU load. While a call
is running all Asterisk is doing is moving a few buffers around.
Without a call running Asterisk is doing nothing.

On both Asterisk and Freeswitch the echo canceller, DTMF detector,
speech codec (if used), and FXS interrupts will dominate CPU load.
Quite a lot could be done to optimise these independent of any switch
from Asterisk to Freeswitch (for example hand optimised assembler).
However at the moment it seems unwarranted.

Yesterday I measured loadav on a MP01 to IP04 call at 0.37 (5 minute
average). Think this means 18% average CPU load? "top" gives me
numbers that bounce all over the place. To be honest I am not sure how
to measure CPU load accurately on a MP01. Only way I really trust is a
cycle counter around high-CPU functions but this is hard on a
multitasking system. Apart from that the best approach is to load up
the network (e.g. make it relay 15 calls) and listen to voice quality.

Perhaps a better area for CPU optimisation is the Linux network/Wifi
code. Work to make the MP more efficient at high packet rates - that is
where the other 82% of the CPU will be used in a high load situation.
Elektra did some good work on this last year by compiling out the
firewall code.

Cheers,

David
> To unsubscribe from this group, send email to village-telco-dev
> +unsub...@googlegroups.com.

Marc Abrams

unread,
May 1, 2010, 2:42:32 AM5/1/10
to village-...@googlegroups.com
David:

With respect to optimization, I actually would be really ruthless and get rid of anything that either buffers or inspects packets unless absolutely necessary for operation of the MP. These things use a lot of interrupts, CPU and bus cycles which you need to process voice packets. I personally would resist recreating a general purpose AP (albeit, with an FXS and Freeswitch) because that's not the calling (pun intended) of the MP. The less it does, besides make and receive phone calls over the mesh, the simpler it will be to setup and more reliable it will be in practice.

I'm excited to see you consider the change to Freeswitch because it will again simplify and modernize the MP even more.

marc.
Reply all
Reply to author
Forward
0 new messages