Update and question on power output
Steve Song
As mentioned previously, we've been having some debugging issues around
the RF performance of the Mesh Potato which has delayed the release of
the 100 beta units originally scheduled for September. I am happy to
say that it appears that those problems are well on track to resolution
and we are back on track for producing the beta units. This is thanks
to a lot of hard work on the part of Edwin and Alen from Atcom and from
David in tracking down the source of some of the erratic performance
that we were experiencing. The downside is that the most optimistic
scenario for shipping the units will be just before the end of the year.
Frustrating but development is never a straight line. I am quite
extraordinarily relieved that we are back on track.
An interesting question emerged out of the debugging process. The DLink
DIR-300s that we have done some of the prototyping with have a power
output of 17dBm. Right now the prototype MPs work fine at 17dBm.
However, according to documentation, the Atheros 2317 chip appears to be
capable of 20dBm. My question to the WiFi gurus in the community is,
"What is the impact in practical terms of 17dBm versus 20dBm power
output for a device like the Mesh Potato?"
Cheers... Steve
sebastian buettrich
On Fri, 2009-10-23 at 10:50 +0200, Steve Song wrote:
...
> An interesting question emerged out of the debugging process. The DLink
> DIR-300s that we have done some of the prototyping with have a power
> output of 17dBm. Right now the prototype MPs work fine at 17dBm.
> However, according to documentation, the Atheros 2317 chip appears to be
> capable of 20dBm.
yes it is, afaik.
> My question to the WiFi gurus in the community is,
> "What is the impact in practical terms of 17dBm versus 20dBm power
> output for a device like the Mesh Potato?"
with all other parameters unchanged,
and in theory :) :) :)
3 dB in TX makes for a
factor 1.4 in reach -
i.e. 1400 m instead of 1000 m
14 km instead of 10 km,
that s why the 26 dBm of the AR2315 are nice to have.
doubles the reach.
theory :)
best,
sebastian
Mike O'Connor
had a 90mw which is about 20dBm, this had a coverage of 500m, using
3dBm antenna's
What sort of antenna are you planing to have on the MPs ?
Mike
Rudolf Meijering
It should also be noted that what is important is not the output power or even the received power, but the Signal to noise ratio. In a sparse network, a higher output power would increase the signal strength relative to the noise. In a dense mesh however, the increased power output would increase the interference between the nodes, which effectively creates more noise and therefore a lower signal to noise ratio. You would therefore want to change the output power depending on your mesh network layout, the antenna's used and the noise levels.
For long distance communication, it would be better to use a good antenna which focus the signal to a smaller area, therefore increasing the power that is received, than to increase the output power of the card while using it with an omni. The omni would simply create a lot of noise for other nodes in the area.
Another factor to take into consideration is the national regulations, I think the DIR-300 is set to 17dBm to comply with the strictest regulations.
Rudolf Meijering
Cell: +27741419409
Steve Song
-Steve
Steve Song
Telecommunications Fellow, Shuttleworth Foundation
email: steve...@shuttleworthfoundation.org
work: +27 21 970 1220
mobile: +27 83 482 2088
skype: steve_l_song
blog: http://manypossibilities.net
twitter: stevesong
Mike O'Connor
elektra
> An interesting question emerged out of the debugging process. The DLink
> DIR-300s that we have done some of the prototyping with have a power
> output of 17dBm. Right now the prototype MPs work fine at 17dBm.
> However, according to documentation, the Atheros 2317 chip appears to be
> capable of 20dBm. My question to the WiFi gurus in the community is,
> "What is the impact in practical terms of 17dBm versus 20dBm power
> output for a device like the Mesh Potato?"
>
The figures are suggesting that the RF frontend of the DIR-300 isn't
very effective, because it is adding a attenuation of -3dB. Since the
signal has to pass through the RF frontend while transmitting and
receiving, our link budget between two similar devices will have a
additional loss of -6dB. The DIR-300 RF frontend loses 3dB transmit
power and 3dB receiver sensitivity. The DIR-300 specs are 17dBm TX power
and -89dBm RX sensitivity @ 1Mbit.
No RF frontend is perfect, but I assume it is possible to design a
frontend with 1dB loss without too much effort. Again we will still lose
1dB when transmitting and receiving, but we could have a link budget
which could be 4dB better. The result would be a range increase of ~50%.
I think for the beta production run it is OK to have similar performance
like the DIR-300. We need to get the devices out soon, so all the folks
out there can work on other aspects (software, Asterisk integration).
However for mass production I think we should try to get better range
and performance.
Particularly RX sensitivity is important in my opinion. With better RX
sensitivity the MPs can use higher data rates which reduces the amount
of airtime consumed and thus provides better network capacity.
The AR2317 chip that the DIR-300 and the MP01 are using has two
symmetrical RF input/outputs. One is TX/RX and the other one is RX-only
to provide receiver diversity between two antennas. (The DIR-300 and the
MP prototype have a small antenna etched on the board.) Actually the
RX-only input has much better receiver sensitivity than the TX/RX port
(-96dBm). It would be great to have two well working antennas inside the
mass production MP, so we can make best use of the RX-only port and RX
diversity.
Looking for PCB antenna designs I have found the following antenna
designs by Texas Instruments:
http://focus.ti.com/lit/an/swra093d/swra093d.pdf
http://focus.ti.com/lit/an/swru120b/swru120b.pdf
http://focus.ti.com/docs/toolsw/folders/print/cc25xxem_ref_des.html
Two of these designs use symmetric antennas - hence we wouldn't need a
balun (balanced to unbalanced) circuitry anymore. This would save some
components in the BOM and also the losses introduced by the balun (a
integrated balun made by TDK has a attenuation of 1dB). However the
F-type antenna design has better omnidirectional radiation pattern and
gain (but would need a balun for each antenna)
Making best use of the RX-only port and improving the TX performance
could increase the link budget by 6-7dB - this would double the range
compared to a DIR-300. And make me smile...
Cheers,
Elektra
David Rowe
> And what dBm will this be ?
suggestions on these specifications? We haven't designed the antenna
yet so please feel free to make suggestions.
The question of power a lot depends on the use case. In my minds eye I
am still seeing closely spaced house/shacks in South African townships.
My biggest concern was multipath off tin roofing and spectrum congestion
plus interference, e.g. from a microwave in every second house. It may
even be mounted indoors in some applications.
I have a feeling that the Village Telco system capacity and performance
may not be a strong function of transmitter power. The packet loss will
perhaps be a stronger function of time-varying interference from other
users and other devices using the spectrum.
Cheers,
David
David Rowe
> The figures are suggesting that the RF frontend of the DIR-300 isn't
> very effective, because it is adding a attenuation of -3dB. Since the
> signal has to pass through the RF frontend while transmitting and
> receiving, our link budget between two similar devices will have a
> additional loss of -6dB.
Nope. The RF power transferred between the antenna and AR2317 in TX and
RX mode depends on many factors. The path for the RF signal is
different in Tx and RX mode due to varying impedance's which are unknown
to us. Without accurate specifications and the right test equipment we
can only speculate.
> The DIR-300 RF frontend loses 3dB transmit
> power and 3dB receiver sensitivity.
I am afraid not. The 20dBm rated output appears to be due to D-link
deliberately setting the power target of 17dBm. Another factor is
supply rail - the Atheros data sheet specifies 20dBm (plus/minus 1dB)
using a 3.5V supply rail. It is not an indicator of receiver
performance.
We need to avoid splitting hairs. Wifi gear is designed to very broad
specifications. In practice the conditions for ideal Wifi radio
performance rarely occur. If your link budget is only 2dB your link is
going to fall over for a variety of reasons in the real world.
To predict or design to a specification of 1 dB is going to be
impossible in production for a variety of reasons. I have seen the
power move by 2dB as the AR2317 warms up! The AR2317 calibration
software has a Tx power tolerance of +/- 1.5dB - so is some cases that's
a 3dB unit-unit variation straight off the assembly line.
I have no doubt the receiver specs on AR2317s (e.g. LNA gain, noise
figure, and impedance of the tx/rx switch ports) vary by several dB from
chip-chip. That's just part of the deal with a low cost CMOS SoC like
the AR2317. It's designed to be inexpensive with adequate radio
performance.
Let not forget the goal here is good end-end VOIP performance. That
means the telephone call should sound good. Once we have a certain link
margin the RF performance has only a minor role in meeting this goal.
Where we can make a big difference is clever algorithms and system
design. Look at end-end factors like Asterisk, the speech codec, number
of hops, batman-performance-with-voip to allow voice to get through
under challenging packet loss conditions.
Cheers,
David
David Rowe
than VOIP-over-Wifi for telephony?
By "better" consider this example:
I can take my GSM handset down the street and around the corner from a
GSM base station and it still works. If I take a Mesh Potato down the
street and around the corner from another mesh node it stops working.
Some ideas:
1/ Propagation of 2.4GHz versus 900MHz. However there are 1800MHz GSM
services I understand that still work OK. Propagation at 1.8GHz can't
be that much different to 2.4GHz.
2/ EIRP. A GSM handset may transmit at 1W (30dBm), compared to a Wifi
AP of say 20dBm.
3/ Bit rate and modulation scheme. I am not sure how GMSK at 250 kbit/s
compares to 1 Mbit/s spread spectrum for Wifi in terms of packet (or
bit) error rate. Roughly 12dB gain at 1/4 of the bit rate?
4/ Protocol and forward error correction. Maybe Wifi depends on big
packets getting thru without error.
5/ Benefits of licensed versus unlicensed spectrum?
6/ Propagation of GMSK versus spread spectrum? Maybe GMSK is more
robust to multipath?
7/ Antenna gain of the sectorial antennas at the GSM base station?
Comments invited......
Thanks,
David
David Carman
<steve...@shuttleworthfoundation.org> wrote:
> "What is the impact in practical terms of 17dBm versus 20dBm power
> output for a device like the Mesh Potato?"
Every time I have increased Tx power to establish good signal for a
mesh WRT, I have ended up setting it back to default (17dBm) once I
have solved the root cause of the signal problem by repositioning the
router. The only situation where increased Tx power has resulted in
better signal is for external antennae on long RF coax cables.
My homebaked explanation for this is that increasing the TX power
increases the noise as well as the signal. Signal-to-noise ratio is
more dependent on antenna quality than Tx power. The MP will be using
a small antenna which will probably generate a lot of noise at high Tx
power. Therefore we should focus on optimizing antenna design, rather
than look for more Tx power.
The etched antenna designs that Elektra has posted are
omnidirectional. I believe that a multipurpose router like the MP
should have a directional component to its signal so that it can point
for long links, with enough backward radiation to service the
immediate surroundings.
There are two ways to propagate signal in a particular direction.
First is to extend the antenna with passive directors (e.g. Yagi) -
not feasible with the MP. The second is with a patch array (panel),
which would suit the MP well.
A small patch array etched onto the board can have an insulator and
ground plane stuck over it, or vice versa. This would not cost much
more than the etched omni designs, but would take up more space inside
the box. A panel also has the advantage of picking up signal over a
larger surface. 2.4GHz translates to a wavelength of 12cm, so nodes
and internodes are centimetres apart. I have found that moving a
standard WRT antenna 3cm can result in massive signal fluctuation,
especially in confined spaces with lots of metal structure causing
diffraction.
So a panel design would make installation easier by avoiding finicky
positioning issues. A small ground plane would allow enough backward
radiation to service immediate surroundings. The front side of the MP
could have a label for unskilled installers, like a Claymore mine:
"front toward friend".
Re Mounting: I am most impressed with the Nanostation's mounts. It
would be great to have similar mounts on the MP. We don't need fancy
mouldings - just 4 eyes to thread some chicken wire through (chicken
wire holds the African continent together). Mounting the MP on a stick
or bamboo instead of a metal pole would also result in better signal.
Re Microwave ovens: The only protection against microwave ovens is
redundant routes. That is why a mesh network is the only viable
architecture for any 2.4GHz network. I see people turning microwave
ovens on at lunch and supper in the routing data. No-one will be able
to make or receive a phone call while their own oven is on. Redundant
routing will allow phone calls while neighbours have their oven on.
Fortunately ovens are only used for a few minutes at a time so, with
redundant routes, the impact of microwave ovens is kept to a minimum,
except for your own.
David
Tim Hanson
most microwaves operate since water absorbs it so well. So, yes,
atmospheric absorption at 1.8Ghz is much less than 2.4GHz, by about a
factor of 4-5. I'm not sure about walls and solid absorption, but my
one-bit approximate is maybe a factor of 2 difference between the two
frequencies.
<rant>The fact of water absorption was why 2.4GHz was left unlicensed
in the first place. It was a 'junk' band - but, despite this, just
look at all the successful uses! All without needing either lobbyists
or the FCC. Methinks more bands should be unlicensed so they, too,
can be well used. </rant> :-)
Speaking of 2.4GHz, I heard a story about some Stanford students who
were trying to set up a zigbee network in Yosemite (not sure why).
They couldn't get their nodes to communicate over the spec'd
difference - no matter how much they increased the output power.
Turned out that the pine needles - water filled - act like little
tuned absorbing antennas to 2.4GHz.
I'd say use another band if you can!
Tim
David Rowe
Some great information in your post from a real live mesh network -
thanks. The point about redundant routes around interferers is
excellent.
Re:
> My homebaked explanation for this is that increasing the TX power
> increases the noise as well as the signal.
in the transmitted signal - just like turning up the volume too far on
your TV. It sounds louder, but the overall signal quality is poorer to
the listener, a similar effect to adding noise as you suggest. It also
adds more interference to adjacent channels.
Cheers,
David
David Carman
> Here is a question for the GSM and Wifi gurus. Why does GSM work better
> than VOIP-over-Wifi for telephony?
Sorry, can't hear you. You're cracking up ;^)
> 1/ Propagation of 2.4GHz versus 900MHz. However there are 1800MHz GSM
> services I understand that still work OK. Propagation at 1.8GHz can't
> be that much different to 2.4GHz.
I agree - 0.9GHz can prolly look almost as far around corners as 2.4GHz.
> 2/ EIRP. A GSM handset may transmit at 1W (30dBm), compared to a Wifi
> AP of say 20dBm.
And don't forget the base station power. GSM deployments have relied
on brute force. Remember the early days of GSM, where the choice was
between a 4W and 8W handset? It reminds me of the early days of the
contraceptive pill - pack them with huge amounts of steroids to ensure
efficacy, and then address safety issues once you're making a profit.
> 3/ Bit rate and modulation scheme. I am not sure how GMSK at 250 kbit/s
> compares to 1 Mbit/s spread spectrum for Wifi in terms of packet (or
> bit) error rate. Roughly 12dB gain at 1/4 of the bit rate?
The trend in digital wireless is towards higher frequencies and
corresponding higher bitrates. The lower frequency of GSM is a core
problem that is being addressed by better modulation and use of higher
frequencies.
> 4/ Protocol and forward error correction. Maybe Wifi depends on big
> packets getting thru without error.
UDP transmissions are more reliable on a mesh, so I suspect packet
size is an issue. The "cracking up" pattern on GSM also suggests small
packet size.
> 5/ Benefits of licensed versus unlicensed spectrum?
Those microwave ovens :((
On Sat, Oct 24, 2009 at 6:40 AM, Tim Hanson <side...@gmail.com> wrote:
> 2.4 GHz is in the water absorption band. It is the frequency at which
> most microwaves operate since water absorbs it so well.
Not so. Water's main microwave absorption is at 22GHz. Microwave ovens
deliberately operate at an order of magnitude lower so that they don't
vapourize the surface of the food, while leaving the inside frozen.
> Turned out that the pine needles - water filled - act like little
> tuned absorbing antennas to 2.4GHz.
Pure water, in the form of rain, hail, sleet or snow, does not absorb
2.4GHz radiation. I don't see any significant signal loss during heavy
rainstorms. However I do see chronic loss a few weeks after
groundsoaking rains (winter here), for the reason you describe above.
Plant sap is not just water. It contains salts, which make the
solution a good conductor. Pure water is a poor conductor. So signal
at any frequency is lost through induction of the plant sap. This
effect is so profound that I estimate that trees triple their mass
from the first ground-soaking rains after our dry summer. Leaf size
and shape don't seem to matter - absorption is empirical, unrelated to
frequency.
The other situation where this absorption occurs is low altitude links
over sea/brackish water. Wind can whip salty spray up into the air and
snuff signal totally.
David
Alexander Chemeris
I'm not an RF engineer, but from what I read - GSM basestation (BS)
is a huge output power, up to 60W and sometimes more. So a phone
does not need to be very sensitive. On the other side, BS has much
more sensitivity then a phone, so phone does not need to have very
high output power. That's why even at comparable powers of a WiFi
gear and a mobile phone, phone will win.
re: 180
It's used in cities, not in rural, because it has smaller range and it
better propagates in buildings because it reflects from walls, thus
increasing coverage.
Again, I may be wrong, but that's what I've read.
Regards,
Alexander Chemeris.
David A. Burgess
On Oct 23, 2009, at 8:19 PM, David Rowe wrote:
>
> Here is a question for the GSM and Wifi gurus. Why does GSM work
> better
> than VOIP-over-Wifi for telephony?
One of my favorite topics. The short answer is that WiFi was built
to operate over 10's of meters and GSM was built to operate over 10's
of kilometers. They are plenty of specific differences.
>
> By "better" consider this example:
>
> I can take my GSM handset down the street and around the corner from a
> GSM base station and it still works. If I take a Mesh Potato down the
> street and around the corner from another mesh node it stops working.
>
> Some ideas:
>
> 1/ Propagation of 2.4GHz versus 900MHz. However there are 1800MHz GSM
> services I understand that still work OK. Propagation at 1.8GHz can't
> be that much different to 2.4GHz.
See the previous post on water absorption. But unless you are in
vegetation or rain, that's just the start of it. (BTW, I would
expect QoS of a wifi mesh to vary with weather. Does it?)
>
> 2/ EIRP. A GSM handset may transmit at 1W (30dBm), compared to a Wifi
> AP of say 20dBm.
Probably not. GSM uses closed loop power control to ease the dynamic
range requirements at the BTS and to maximize handset battery life.
If you are just around the corner from the BTS, you are probably well
below 30 dBm.
>
> 3/ Bit rate and modulation scheme. I am not sure how GMSK at 250
> kbit/s
> compares to 1 Mbit/s spread spectrum for Wifi in terms of packet (or
> bit) error rate. Roughly 12dB gain at 1/4 of the bit rate?
GMSK is very similar to DQPSK, with the difference being that in GMSK
the signal trajectory is on the unit circle instead of in straight
lines. So the required link margin is low. GSM is receivable down
to around -110 dBm in most handsets and down to -113 dBm in the BTS.
Also, the actual total payload rate for a full GSM channel is closer
to 108 kb/s and they error code like hell. See next item.
>
> 4/ Protocol and forward error correction. Maybe Wifi depends on big
> packets getting thru without error.
Not only are GSM L2 frames very small (23 bytes), but the coding in
L1 allows the system to maintain low frame error rates even when
channel rate bit error rates are 10%-15%. In fact, given good soft
information between the demodulator and the error correction, you can
discard *half* of the radio bursts and still decode flawlessly.
This may be the most important single difference.
Also, because the GSM channel is essentially an ISDN-style
synchronous line, you have reliable QoS as long as the radio link
stays above the minimum service threshold.
>
> 5/ Benefits of licensed versus unlicensed spectrum?
You mean interference? It depends on your specific environment. Has
anyone done an RF survey at 2.4 GHz to see what else is out there?
Given the absorption qualities of 2.4 GHz, I would be surprised to
find strong interferers. Except maybe microwave ovens, which operate
in this band precisely because it's a good frequency for pumping
energy into water molecules.
>
> 6/ Propagation of GMSK versus spread spectrum? Maybe GMSK is more
> robust to multipath?
Yes. The narrower GSM bandwidth makes it naturally more robust in
cluttered environments, but it also includes an equalizer. CDMA/
WCDMA cellular uses rake receivers to combat multipath, with some
success. I don't know what the standard practice is in wifi, though.
>
> 7/ Antenna gain of the sectorial antennas at the GSM base station?
Maybe. The BTS antennas are probably 10-15 dBi. But you could put a
10 dB antenna on a wifi node, too. It wouldn't be *that* big.
>
> Comments invited......
Done. Counter-comments invited.
>
> Thanks,
>
> David
>
-- David
David A. Burgess
Kestrel Signal Processing, Inc.
David A. Burgess
On Oct 23, 2009, at 11:14 PM, David Carman wrote:
> And don't forget the base station power. GSM deployments have relied
> on brute force. Remember the early days of GSM, where the choice was
> between a 4W and 8W handset? It reminds me of the early days of the
> contraceptive pill - pack them with huge amounts of steroids to ensure
> efficacy, and then address safety issues once you're making a profit.
Most GSM systems are limited by uplink performance, not downlink.
Once the BTS EIRP is more than about 10 dB above the handset,
downlink power just doesn't matter anymore. The hard part is
receiving the uplink.
>
>> 3/ Bit rate and modulation scheme. I am not sure how GMSK at 250
>> kbit/s
>> compares to 1 Mbit/s spread spectrum for Wifi in terms of packet (or
>> bit) error rate. Roughly 12dB gain at 1/4 of the bit rate?
>
> The trend in digital wireless is towards higher frequencies and
> corresponding higher bitrates. The lower frequency of GSM is a core
> problem that is being addressed by better modulation and use of higher
> frequencies.
But it this same low bitrate that makes GSM so robust for classic
telephony. "It ain't sexy", but it works.
elektra
> 2.4 GHz is in the water absorption band. It is the frequency at which
> most microwaves operate since water absorbs it so well.
This is a popular myth. The lowest resonant frequency of water (fluid) is
18GHz, λ = 1,667 cm. Water vapor or steam has 22,23508 GHz resonance
frequency.
http://en.wikipedia.org/wiki/Microwave_oven:
The frequencies used in microwave ovens were chosen based on two constraints.
The first is that they should be in one of the ISM bands set aside for non-
communication purposes. Three additional ISM bands exist in the microwave
frequencies, but are not used for microwave cooking. Two of them are centered
on 5.8 GHz and 24.125 GHz, but are not used for microwave cooking because of
the very high cost of power generation at these frequencies. The third,
centered on 433.92 MHz, is a narrow band that would require expensive
equipment to generate sufficient power without creating interference outside the
band, and is only available in some countries. For household purposes, 2.45
GHz has the advantage over 915 MHz in that 915 MHz is only an ISM band in the
ITU Region 2 while 2.45 GHz is available worldwide.
Cheers,
Elektra
David Rowe
informative reply.
So given the lessons from a very successful system (GSM), what, I
wonder, can we do to make Wifi more conducive to telephony? How can I
get around that corner? Or am I stuck?
Here is what we have to work with:
- as an experiment, lets assume we have the whole Wifi channel available
to ourselves (no non village telco interferers).
- Lets assume that we have only a few voice channels, so we can choose
any rate between 1Mbit and 54Mbit and still have plenty of bandwidth.
- the AR2317 (which is probably typical) can detect packets at 1Mbit/s
down to -96dBm, 54 Mbit/s at -74dBm. I think the firmware guarantees if
you get a packet, it is error free.
- we can choose to send rather short packets if we like to reduce the
PER (at an expense of Ethernet/IP overhead). Or long ones for protocol
efficiency.
- long shot: we maybe don't have to use IP, we could send our own
Ethernet frames.
- we can add redundancy in the form of FEC, for example send FEC packets
or mix FEC with payload data. But it would have to be on top of the
Ethernet of IP packets. Weird, I know. Not sure if it's useful.
However we could reconstruct missing codec frames that way.
Cheers,
David
David A. Burgess
On Oct 24, 2009, at 12:06 AM, David Rowe wrote:
>
>> One of my favorite topics.
>
> Thanks David I was hoping you would say that :-) Thank you for your
> informative reply.
>
> So given the lessons from a very successful system (GSM), what, I
> wonder, can we do to make Wifi more conducive to telephony? How can I
> get around that corner? Or am I stuck?
>
> Here is what we have to work with:
>
> - as an experiment, lets assume we have the whole Wifi channel
> available
> to ourselves (no non village telco interferers).
>
> - Lets assume that we have only a few voice channels, so we can choose
> any rate between 1Mbit and 54Mbit and still have plenty of bandwidth.
>
> - the AR2317 (which is probably typical) can detect packets at 1Mbit/s
> down to -96dBm, 54 Mbit/s at -74dBm. I think the firmware
> guarantees if
> you get a packet, it is error free.
Definitely use the lower rate. Here's some math. The Hata models
are OK for predicting cellular coverage. I've never tried them on
wifi, but here's a shot. Your tx is 20 dBm. Your antennas are
probably 0 dB. Both antennas are at about 1 meter AGL.
Case 1: Your receiver 54 MB/s sensitivity is -74 dBm, but you want a
10 dB margin above that (-64 dBm) for fading and multipath, so your
allowed path loss is 84 dB (20 dBm - 84 dB = -64 dBm). The Hata
model predicts a range of about 22 meters for reliable (-64 dBm)
service, maybe 35 meters for marginal (-74 dBm) service.
Case 2: Your receiver 1 MB/s sensitivity is -96 dBm, but you want a
10 dB margin above (-86 dBm) that for fading and multipath, so your
allowed path loss is 106 dB (20 dbm - 106 = -86 dBm). The Hata model
predicts a range of 65 meters for reliable (-86 dBm) service and 105
meters for marginal (-96 dBm) service.
I have no idea if these estimated ranges reflect your field
experience. That 10 dB fading margin is probably too optimistic.
But even if the absolute ranges are unrealistic, the *ratio* is
probably accurate: a factor of three improvement by pushing down to 1
Mb/s.
>
> - we can choose to send rather short packets if we like to reduce the
> PER (at an expense of Ethernet/IP overhead). Or long ones for
> protocol
> efficiency.
Shorter is better if you can get away with it, since the lower layer
accepts/rejects whole packets. As a packet gets longer its
probability of successful transmission falls exponentially. It will
probably take a lot of experimentation to chose the right size. What
codec are you using? Can/do you send one packet per codec frame?
>
> - long shot: we maybe don't have to use IP, we could send our own
> Ethernet frames.
> - we can add redundancy in the form of FEC, for example send FEC
> packets
> or mix FEC with payload data. But it would have to be on top of the
> Ethernet of IP packets. Weird, I know. Not sure if it's useful.
> However we could reconstruct missing codec frames that way.
OK, I think I see what you are saying: Run raw ethernet with your own
FEC coding between the ethernet layer and the IP layer? That would
probably cut your effective data rate in half again, but would also
effectively make your receiver more sensitive.
But yes, one of the problems of the wifi radio interface is that it
relies on L3 and higher for all of the reliability. If you can put
some kind of variable-rate coding in L1 or L2 you can, in effect,
change your modulation in software to trade range for bandwidth at
rates well below 1 Mb/sec. But then you run into the next problem
with wifi, which is that CSMA/CA is not nearly as efficient as lock-
step TDMA, especially over significant distances and especially in
multi-node, shared-medium networks.
Also, if you can send one packet per codec frame, you might consider
turning off parity checking on media packets. SIP signaling can rely
on retransmission in the higher layers. The problem with that,
though, is that you are running a multihop network and speech quality
might degrade very quickly over multiple links. You can probably
tolerate one such link in the call path, but probably not more.
Here's a very important question before you put any effort into
anything novel: How are data losses distributed over time? Do whole
packets just disappear in the radio link, or is there a steady
occurrence of bad bits? Does the link switch between good and bad
from one packet to another, or is it just steadily mediocre? Does
the bit error probability vary a lot from packet to packet? (All the
same question, I know.)
>
> Cheers,
Alexander Chemeris
> On Oct 24, 2009, at 12:06 AM, David Rowe wrote:
>> - we can choose to send rather short packets if we like to reduce the
>> PER (at an expense of Ethernet/IP overhead). Or long ones for
>> protocol efficiency.
>
> Shorter is better if you can get away with it, since the lower layer
> accepts/rejects whole packets. As a packet gets longer its
> probability of successful transmission falls exponentially. It will
> probably take a lot of experimentation to chose the right size. What
> codec are you using? Can/do you send one packet per codec frame?
All VoIP codecs can be used in 1 frame per packet mode, and that's
how they're usually used. You put more frames into packet only when
you want to save bandwidth, or when you're doing RED (redundant
enccoding).
Here is my VoIP-side of the view. You can try using UDPlite, which
offers partial checksum protection, e.g. you can protect RTP header
and some essential bits of a codec and leave the rest unprotected.
But you will need to turn off checksum checking on WiFi driver level,
and I don't know how easy is this, probably some patching is needed?
Then, FEC for VoIP streams is a pretty well developed area, there
are even RTP payload type for FEC-encoded data. But it will increase
header size even more. You can also consider redundant encodings,
but they are probably less effective then FEC.
To reduce packet overhead you can use RTP header compression
or trunking protocols. The most famous trunking protocol so far
is IAX (inter-Asterisk eXchange), which is natively supported by
Asterisk. This can help you greatly reduce RTP header overhead.
What for using bare Ethernet without IP... well, I'd first try to go
without that much hacking. I.e. try UDP-lite, FEC and trunking first.
> Here's a very important question before you put any effort into
> anything novel: How are data losses distributed over time? Do whole
> packets just disappear in the radio link, or is there a steady
> occurrence of bad bits? Does the link switch between good and bad
> from one packet to another, or is it just steadily mediocre? Does
> the bit error probability vary a lot from packet to packet? (All the
> same question, I know.)
I agree that you should do more testing and get the answer on these
questions. Yet, this does not stop you from doing other research and
testing.
--
Regards,
Alexander Chemeris.
David Rowe
> Shorter is better if you can get away with it, since the lower layer
> accepts/rejects whole packets. As a packet gets longer its
> probability of successful transmission falls exponentially. It will
> probably take a lot of experimentation to chose the right size. What
> codec are you using?
> Can/do you send one packet per codec frame?
efficiency. It is also a lighter load on our little CPU - more packets
mean more interrupts/second which have a lot of CPU overhead. However
those issues are another story - we shouldn't worry about them for this
thought experiment.
> Here's a very important question before you put any effort into
> anything novel: How are data losses distributed over time? Do whole
> packets just disappear in the radio link, or is there a steady
> occurrence of bad bits? Does the link switch between good and bad
> from one packet to another, or is it just steadily mediocre? Does
> the bit error probability vary a lot from packet to packet? (All the
> same question, I know.)
Carman might have some data from their mesh networks. It might be the
key question - perhaps the whole issue of speech quality over the mesh
can be reduced to a packet loss model.
My basic understanding is that a packet gets thru with 0 bit errors, or
it doesn't get thru.
My limited understanding is this: the Wifi protocol makes several
attempts to re-send every packet up to 7 times, pending an ack from the
other side. Unlike the wired Internet, this operates even for UDP
packets. There is also rate adaption logic shifting rates up and down
based on some metric.
I am not sure if these data-oriented Wifi protocols ware working for us
or against us when it comes to VOIP over Mesh Networks.
For example if there is a momentary outage on the channel resending a
packet several times will cause delay and consume bandwidth. For voice
it might just be better to forget the packet and move on. Much better
to lose one 20ms packet than clog up the channel or wait 2000ms trying
to resend something and queuing other packets behind you.
Couple of other thoughts:
1/ The PER specs for Wifi are based on 1000 byte packets. As you say
David, we may be able to do much better (better chance or receiving a
packet) with smaller packets. The trade off is more overhead and higher
collision probability on the channel.
Perhaps this can be tested with beacon packets, I understand that they
are transmitted at a low rate and are quite short.
2/ Interleaving of data and FEC might be helpful. For example spread
the payload data from 4 voice frames randomly over 4 packets and add
FEC. If one frame is lost entirely we can then reconstruct it.
Alexander re your comment:
>To reduce packet overhead you can use RTP header compression
>or trunking protocols. The most famous trunking protocol so far
>is IAX (inter-Asterisk eXchange), which is natively supported by
>Asterisk.
wondering what can be done from a trunking point of view. In the Mesh
we might find a small number of nodes carrying a lot of trunked voice
traffic. Perhaps there are some batman optimisations for VOIP possible
to make this more efficient or reliable.
Thanks,
David
David A. Burgess
One important topic not covered here is antennas. What do you really
know about your antennas, like their dBi and pattern shape? And how
are they typically placed?
On Oct 24, 2009, at 3:31 PM, David Rowe wrote:
>
> GSM at 13 kbit/s and Speex at 8 kbit/s are our current options
As much as I like GSM, it looks like Speex is a winner here.
>
>> Can/do you send one packet per codec frame?
>
> Yes, although currently we have it set at 4 packets/frame for
> bandwidth
> efficiency. It is also a lighter load on our little CPU - more
> packets
> mean more interrupts/second which have a lot of CPU overhead. However
> those issues are another story - we shouldn't worry about them for
> this
> thought experiment.
If you cut the packet length in half it will be like taking the
square root of the packet success rate. How much that is worth
depends on the current success rate. Roughly speaking, you will cut
the packet failure rate in half. Maybe 2 packets/frame is a good
tradeoff. I don't know.
>
> My basic understanding is that a packet gets thru with 0 bit
> errors, or
> it doesn't get thru.
Sure, but the question is, "What do the bit error statistics look
like down in the radio, before the parity check is applied?" To
determine that you will need to send packets with known content and
then check for differences on the receive end, way down in the lowest
sublayers available to you.
>
> My limited understanding is this: the Wifi protocol makes several
> attempts to re-send every packet up to 7 times, pending an ack from
> the
> other side. Unlike the wired Internet, this operates even for UDP
> packets. There is also rate adaption logic shifting rates up and down
> based on some metric.
Ick.
>
> I am not sure if these data-oriented Wifi protocols ware working
> for us
> or against us when it comes to VOIP over Mesh Networks.
Probably against you.
>
> For example if there is a momentary outage on the channel resending a
> packet several times will cause delay and consume bandwidth. For
> voice
> it might just be better to forget the packet and move on. Much better
> to lose one 20ms packet than clog up the channel or wait 2000ms trying
> to resend something and queuing other packets behind you.
Yes. For real time media, retransmission wastes bandwidth and
introduces brutal jitter. It makes overall quality worse, not better.
In cellular, the control channels have robust parity checking and
retransmission, but the media channels have only partial parity
checking and receiver just extrapolates lost vocoder frames.
>
> Couple of other thoughts:
>
> 1/ The PER specs for Wifi are based on 1000 byte packets. As you say
> David, we may be able to do much better (better chance or receiving a
> packet) with smaller packets. The trade off is more overhead and
> higher
> collision probability on the channel.
>
> Perhaps this can be tested with beacon packets, I understand that they
> are transmitted at a low rate and are quite short.
Yes, that's a good idea. The error statistics there should allow you
to predict them for other traffic types.
>
> 2/ Interleaving of data and FEC might be helpful. For example spread
> the payload data from 4 voice frames randomly over 4 packets and add
> FEC. If one frame is lost entirely we can then reconstruct it.
You might get good results just by applying FEC, without
interleaving. It depends on the error statistics. The problem,
though, is that the decoding can be computationally heavy for a small
processor. There are some decent Viterbi decoders in the CommonLibs
directory of OpenBTS that you might find useful... if you can work
with C++.
If you do opt for interleaving, consider a streaming interleaver over
a block interleaver.
>
> Alexander re your comment:
>
>> To reduce packet overhead you can use RTP header compression
>> or trunking protocols. The most famous trunking protocol so far
>> is IAX (inter-Asterisk eXchange), which is natively supported by
>> Asterisk.
>
> Yes we should try and compare IAX, that is a good idea. I am also
> wondering what can be done from a trunking point of view. In the Mesh
> we might find a small number of nodes carrying a lot of trunked voice
> traffic. Perhaps there are some batman optimisations for VOIP
> possible
> to make this more efficient or reliable.
>
> Thanks,
>
> David
Hope I can help.
David Rowe
> know about your antennas, like their dBi and pattern shape? And how
> are they typically placed?
That's actually what we are thinking about at the moment. We intend to
design an internal antenna but are calling for ideas about the sort of
gain and pattern.
A typical placement might by 3m above ground, like a TV antenna.
- David
Alexander Chemeris
On Sun, Oct 25, 2009 at 06:14, David A. Burgess <dbur...@jcis.net> wrote:
> On Oct 24, 2009, at 3:31 PM, David Rowe wrote:
>>
>> GSM at 13 kbit/s and Speex at 8 kbit/s are our current options
>
> As much as I like GSM, it looks like Speex is a winner here.
Right. Yet, if you go with partial checksum protection, you
will have to develop own Speex packaging format, because
current one is not suitable for this. GSM already have an
option to support partial checksums protection, IIRC.
>> My limited understanding is this: the Wifi protocol makes several
>> attempts to re-send every packet up to 7 times, pending an ack from
>> the
>> other side. Unlike the wired Internet, this operates even for UDP
>> packets. There is also rate adaption logic shifting rates up and down
>> based on some metric.
>
> Ick.
>
>>
>> I am not sure if these data-oriented Wifi protocols ware working
>> for us
>> or against us when it comes to VOIP over Mesh Networks.
>
> Probably against you.
>
>>
>> For example if there is a momentary outage on the channel resending a
>> packet several times will cause delay and consume bandwidth. For
>> voice
>> it might just be better to forget the packet and move on. Much better
>> to lose one 20ms packet than clog up the channel or wait 2000ms trying
>> to resend something and queuing other packets behind you.
>
> Yes. For real time media, retransmission wastes bandwidth and
> introduces brutal jitter. It makes overall quality worse, not better.
>
> In cellular, the control channels have robust parity checking and
> retransmission, but the media channels have only partial parity
> checking and receiver just extrapolates lost vocoder frames.
Well, this depends on how bad loss pattern is. Sometimes it's
better to tolerate 200ms jitter then loss 50% of data, making
conversation completely unintelligible. So again, you need to
collect real data to make decisions. And actually that may be
a PhD size of work. :) Jitter-fighting is a hardest part of VoIP!
BTW, interesting question is how jitter will increase in mesh
multi-hop environment, where every link may have very high
jitter.
>> 2/ Interleaving of data and FEC might be helpful. For example spread
>> the payload data from 4 voice frames randomly over 4 packets and add
>> FEC. If one frame is lost entirely we can then reconstruct it.
>
> You might get good results just by applying FEC, without
> interleaving. It depends on the error statistics. The problem,
> though, is that the decoding can be computationally heavy for a small
> processor. There are some decent Viterbi decoders in the CommonLibs
> directory of OpenBTS that you might find useful... if you can work
> with C++.
>
> If you do opt for interleaving, consider a streaming interleaver over
> a block interleaver.
Simplest way, often used in VoIP is a repetition of a frame in the
next packet. But it roughly doubles bandwidth requirements.
--
Regards,
Alexander Chemeris.
David Rowe
> like down in the radio, before the parity check is applied?" To
> determine that you will need to send packets with known content and
> then check for differences on the receive end, way down in the lowest
> sublayers available to you.
Yes, very good idea.
I also like the idea (in yr previous email) of turning off parity checks
on media packets, i.e. getting the Wifi layer to pass through all
packets, even those of dubious quality. This would help in a lot of
ways:
i) Voice packets can tolerate a much higher bit error rate than data so
we would avoid unneeded retransmissions.
ii) It would stop a bunch of retransmissions piling up which as you
suggest may be working against us.
iii) It would let us apply our own FEC and/or interleaving schemes,
optimised for voice traffic.
I am not sure if we have access to those layers. If we are lucky they
are part of the MadWifi driver, if we are unlucky it's part of the
Atheros "secret sauce", i.e. buried in the hardware or the binary blob
called the HAL. Elektra do you have any thoughts on this?
> In cellular, the control channels have robust parity checking and
> retransmission, but the media channels have only partial parity
> checking and receiver just extrapolates lost vocoder frames.
Yes, well we have TCP for the control channels which should be OK.
Alexander, this question you raise is a very good one:
> BTW, interesting question is how jitter will increase in mesh
> multi-hop environment, where every link may have very high
> jitter.
It's probably time for some tests to characterise the channel, for
example:
1/ Set up a two node mesh with good signal, send UDP (RTP?) packets over
it and then perturb the link in such a way that Wifi packet's are lost,
forcing the re-transmission system to act. Gather statistics on UDP
packet arrival time and/or loss. The UDP packet rate could be set to
simulate a single call or a mesh node trunking many phone calls.
2/ Repeat (1) over a multiple hop mesh, say 5 hops.
3/ See if we can get low level stats on the error patterns prior to the
Wifi protocols doing their thing.
4/ Put up a test signal from a Mesh Potato or router mounted at a
typical mesh-potato-installation height then walk around the
neighbourhood with another router or spectrum analyser to measure path
loss in an urban environment. Some real numbers might tell us what is
possible by messing with data rates and FEC schemes.
For example is the channel power limited? Or perhaps limited by
multi-path effects, i.e. large signal and hence PER variations with
small changes in router location, orientation, or movement of nearby
objects (which I suspect).
Thanks,
David
Alexander Chemeris
>> In cellular, the control channels have robust parity checking and
>> retransmission, but the media channels have only partial parity
>> checking and receiver just extrapolates lost vocoder frames.
>
> Yes, well we have TCP for the control channels which should be OK.
Oh, missed this in my previous e-mail. IIRC, TCP is not recommended
for SIP, because SIP provides its own means of control packet
retransmission. Yet, while SIP timers are tunned to a usual wired-Internet,
I think they'll be a good start for mesh-network too. I believe you can
raise this question on IETF's SIP WG mailing list - that's the best place
to get real SIP expertise. IAX also provides retransmission of control
packets and most important media-packets (e.g. I-frames in MPEG
video stream).
--
Regards,
Alexander Chemeris.
elektra
> I am not sure if we have access to those layers. If we are lucky they
> are part of the MadWifi driver, if we are unlucky it's part of the
> Atheros "secret sauce", i.e. buried in the hardware or the binary blob
> called the HAL. Elektra do you have any thoughts on this?
Atheros has released one version of the source code for the HAL. However the
HAL used in Openwrt's Madwifi driver is different and not publicly available.
Some functions are directly performed in the hardware.
http://madwifi-project.org/wiki/news/20080929/hal-source-code-released
<Quote>
It might be worth mentioning that the HAL Atheros has released the source for
differs from the HAL versions we use in MadWifi, and thus is no drop-in
replacement. In theory it should be possible to integrate the HAL and thus
make MadWifi a fully open-source driver. However, in the light of recent
discussions and the amount of work required for such a step, it seems
questionable if it will happen. It's likely that this will be subject of
discussions in the madwifi-devel list.
</Quote>
Forward error correction is used in 802.11g and 802.11a.
Even with the GSM codec the actual payload is quite small compared to the
protocol overhead. As David has mentioned before we'll have problems sending
many small packets. We get more load from interrupts and we'll spend more
time on negotiating who is going to transmit packets.
> > In cellular, the control channels have robust parity checking and
> > retransmission, but the media channels have only partial parity
> > checking and receiver just extrapolates lost vocoder frames.
>
> Yes, well we have TCP for the control channels which should be OK.
>
TCP is performing very bad in wireless environments, because it will
erroneously consider packetloss as network congestions and fall back to
exponential back off. It is great that WiFi cards perform retransmissions on
a per-hop basis.
> Alexander, this question you raise is a very good one:
> > BTW, interesting question is how jitter will increase in mesh
> > multi-hop environment, where every link may have very high
> > jitter.
It depends on how bad links are. With a chain of good links all is fine. Even
with many hops. With a chain of miserable or not-so well working links jitter
and latency is growing. The WiFi layer has to perform retransmissions for
each weak link in the chain. An adaptive jitter buffer is a must. However the
real disaster for VOIP is packet loss beyond 5%. That can happen easily if
you have to rely on unreliable links.
For many people round trip latency in VOIP is a holy cow. I often hear quotes
like "Latency beyond 100ms is unbearable". I don't think so. My personal
opinion is that round trip latency of 300ms is still comfortable. Even more
latency is bearable if the system you are using is the only means to do voice
communication ;-)
Every time you communicate over a satellite link the latency is far higher.
The round trip delay I measured was ~600 ms at best. Uncomforable but one can
get used to it.
Btw: I'm using a multi-hop mesh network as my Internet uplink since 2004 and
David and I have spent many hours talking at night between Berlin and
Adelaide. The latency introduced by the mesh is very low compared to the
latency between Germany and Australia.
The WiFi range from the top of a house or shack is several hundred meters,
even in the center of Berlin in a fully overcrowded and noisy 2.4 GHz band.
(Using the default simple sleeve-dipole antennas that are shipped with most
routers.) It is great to have the WiFi device 1-3 meters above the roof.
Regarding mobile VOIP mesh phones: The main problem is that WiFi is hard on
battery, particularly in ad-hoc mode. There is no appropriate power saving
mechanism for ad-hoc. Also most mobile WiFi devices don't offer great range.
They usually have low transmit power to save battery and ineffective
antennae. My Nokia N810 is a good exception from this, because it has a
really nice 100mW WiFi interface that works nicely in Ad-Hoc mode. However
the battery only lasts 4 hours if I use it in the mesh. Range is a few
hundred meters outdoors. With a mesh routing protocol tuned to converge
quickly it is usable as a mobile mesh phone. (As long as you are not moving
fast)
It could be great to have a less strict policy on retransmitting voice packets
in case of marginal errors in the VOIP packet payload. This could be a great
improvement.
Elektra
David A. Burgess
existing wifi parity checks and L1 ack/repeat mechanism.
Alexander Chemeris
On Sun, Oct 25, 2009 at 20:23, elektra <onel...@gmx.net> wrote:
> Even with the GSM codec the actual payload is quite small compared to the
> protocol overhead. As David has mentioned before we'll have problems sending
> many small packets. We get more load from interrupts and we'll spend more
> time on negotiating who is going to transmit packets.
Seems trunking may be a solution for you, but you'll have a big trade-off
between PER and latency on one hand and CPU load on the other.
>> > In cellular, the control channels have robust parity checking and
>> > retransmission, but the media channels have only partial parity
>> > checking and receiver just extrapolates lost vocoder frames.
>>
>> Yes, well we have TCP for the control channels which should be OK.
>>
>
> TCP is performing very bad in wireless environments, because it will
> erroneously consider packetloss as network congestions and fall back to
> exponential back off. It is great that WiFi cards perform retransmissions on
> a per-hop basis.
>
>> Alexander, this question you raise is a very good one:
>> > BTW, interesting question is how jitter will increase in mesh
>> > multi-hop environment, where every link may have very high
>> > jitter.
>
> It depends on how bad links are. With a chain of good links all is fine. Even
> with many hops. With a chain of miserable or not-so well working links jitter
> and latency is growing. The WiFi layer has to perform retransmissions for
> each weak link in the chain. An adaptive jitter buffer is a must. However the
> real disaster for VOIP is packet loss beyond 5%. That can happen easily if
> you have to rely on unreliable links.
I would say that with decent PLC you can tolerate up to 15% of loss and
still have intelligible speech (at least for English and Russian languages
and uniformly distributed losses of 20ms frames). We wrote such generic
PLC for use with G.711 and I think Speex's PLC is no worse (yet I haven't
tested it). BUT it really depends on loss pattern, you you lose, say five
20ms frames at once - you've definitely lost few phonemes, which is not
recoverable without AI. :)
> For many people round trip latency in VOIP is a holy cow. I often hear quotes
> like "Latency beyond 100ms is unbearable". I don't think so. My personal
> opinion is that round trip latency of 300ms is still comfortable. Even more
> latency is bearable if the system you are using is the only means to do voice
> communication ;-)
>
> Every time you communicate over a satellite link the latency is far higher.
> The round trip delay I measured was ~600 ms at best. Uncomforable but one can
> get used to it.
>
> Btw: I'm using a multi-hop mesh network as my Internet uplink since 2004 and
> David and I have spent many hours talking at night between Berlin and
> Adelaide. The latency introduced by the mesh is very low compared to the
> latency between Germany and Australia.
In general I agree with you that trained people may tolerate much higher
latency that is usually stated (ITU-T standard for voice communication states
that excellent quality may be achieved only bellow 150ms). And I also believe
that for very-low-cost solution at which you aim that's fine, as long
people won't
expect excellent quality (am I right?). Latency is vital, when you ship product
to a well developed market, where people can choose between good, better
and best quality. That's from non-technical point of view. From technical point
of view, latency in VoIP is essential when you face (1) echo and (2) conference
calls. In conference call latency plays its bad role much more and actually
with 600ms latency conference looks more like push-to-talk, where you need
a well defined protocol of talking permission passing.
BTW, what do you plan to do with echo? Just rely on analog phone? I fear
there may be a lot of troubles - echo is veeeery bad thing and makes
conversation almost impossible. So you plan at least echo suppression
if not echo cancellation?
--
Regards,
Alexander Chemeris.
Sjur Eivind Usken
> for SIP, because SIP provides its own means of control packet
> retransmission. Yet, while SIP timers are tunned to a usual wired-Internet,
> I think they'll be a good start for mesh-network too. I believe you can
> raise this question on IETF's SIP WG mailing list - that's the best place
> to get real SIP expertise. IAX also provides retransmission of control
> packets and most important media-packets (e.g. I-frames in MPEG
> video stream).
SIP TCP is not that widely deployed and not all SIP UA that supports this.
SIP standard (RFC 3262) defines Provisional Response.
This is a standard that sends an ACK for every event.
In case it does not see the ACK, it resends the packet after 0,5
seconds, then 1 second, 2 seconds, then 4 seconds, 4 again until you
reach the 20 second time-out for the message.
regards
sjur
David A. Burgess
cellular carriers often have latencies exceeding 200 ms. We get
along just fine.
On Oct 25, 2009, at 9:53 AM, Alexander Chemeris wrote:
>
>> For many people round trip latency in VOIP is a holy cow. I often
>> hear quotes
>> like "Latency beyond 100ms is unbearable". I don't think so. My
>> personal
>> opinion is that round trip latency of 300ms is still comfortable.
>> Even more
>> latency is bearable if the system you are using is the only means
>> to do voice
>> communication ;-)
>>
>> Every time you communicate over a satellite link the latency is
>> far higher.
>> The round trip delay I measured was ~600 ms at best. Uncomforable
>> but one can
>> get used to it.
Alexander Chemeris
>> Oh, missed this in my previous e-mail. IIRC, TCP is not recommended
>> for SIP, because SIP provides its own means of control packet
>> retransmission. Yet, while SIP timers are tunned to a usual wired-Internet,
>> I think they'll be a good start for mesh-network too. I believe you can
>> raise this question on IETF's SIP WG mailing list - that's the best place
>> to get real SIP expertise. IAX also provides retransmission of control
>> packets and most important media-packets (e.g. I-frames in MPEG
>> video stream).
>
> SIP TCP is not that widely deployed and not all SIP UA that supports this.
may easily have SDP body which will make your INVITE bigger then
allowed 1.5K bytes for UDP. Such INVITEs are to be sent over TCP.
> SIP standard (RFC 3262) defines Provisional Response.
> This is a standard that sends an ACK for every event.
> In case it does not see the ACK, it resends the packet after 0,5
> seconds, then 1 second, 2 seconds, then 4 seconds, 4 again until you
> reach the 20 second time-out for the message.
>
>
> regards
> sjur
>
> >
>
Regards,
Alexander Chemeris.
elektra
> Seems trunking may be a solution for you, but you'll have a big trade-off
> between PER and latency on one hand and CPU load on the other.
Indeed. I'm particularly concerned about the CPU load.
> I would say that with decent PLC you can tolerate up to 15% of loss and
> still have intelligible speech (at least for English and Russian languages
> and uniformly distributed losses of 20ms frames). We wrote such generic
> PLC for use with G.711 and I think Speex's PLC is no worse (yet I haven't
> tested it). BUT it really depends on loss pattern, you you lose, say five
> 20ms frames at once - you've definitely lost few phonemes, which is not
> recoverable without AI. :)
Are there open source packet loss concealment solutions available?
> In general I agree with you that trained people may tolerate much higher
> latency that is usually stated (ITU-T standard for voice communication
> states that excellent quality may be achieved only bellow 150ms). And I
> also believe that for very-low-cost solution at which you aim that's fine,
> as long people won't
> expect excellent quality (am I right?).
Absolutely.
> Latency is vital, when you ship
> product to a well developed market, where people can choose between good,
> better and best quality. That's from non-technical point of view. From
> technical point of view, latency in VoIP is essential when you face (1)
> echo and (2) conference calls. In conference call latency plays its bad
> role much more and actually with 600ms latency conference looks more like
> push-to-talk, where you need a well defined protocol of talking permission
> passing.
We are aiming at developing countries with this project. People are used to
overprized services of low quality. If any.
> BTW, what do you plan to do with echo? Just rely on analog phone? I fear
> there may be a lot of troubles - echo is veeeery bad thing and makes
> conversation almost impossible. So you plan at least echo suppression
> if not echo cancellation?
Currently the MP is using David's Open Source Line Echo Canceller (OSLEC).
Thanks!
Elektra
Alexander Chemeris
On Sun, Oct 25, 2009 at 21:28, elektra <onel...@gmx.net> wrote:
>> I would say that with decent PLC you can tolerate up to 15% of loss and
>> still have intelligible speech (at least for English and Russian languages
>> and uniformly distributed losses of 20ms frames). We wrote such generic
>> PLC for use with G.711 and I think Speex's PLC is no worse (yet I haven't
>> tested it). BUT it really depends on loss pattern, you you lose, say five
>> 20ms frames at once - you've definitely lost few phonemes, which is not
>> recoverable without AI. :)
>
> Are there open source packet loss concealment solutions available?
Humm... I'm sure there are a lot of them, because every VoIP UA should
have it, or it won't be usable. But I have no feel of their quality, as we never
evaluated them. Skilled DSP engineer can develop one for about 2 months
from scratch.
But you don't need generic-purpose PLC about which I've been talking.
Most todays codecs have internal PLC which is usually *better* then
generic ones, because they use internal codec state knowledge. If you're
aiming at Speex, I would recommend you to evaluate its internal PLC
and decide whether it's fine for you. If not - it's likely better to work on
improving it rather then using generic purpose one. Talk to Jean-Marc
Valin (Speex author) - he's a nice and supportive guy.
BTW, have you measured Speex performance on your CPU in presence
of lost frames? I may be different (may be higher) then under normal
conditions.
>> BTW, what do you plan to do with echo? Just rely on analog phone? I fear
>> there may be a lot of troubles - echo is veeeery bad thing and makes
>> conversation almost impossible. So you plan at least echo suppression
>> if not echo cancellation?
>
> Currently the MP is using David's Open Source Line Echo Canceller (OSLEC).
AEC (Acoustic Echo Canceler) is different from LEC (Line Echo Canceler)
and I doubt you can use LEC instead of AEC with a good results. Yet, I'm not
an expert in echo cancellation and can't tell you more.. Take a look at Speex's
AEC - it's the best known open-source AEC out there. What I worry about is
CPU consumption - AEC is very CPU-intensive and is very sensitive to acoustic
channel non-linearity like frame dropouts. So it's best done as close
to hardware
as possible - some even do it in separate DSP in-between of audio codec and
CPU. But I doubt you can afford that, so you'll have to go with software AEC
and you should measure do you have enough CPU power for it.
Again, I'm sure Jean-Marc will answer questions if any.
Btw, what algorithm/implementation do you use for Jitter Buffer (JB)?
What do you do with input audio gain - do you have Adaptive Gain Control (AGC)
or rely on external volume control?
--
Regards,
Alexander Chemeris.
David A. Burgess
etc.) typically used in cellular, including GSM, you can get good
results by interpolating or extrapolating vocoder parameters prior to
decoding. Most published cellular codecs include explicit algorithms
for bad frame processing, normally based on extrapolation instead of
interpolation because that has lower latency. All such
specifications include guidelines for bad frame concealment even if
no specific example algorithm is given. All of that knowledge is
readily applicable to PLC for this application.
(I've even seen some iDEN systems that deliberately discarded every
other frame so that they could over-allocate a channel and support an
extra call on the cell. It sounded like hell, but paying subscribers
were using it.)
On Oct 25, 2009, at 11:55 AM, Alexander Chemeris wrote:
>>
>> Are there open source packet loss concealment solutions available?
>
> Humm... I'm sure there are a lot of them, because every VoIP UA should
> have it, or it won't be usable. But I have no feel of their
> quality, as we never
> evaluated them. Skilled DSP engineer can develop one for about 2
> months
> from scratch.
>
> But you don't need generic-purpose PLC about which I've been talking.
> Most todays codecs have internal PLC which is usually *better* then
> generic ones, because they use internal codec state knowledge. If
> you're
> aiming at Speex, I would recommend you to evaluate its internal PLC
> and decide whether it's fine for you. If not - it's likely better
> to work on
> improving it rather then using generic purpose one. Talk to Jean-Marc
> Valin (Speex author) - he's a nice and supportive guy.
Alexander Chemeris
> Again, I concur. For the CELP-family codecs (VSELP, ACELP, RPE-LTP,
> etc.) typically used in cellular, including GSM, you can get good
> results by interpolating or extrapolating vocoder parameters prior to
> decoding. Most published cellular codecs include explicit algorithms
> for bad frame processing, normally based on extrapolation instead of
> interpolation because that has lower latency.
Well, actually in VoIP you can use interpolation in many cases, just
because you'll likely have JB length of more then 3 packets in real
environment and interpolation is just better then extrapolation. Sure,
you need to support extrapolation in case next packet is missing or
you're doing same-city call.
--
Regards,
Alexander Chemeris.
elektra
Thanks for your interesting suggestions and comments. It is particularly
interesting to learn that Speex and probably other codecs are going to
consume more CPU load in the face of packet loss.
Your questions regarding VOIP are well beyond my humble knowledge about the
subject. I feel that I'm learning a lot following this discusion, but I have
to refer to David. He can surely answer your questions.
Elektra
David Rowe
Thanks for your comments re the HAL. So is it possible to (i) control
the number of retransmission attempts (down to zero) and (ii) pass
through packets that fail the checksum using Madwifi? It would be
useful to check out the bit error patterns on packets that fail to be
received correctly.
> Forward error correction is used in 802.11g and 802.11a.
It would be interesting to test the PER for comparable 11g and 11b modes
to test the effect of the built in FEC. For example is the PER better
at 6 Mbit 11g than 5 Mbit 11b?
> Even with the GSM codec the actual payload is quite small compared to the
> protocol overhead. As David has mentioned before we'll have problems sending
> many small packets. We get more load from interrupts and we'll spend more
> time on negotiating who is going to transmit packets.
Yes this is an interesting question. Does the protocol overhead of many
small packets result in poorer voice quality than fewer large packets?
We probably need a model or experimental data to test this question.
There a bandwidth/efficiency hit with smaller packets, but if we have
plenty of bandwidth compared to the voice traffic this may be OK.
I assume we start with good quality links - as the Village Telco
Entrepreneur will be installing them to some standard, e.g. a link
margin of 10-20dB. They might be 300m apart in line of sight (but with
crappy Fresnel zone clearance and lots of multipath). However then
something screws up the channel, for example a sudden burst of traffic
or interference, or a physical change to a multipath reflection that
wipes out a packet.
In a good channel small and large packets will sound fine. As we add
errors or lose packets I would like to understand what happens with
different configurations.
Our previous discussions on GSM v Wifi might also help in this scenario,
e.g. perhaps a lower bit rate or the use of FEC can punch thru
interference.
> For many people round trip latency in VOIP is a holy cow. I often hear quotes
> like "Latency beyond 100ms is unbearable". I don't think so. My personal
> opinion is that round trip latency of 300ms is still comfortable. Even more
> latency is bearable if the system you are using is the only means to do voice
> communication ;-)
Yes I am not too worried about latency. I feel one parameter we have to
play with is increasing end-end delay (if it helps, e.g. using a jitter
buffer).
Actually the delay in a MP to MP call is probably already 200ms,
although I haven't measured it. No who has used a MP has mentioned the
delay during conversational tests (unless they were in the same room).
However it's something we should try to measure and optimise one day.
After we get some data on the RTP traffic I would like to look at the
jitter buffers available to us in Asterisk. Make sure they fit our
traffic and make a meaningful improvement. Tweak them if necessary.
Cheers,
David
David Rowe
> between PER and latency on one hand and CPU load on the other.
Actually we have some flexibility with CPU load. The DSP algorithms
which chew up a lot of the CPU (codec and echo canceller) haven't been
optimised for the AR2317 (MIPs 4K core) yet. There is also the
possibility of making the Wifi driver more efficient for smaller packets
(although that's more of a long shot).
So I don't consider CPU load a show stopper for smaller packets (yet).
> BTW, what do you plan to do with echo? Just rely on analog phone? I fear
> there may be a lot of troubles - echo is veeeery bad thing and makes
> conversation almost impossible. So you plan at least echo suppression
> if not echo cancellation?
We are using Oslec: http://www.rowetel.com/ucasterisk/oslec.html
The echo problem is similar to an ATA:
VOIP - AR2317 CPU - FXS Port - Analog Phone
There is line echo from the 2-wire FXS port/analog phone interface that
must be cancelled. However the echo path is short as we are talking
about a few metres of phone cable rather than a few km for cable back to
a central office. We currently run a 8ms tail on the MP which works
fine and the MIPs used don't seem to upset the AR2317 under our maximum
load model (MP routing 15 call while making a call of it's own).
Oslec works fine in practice - there is no echo from the FXS port.
Cheers,
David
David Rowe
> But you don't need generic-purpose PLC about which I've been talking.
> Most todays codecs have internal PLC which is usually *better* then
> generic ones, because they use internal codec state knowledge. If you're
> aiming at Speex, I would recommend you to evaluate its internal PLC
> and decide whether it's fine for you.
Yes it's worth checking out Speex with PLC enabled once have an
objective way to compare. For example we could sample some RTP packets
off air on a bad channel, then run them thru the Speex decoder with and
without PLC switched on.
> BTW, have you measured Speex performance on your CPU in presence
> of lost frames? I may be different (may be higher) then under normal
> conditions.
Nope, but in general the CPU load for PLC is trivial in CELP codecs
compared the other DSP work going on. But worth checking, I am not
familiar with the Speex PLC algorithm.
> Btw, what algorithm/implementation do you use for Jitter Buffer (JB)?
What ever Asterisk has internally. I understand this has been upgraded
in recent versions.
> What do you do with input audio gain - do you have Adaptive Gain Control (AGC)
> or rely on external volume control?
Nope, but it doesn't seem to be a problem. My land line phone at home
doesn't have a volume control and works just fine. In practice the
person deploying the MPs will probably use the same analog telephone to
help avoid variations.
Cheers,
David
Alexander Chemeris
(Hum.. I've counted at least three Davids actively participating here..)
On Mon, Oct 26, 2009 at 00:32, David Rowe <da...@rowetel.com> wrote:
>> For many people round trip latency in VOIP is a holy cow. I often hear quotes
>> like "Latency beyond 100ms is unbearable". I don't think so. My personal
>> opinion is that round trip latency of 300ms is still comfortable. Even more
>> latency is bearable if the system you are using is the only means to do voice
>> communication ;-)
>
> Yes I am not too worried about latency. I feel one parameter we have to
> play with is increasing end-end delay (if it helps, e.g. using a jitter
> buffer).
>
> Actually the delay in a MP to MP call is probably already 200ms,
> although I haven't measured it. No who has used a MP has mentioned the
> delay during conversational tests (unless they were in the same room).
> However it's something we should try to measure and optimise one day.
I can easily believe in this. Without a special tunning, that's what you
get just for audio in/out. This can be dropped to about 50ms with heavy
tunning and even lower, if you know your hardware. But this tunning is
harder then it looks first and in some cases may require even new audio
driver. Sure it depends, but I want to warn you beforehand. In SIPez we've
spent months, tweaking audio for low latency on some embedded platform.
--
Regards,
Alexander Chemeris.
Alexander Chemeris
>> BTW, what do you plan to do with echo? Just rely on analog phone? I fear
>> there may be a lot of troubles - echo is veeeery bad thing and makes
>> conversation almost impossible. So you plan at least echo suppression
>> if not echo cancellation?
>
> We are using Oslec: http://www.rowetel.com/ucasterisk/oslec.html
>
> The echo problem is similar to an ATA:
>
> VOIP - AR2317 CPU - FXS Port - Analog Phone
>
> There is line echo from the 2-wire FXS port/analog phone interface that
> must be cancelled. However the echo path is short as we are talking
> about a few metres of phone cable rather than a few km for cable back to
> a central office. We currently run a 8ms tail on the MP which works
> fine and the MIPs used don't seem to upset the AR2317 under our maximum
> load model (MP routing 15 call while making a call of it's own).
>
> Oslec works fine in practice - there is no echo from the FXS port.
Sorry, I wasn't clear enough - I meant Acoustic Echo Cancelling:
http://en.wikipedia.org/wiki/Acoustic_Echo_Cancelling
It requires 50-200ms tail length and thus is much more CPU intensive.
For usual phones you probably will need lower values from this interval,
yet they might beat your CPU to the ground.
--
Regards,
Alexander Chemeris.
Alexander Chemeris
On Mon, Oct 26, 2009 at 00:59, David Rowe <da...@rowetel.com> wrote:
>> But you don't need generic-purpose PLC about which I've been talking.
>> Most todays codecs have internal PLC which is usually *better* then
>> generic ones, because they use internal codec state knowledge. If you're
>> aiming at Speex, I would recommend you to evaluate its internal PLC
>> and decide whether it's fine for you.
>
> Yes it's worth checking out Speex with PLC enabled once have an
> objective way to compare. For example we could sample some RTP packets
> off air on a bad channel, then run them thru the Speex decoder with and
> without PLC switched on.
You can't turn off and on PLC in it, because it automatically perform PLC
when you pass empty frame to it. So I'd recommend (1) compare original
audio after encoding/decoding with audio after encoding/loss/decoding to
get relative value and (2) just listen to the audio after encoding/loss/decoding
to see is it intelligible enough or not.
>> BTW, have you measured Speex performance on your CPU in presence
>> of lost frames? I may be different (may be higher) then under normal
>> conditions.
>
> Nope, but in general the CPU load for PLC is trivial in CELP codecs
> compared the other DSP work going on. But worth checking, I am not
> familiar with the Speex PLC algorithm.
>
>> Btw, what algorithm/implementation do you use for Jitter Buffer (JB)?
>
> What ever Asterisk has internally. I understand this has been upgraded
> in recent versions.
I'd recommend you to valuate it carefully after you get some experimental
data on delay distribution. I guess you need some sort of fast-reacting
algorithm in your environment, like something Ramji-based, not something
histogram-based (like Speex's JB).
--
Regards,
Alexander Chemeris.
David Rowe
> > Oslec works fine in practice - there is no echo from the FXS port.
>
> Sorry, I wasn't clear enough - I meant Acoustic Echo Cancelling:
> http://en.wikipedia.org/wiki/Acoustic_Echo_Cancelling
> It requires 50-200ms tail length and thus is much more CPU intensive.
> For usual phones you probably will need lower values from this interval,
> yet they might beat your CPU to the ground.
Why do we need an acoustic echo cancelling? Analog telephones require
only line echo cancellers. Analog speaker phones have their own echo
control built in.
Cheers,
David
David Rowe
http://mjrainey.googlepages.com/elsilbo
A talented Radio Amateur has developed an analog double-sideband
transmitter powered entirely by the voice energy of a person talking (no
battery or power supply). Coupled with a decent antenna it has made
160km range contacts with a power output of 5mW!
And here I am trying to get around corners at 2.4GHz with 200 MHz
processors, millions of lines of code, an umpteen-million transistor
SoC..... :-)
- David
Antoine van Gelder
On 24 Oct 2009, at 08:30 , elektra wrote:
> http://en.wikipedia.org/wiki/Microwave_oven:
http://imgs.xkcd.com/comics/nachos.png
- a
--
http://7degrees.co.za
"Libré software for human education."
Alexander Chemeris
Hum, echo control in analog phone? Do you mean echo suppression, i.e.
when mic is muted when it's not active?
re: analog phone
You mean that handset provides enough coupling to prevent echo? Are you
sure that's true for low-cost handsets too?
PS Just wonder - do you plan to use DTX or VBR with Speex?
--
Regards,
Alexander Chemeris.
Alberto Escudero
>>
>> 5/ Benefits of licensed versus unlicensed spectrum?
>
> You mean interference? It depends on your specific environment. Has
> anyone done an RF survey at 2.4 GHz to see what else is out there?
> Given the absorption qualities of 2.4 GHz, I would be surprised to
> find strong interferers. Except maybe microwave ovens, which operate
> in this band precisely because it's a good frequency for pumping
> energy into water molecules.
>
The worse situation i have suffered has been the presence of cordless
phones working in 2.4 Ghz in a hotel. You could not use your wireless
network while receiving a call in the hotel room.
-aep
elektra
> Thanks for your comments re the HAL. So is it possible to (i) control
> the number of retransmission attempts (down to zero)
Yes, but...
Retransmissions are controlled by the rate control algorithm in Madwifi. The
decision to retransmit packets is done on MAC layer. If a packet doesn't
match the Frame Check Sequence (32 bit CRC) of the 802.11 packet it will not
be ACKed. So as a consequence we would have to decide to send ACKs for broken
MAC packets. Or - more simple - send them as broadcasts. However this would
mean to implement rate control from scratch again. This is unlikely a
feasible approach.
If the Frame Check Sequence is incorrect we have no clue which part of the
information is broken. I don't think it would make much sense to circumvent
this process.
Rate control is a rather complex field and definitely a must-have. There are
at least three different rate control algorithms for Madwifi, with Ministrel
being the most sophisticated and best algorithm. It is now also the default
in mac802.11 / Linux Wireless. Ministrel does much more than just simply
repeat a package 7 times when it misses acknowledgments. It has intelligent
strategies to deal with packetloss. It will quickly decide to change the
unicast data rate, to avoid excessive retransmissions.
Without retransmissions the packetloss is very high - even if packets are very
small. What is worse: These losses are multiplying on multihop routes in a
mesh network. Take the Batman protocol messages for example. They are very
small - old versions were just 12 byte payload plus IP and MAC overhead. They
are sent as broadcasts at basic / lowest data rate, so 1Mbit in a mixed
802.11bg network.
Older versions of Batman didn't perform any retransmissions of protocol
messages. From my practical experience I have to say that even well working
wireless single hop links (you get several 100 kByte per seconds through it)
are likely to lose at least 10% of the Batman protocol packets - on the first
hop. After 2 equally good hops we were losing 19% packetloss and so on.
> and (ii) pass
> through packets that fail the checksum using Madwifi? It would be
> useful to check out the bit error patterns on packets that fail to be
> received correctly.
I'm not sure whether we really want to ACK packets with broken MAC Frame Check
Sequence. How do we know which part of the message is broken if the checksum
of an 802.11 packet doesn't match?
> > Forward error correction is used in 802.11g and 802.11a.
>
> It would be interesting to test the PER for comparable 11g and 11b modes
> to test the effect of the built in FEC. For example is the PER better
> at 6 Mbit 11g than 5 Mbit 11b?
The receiver sensitivity specs of the DIR-300 indicate that 802.11g has a
significant advantage over 802.11b (rates 1 2 5.5 11Mbit):
RX sensitivity:
• 54 Mbit/s OFDM, 10 % PER, -68 dBm)
• 48 Mbit/s OFDM, 10 % PER, -68 dBm)
• 36 Mbit/s OFDM, 10 % PER, -75 dBm)
• 24 Mbit/s OFDM, 10 % PER, -79 dBm)
• 18 Mbit/s OFDM, 10 % PER, -82 dBm)
• 12 Mbit/s OFDM, 10 % PER, -84 dBm)
• 11 Mbit/s CCK, 8% PER, -82 dBm)
• 9 Mbit/s OFDM, 10 % PER, -87 dBm)
• 6 Mbit/s OFDM, 10 % PER, -88 dBm)
• 5,5 Mbit/s CCK, 8% PER, -85 dBm)
• 2 Mbit/s QPSK, 8% PER, -86 dBm)
• 1 Mbit/s BPSK, 8% PER, -89 dBm)
Elektra
Alexander Chemeris
Is it possible to control maximum latency, induced by retransmission? Kinda
get real-time promise from the driver. Also, it might be useful to get
information
about the current state of the link from low-levels, so application could adjust
its transmission params. E.g. adjust number of frames per packet, codec
bitrate, etc. Though that's the question - how to signal this information back
to stream originator if bad link happens on the Nth hop. This might lead to
over-engineering. Or.. Has anyone looked at DCCP, SCTP and other protocols
smarter then venerable UDP?
--
Regards,
Alexander Chemeris.
David Rowe
> re: analog phone
> You mean that handset provides enough coupling to prevent echo? Are you
> sure that's true for low-cost handsets too?
Acoustic echo does not seem to be a problem with analog phones. I guess
the acoustic coupling of signal from speaker to microphone in the
handset is small or the delay is small enough so that the small amount
of acoustic coupling is seen as part of the line echo on the FXS
port/analog phone 2 wire interface.
> PS Just wonder - do you plan to use DTX or VBR with Speex?
Haven't really thought about it yet.....
Cheers,
David
David Rowe
Thank you for your informative reply:
> If the Frame Check Sequence is incorrect we have no clue which part of the
> information is broken. I don't think it would make much sense to circumvent
> this process.
Well as David B suggested our purpose is to characterise the channel,
seeing a broken packet would tell information about the pattern of bit
errors.
> Rate control is a rather complex field and definitely a must-have. There are
> at least three different rate control algorithms for Madwifi, with Ministrel
> being the most sophisticated and best algorithm. It is now also the default
> in mac802.11 / Linux Wireless. Ministrel does much more than just simply
> repeat a package 7 times when it misses acknowledgments. It has intelligent
> strategies to deal with packetloss. It will quickly decide to change the
> unicast data rate, to avoid excessive retransmissions.
Yes I am sure they are great algorithms for data. Not sure if they are
ideal for media like voice. For example a voice packet with a 1% bit
error rate will be decoded OK, for data a 1% bit error rate is
unacceptable. Delaying data for a few 100ms due to retransmissions is
fine, but a bad idea for voice.
Then again the current Wifi algorithms may be 100% necessary even for
for media given it's an Ethernet-style collision sense channel (I forget
the correct acronym).
At this stage we are just exploring ideas, I would like to measure and
understand how the Wifi algorithms affect voice quality.
> Older versions of Batman didn't perform any retransmissions of protocol
> messages. From my practical experience I have to say that even well working
> wireless single hop links (you get several 100 kByte per seconds through it)
> are likely to lose at least 10% of the Batman protocol packets - on the first
> hop. After 2 equally good hops we were losing 19% packetloss and so on.
OK that is really interesting. So with a large link margin (say 20dB)
10% of packets are still lost requiring re-transmission?
Is this because collisions with other packets on the channel is the
dominant packet loss mechanism?
> > and (ii) pass
> > through packets that fail the checksum using Madwifi? It would be
> > useful to check out the bit error patterns on packets that fail to be
> > received correctly.
>
> I'm not sure whether we really want to ACK packets with broken MAC Frame Check
> Sequence. How do we know which part of the message is broken if the checksum
> of an 802.11 packet doesn't match?
We can send data payload of all 0's and count the 1's at the receiver.
This will tell us something about the error distribution of the Wifi
radio channel. Remember this is just a measurement exercise at this
stage.
Interesting to compare to GSM which has no ACK packets. GSM must be way
more efficient. Perhaps the key here is that GSM uses TDMA in licensed
spectrum so it has a small chance of having it's packets squashed by
another transmitter.
Cheers,
David
Alexander Chemeris
On Tue, Oct 27, 2009 at 00:04, David Rowe <da...@rowetel.com> wrote:
>> re: analog phone
>> You mean that handset provides enough coupling to prevent echo? Are you
>> sure that's true for low-cost handsets too?
>
> Acoustic echo does not seem to be a problem with analog phones. I guess
> the acoustic coupling of signal from speaker to microphone in the
> handset is small or the delay is small enough so that the small amount
> of acoustic coupling is seen as part of the line echo on the FXS
> port/analog phone 2 wire interface.
IIRC, acoustic echo in analog phones only happens on long range calls,
because if echo delay is small enough, our ear does not recognize it
as echo. So I would guess that cheap analog handsets don't have any
echo cancellation in them. In VoIP you easily get delays which outweight
oversees analog line delays, that's why echo is so a problem in VoIP.
Handset design also aims at decoupling speaker from mic, but I wonder
how good is it in cheap an old ones?
I would be happy to be wrong, but that's what I read about this.
>> PS Just wonder - do you plan to use DTX or VBR with Speex?
>
> Haven't really thought about it yet.....
You should usually think about it, because VBR/DTX will decrease
size of a good part of packets.
--
Regards,
Alexander Chemeris.
Steve Song
I sent a link a little while ago to a study for OFCOM, the UK
communications regulator, on WiFi interference. It makes me think that
getting behind a "WiFi Friendly" campaign might not be a bad idea.
-Steve
http://www.ofcom.org.uk/research/technology/research/exempt/wifi/wfiutilisation.pdf
Abstract
---------
A survey of IEEE 802.11b/g 'WiFi' usage has been carried out at various
urban locations in the UK. This has revealed a wide variety of problems
encountered by users, many of which are due to causes other than
spectrum issues. These include problems with the wired Internet and
device configuration errors.
Where the users' problems are spectrum-related they tend to be due to
interference between devices in the 2.4 GHz ISM band rather than
congestion, as was initially believed. Such problems are common but
geographically dispersed. In the centre of London, however, it has been
shown that the demands on this band are much higher than other locations
surveyed and users are experiencing the combined effects of interference
and congestion.
The problems of interference between different types of radio device in
the 2.4 GHz band lead us to conclude that a certification scheme is
highly desirable. Some non-WiFi devices already sport 'WiFi-friendly'
claims on their datasheets. We propose extending this concept to a '2.4
GHz friendly' logo, which would help drive acceptance of innovative
technologies in this band.
elektra
> For example a voice packet with a 1% bit
> error rate will be decoded OK, for data a 1% bit error rate is
> unacceptable.
I think we can not tolerate any errors that are not recoverable in the data
overhead that our voice data is encapsulated in. To the MAC layer of 802.11 it
doesn't matter which part of a received package is damaged.
Compared to the voice payload of a well compressing voice codec the overhead
introduced by MAC, IP and UDP/RTP is huge, given that we operate with small
audio chunks.
The 802.11-specific header adds 34 bytes overhead to a transmission.
Further we have to add IP (20 bytes) and UDP (8 bytes) plus RTP (12 bytes)
overhead. Resulting in a total of 74 bytes overhead if I haven't missed
something.
The payload of a 20ms GSM 06.10 audio sample is 33 byte.
> OK that is really interesting. So with a large link margin (say 20dB)
> 10% of packets are still lost requiring re-transmission?
>
> Is this because collisions with other packets on the channel is the
> dominant packet loss mechanism?
Yes, these errors are caused by collisions and interference from other
stations. Bruno Randolf has verified for 4G Systems that SNR alone is a
terrible metric for a mesh routing protocol. You can have a 30 dB SNR - it
doesn't matter if a station in close proximity transmits while you try to
receive data.
> Interesting to compare to GSM which has no ACK packets. GSM must be way
> more efficient. Perhaps the key here is that GSM uses TDMA in licensed
> spectrum so it has a small chance of having it's packets squashed by
> another transmitter.
CSMA/CA (Carrier Sense Multiple Access with Collision Avoidance) is probably
not as good as TDMA with this regard, but we are building a mesh i.e.: a non-
hierarchical multipoint-to-multipoint network. I wonder how it could be
possible to organize TDMA in a mesh?
Cheers,
Elektra
David Rowe
> I think we can not tolerate any errors that are not recoverable in the data
> overhead that our voice data is encapsulated in. To the MAC layer of 802.11 it
> doesn't matter which part of a received package is damaged.
address or wifi rate data is messed up.
> The 802.11-specific header adds 34 bytes overhead to a transmission.
> Further we have to add IP (20 bytes) and UDP (8 bytes) plus RTP (12 bytes)
> overhead. Resulting in a total of 74 bytes overhead if I haven't missed
> something.
single packets.
> > Is this because collisions with other packets on the channel is the
> > dominant packet loss mechanism?
>
> Yes, these errors are caused by collisions and interference from other
> stations. Bruno Randolf has verified for 4G Systems that SNR alone is a
> terrible metric for a mesh routing protocol. You can have a 30 dB SNR - it
> doesn't matter if a station in close proximity transmits while you try to
> receive data.
approximation we don't need to worry much about the radio power and link
budget.
So the nominal channel always has some jitter due to a constant low
level packet loss and retransmission. As interference and/or more
capacity is used jitter gets worse and eventually packets are lost.
OK so now I can understand some trade offs between sending single and
multiple speech frames in a single Wifi packet.
When sending multiple frames of speech data in a single Wifi packet
there is a big increase in bandwidth efficiency due to protocol
overhead. Does this means the channel is occupied for less time overall
so reducing the overall probability of a collision?
However when a collision does occur we have more data to re-transmit.
Several re-transmits of a packet containing 80ms of speech data could
lead to significant jitter.
I wonder if there is some sort of model or experiment we can develop to
test these trade offs.
The question of re-transmission for voice packets is still open to me.
Might be better to drop them and let the codec work it out. Would
certainly help channel congestion. Also adding FEC to voice packets has
very little BW hit given the huge protocol overhead. FEC bits in
packets n-1 and n+1 could let us rebuild packet n, at the expense of
some delay.
If the main aim is reducing collisions this suggests higher bit rates
are better to shorten channel occupation time (As you have mentioned
before Elektra). Hmmm, although I guess a big chunk of the wifi packet
is transmitted at the baseline bit rate, i.e. the length of a packet in
ms is not directly proportional to the bit rate.
If we know the clear-channel link margin in advance, is it a good idea
to let ministrel reduce the bit rate when packets don't get thru? Would
we be better off fixing the bit rate?
> CSMA/CA (Carrier Sense Multiple Access with Collision Avoidance) is probably
> not as good as TDMA with this regard, but we are building a mesh i.e.: a non-
> hierarchical multipoint-to-multipoint network. I wonder how it could be
> possible to organize TDMA in a mesh?
Here is a scenario:
Say node A is relaying VOIP media packets to nodes B & C. Is it
possible to send them all in one Wifi packet? Node B will always hear
traffic going to node C anyway. Is this a multi-cast/IP-layer question?
Maybe batman does this already, not sure. In this case node A would
just send one packet every 20ms, but the packet would be of variable
length.
A's transmit "timeslot" could be adjusted away from other nodes who also
transmit periodically every 20ms (if they have any traffic).
This simple scheme would get messy if B gets the packet but C doesn't.
TDMA v GSM
----------
I can start to see the advantages of GSM over VOIP-over-Wifi. With a
TDMA system much of the protocol overhead goes away:
- TDMA gets rid of collision issues, for a few ms the channel is all
yours
- RTP, UDP, and even IP is not needed as the data is always delivered in
the right order, your time slot uniquely identifies you
- much of the physical-layer modem synchronisation overhead is not
needed, as its the same base station transmitting to you all the time
- handset power control which would improve cell-cell isolation and
performance of adjacent receivers (nodes)
Thanks,
David
> Cheers,
> Elektra
>
>
David Rowe
A question for the mesh-networkers:
Why is it that bandwidth of a link is inversely proportional to the
number of hops?
I understand why this is so for a simple case like 3 nodes:
A-B-C
If B is transmitting the channel will be busy, but how about:
A-B-C-D-E
If A and E can't "hear" each other does it mean the channel can be
re-used, e.g. can a UDP packet can flow between A-B and the same time as
D and E?
Thanks,
David
David Carman
On Thu, Oct 29, 2009 at 4:00 AM, David Rowe <da...@rowetel.com> wrote:
> Why is it that bandwidth of a link is inversely proportional to the
> number of hops?
Some raw measurements for you from a very quiet OLSR network at 5am:
The route tested (with ETX values in between) is:
(74) <---1.157---> (75) <---1.040---> (4) <---2.298---> (26) <---1.184
---> (161) <---1.366---> (41)
I am working from a machine cabled to 192.168.10.41.
Tracing back from the distant router:
root@scarborough-74:~# traceroute 192.168.10.41
traceroute to 192.168.10.41 (192.168.10.41), 30 hops max beginning
with hop 1, 40 byte packets
1 scarborough-75.olsr (192.168.10.75) 8.022 ms 2.58 ms 1.588 ms
2 192.168.10.4 (192.168.10.4) 16.815 ms 8.801 ms 6.714 ms
3 scarborough-26.olsr (192.168.10.26) 20.945 ms 39.606 ms 8.444 ms
4 scarborough-161.olsr (192.168.10.161) 12.557 ms 13.344 ms 16.273 ms
5 scarborough-41.olsr (192.168.10.41) 44.541 ms 21.604 ms 41.157 ms
Copying the busybox binary from each router:
david@design:~$ scp ro...@192.168.10.74:/bin/busybox ./
busybox
100% 449KB
29.9KB/s 00:15
david@design:~$ scp ro...@192.168.10.75:/bin/busybox ./
busybox
100% 449KB
37.4KB/s 00:12
david@design:~$ scp ro...@192.168.10.4:/bin/busybox ./
busybox
100% 518KB
57.6KB/s 00:09
david@design:~$ scp ro...@192.168.10.26:/bin/busybox ./
busybox
100% 449KB
149.6KB/s 00:03
david@design:~$ scp ro...@192.168.10.161:/bin/busybox ./
busybox
100% 449KB
448.9KB/s 00:01
Pinging each router:
david@design:~$ ping -c 5 192.168.10.74
--- 192.168.10.74 ping statistics ---
5 packets transmitted, 4 received, 20% packet loss, time 4006ms
rtt min/avg/max/mdev = 12.509/17.029/25.928/5.252 ms
david@design:~$ ping -c 5 192.168.10.75
--- 192.168.10.75 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4006ms
rtt min/avg/max/mdev = 10.080/11.273/12.170/0.749 ms
david@design:~$ ping -c 5 192.168.10.4
--- 192.168.10.4 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4003ms
rtt min/avg/max/mdev = 7.091/13.335/16.931/3.343 ms
david@design:~$ ping -c 5 192.168.10.26
--- 192.168.10.26 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4006ms
rtt min/avg/max/mdev = 3.299/3.582/3.887/0.247 ms
david@design:~$ ping -c 5 192.168.10.161
--- 192.168.10.161 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4007ms
rtt min/avg/max/mdev = 2.069/4.190/8.803/2.381 ms
David Carman
> The route tested (with ETX values in between) is:
>
> (74) <---1.157---> (75) <---1.040---> (4) <---2.298---> (26) <---1.184---> (161) <---1.366---> (41)
Note the high ETX between 4 and 26. The link is not especially long.
The loss is symmetrical. The reason for this poor link is that 4 and
26 have at least twice as many neighbours as the other routers.
Which brings me back to (yawn) the choice of antenna.
An acoustic analogy: In a noisy room, or listening into the distance,
we direct our best ear at the source we want to hear, sometimes using
our hand to focus our hearing even more. If we had one ear on the top
of our heads, cocktail parties wouldn't exist. Arrays are now
considered best for public address acoustics too. Why are we not
giving the MP the equivalent capability?
In multipath environments, point sources are less reliable and require
more accurate positioning than arrays. As explained previously, the
distance between nodes and internodes in complex environment can be a
few centimetres, and can mean the difference between great signal and
no signal at all. Double point source antennae with diversity can
reduce the chance of a MP finding itself in a 2.4GHz black hole, but
an array will do better.
There isn't much cost difference between two points and an array being
etched onto the PCB, and we shall be using suboptimal materials to do
so. Even without a ground plane, an array will provide more reliable
reception and better range than two point sources.
Lastly, arrays are simply beautiful -
http://en.wikipedia.org/wiki/File:Antenna_flat_panel.png
David
David Rowe
The bandwidth figures are roughly
Hops BW (kB/s)
1 449/1
2 449/3
3 449/8
4 449/12
5 449/16
I guess this is TCP/IP with a MTU (packet size) set to the maximum.
Cheers,
David
David Carman
> I guess this is TCP/IP with a MTU (packet size) set to the maximum.
Yup - MTU=1500
d
David Rowe
> An acoustic analogy: In a noisy room, or listening into the distance,
> we direct our best ear at the source we want to hear, sometimes using
> our hand to focus our hearing even more. If we had one ear on the top
> of our heads, cocktail parties wouldn't exist. Arrays are now
> considered best for public address acoustics too. Why are we not
> giving the MP the equivalent capability?
Well using a phased array of smaller elements is a method for building
an antenna of a certain radiation pattern. An array is not a capability
in itself. Is there a certain radiation (reception) pattern you had in
mind? For example some directional pattern compared to am omni?
Multiple lobes maybe?
To use a direct analogy with your movable head scenario steerable arrays
are also possible, but probably out of scope for the MP antenna design.
> In multipath environments, point sources are less reliable and require
> more accurate positioning than arrays. As explained previously, the
> distance between nodes and internodes in complex environment can be a
> few centimetres, and can mean the difference between great signal and
> no signal at all.
Yes I have measured this. The other day I connected an antenna to a
spectrum analyser then started a MP transmitting in the same room. Just
waving my arm around in the same room made the signal jump up and down
by 10dB.
A roof mounted MP in a multi-path environment would have similar large
signal variations with small changes in it's position, or the
environment.
I am not sure what specification this translates to in antenna design.
Perhaps it is the reason for small diversity antennas we find on
routers, or the twin antenna of a WRT54G?
Not sure an array of patch elements on a PCB will do any better than a
simple dipole in this respect. The patches may synthesise a point
source and have their own problems in multipath. The microwave patch
antennas I have seen are designed for lots of gain in one direction,
i.e. functionally equivalent to a dish.
But lets check it out. Perhaps the antenna problem is less about gain
and directivity and more about performance in large signal multipath
environments.
Cheers,
David
Vickram Crishna
Arrays are now considered best for public address acoustics too.
Vickram
http://communicall.wordpress.com
David Carman
> Well using a phased array of smaller elements is a method for building
> an antenna of a certain radiation pattern. An array is not a capability
> in itself.
Yes, an array confines the signal in two dimensions, while a dipole
confines in one. So an array can transmit further and receive more
clearly (due to both directed gain and reduction of side interference)
in two directions: front and back.
But an array also has a wider area exposed to induction. Because of
the small distance between great- and no-signal in 12cm wavelength
diffraction, an array can ensure that the antenna is not sitting
inside a 4cm wide internode.
I expect that this is why a diversity switch was considered necessary
for WRT and other routers. However a diversity switch chooses between
two points, both of which could happen to be in internodes. An array
would cover a 10cm x 10cm area and would be less prone to poor signal
in this way.
The Nano panel is the best antenna I have ever worked with. One distal
link on the mesh is about 1000m away from a 14dBi panel facing 60
degrees off midline. The fresnel zone is obstructed by a rocky ridge
at 500m with a house and a metal water tower right in between. I
struggled with a 13dBi Yagi, eventually putting a standard WRT on the
water tower, resulting in a great link with an extra hop. Recently I
replaced the WRT/Yagi with a Nano and took down the water tower link.
The ETX is now 1.057.
A panel array will cost marginally more than dipoles, because it
requires a stick-on layer as well as the etch. But I do believe that
it will result in far better reception.
As previously posted
http://wiki.xprize.frednet.org/index.php/Picorover_Terrain_Scanning_Subsystem#Previous_design_.40_2.4_GHz
taken from http://www.proyectoradio.com/html/Engineering/Articles/SpAntenna.html
looks ideal for the MP. The design does not require a ground plane to
achieve higher directional gain and reception coverage area. The
radiation pattern accomodates point-to-point-to-point links. If the
outside of the casing is exactly 8mm away from the patch array,
whatever the MP is strapped to can provide a ground plane, further
enhancing directional gain. Positioning the array can be as simple as
facing the MP in the direction of a known neighbour.
> To use a direct analogy with your movable head scenario steerable arrays
> are also possible, but probably out of scope for the MP antenna design.
Stepper motors aside, adaptive antenna technology can shift the phase
of each patch in the array to do exactly that. I expect that such
circuitry will become de facto, as the battle of the airwaves
continues.
> But lets check it out. Perhaps the antenna problem is less about gain
> and directivity and more about performance in large signal multipath
> environments.
Thanks for considering a patch array for the MP. Point sources with
diversity are so September the 10th :^)
David
Sjur Eivind Usken
ping time when you have full 5 bar 3G connection.... )
3G has its own problems with packet data... too large buffers...
http://blogs.broughturner.com/2009/10/is-att-wireless-data-congestion-selfinflicted.html
regards
sjur
David Rowe
question. Say we have a bunch of nodes in a Village Telco mesh leading
to a gateway GW:
A-B-C-D-E-GW
Lets say there are two streams of traffic on the mesh (i) a single call
from A to GW via all the intermediate hops and (ii) several calls
between E-GW, coming from other nodes connected to E, not illustrated
above.
Node E being close to the GW will be likely to carry traffic for many
calls. I am assuming most calls are terminated off the local mesh, so
am not concerned about calls between say C and E.
So here are the actual questions:
1/ What is the bandwidth we see of the A-GW link?
2/ What is the bandwidth we see on the E-GW link?
3/ Does the presence of traffic on A-GW significantly reduce the
bandwidth on E-GW?
This scenario is important as we need more bandwidth on links closer to
the GW.
Thanks,
David
David Carman
> This scenario is important as we need more bandwidth on links closer to
> the GW.
WiFi bandwidth is huge compared to (South African) Internet bandwidth.
The moment 16kbps audio becomes a problem, it will be time for someone
to establish a second GW. If the network is also used for data, then
QoS the IAX port.
As a general rule, those closest to the trough get to eat first. When
a mesh congests, those furthest from the GW complain first.
David Rowe
> WiFi bandwidth is huge compared to (South African) Internet bandwidth.
> The moment 16kbps audio becomes a problem, it will be time for someone
> to establish a second GW. If the network is also used for data, then
> QoS the IAX port.
starting with a bunch of papers that a gentleman called Saritha posted
on my blog:
http://www.rowetel.com/blog/?p=70#comment-49682
Effective Wifi bandwidth for VOIP is surprisingly small, mainly due to
the big gaps between packets imposed by the 802.11 MAC protocol. The
throughput of 200 byte UDP packets appears to be around 1 Mbit/s. The
baseline VOIP over Wifi performance reported in one paper is 8 g729
calls over a single hop, and just 1 call over 5 hops.
Elektra wisely suggested that we aggregate VOIP packets at the source
(e.g. place 4x 20ms GSM speech codec frames in a single Wifi packet)
which lowers the all-important packet rate. She has demonstrated 15-45
calls over two hops on our test bed:
http://wiki.villagetelco.org/index.php/How_to_use_the_software_build_environment#Configuring_Load_Test
(Results section). Although in this test all of the calls came from one
node so the traffic was highly structured compared to random access by
multiple nodes on the same channel.
So I am concerned about:
(i) what happens when their is activity over many hops? Does the whole
mesh slow down? Or do we still have good bandwidth on some links?
(ii) what happens when we add 1 call to a system at capacity. Does it
fall over entirely (all calls get trashed) or degrade gracefully?
Cheers,
David R
Lew Pitcher
wiser and more experienced heads.
On Thu, October 29, 2009 6:22 pm, David Rowe wrote:
[snip]
> So I am concerned about:
>
> (i) what happens when their is activity over many hops? Does the whole
> mesh slow down? Or do we still have good bandwidth on some links?
ISTM that the answer depends a lot on the network path the packets take.
If the network is linear (that is to say, routing works by "merging"
packets into a single "best route"), then there will be a core of
overworked nodes where bandwidth is at a premium, and the edges will have
better bandwidth. In a simple
A---B---C---D---E
network, C participates in most of the conversations, while A & E
participate in only a few conversations. Or, conversations between nodes
will occupy more of the bandwidth between B&C, and C&D than they will
between A and B or D and E.
If the network is interconnected (that is to say, packets may follow a
variety of routes between endpoints), then there is less congestion in the
centre, and those central nodes would have less use of their bandwidth.
This would look something like
B --C
/ \
A---D---E
Now, because from any one node, there are multiple paths available,
packets can route around congestion, and bandwidth is preserved.
To me, all this makes questions on bandwidth use very much dependent on
network topology and how the routing protocol works.
We can't do much about network topology here; while we can make
suggestions, we have no real control over a VT operator's decisions wrt
the physical and logical placement of nodes. However, network routing /is/
under our control, in a way. This may be a question that only BATMAN can
answer ;-)
> (ii) what happens when we add 1 call to a system at capacity. Does it
> fall over entirely (all calls get trashed) or degrade gracefully?
Assuming that our VoIP packets are carried in UDP, then I /think/ that
we'll see a "graceful" degradation of calls that traverse the bottleneck
routes. UDP packets do not have "guaranteed delivery", and may be silently
dropped (discarded) at any point on the route. If a midstream node can't
handle the traffic it receives, it will drop packets. It is up to chance
(that is to say, the instantaneous measure of packet volume) as to /which/
packets get dropped, and thus /which/ conversations suffer degradation.
In one sense, the smaller the packet, the better the chances of surviving
through a congested network, because the decision to drop or not /is/
instantaneous, and the conditions may not be there when the next small
packet arrives. Further, small (single voice frame) packets dropped won't
degrade the resulting audio output as much as would happen with larger
(multiple frame) packets.
Just my 2 cents.
--
Lew Pitcher
Lew Pitcher
On Thu, October 29, 2009 7:30 pm, Lew Pitcher wrote:
[snip]
> To me, all this makes questions on bandwidth use very much dependent on
> network topology and how the routing protocol works.
>
> We can't do much about network topology here; while we can make
> suggestions, we have no real control over a VT operator's decisions wrt
> the physical and logical placement of nodes. However, network routing /is/
> under our control, in a way. This may be a question that only BATMAN can
> answer ;-)
It occurs to me that I need to explain this last remark.
If our network (at the wifi level) looks like
B---C
/ \
A---D---E
but looks like
B---C
/ \
A D---E
to the routing software, then the routing software has effectively
flattened our network out into a linear "best route" highway. Where the
intermediate MP's might have been able to take advantage of uncongested
connections to route packets, it now has to route packets /through/ the
congestion.
So, to me, it comes down to a question about BATMAN: does BATMAN choose a
single "next-hop" destination for packet forwarding, or can it maintain a
multiplicity of alternate "next-hop" packet destinations?
If BATMAN can handle multiple "next-hop" choices for any single
destination, how does it choose between which one to use? Can it use
congestion/bandwidth information to "prefer" one "next-hop" over another
for a given destination?
--
Lew Pitcher
Lew Pitcher
attempt to answer my own questions ... sort of.
On Thu, October 29, 2009 7:52 pm, Lew Pitcher wrote:
>
>
[snip]
> So, to me, it comes down to a question about BATMAN: does BATMAN choose a
> single "next-hop" destination for packet forwarding, or can it maintain a
> multiplicity of alternate "next-hop" packet destinations?
>
> If BATMAN can handle multiple "next-hop" choices for any single
> destination, how does it choose between which one to use? Can it use
> congestion/bandwidth information to "prefer" one "next-hop" over another
> for a given destination?
OK, I've read through the BATMAN RFC, and think I have a partial answer.
BATMAN does not explicitly record multiple optional "next-hop" choices for
any destination. Instead, it only records /one/.
But that one "next-hop" choice is dictated by the neighbour node which
transfered the destination node's OGM to the MP. Thus, the "next-hop" is
dynamic, and can change according to external conditions.
One of those conditions that would affect the "next-hop" is network
congestion. If we have two physical routes between this MP and the target
MP, the route that got us the OGM first wins. Congestion within a route
would, by necessity, delay an OGM sent through that route, and thus the
"winning" OGM is the one with the least obstacles, congestion included.
The more frequently the target MP broadcasts it's OGM, the faster the
intermediate MP can react to congestion.
But, as each OGM is, in itself, network traffic, the more frequently the
target MP broadcasts it's OGM, the worse congestion along common routes
will be.
My guess is that there is a balance point between too-frequent OGM
broadcasts and too-infrequent broadcasts that can only be established
empirically.
--
Lew Pitcher
David Carman
> (i) what happens when their is activity over many hops? Does the whole
> mesh slow down? Or do we still have good bandwidth on some links?
> (ii) what happens when we add 1 call to a system at capacity. Does it
> fall over entirely (all calls get trashed) or degrade gracefully?
With data congestion, the links with the most hops deteriorate far
more than links with few hops.
Perhaps we need to look at that bugbear of VoIP, Skype. While we are
developing a VoIP system out of a mesh network, it appears that Skype
are developing a mesh network out of their VoIP system. I suspect that
Skype packet size and transmission rate may offer clues as to how to
proceed.
I dabbled with mesh VoIP for about a year, installing IAX and SIP
softphones on laptops and offering a single analogue phone line for
calls. Briefly, the findings were:
1. Call quality depended more on the quality of the laptop audio
chipset than anything else.
2. IAX (iaxcomm) was more reliable than SIP (ekiga) - much less jitter
close to capacity.
3. When the network was quiet, VoIP quality was good everywhere.
3. At capacity, the links with many hops would not work at all, while
links with few hops were unaffected.
The Red Hill valley down the road has an appalling analogue wireless
POTS system, half-maintained by Telkom. The farming community there
took a lot of the initiative in establishing the mesh in their area.
Before GPRS got to them, it was their only Internet connection.
I plan to put one of the 2 MPs at a pottery there and the other at the
potter's house in Scarborough (+/-6 hops). The connection should be
used a lot and I'll get a lot of user feedback (the potter is my Red
Hill mine canary).
If Atcom can supply 2 of those yummy AT-620P SIP/IAX phones, I can
install them on the existing FF/OLSR routers and set up ssh routes
into the WRTs for you. You can then experiment with IAX UDP vs SIP,
different packet sizes, etc. Otherwise I can do the same once the MPs
arrive.
Red Hill also has an underserviced shack settlement that could do with
telephony. David Savage provided 5.8GHz gear when he wanted to link
through us to Hout Bay, so we have a live 5.8GHz link from Red Hill
across False Bay to Tygerberg. David works closely with Alan Levine,
and I envisage using Dabba as upstream VoIP provider for Red Hill in
the future.
David
David Rowe
Thank you for your comments.
Lew:
Yes I think there is some possibility for batman to route around
congestion. However what I am looking at right now is the fundamental
ability of the 802.11 MAC architecture to handle VOIP packets over a
mesh. For my thought experiments I am assuming the routes are static
for the duration of the call.
David C:
The mission Steve has set for us is broadly to deliver a Skype-like
experience for the Village Telco. Before Skype there were plenty of
soft phone clients but Skype "just works" in many cases.
On a practical note looking at the statistics of Skype packets might be
a useful exercise.
Your practical experiences with VOIP over your mesh are very
interesting, and the test sites sound great. Running some other
characterisation tests might also be useful, for example measuring UDP
packet delay and jitter over your network for varying number of
simulated calls (UDP packet streams).
I think for a phone system that people pay money for we need something
close to a GSM experience from a non-mobile handset, i.e. calls get thru
most of the time and don't drop out, but occasionally you get a few
clicks and pops. So I am trying to match this use case with the channel
we have on a mesh network.
Cheers,
David R
elektra
> Yes I think there is some possibility for batman to route around
> congestion.
Congested links will show a detoriating metric. So Batman will look for
alternatives.
Batman does select a single next-hop node to forward transmissions for each
destination in the network. It will use symmetric (same path to and from
destination) or asymmetric (different path to and from destination) routes,
depending on network topology and link metrics.
Indeed, the 802.11 MAC layer is ineffective when transporting a small payload.
Sending 20 bytes = 1 voice sample of 20ms per transfer is ludicrously
ineffective and thus a perfect argument for a thesis ;-)
We should be able to get a voice chunk through our multi-hop routes in less
time than our voice sample is long. I think we should be several times faster
than that.
The number of calls detoriates with the number of hops because the number of
transmissions (airtime consumption per call) increases. Also more
transmissions lead to more collisions in the media and thus retransmissions.
Call detoriation will affect those users that need many hops first.
My estimation would be that with 2 voice samples per transmission we get twice
the call capacity. With 4 voice samples per transmission we get four times the
call capacity.
Naturally there is a limit somewhere and we can't increase the length of audio
chunks endlessly. We will probably have to find the best settings empirically.
It could be helpful to use UDP/RTP header compression and a well compressing
voice codec to save bandwidth, but the tradeoff will be CPU load. Also smaller
packets have a higher chance to get through and need less retransmissions.
However I think the payload should not be smaller than the overhead. A total
packet size of 300-500 bytes is probably fine.
Next optimization step would be to implement packet aggregation, to forward
VOIP packets from multiple calls within one data packet. But that is probably
a non-trivial task.
Elektra
David Rowe
> My estimation would be that with 2 voice samples per transmission we get twice
> the call capacity. With 4 voice samples per transmission we get four times the
> call capacity.
packet rate over the channel. The packet rate is the critical driver
for 802.11 MAC bandwidth.
Every time a packet is sent the MAC inserts big gaps to deal with
contention and acknowledgement. This is "dead air" for the channel - no
one is transmitting most of the time.
For example a 60 byte g729 packet (including 20 byte payload data, IP,
UDP, RTP overhead) takes 44uS to transmit at 11 Mbit/s. However when
the various MAC overheads and contention timers and delays and ACK
processes are added the total elapsed time before we can send another
packet is 800us. Most of this time no one is transmitting.
The effective payload data rate is 20 bytes/800uS * 8 bits/byte = 200
kbit/s on an "11 Mbit/s" channel.
Increasing the channel bit rate, compressing UDP/IP/RTP overheads etc
help but not as much as you would think. Doubling the bit rate doesn't
double the throughput, as it doesn't affect any of the dead time on the
channel, just the small amount of time spent actually transmitting.
So the best way to increase VOIP capacity is reduce the number of
packets on the channel, for example via packet aggregation at the source
or between hops.
Or maybe hack the 802.11 MAC protocol.
Cheers,
David
Alexander Chemeris
Here are my 2 VoIP cents.
On Fri, Oct 30, 2009 at 09:52, David Carman <tid...@gmail.com> wrote:
> I dabbled with mesh VoIP for about a year, installing IAX and SIP
> softphones on laptops and offering a single analogue phone line for
> calls. Briefly, the findings were:
> 1. Call quality depended more on the quality of the laptop audio
> chipset than anything else.
Heh, that's the case, yes. Audio quality depends a lot on chipset
quality, but the most - on chipset's drivers quality.
> 2. IAX (iaxcomm) was more reliable than SIP (ekiga) - much less jitter
> close to capacity.
From my VoIP developer experience, Ekiga is sadly not very well
written softphone. So it's not a good measure of generic SIP/RTP
quality. Still, as I spoke earlier, I like how IAX performs and I think
the must to be tested along with SIP/RTP. It worth it justs for
bandwidth savings.
--
Regards,
Alexander Chemeris.
Alexander Chemeris
On Fri, Oct 30, 2009 at 16:56, elektra <onel...@gmx.net> wrote:
> Next optimization step would be to implement packet aggregation, to forward
> VOIP packets from multiple calls within one data packet. But that is probably
> a non-trivial task.
That's exactly what trunking does (read, what IAX was invented for).
Basically it works as following:
1) Start collecting all voice frames
2) If we're collecting more then N ms, send all collected frames as an
aggregated packet.
3) If we've collected more then M bytes, send aggregated packet even
if N ms haven't passed.
4) Goto (1)
The problem here would be that we should know which UDP packets
flowing by are RTPs and which are not.
But thinking this thought I came to an idea (may be inventing a wheel?).
What if we apply trunking on every direct link for all UDP traffic? I.e.
collect *all* UDP packets smaller then M bytes, pack them to a bigger
meta-UDP packet, send over the air and then unpack on the other
side. I believe this can be done transparently under Linux with some
hacking. ;) Yes, this will introduce more delay and jitter, but if David
Rowe is correct, it should drastically improve overall system behavior.
That'll be something like TCP Nagle algorithm for UDP, but with deadlines.
What would you say?
--
Regards,
Alexander Chemeris.
David A. Burgess
So going back to a previous question about what makes cellular
performance so different from wifi, this is a great example. Your
capacity figures are completely dominated by per-packet overhead.
After seeing this it is clear why cellular networks use synchronous
channels. GSM uses guard periods between bursts to absorb timing
errors at the handset, but these account for only about 5% of the
time on the channel -- even though each burst only carries about 7
bytes of payload data. And, thanks to interleaving, the loss of a
single burst on the radio channel does not prevent the recovery of
the vocoder frame in the LEC decoder.
So for wifi you make the packets big to get efficient use of the
channel, but these bigger packets are much more susceptible to bit
errors in the MAC's all-or-none parity checker. This will limit your
range. One technique suggested to me for dodgy wired networks is to
use IAX trunking to combine traffic streams and then send two copies
of every RTP packet. I've never tried it, but it is consistent with
a lot of the other suggestions here, it is simple to implement, and
it has low computational complexity.
-- David
On Oct 30, 2009, at 1:36 PM, David Rowe wrote:
>
>> My estimation would be that with 2 voice samples per transmission
>> we get twice
>> the call capacity. With 4 voice samples per transmission we get
>> four times the
>> call capacity.
>
> I agree with this model. The reason it works is that it lowers the
> packet rate over the channel. The packet rate is the critical driver
> for 802.11 MAC bandwidth.
>
> Every time a packet is sent the MAC inserts big gaps to deal with
> contention and acknowledgement. This is "dead air" for the channel
> - no
> one is transmitting most of the time.
David A. Burgess
Kestrel Signal Processing, Inc.
Alexander Chemeris
On Sat, Oct 31, 2009 at 03:31, David A. Burgess <dbur...@jcis.net> wrote:
> So going back to a previous question about what makes cellular
> performance so different from wifi, this is a great example. Your
> capacity figures are completely dominated by per-packet overhead.
> After seeing this it is clear why cellular networks use synchronous
> channels. GSM uses guard periods between bursts to absorb timing
> errors at the handset, but these account for only about 5% of the
> time on the channel -- even though each burst only carries about 7
> bytes of payload data. And, thanks to interleaving, the loss of a
> single burst on the radio channel does not prevent the recovery of
> the vocoder frame in the LEC decoder.
You said that OpenBTS users will likely use WiFi for backhaul, so
it seems that this discussion makes a lot of sense for OpenBTS
too. We may end up in a situatioin when GSM part works smoothly,
but VoIP part over WiFi just hangs.
> So for wifi you make the packets big to get efficient use of the
> channel, but these bigger packets are much more susceptible to bit
> errors in the MAC's all-or-none parity checker. This will limit your
> range. One technique suggested to me for dodgy wired networks is to
> use IAX trunking to combine traffic streams and then send two copies
> of every RTP packet. I've never tried it, but it is consistent with
> a lot of the other suggestions here, it is simple to implement, and
> it has low computational complexity.
I second this.
Exactly what I'm advocating for the first tries.
--
Regards,
Alexander Chemeris.
David Rowe
> So going back to a previous question about what makes cellular
> performance so different from wifi, this is a great example. Your
> capacity figures are completely dominated by per-packet overhead.
like the MAC overhead is the dominant factor - more significant than bit
rate, modulation schemes, speech compression, link budget, and antennas
when it comes to setting VOIP capacity of the mesh.
> So for wifi you make the packets big to get efficient use of the
> channel, but these bigger packets are much more susceptible to bit
> errors in the MAC's all-or-none parity checker.
investigating. My current Use Case assumes MPs on poles a few 100m
apart with good link budgets, so I had assumed PER would be low, even
for large frames. The dominant "interference" is traffic from our own
mesh, i.e. collisions and the MAC overhead issue.
Alex & David B:
Re the IAX trunking AFAIK this is a form of source aggregation. We will
have each MP running Asterisk, which will then send RTP or IAX packets
to a gateway. With only one voice stream from the MP we can only
aggregate multiple packets from a single source. IAX trunking gets a
lot more efficient when you have say 10 channels moving between one
Asterisk box and another, like two branch offices.
We can actually do the same thing with RTP in asterisk, e.g. in sip.conf
[sip trunk]
allow=gsm:80
will create RTP packets with 80ms of GSM codec data.
What would be really nice would be hop-hop aggregation. For this I
think we need something clever in the packet routing code in the kernel.
BTW one other factor that sets Wifi and GSM apart - great big tall
masts! This must help the link budget hugely by reducing the number of
reflections. I don't know about you guys but I find myself noticing GSM
towers and spotting good wifi antenna locations everywhere I go these
days ;-)
Cheers,
David
elektra
You guys are breaking in open doors ;-)
After running the VOIP performance tests last year between three Nanostations
I drew a few conclusions to achieve more calls in the Villagetelco
environment. I'll add other points that are more clear to me now and sum them
up.
1) Send multiple audio samples from a single source in one WiFi packet. Apart
from the fact that the limiting factor is the number of packets on the WiFi
channel there is no point sending 20ms over a multihop route that will add a
average delay of say 40ms after 6 hops. The experiments have shown that this
works as expected. However using packets larger than 5 samples with GSM
resulted in a enormous CPU load. May be a bug? So it would be great to learn
why this happened and probably fix it. Probably not everyone will agree with me
that it may be sensible to go higher than 5 samples, but I think we should
nevertheless find it out to have the option to test it. It could be an easy way
to keep up with high demand at the cost of latency. Maybe we can even change
the number of samples dynamically to keep up with the demand during rush
hours.
2) Larger packets have higher error rates, so get the best effect from the
smallest amount of data (header compression, good codec). This also allows to
put more voice calls in a single packet if we manage to do trunking. However
given the relatively high overhead saving a few bytes at the expense of heavy
CPU load is not a sensible way to go.
3) Sending multiple calls within one WiFi packet would be a great optimization
(trunking). Everyone commenting on this list is advocating for it, so I see
consensus here. But...
I wonder how this can be implemented. I'm not at all an Asterisk/IAX2 expert.
I see several problems and would be glad to learn that I'm wrong. We have
Asterisk running on all Mesh-Potatoes. They have to deal with one call
maximum, if someone picks up the phone connected to their FXS port. So MP A
will call MP Z via a route A - B - C - D - E - Z. There is a single call on
this path - hence no trunking.
Now someone starts an additional call from B to E, using partly the same path.
Can we use trunking?
Now lets assume Z is acting as Internet uplink and most calls go there.
Multiple calls from A, F, G, H, J, K, L follow the path B - C - D - E - Z. Can
we use trunking in this situation?
I did a quick search on this. Here is a quote from http://www.voip-
info.org/wiki/view/Asterisk+bandwidth+iax2:
"Of course, this mode (IAX trunking) can only be used if all the calls are
between two specific Asterisk servers, but this is frequently the case with
toll-avoidance situations or between two branch offices where there is an
Asterisk server at each location."
4) Fix the problems of 802.11bg with RTS/CTS in multihop ad-hoc environment,
so we can make use of RTS/CTS (Get rid of RTS broadcast storms)
5) Make best use of the Madwifi/Atheros/Openwrt capabilities to prioritize VOIP
over data traffic using TC so the Villagetelco environment works well as
Internet and data network without degrading call quality.
Help is on the way regarding 4) and 5). I'll work on it in cooperation with
Felix Fietkau who is maintaining the QOS scripts and the Madwifi driver in
Openwrt.
I'll very much appreciate any comments and suggestions on 1), 2) and 3)
I'm going to be offline over the weekend.
Looking forward to read your comments.
Cheers,
Elektra
David Rowe
Re (1) - source aggregation yes. We should investigate the fall over
point when aggregating 5 20ms frames of speech data, could be a buffer
size issue somewhere in Asterisk. Scaling aggregation with channel load
is a neat idea.
Re (3) yes I agree IAX just does source aggregation. Nice, but doesn't
help us with hop-hop aggregation, we need some cleverer magic for that.
Re (4) the papers I have been reading suggest a 20% bandwidth hit with
RTS/CTS even in infrastructure mode. Can a working version of RTS/CTS
in ad-hoc mode really help VOIP channel capacity?
Qos (5) is nice as an additional feature but doesn't address mesh
capacity for voice calls. To be honest I am wary of allowing data onto
a Village Telco mesh at this stage.
Cheers,
David
David Rowe
I have been testing the maximum packet rate of Wifi links for short
VOIP-type packets over single hops. I used ns2 (a network simulator)
and some real world Wifi links.
I get around 1000 packets/s on ns2 (using the 802.11b model at 11 Mbit),
and between 900 and 1100 packets/s over real 11M & 54M Wifi links.
1000 packets/s is also close to the numbers I have found in the
VOIP-over-mesh papers I have been reading.
To measure a real single-hop link I used ping floods from my laptop to
my AP, e.g.
[root]# ping linksys -f -q -c 5000
I ran iptraf on the wireless interface to measure packets/s.
I varied the ping packet size using the -s option to approximate
different packet sizes (e.g. simulating payloads of 1-4 codec frames),
although the packet rate doesn't change much for small packets.
So here is a model for VOIP capacity over mesh Wifi:
1/ We have a budget of 1000 Wifi packets/s across the mesh.
2/ We aggregate (pack) 4 codec packets into one Wifi packet. So with
1000 Wifi packets/s we can send 4000 codec packets/s.
3/ For a call we need 100 codec packets/s (50 codec packet/s in each
direction for a full duplex call). So for single hop calls we have a
total of 4000/100 = 40 simultaneous phone calls.
4/ A 2 hop link consumes two Wifi packets, a 3 hop link three Wifi
packets etc. This models the way bandwidth decreases with the number of
hops.
5/ Consider 20 single-hop calls and 10 2-hop calls all running at the
same time. This would consume 20(100 codec packet/s) + 10(2 packets for
two hops)(100 codec packets/s) = 4000 codec packets/s.
Based on this example we can estimate the number of VOIP calls the mesh
can support is 40/(average number of hops). For example with an average
of 4 hops 10 simultaneous calls can be supported.
Next steps:
(i) Test a simple 3-node mesh network to see if we can repeat these
results.
(ii) Replace ping with a tool that sends UDP or RTP packets at a
constant rate but doesn't wait for replies like ping does.
(iii) Look at the patterns of packet loss when the mesh is overloaded.
Based on (4) adding a call from a multi-hop node could overload the mesh
quickly.
Note that 4000 GSM speech codec packets/s is only (4000 packets/s)(33
bytes/packet)(8 bits/byte) = 1.06 Mbit/s. Rather low considering the
raw bit rate of 11Mbit/s.
I would appreciate comments on the model, and even repeating the ping
tests on nodes on real world mesh networks (Elektra and David C). My
real world test results are still tentative as they may be affected by
interference and possibly the CPU load of the router.
Cheers,
David R
Donald Gordon
The Mikrotik routers have an option to aggregate multiple packets together (at the IP or ethernet layer, not sure which). I guess you'd have to do it in the kernel to get decent performance, but something like that might be good. For one thing, you're going to want to be able to do the aggregation at intermediate nodes in the network, as most mesh potatoes are only going to be active on a single call at a time, and the packet isn't going to see another asterisk until it hits the gateway.
It would probably be reasonable to only group packets going to the same IP together; this will save on packets-per-second heading towards the gateway, and would mean that the aggregation wouldn't have to be redone at each intermediate router, which could help with keeping the CPU load down.
You could prototype this by putting in some NAT rules (or a tun/tap and a port-specific route) to redirect IAX packets passing through to localhost, aggregate them and send them to the gateway on a different port to a daemon to deaggregate them and pass them to the local asterisk.
donald
elektra
> Re (1) - source aggregation yes. We should investigate the fall over
> point when aggregating 5 20ms frames of speech data, could be a buffer
> size issue somewhere in Asterisk. Scaling aggregation with channel load
> is a neat idea.
>
I wonder whether we can change these settings on-the-fly?
> Re (3) yes I agree IAX just does source aggregation. Nice, but doesn't
> help us with hop-hop aggregation, we need some cleverer magic for that.
>
Whatever we do here - it must be implemented in kernel space, without
interaction from user space, because copying data from kernel space to
user space and vice versa is a massive performance bottleneck.
> Re (4) the papers I have been reading suggest a 20% bandwidth hit with
> RTS/CTS even in infrastructure mode. Can a working version of RTS/CTS
> in ad-hoc mode really help VOIP channel capacity?
>
RTS/CTS is all about mitigating the hidden nodes / exposed nodes
problem. RTS/CTS consumes airtime, without offering any benefits if
there are no/few hidden/exposed nodes. So in infrastructure mode / a
typical hotspot situation it doesn't make sense since most stations are
in the same area and can effectively work with CSMA. The idea is to get
RTS/CTS work right for multihop communcation and check whether it is
helpful. It is a minor modification in the Madwifi driver.
> Qos (5) is nice as an additional feature but doesn't address mesh
> capacity for voice calls. To be honest I am wary of allowing data onto
> a Village Telco mesh at this stage.
>
Agreed. However, even if the VT mesh will only serve VOIP services we'll
have to deal with other network traffic, too.
Cheers,
Elektra
Alexander Chemeris
On Mon, Nov 2, 2009 at 16:24, elektra <onel...@gmx.net> wrote:
>> Re (3) yes I agree IAX just does source aggregation. Nice, but doesn't
>> help us with hop-hop aggregation, we need some cleverer magic for that.
>>
>
> Whatever we do here - it must be implemented in kernel space, without
> interaction from user space, because copying data from kernel space to
> user space and vice versa is a massive performance bottleneck.
Good point. Kernel-to-userspace copying is a big overhead indeed.
And this is in line with my thoughts about IP-level aggregation right in
the kernel. I wonder, is there anyone with experience of using Click Modular
Router [1]? It seems it may be a good way to implement such aggregation.
Yet, I've read their papers briefly and have zero real usage experience
with it.
1. http://read.cs.ucla.edu/click/
--
Regards,
Alexander Chemeris.
Alexander Chemeris
I've repeated your tests in my environment with slightly different results.
So, I used following setup:
IBM T60 laptop (with Intel PRO/Wireless 3945ABG) = WiFi 802.11g =>
ASUS WL-330gE (in bridge mode) = 1Gbps LAN =>
Core2Duo 1.8GHz
To follow your test setup I used flood ping plus iptraf to test packet rate.
But I did small modification - I ran several pings simultaneously, because
I found that I have no packet loss and thus concluded I haven't reached
possible WiFi connection limit. Interesting thing is that even with 5 (!)
flood pings running simultaneously I still have 0% loss on *all* of them
so it seems that something prevents ping to be a real flood. Probably
driver blocks socket or such. So it seems ping is not a good tool for
that kind of testing and we need something better.
So, here is what I've gotten (all results are averaged with just my eyes
over 10-20secs of run)
1) "ping -f -q -s 56"
1 ping stream - 1150pkt/s - 109KB/s
2 ping streams - 1700-1800pkt/s - 165-170KB/s
3 ping streams - 1810-1870pkt/s - 209-215KB/s
4 ping streams - 1890-1910pkt/s - 230KB/s
5 ping streams - 1900-1920pkt/s - 245KB/s
You can see that 5 pings almost doubled packet rate and more then
doubled bitrate.
Following tests intended to test dependency on packet size and
involve only 1-2 ping streams:
2) "ping -f -q -s 112"
1 ping stream - 1082 pkt/s
2 ping streams - 1100-1500 pkt/s
3) "ping -f -q -s 224"
1 ping stream - 960-970 pkt/s
2 ping streams - 1000-1400 pkt/s
So yes, with 4x packet size increase packet rate decrease only
by few percents.
Regards,
Alexander Chemeris.
Alexander Chemeris
outgoing rates were the same. So you should double the rates to get
real bandwidth.
Regards,
Alexander Chemeris.
Alexander Chemeris
So, I've also done one more test with the same setup using 'iperf'.
Core2Duo was running "iperf -u -s" and laptop was running
"iperf -u -l56 -b20M -t60 -d -c <serverIP>". I set bandwidth to
20Mbit/s to ensure that I saturate link. UDP payload size is set to
56 bytes to simplify comparison with flood ping. Test has been
running for 60secs. Results were measured with "iptraf" on
laptop side:
Incoming:
407 KB/s
4300 pkt/s
Outgoing:
120 KB/s
1250 ptk/s
I'm not sure why incoming is not symmetrical to outgoing in this case,
but I guess it's due to link congestion or iperf bad implementation.
You can see that here I reach 5550 pkt/s and 527 KB/s = 4216 Kbit/s,
which is slightly more then with 5 ping streams in previous test. And
it seems that if I ran more ping streams I would reach the same values.
Seems like you should retry your tests with 'iperf' to get better
estimation, of MP performance.
Regards,
Alexander Chemeris.
elektra
> I get around 1000 packets/s on ns2 (using the 802.11b model at 11 Mbit),
> and between 900 and 1100 packets/s over real 11M & 54M Wifi links.
>
> 1000 packets/s is also close to the numbers I have found in the
> VOIP-over-mesh papers I have been reading.
>
The tests that I have performed last year have shown a throughput of
~3.4 MByte/s on a single hop 54Mbit link and ~1.69Mbit on a two hop mesh
for FTP transfers (under ideal conditions). I was using a MTU of 1500
bytes per package, including IP and TCP overhead (40 bytes). Hence I
measured a transfer of 2330 data packets per second with 1500 bytes.
This figure is not taking the smaller TCP SACKs into account, that also
need channel coordination time. Plus 9 Batman protocol broadcast
packets/second. The total number of payload transmissions in the channel
was therefore about 2400 packets per second using TCP.
Let's not forget that ping is measuring round trip packets (hence you
have to multiply the number of packets by two). Also end-to-end
measurements between MPs/DIR-300s/Nanostations will yield lower results
because of the limited CPU power of 180MHz MIPS processors. I'd
recommend to let PCs/laptops, connected to the LAN ports, do the CPU
intensive tasks (like answering/sending a flood of pings).
> 3/ For a call we need 100 codec packets/s (50 codec packet/s in each
> direction for a full duplex call). So for single hop calls we have a
> total of 4000/100 = 40 simultaneous phone calls.
>
According to our measurements last year the capacity is 45 calls on a
two hop mesh route, with GSM, 4 * 20ms samples per packet. Therefore I
assume the single hop capacity is 90 calls. With 100ms / 5 GSM samples
per packet the call capacity increased to 53 with two hops. Hence we
could estimate 106 calls on a single hop route. Since the number of
calls is rather limited by the WiFi channel access coordination we can
expect similar results for lower bitrates.
I'd recommend to those interested in the subject to check the "Results"
section in the Wiki:
http://wiki.villagetelco.org/index.php/How_to_use_the_software_build_environment
Can we use silence detection with Asterisk to increase the capacity
further? That could almost double the call capacity.
>
> I would appreciate comments on the model, and even repeating the ping
> tests on nodes on real world mesh networks (Elektra and David C). My
> real world test results are still tentative as they may be affected by
> interference and possibly the CPU load of the router.
>
Unfortunately I have only root access to a few distant nodes in the
Berlin mesh. But I will locally set up a testbed with as many nodes I
can get hold of.
Cheers,
Elektra
Alexander Chemeris
> Can we use silence detection with Asterisk to increase the capacity
> further? That could almost double the call capacity.
Yes, DTX (Discontinuous Transmission) should considerable increase
link capacity. But it *might* increase CPU load - I haven't done any
load testing with DTX, just telling from general sense. You should try
enabling it in Speex, it's as easy as 1 function call.
BTW, what do you use as a SIP UA? Is it a separate application or you
use Asterisk with some fancy module for this?
--
Regards,
Alexander Chemeris.
elektra
>
> BTW, what do you use as a SIP UA? Is it a separate application or you
> use Asterisk with some fancy module for this?
>
David Rowe
Thank you for performing your own packet rate tests and your comments.
Alex I repeated your multiple-ping tests and saw a similar increase. I
hit a limit as I increased the ping sessions past 5.
54M 2200 packets/s
11M 1100 packets/s
2M 870 packets/s
I changed the bit rate on my laptop and WRT5GL AP (e.g. "iwconfig eth1
rate 11M") while the tests were running. I had to change the rate at
both ends, e.g. it is possible to have one direction running at 54M and
the other at 2M. The 11M 1100 packet/s is consistent with the ns2
simulations and the VOIP-over-802.11b papers I have been reading.
2200 packets/s is consistent with Elektra's 3.4Mbyte/s 1500 MTU (2400
packets/s) and 45 calls over a 2 hop mesh:
(45 calls)(25 packet/s/call)*(2 hops) = 2250 packet/s.
Note a 100 codec packet/s call with 4 codec packets per Wifi packet
consumes 25 packets/s.
In my case these packet rates were total number of packets, summed by
iptraf. I can see two ICMP packets per ping (forward and return) so I
think I am counting all packets. It is important to use an IP, not a
host name, e.g.
ping 192.168.1.22 -f -q
rather than
ping linksys -f -q
to avoid 2 extra DNS packets per ping.
I haven't managed your 5000 packets/s yet Alex, but will keep trying,
perhaps with iperf.
I am assuming the packet rate is MAC-limited not CPU limited but as
Elektra suggests we will hit a wall some time with a fully loaded MP.
2500-5000 packet/s is a lot of interrupts for such a little CPU.
Re DTX I am not sure if it helps. During "silence" GSM codecs send low
bit rate packets that have just enough information to code the
background noise (complete silence sounds unnatural). These packets can
be sent at a lower rate, however if we are already aggregating frames
this won't give us much advantage. For example if we need to send one
background packet every 80ms it will be the same packet rate as 4
full-rate packets every 80ms. Need to think about this some more.
Elektra re further source aggregation (5 codec packets etc) I think we
have gained most of the benefit from your original idea of aggregating 4
packets. This has increased the capacity by a factor of 4 and reduced
CPU load (lower interrupt rate) by the same factor. We are now close to
other limits (latency and the effects of packet loss with large
packets).
Question: All of these tests were performed using just two radios on the
channel. What happens when 10 radios are contending - do we still get
the same total packet rate (assuming no collisions)? What I mean is
does the MAC protocol insert additional gaps when other transmissions
are heard?
Cheers,
David
David Rowe
I am thinking about ways to deal with an overloaded mesh.
The potential for overload is large, as we have 150 nodes on a mesh.
Our previous discussions suggests we will have the capacity for 10-20
multi-hop calls (or 40-90 single hop calls).
If we overload the mesh, the buffers in the routers will fill, and
packets will be discarded. The effect should be uniform, i.e. new calls
will experience as many dropped packets as existing calls on the mesh.
If each packet contains 80ms of speech, there won't be much the codec
can do about lost packets.
So if we overload the mesh by more than a few %, I would suggest that
all calls will fall over, an undesirable situation. Better to prevent
new calls from going through while maintaining the quality of existing
calls.
One way to do this would be using the Asterisk server to process (via a
dialplan and some shell script) each new call. It could keep a count of
active calls, and prevent a new call from being connected once a certain
threshold is reached.
The Asterisk box could also "listen" to the packet rate of the channel
to sense when we are getting near capacity. It may also be able to
determine if interference (e.g. from Wifi signals off our mesh or other
services) are affecting capacity and reduce the call-limit accordingly.
Cheers,
David
David Carman
On Tue, Nov 3, 2009 at 1:00 AM, David Rowe <da...@rowetel.com> wrote:
> The Asterisk box could also "listen" to the packet rate of the channel
> to sense when we are getting near capacity. It may also be able to
> determine if interference (e.g. from Wifi signals off our mesh or other
> services) are affecting capacity and reduce the call-limit accordingly.
The existing measure of link quality (counting BATMAN/OLSR OGMs)
should be a reliable indicator of spare capacity as well as
interference/poor links.
When a FF/OLSR link is saturated, more OGMs are lost and this is
reflected in the LQ/ETX. An initial polling of routers along the call
path, and multiplication of the LQ values would provide a measure of
the ability of the mesh to handle the call. There would be no
difference in the algorithm for single or multihop links.
David
Alexander Chemeris
This sounds very much like some channel reservation protocol may
be utilized, like RSVP[1] or such. DCCP[2] may also be of any use
here to monitor congestion and it is readily available in Linux.
That's just an idea, as I never used those in real life, only read about
them.
1. http://en.wikipedia.org/wiki/Resource_reservation_protocol
2. http://en.wikipedia.org/wiki/Datagram_Congestion_Control_Protocol
--
Regards,
Alexander Chemeris.
Corinna Elektra Aichele
> Re DTX I am not sure if it helps. During "silence" GSM codecs send low
> bit rate packets that have just enough information to code the
> background noise (complete silence sounds unnatural). These packets can
> be sent at a lower rate, however if we are already aggregating frames
> this won't give us much advantage. For example if we need to send one
> background packet every 80ms it will be the same packet rate as 4
> full-rate packets every 80ms. Need to think about this some more.
do other codecs do the same? These transmissions seem to be unnecessary. A
codec could just play "comfort noise" if there are no voice packets coming in,
as long as the call is going on - rather than make the receiving audio channel
completely silent.
> Elektra re further source aggregation (5 codec packets etc) I think we
> have gained most of the benefit from your original idea of aggregating 4
> packets. This has increased the capacity by a factor of 4 and reduced
> CPU load (lower interrupt rate) by the same factor. We are now close to
> other limits (latency and the effects of packet loss with large
> packets).
I still think we should investigate the feasibility of larger audio chunks.
We'll have less packets in the air, which also means less collisions. Our
packets are not getting really large with 5 or even 6 samples. Still the
overhead from WiFi/IP/UDP/RTP is comparatively large. Of course, if we lose
120ms of audio compared to 80ms this is more noticeable. But it has to be
proven empirically whether we will lose less 80ms packets than 120ms packets.
With larger packets we can deal with more calls before our network becomes
congested. Congestion can lead to excessive retransmissions and high latency
on a multi-hop link. As a consequence we may have to increase the size of the
jitter buffers. Of course this discussion will become superfluous, once we
manage to perform packet-aggregation on a per-hop basis. However we can not
say how much this will affect CPU load.
> Question: All of these tests were performed using just two radios on the
> channel.
I was using three, but they were all in the same room. Hence they could
coordinate media use pretty well.
> What happens when 10 radios are contending - do we still get
> the same total packet rate (assuming no collisions)? What I mean is
> does the MAC protocol insert additional gaps when other transmissions
> are heard?
If they all want to transmit something they'll need more time to coordinate
media access. Things get worse if there are hidden/exposed nodes. That may be
the moment when RTS/CTS can becomes handy.
Reducing the number of packets in the air will ease things for the WiFi MAC
layer, particularly with many nodes in range and hidden/exposed nodes.
Cheers,
Elektra
Alexander Chemeris
On Tue, Nov 3, 2009 at 19:09, Corinna Elektra Aichele <onel...@gmx.net> wrote:
>> Re DTX I am not sure if it helps. During "silence" GSM codecs send low
>> bit rate packets that have just enough information to code the
>> background noise (complete silence sounds unnatural). These packets can
>> be sent at a lower rate, however if we are already aggregating frames
>> this won't give us much advantage. For example if we need to send one
>> background packet every 80ms it will be the same packet rate as 4
>> full-rate packets every 80ms. Need to think about this some more.
>
> do other codecs do the same? These transmissions seem to be unnecessary. A
> codec could just play "comfort noise" if there are no voice packets coming in,
> as long as the call is going on - rather than make the receiving audio channel
> completely silent.
Usually codecs send some infrequent packets with some kind of comfort
noise. They do so for the following reasons:
1) They try to reproduce real ambient noise of sender to make switch between
generated noise and active speech less confusing for listener. During active
speech you will likely hear real background noise, so it will sound
awkward if you
will hear just white noise inbetween,
2) It's always a good idea (in general, uncontrolled environment) to keep
connection alive. This helps (1) to maintain NAT hole for your stream live and
(2) to detect stream stop by the other side. Default SIP profile (without SIP
keep-alives) just can't detect broken connection, so you need to monitor
RTP stream to detect it.
In Speex in DTX mode encoder's return value tells you does encoded packet
contain speech or it is a silence filler. In latter case you could
drop it if you
want real DTX. Normal RTP implementations just send every-Nth frame where
N is a configuration parameter. So if you want, you may send it every, say,
400-500ms and save a lot of bandwidth and packets/s.
--
Regards,
Alexander Chemeris.
David Rowe
I have been doing some more packet rate tests and have some results.
First some Ethernet tests. Each test involved setting up 5 ping floods
from eth0 on my laptop:
$ ping -f -q ip.of.target.box & <five times>
Test Machine Interface packets/s loadav
-------------------------------------------------------
1 Nanostation 2 eth0 2500 1.5
2 IP04 eth0 10000 1.8
3 DIR-300 eth0 2400 1.3
4 x86 box eth0 40000 0.1
5 WRT54GL eth0 3000 1.05
All the boxes with loadav > 1 had very slow command line response. I
wouldn't trust them for any real time VOIP. A DIR-300 is pretty much
the same as a Mesh Potato (same CPU, Ethernet, Wifi).
Now I know ping is not ideal for these tests. However if the embedded
processor can't process a ping reply at a reasonable loadav over
Ethernet then it will probably struggle with any Wifi and VOIP
processing at the same packet rate. Pls correct me if this assumption
is incorrect, i.e. is ping a known CPU hog?
However maybe I messed up somewhere. Could some one please repeat some
of these tests?
Wifi Mesh: I also tried a similar 5 x ping-flood test over a mesh built
from 3 Nanostation 2's:
1 hop, 1460 packets/s, loadav 1.93
2 hop, 1160 packet/s, loadav 3.33
An iptables rule was used to force Batman to select a two hop path.
However I am not sure what bit rate the radios are using for each mesh
hop, iwconfig says 0 bit/s. Perhaps as they are in the same room they
are being overloaded and using a low channel bit rate. The Nanostations
were also very slow with this loadav. They would need to be throttled
back for real-world operation.
Cheers,
David