MAVlink over half-duplex radiolink

[Alex Chen]

I have setup with Arduplane 3.3 and Mission Planner connected via serial radio modems. I tested remote control with joystick and find that my link has permanent drop rate and the greater MAVlink data throughput the grater packet drop rate. Changing packet size, air data rate, serial baudrate changed almost nothing.

I think that the problem is in MAVlink message fragmentation when packing data for radiotransmission. Data sent over radiolink in halfduplex mode in radiopackets. Data packed to packet when packet size reached or character timeout is reached. If time interval between MAVlink messages shorter than character timeout MAVlink messages should fragment message overflowed packet buffer. When fragmented message received it put to serial and if next fragment haven't received untill last character put to serial delay between characters happens. When MAVlink reads port for message character timeout may occur and packet dropped.

SiK based modems are aware of MAVlink and thus (I believe) send each MAVlink message separately. But modems that unaware of MAVlink message format are prone to this problems. So the question is: do we have any means to a) set large inter character timeout on receive end to wait for next fragment (for larges possible fragment it should be near 260 characters at port baudrate), or b) set delay between MAVlink messages greater than modems character timeout to force modem send packet. Most modem have inter character timeout from 1 ms. So delay between messages should be greater than 1,5-2 ms for reliable detection.

Does it make sense? May I missed something trivial?

[Andrew Tridgell]

Hi Alex,

> SiK based modems are aware of MAVlink and thus (I believe) send each
> MAVlink message separately.

The SiK firmware will happily send multiple MAVLink messages in a single
radio frame. It just tries to align the MAVLink messages boundaries with
the radio frame boundaries. If multiple messages fit in one radio frame
it will do that. It helps a lot. That also means that if it thinks there
are more bytes needed to complete a MAVLink message then it will wait
for a few ms before sending the packet.

The other big factor that helps the SiK radios is the relatively large
serial buffers it uses. The default is a 1900 byte receive buffer and
650 byte transmit buffer.

> set delay between MAVlink messages greater than modems character
> timeout to force modem send packet.

that is a bit tricky to organise due to the multiple layers of
buffering between the autopilot code and the radio. On something like
the Pixhawk there is the mavlink level buffering, followed by the AP_HAL
uart buffer, followed by a NuttX level uart buffer. To be really
effective this delay would need to be at the lowest level.

What radio are you using? Does it have hardware flow control? Can you
modify the firmware to be mavlink aware?

Cheers, Tridge

[Alex Chen]

I tried different modems Digi, Microhard, etc. any half-duplex modem should be affected. And we have no access to firmware if we take any vendor's standalone modem and most OEM modem modules. The only solution free of this effect is a high speed broadband modems but they has no even half range comparatively to RFD900+.

So If we want to use third party half-duplex modems we have to manage packetization or fragment assembly. End the later is more preferable, I think because along with small packet size it can decrease latency.

[Craig Elder]

Alex

What specific issue are you seeing with the Microhard radios?  I have worked on several different vehicles that have a Microhard installed and have not seen any problems with it.

The solution to your situation might be a simple as using hand shaking.

Craig

On Mon, Sep 7, 2015 at 11:28 PM, Alex Chen <chen...@gmail.com> wrote:

I tried different modems Digi, Microhard, etc. any half-duplex modem should be affected. And we have no access to firmware if we take any vendor's standalone modem and most OEM modem modules. The only solution free of this effect is a high speed broadband modems but they has no even half range comparatively to RFD900+.

So If we want to use third party half-duplex modems we have to manage packetization or fragment assembly. End the later is more preferable, I think because along with small packet size it can decrease latency.

--
You received this message because you are subscribed to the Google Groups "drones-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to drones-discus...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Alex Chen]

Craig,

Nothing specific to Microhard. I think it's specific to half-duplex. Half-duplex transmission by-design has a Tx-Rx cycle which leads to necessity to fragment data stream to packets and makes delay between radio packets. Internal modem receive buffers and interface port transmit buffers may stitch packets into data stream but may not. It depends on air data rate and modems port data rate. Now we find solution with some fine tuning of modems and setting port rates significantly lower than air data rate, so serial port transmit buffer stitches fragmented packet inside modem. But we lost more than half of available data rate.

So I state that there's a potential issue when using half-duplex radio modem, and fine tuning of link may manage with it. I think it would help greatly if we can tune receive character timeout on both sides of data link. But fr now I can't find where in code it checked. And there's a question what circumstances may happen with this timeout increase (remote control latency, glitches on high BER, etc.). 

[Andrew Tridgell]

Hi Alex,

> Nothing specific to Microhard. I think it's specific to half-duplex.

pretty much all radios are half-duplex. The half-duplex nature of the
link is not the cause of your issue I think. Full-duplex radios are
possible, but are exceedingly rare as they are difficult to build.

Have you analysed some tlogs to see what sort of loss you are actually
getting?

I can certainly see that if we knew that the radio had a specific wait
time for packet completion and we could arrange for packets to be sent
out the uart so that this wait time was triggering on mavlink packet
boundaries then that may help with some radio types. It will also make
things worse in some (quite common) circumstances.

A MAVLink enabled vehicle tends to send a bunch of short packets in
bursts. If the radio has a maximum frame length of 256 bytes then it
would be quite wasteful to send each small packet (which may be less
than 20 bytes) as a separate frame.

If you wanted to experiment with doing this on a Pixhawk then the code
you would need to change is here:

https://github.com/diydrones/ardupilot/blob/master/libraries/AP_HAL_PX4/UARTDriver.cpp#L503

and the logic would be similar to what we use to packetise MAVLink
packets on UDP boundaries on Linux:

https://github.com/diydrones/ardupilot/blob/master/libraries/AP_HAL_Linux/UARTDriver.cpp#L499

Cheers, Tridge

[Bill Bonney]

Full-Duplex is difficult if actually almost impossible, though recently some Boffins are getting close http://m.theregister.co.uk/2015/08/13/full_duplex_radio_tech_bristol_uni/

Practically all current tech half-duplex unless transmit and receive is on different frequencies.

That said, is the issue present if you make sure the receiving radio is not transmitting any data. Or disable heartbeat sending from The GCS if you have it enabled.

Bill

[Alex Chen]

Bill,

I think turning off heartbeats from GCS changes almost nothing when using remote control with joystick. We have down rate about 1.9kBps and up rate 17Bps without joystick and about 490Bps with joystick control as shown in MIssion Planner link quality widget. Considering air rate is simplex rate, we need about 19kbps pure data air rate. Also we have transmission control overhead and Tx/Rx switch delay, so it's difficult to estimate theoretical needed air data rate. Having 115/230 kbps air data rate we had excessive drops at default modem config. With port rates restricting to 1/4 air rate and some unclear internal timing tuning we got almost no drops. 

Anyway if we had MAVlink data burst size tuning and adjust it to modem's packet size and also inter burst delay tuning and adjust it to modem's inter character timeout we will have best possible throughput/latency performance on half-duplex modems.