Error in APRS Tracker Code

[John Graham-Cumming]

Folks,

The current Phoenix 1 flight is exhibiting an odd behavior where the altitude is dipping sometimes. This is a software bug.


The code being used appears to be from here: http://sharon.esrac.ele.tue.nl/users/pe1rxq/aprstracker/aprstracker.html I have examined the most recent version (0.8) and there is a serious problem in the altitude calculation as follows: https://gist.github.com/3476754. The offending line is #15 where the following calculation is performed:

t=(d-'0')*33;

Both t and d are unsigned chars (i.e. can hold 0 to 255) and d will be a single ASCII number digit. Any time a digit of the altitude exceeds 7 t will underflow and the altitude will be wrong (sometimes woefully wrong). This effect can be seen in the chart above. The first large dip corresponds to 8,000 metres because the GPS is providing altitudes in metres that are converted (erroneously) to feet. Notice how it recovers once the altitude goes above 10,000 metres. Then next large drop starts at 18,000 metres and recovers at 20,000 metres. Smaller dips occur when a digit with low significance is > 7.

I would suggest _not_ using this APRS code. This bug is serious and the code needs to be rethought.

John.

[MikeB]

At the risk of being seeming OT, this type of software bug could have been detected by using a GPS simulator. Paul Verhage has developed a low cost PICAXE based design that produces GPS sentences from a simulated HAB flight. A description for this useful development tool can be found at...

http://nearsys.com/catalog/sensor/gpssim.htm 

MikeB

[John Graham-Cumming]

At the conference I will be talking about this sort of bug and how to detect it.

You do not need a GPS simulator, just simple unit testing.

John.

--
You received this message because you are subscribed to the Google Groups "UKHAS" group.
To view this discussion on the web visit https://groups.google.com/d/msg/ukhas/-/cOjziLP3gyUJ.
To post to this group, send email to uk...@googlegroups.com.
To unsubscribe from this group, send email to ukhas+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/ukhas?hl=en.

--
My book: The Geek Atlas, 128 Places Where Science and Technology Come Alive
http://geekatlas.com/  Signed copies: http://bit.ly/eSvmsV

Brochure d'Histoire d'Art: http://www.etsy.com/listing/98699897/frise-chronologique-de-la-peinture

[John Graham-Cumming]

Also, note that this code contains other errors (no surprise!) that at specific altitudes causes vast overreporting of the altitude. Here's a graph showing the GPS reported altitude in metres, the equivalent in feet and the output of the faulty code for all altitudes from 0 to 40,000 metres. The enormous leaps upwards occur with the right combination digits in the altitude in metres because carrying is incorrectly implemented.

This is really poor code!

[Mike Bessant]

John,

I would think few people have more HAB experience than Paul Verhage and he recommends testing code using simulated GPS data before flight. However, I would be interested to learn what meant by 'simple unit testing'.

MikeB

--

You received this message because you are subscribed to the Google Groups "UKHAS" group.

[John Graham-Cumming]

On Sun, Aug 26, 2012 at 3:01 PM, Mike Bessant <mikebes...@gmail.com> wrote:

I would think few people have more HAB experience than Paul Verhage and he recommends testing code using simulated GPS data before flight. However, I would be interested to learn what meant by 'simple unit testing'.

I will talk about this in my talk at the conference, but, briefly,

1. The code to do the metres -> feet calculation should be separated into a function of its own, say char * ft_to_mtrs( char * s). This isolates the code nicely for reuse and testing.

2. A test suite can then be written that takes various inputs, calls the function and check that the output is what is to be expected. For example, testing just every 1,000 meters up to 40,000 would have shown the error that's present.

Unit testing: http://en.wikipedia.org/wiki/Unit_testing

John.

[David Akerman]

As John said, this type of bug can be quickly and easily found simply by testing the errant code in isolation.  Cheaper and much, much quicker than using a simulator.  Also much easier to set up automated tests this way.

That said, it's worth running a completed (and unit tested) tracker through some sample flights.  I've done this using a simple PC program to generate the NMEA, sent out to the tracker via a USB-TTL serial adapter.  Some may remember me drawing a flying saucer on spacenear.us using it :-).  This way is cheaper than a hardware simulator and makes it simpler to set up different flight scenarios.  However if any bugs do show up at this stage, you have to ask why they didn't show up in the earlier unit testing.

Dave

--
You received this message because you are subscribed to the Google Groups "UKHAS" group.

To view this discussion on the web visit https://groups.google.com/d/msg/ukhas/-/cOjziLP3gyUJ.

[David Akerman]

I hope he doesn't recommend not doing the unit testing first.

[Anthony Stirk]

Also as usual the Wiki is your friend :

http://ukhas.org.uk/guides:common_coding_errors_payload_testing#simulating_a_flight 

Anthony

[MikeB]

Of course every effort should be made to eliminate errors during software development but a good simulator can detect fault conditions that you had not thought about (the unknown -unknowns as an infamous US politician once said).    

Here is another PC based GPS simulator for those who would rather not use a dedicated version...http://www.labsat.co.uk/index.php/en/products/free-satgen-nmea-software.html

MikeB

[John Graham-Cumming]

On 28 Aug 2012, at 09:15, MikeB <mikebes...@gmail.com> wrote:

> Of course every effort should be made to eliminate errors during software development but a good simulator can detect fault conditions that you had not thought about (the unknown -unknowns as an infamous US politician once said).

Can you give an example of that? In general testing means verifying that the results are expected and hence the output would be 'known'. It is possible to do some crash detection by fuzzing inputs with random data, but that's very different from these devices that generate NMEA strings. In my view they are useful for testing the serial connection and little else because the rest (string recognition, parsing, unit conversion) is ripe for unit testing. That can be done entirely in software and in an automated fashion.

John.

[Mike Bessant]

John

I am not trying to suggest that the two approaches are mutually exclusive.

Both software testing and complete systems testing (using simulators) have their part to play in increasing reliability. To reject the latter would be to ignore years of proven experience in the field of developing system for reliable remote operation, which I would think could be applied directly to HAB projects. 

MikeB 

--
You received this message because you are subscribed to the Google Groups "UKHAS" group.

[Steve Aerospace]

Hi HAB folk:
Just a heads up that I expect to do a pico (well more micro really) balloon flight out of the Buckinghamshire/Cambridgeshire area on Thursday morning. This will be using a 100g Latex balloon. 434.650MHz XABEN1 format (7n1 425Hz shift). Hoping to reach 50Kft with a small video payload.

Steve G8KHW

[MikeB]

Steve,

Will that be at the EARS site?

MikeB

[Steve Aerospace]

Hi Mike 

I'm not planning for it to be there - I was going to launch it from EMF at Milton Keynes - but thats a bit too far over - so I was thinking more like Cambridge - Haverhill way.  Not too far to drive - but far enough away from the coast to be reasonably sure of getting it back.  Allowing for  lot of uncertainty about the characteristics of this balloon.

Steve

--
You received this message because you are subscribed to the Google Groups "UKHAS" group.

To view this discussion on the web visit https://groups.google.com/d/msg/ukhas/-/BG8P4tAX2WQJ.

To post to this group, send email to uk...@googlegroups.com.
To unsubscribe from this group, send email to ukhas+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/ukhas?hl=en.

Steve Randall

Random Engineering Ltd

st...@randomaerospace.com

+44 7802 242135

[Dave B]

------- Original Message -------
John Graham-Cumming Aug 28 12:49PM +0200  

> Of course every effort should be made to eliminate errors during
software development but a good simulator can detect fault conditions
that you had not thought about (the unknown -unknowns as an infamous US
politician once said).
 
Can you give an example of that? In general testing means verifying that
the results are expected and hence the output would be 'known'. It is
possible to do some crash detection by fuzzing inputs with random data,
but that's very different from these devices that generate NMEA strings.
In my view they are useful for testing the serial connection and little
else because the rest (string recognition, parsing, unit conversion) is
ripe for unit testing. That can be done entirely in software and in an
automated fashion.
 
John.

--------------------------------

Well, we once supported a Dutch software company, that made a great
ballyho about how their software was fully tested before release, by an
automated system. I even visited their development offices and saw such
a test in progress. Impressive it has to be said..

But...

They were only checking for expected correct responses, for known correct
inputs. I proved to them (in about 5 seconds flat!) That give their
latest and greatest some invalid input (an extra space in a string input
I seem to remember in one instance) and it would at best give you garbage
output, at worst BSoD the PC! Not good, for something that was intended
to control a multi million dollar high power RF test facility!

(I was escorted out of the office shortly after that, while they sat down
and "had a meeting to discus the issue".)

We don't support that software any more. (Thankfully! C++ code with
Dutch language procedure/variable etc names, and comments. It's amazing
what Google Translate can handle, I was adding English translations to
the comments in their code, another black mark to my name...)


I'm probably preaching to the converted, but....

Take care with any software that takes input from something not of your
doing. Heck, even if it is your code creating the incoming data, check
it...

All input (especialy wetware created) should be sanitised for gross
errors (missing fields, out of bounds values, string data instead of
numeric etc, and so on) before the main processing routines.

Then, whatever routine does the processing, should have robust error
catching, much like the Try/Except constructs in Delphi.

Don't forget the 'Else' clause, at the end of an 'If' etc. Even if you
don't "Need" it, code it, to make any data values passed downstream
"sane" or indicate a problem in a safe way for later code, if that
happens.

Preset/Clear any common scratchpad variable's that are shared between
routines, that are "NOT" used to transfer data from one routine to
another. Be consistant with that, do you preset on entry, or garbage
collect on exit, do it one way, and one way only througout your code.

Take care of your string buffers too, no blind copying, always limit the
possible number of bytes coppied, to be equal or less than the buffer
size! Obvious, but that's the single biggest single cause of a lot of
'C/C++' code woes, (Buffer Overun's) when dealing with data from the
great unwashed outside world. GPS and other sensor data etc.

Lastly, if you must use an automated software test scheme, after checking
it does what you expect with "Good" data, throw some bad data at it, and
see what fall's over. Start with subtly bad data, extra embedded spaces,
empty data fields etc, such as would happen if a GPS looses lock
unexpectedly for example, then progress to random garbage.

Of course, it's for you to decicde what your code does with bad data,
silently ignore it, or signal an error somewhow.

Making any software robust, usually takes much more time and effort, than
making it do what you want in the first place. Sometimes more so for
simple code. But it pays off in the long term.

If you can, implement a watchdog type reset/recovery. If something bad
happens that locks your microcontroler, at least it could reset itself
after some delay, and if the problem was transient, it could come back to
life. Not always as easy to do as it is to say however...

73.

DJB.

[John Graham-Cumming]

On Fri, Aug 31, 2012 at 9:24 AM, Wouter Weggelaar [PA3WEG] <wout...@gmail.com> wrote:

I agree that it should have been tested with all inputs, but you cannot blame any volunteer that publishes his home grown code for personal education without any warranty.

And I did not seek to blame the original author, only to warn members of the UKHAS group to not use that code.

 

Has someone actually bothered to contact the author?

I have not made any contact.

John.