BETA TEST

[Paul Barkley]

CS> PB>values, hitting control keys at odd times, etc.).
CS> PB>Today the methodology is more rigorous (when done
CS> PB>right) and the team works from a plan. When changes
CS> PB>are made to the program in

CS> And therein lies the MAJOR flaw in debugging/acceptance
CS> testing. The old way may not have been reproducible
CS>(and rarely was, in fact), but it

Cheryl, I don't at all agree that having a test plan is a
major flaw, but I _will_ say I'm not certain I've ever seen
ANYTHING well enough tested with or without a plan <g>...
One of the best methods is to get a total fool to test the
fool-proof program because, as the cliche goes, fools are so
ingenious... That's where your random banging on the
program works well. The advantage of the test plan, if it's
written by professionals and not just written by a
programmer to satisfy some management requirement that he or
she doesn't approve of, is that it SHOULD at least test all
the subsystems and _all_ the paths that are known from the
design.

My complaint about random testing is that the tester often
misses whole sections because they may not be obvious. The
last project I did was tested that way, and the tester
didn't manage to get the Data Guardian to fire, a very
important piece of code that detects out of date data in
subsystems and alerts the user. Astoundingly, it turned out
not to have any errors in it, a surprise happy ending...

Genealogy software (to make this at least a little more
topical) currently tends not to have very many hidden paths
when compared to some custom software that is more complex,
but I expect that will change as the gen systems start
behaving in more sophisticated ways.

The new Windows gen programs, though, are inherently more
difficult to test because of the nature of event driven
programming. It's hard to test stuff when you don't know
WHAT the user is likely to have open on the desktop. The
result is that you try to be a lot more careful and keep all
the routines contained so they don't react badly with other
stuff.

CS> PB>response to the errors found, the entire system has
CS> PB>to be tested AGAIN; this is called regression
CS> PB>testing. Bummer.

CS> And sometimes picks up the fact that in fixing Bug E,
CS> you created Bug Z.

What do you mean _sometimes_? <g> Actually there's an
interesting software engineering statistic from measuring
"fixes" that shows that the probability of introducing a new
bug is inversely proportional to the size of the fix. If
you change only 1 line, there's an 80% probability of
introducing a new bug! Pretty astounding. The percent
drops as you change 2 lines, 3 lines, etc., until, as I
recall, changing 5-10 lines only has a 40% probability of
introducing a new bug!

Regards,
Paul

--- FLAME v1.1
* Origin: Nat'l Genealogical Society, Arlington VA 703-528-2612 (1:109/302)

[Paul Barkley]

GP> [... Four paragraphs of a (unsolicited), somewhat
GP> condescending, treatise on software development
GP> omitted to save bandwidth, and possible knuckle-rapping
GP> by moderator...]

Hey Gary!

Unsolicited, but definitely not condescending.
Condescending is when I say "Only a moron would believe
that..." <g> I don't believe in making terse comments
without enough explanation for the reader to understand the
point. My least favorite sentence in exchanges like this is
"Is not!"; that doesn't help anyone understand the point
very well.

GP> Takes more than that to insult me.

Good. No insult intended.

GP> 1986 - v1.00 was originally a 1986 Science/Math Fair
GP> project by Adam Hudson. (He later released v1.01,
GP> v1.02, v2.00, v2.01, v2.03, and v2.04. At the time, he
GP> was 16 and

I think we just found the problem! Why is not surprising
that a 16 year old and his successors may have had a small
problem with software engineering concepts?

GP> 2) My intent is to release QuickBBS v2.80 Zeta-2 within
GP> the next 30 days.

Hmmm. I wonder is that's a Second Zeta or actually a Zeta
Twice Removed? These guys are REALLY creative!

GP> * V4.0 Gamma-x

Uh oh, that's the version you have to wear a lead apron
for...

Gary, you've been a good sport about this. I know it's very
annoying to accept something at face value, try to be
helpful by passing it along, and then find out other people
think it's really silly.

GP> I hope this has provided you more entertainment today
GP> Paul. <G>

It certainly has! Your source material was even better than
I expected! Thanks again! <vbg>

[Paul Barkley]

TS> Selling "beta" stuff to the public is known in our shop
TS> as "X testing."

TS> You had to have experienced Xerox Write to believe it.

Tom, that's a good name for it! So sorry I missed Xerox
Write <g> but I've seen plenty of other messed up stuff.

What's REALLY REALLY _REALLY_ annoying, though, is when a
user finds a bug in something _I_ wrote. I hate it when
that happens, do everything I can to keep it from happening,
and it still happens anyway from time to time.

The trick, I think, is to avoid FUBARS like screwed up data
models, so that the software errors are small enough that
they're easily fixed without having to rewrite many modules
of stuff. I think I can at least say I've reached that
point, but it sure is hard to get ALL the screwups (er,
undocumented features) out of code...