What is Smart Monkey?
Bernhard Schallenmüller
information? Thanks in advance.
Bernhard
mrfalafel
unexpected user behaviour.
When I was a developer, we had to make applications 'monkey
proof', that is, we had to expect the users to use the
application in all sorts of weird ways (unexpected key
combinations, powering off the terminal in mid-transaction) etc.
A monkey test was where we would literally pound the keyboard,
power-cycle and otherwise thrash the system in any way possible.
'Smart monkey' testing is along the same lines, but more
sophisticated scenarios are designed. As if the user knew a lot
about the application to get themselves into strange areas but
would then do something totally weird that no developer would
have thought possible.
* * * * * * * * * * * * *
"Without deviation from the norm,
'progress' is not possible." F. Zappa
-----------------------------------------------------------
Got questions? Get answers over the phone at Keen.com.
Up to 100 minutes free!
http://www.keen.com
boris beizer
news:3b328740...@usw-ex0103-085.remarq.com...
> Dumb-monkey, smart monkey etc are just terms to define
> unexpected user behaviour.
That may be the usage in some quarters, but among many testers and in the
literature, the definition of the terms are much more precise.
1. "monkey testing" generic term used to cover many different kinds of
tests in which inputs are created by some random process.
2. "dumb monkey testing" Inputs are generated from a uniform probability
distribution without regard to the actual usage statistics.
3. "smart monkey testing" Input are generated from probability
distributions that reflect actual expected usage statistics -- e.g., from
user profiles. There are different levels of IQ in smart monkey testing.
In the simplest, each input is considered independent of the other inputs.
That is, a given test requires an input vector with five components. In low
IQ testing, these would be generated independently. In high IQ monkey
testing, the correlation (e.g., the covariance) between these input
distribution is taken into account. In all branches of smart monkey
testing, the input is considered as a single event.
4. Brilliant monkey testing: the inputs are created from a stochastic
regular expression or stochastic finite-state machine model of user
behavior. That is, not only are the values determined by probability
distributions, but the sequence of values and the sequence of states in
which the input provider goes is driven by specified probabilities.
As an example, consider providing inputs to a telephone for testing.
a. dumb-monkey testing. dial ten digits with each digit selected
independently of the other digits. Obviously, many of these will not
conform to valid telephone numbers.
b. smart monkey, low iq -- dial ten digits taking into account that the
first three digits are an area code and have statistical properties unlike
subsequent digits. (e.g., begins with a 2, 6, or 9; high frequency of 1's,
etc.)
c. smart monkey, high iq. Like a smart monkey, but there are probability
distributions that govern the likelyhood of a given digit following another
digit and of having specific digits in specific places in the ten number
sequence.
d. brilliant monkey. Generates numbers that take into account all the rules
that govern the creation of telephone numbers -- i.e., what is known in
telephony as the "numbering plan." Random digits within the strict rules of
the numbering plan for clean tests -- similar but different rules to force
the creation of numbers that are known to be faulty.
e. very brillian monkey testing. Considers the entire telephone call
scenario and includes such things as hanging up, length of call, dialing
errors, responses from the other side, etc.
You will find muct of this covered in the more recent
literature under the prefered term "stochastic testing"; also in the earlier
literature under "probabilistic testing", and (worst) "random testing."
See my book, "Software Testing Techniques" and also, "Software System
Testing and Quality Assurance' -- but don't look for them under "monkey
testing" since the term was not in general use when I wrote these books.
The term "monkey testing" has been used for several decades (forty
years that I know of) and was coined independently by many, many different
people. In its earlier use, it is almost always intended as a pejorative --
not against the user that is simulated, but against the tester who has too
much reliance on random testing methods.
I first heard the term "smart monkey" and "brilliant monkey" at
Microsoft about 10 years ago.
My final comment on dumb monkey testing is:
Question: "What happens when you let a million monkeys loose on a million
keyboards?"
Answer: " You get a lot of very filthy keyboards." (keyboard description
cleaned up for politeness' sake. )
Boris Beizer
-------------------------------------
Boris Beizer Ph.D. Seminars and Consulting
1232 Glenbrook Road on Software Testing and
Huntingdon Valley, PA 19006 Quality Assurance
TEL: 215-572-5580
FAX: 215-886-0144
Email direct: bbe...@sprintmail.com
Email (Forwarded): bbe...@acm.org, bbe...@ieee.org
------------------------------------------
Mark Engels
this:
There is an article appearing in the January/February 2000 issue of
Software Test & Quality Engineering magazine entitled "Using Monkey
Test Tools." Author Noel Nyman discusses several test tools in use at
Microsoft emulating the monkey-like behavior Mr. F. and Dr. B. describe.
You can try to access the article from the STQE archives at:
Since I know for fact many articles from STQE never make it to the
archives (why would you want to buy the magazine then?) you may want to
try to contact Mr. Nyman directly at:
Hope this helps you Bernard.
--me
Mark Engels
Software Test Engineer
Cardiac Rhythm Management Division
Guidant Corp., St. Paul, Minnesota USA
In article <3b328740...@usw-ex0103-085.remarq.com>,
mrfalafel <mrfalafe...@yahoo.com.invalid> wrote:
> Dumb-monkey, smart monkey etc are just terms to define
> unexpected user behaviour.
>
Sent via Deja.com http://www.deja.com/
Before you buy.
Mark McWhinney
>
> I first heard the term "smart monkey" and "brilliant monkey" at
> Microsoft about 10 years ago.
>
> My final comment on dumb monkey testing is:
>
> Question: "What happens when you let a million monkeys loose on a million
> keyboards?"
Windows 2000
Sorry, that one was just too hard to pass up. :-)
Bob Gagan
in Tom Arnold's book, Visual Test 6 Bible, that was written by Noel Nyman.
It also provides the actual code if you want to give it a try and have
access to VT6.
Bob Gagan
Mark Engels <mark....@guidant.com> wrote in message
news:8jvb8b$54l$1...@nnrp1.deja.com...