Usability testing IF
Branko Collin
Usability testing is a form of testing a product in which average
users run through the paces of interacting with that product while
being filmed.
Often, the test subject is being asked to perform certain tasks,
although that's not necessary. "Play this game" is a valid task.
You can compare usability testing with what's generally understood as
beta-testing in this newsgroup, but it is better in one respect: when
you ask somebody to beta-test your game, you generally do not monitor
them. The beta-tester may take ten minutes to get around an interface
problem without noticing the trouble (s)he is having, simple because
(s)he has had that same problem in numerous games before. It is the
task of the tester to notice these problems and put them in the test
report.
Also, a beta-tester may be an experienced adventure game player who is
used to all the conventions of this genre.
If you were running a usability test using relatively novice players
in order to find out what the basic usability problems of the genre
are, which games would you use?
(No, I do not have concrete plans to perform such tests.)
--
branko collin
- dr: "have you been exposed to any user interfaces designed by engineers?"
- woman: "yes"
- dr: "you have interface poisoning. you'll be dead within a week" (scott adams, dilbert)
Andrew Plotkin
>
> You can compare usability testing with what's generally understood as
> beta-testing in this newsgroup, but it is better in one respect: when
> you ask somebody to beta-test your game, you generally do not monitor
> them.
If you want to come close, ask the tester to type "script on" at the
start of the game session. This doesn't give you timing, but it lets
you see everything else that went wrong or right.
> If you were running a usability test using relatively novice players
> in order to find out what the basic usability problems of the genre
> are, which games would you use?
The games which are commonly recommended for novice players, when they
come around here asking. That is, by definition, their sample of the
genre as a whole.
--Z
"And Aholibamah bare Jeush, and Jaalam, and Korah: these were the borogoves..."
*
* Make your vote count. Get your vote counted.
Mike Rozak
1) Usability testing was done way before beta, often at alpha or pre-alpha
stage (with prototypes). Beta testing was used for stability issues and
configuration issues (sound cards, video cards, etc).
2) The user was placed in a room with a one-way mirror. They were given a
list of tasks to accomplish on somewhat vague levels. For example: "Open the
wave file, hello.wav and play it." or "Record a sample of your voice." They
weren't given documentation (on the thought that real users won't read it
anyway), help was available but usually not used (people don't usually use
help).
3) The whole thing was video taped. The designers would often sit on the
other side of the mirror and watch (and cringe).
4) Users are asked to speak aloud what they're thinking as the progress.
Usability tests are very enlightening. It's amazing how tasks that you (as a
designer) think are trivial, end up being real obstacles to user.
IF has slightly different usability issues, which may require slightly
different tests. Usability tests would also occur around alpha/beta for IF:
1) Do people understand the basic UI if they've never played it before...
probably not.
2) Guess the verb - Logs are very good for this.
3) Is the puzzle too difficult - Time might be handy. Having the user "speak
aloud" or somehow indicate what they're thinking would be really handy.
4) You might want to build in some statistical analysis, to see how long it
takes someone to solve a puzzle. (This is difficult because you don't know
when they start/stop.) Or what rooms they visit, etc. Then, combine the
statistics from a large numbers of users.
5) Computerized telephony (to do X, press 1. to do Y, press 2) does a lot of
usability testing. They measure their success by a) the number of people
that complete their transaction without talking to an operator, and b) the
shorter the call the better.
--
Mike Rozak
http://www.mxac.com.au
"Branko Collin" <col...@xs4all.nl> wrote in message
news:514f901kmteferkev...@4ax.com...
Grue
> users run through the paces of interacting with that product while
> being filmed.
>
> Often, the test subject is being asked to perform certain tasks,
> although that's not necessary. "Play this game" is a valid task.
>
I did this with my roommate, who
1) Doesn't know English well.
2) Never played IF before.
I made him play my (still unfinished) game. Of course he didn't get
very far and I had to explain all the genre conventions to him (for
example he always used commands like PICK ITEM (which doesn't (and
shouldn't) work, probably ADOM influence.) instead of TAKE ITEM (or
even PICK UP ITEM), but then he typed GO NORTH instead of N). I
spotted some typos using this method, but I'd prefer more professional
beta-testing.
Richard Bos
> A Microsoft (and probably other companies), here's how it went:
>
> 1) Usability testing was done
Oh, come on! Two lines in, and you're pulling our leg already.
Richard
Carl D Cravens
> shouldn't) work, probably ADOM influence.) instead of TAKE ITEM (or
Do most people use TAKE? I use GET... I don't know why, I just always
have. Maybe because it's shorter.
--
Carl D Cravens (ra...@phoenyx.net)
Toto, I don't think we're online anymore...
M.D. Dollahite
>
>Do most people use TAKE? I use GET... I don't know why, I just always
>have. Maybe because it's shorter.
I use GET, because that's how the command was listed in most Infocom manuals,
even though it's always been considered the secondary alternative to TAKE on
the programming side of things.
Quintin Stone
> On Wed, 5 May 2004, Grue wrote:
>
> > shouldn't) work, probably ADOM influence.) instead of TAKE ITEM (or
>
> Do most people use TAKE? I use GET... I don't know why, I just always
> have. Maybe because it's shorter.
When I was but a lad, I used TAKE exclusively. For some reason, I
regarded GET as less pure, less reliable. Now that I am older, I am lazy,
and GET only has 3 letters.
/====================================================================\
|| Quintin Stone O- > "You speak of necessary evil? One ||
|| Code Monkey < of those necessities is that if ||
|| Rebel Programmers Society > innocents must suffer, the guilty must ||
|| st...@rps.net < suffer more." -- Mackenzie Calhoun ||
|| http://www.rps.net/QS/ > "Once Burned" by Peter David ||
\====================================================================/