The proper way to test a game

9 views
Skip to first unread message

Greg Falcon

unread,
Sep 23, 1996, 3:00:00 AM9/23/96
to

I have waited a while to ask this question. I have been working on
Escape From Planet Thid for a good chunk of time now, and it is really
starting to come together. What I want to know is, what is the proper
way to test it?

I'm letting a few of my friends play it as I work on it. These are
close friends of mine, who I know will give me blunt opinions and what
they feel needs to be fixed. But they're not real beta testers;
their intent is playing the game ("When will the next release be
done?") and not so much looking for bugs.

What I want to know is, where can I find good, knowledgable,
trustworthy beta testers? How many will I need? (The game has [will
have] fifty rooms or so, and many puzzles.) This is my first project,
so I really have no gague as to how good it is. (I am too involved in
it to give any objective opinion.)

Also, should I wait until the very last room and object is created?
Or is it okay to have testers work on finished portions of the game
while I build the next portion? (I am finishing the rooms in an order
which allows me to play through many of the puzzles right now.)

Finally, if you are an author who has gone through the writing of a
first game before, is there anything helpful you could tell me that
immediately springs to mind?

Thanks.

Greg

--
i've never stolen a .sig idea - wait, i guess i have


Andrew Plotkin

unread,
Sep 23, 1996, 3:00:00 AM9/23/96
to

Greg Falcon (professo...@pnx.com) wrote:
> I have waited a while to ask this question. I have been working on
> Escape From Planet Thid for a good chunk of time now, and it is really
> starting to come together. What I want to know is, what is the proper
> way to test it?

Post here and ask for beta-testers. Usually gets plenty of response.

> What I want to know is, where can I find good, knowledgable,
> trustworthy beta testers?

Well, you can always ignore the comments from the stupid ones. :-)

> How many will I need? (The game has [will
> have] fifty rooms or so, and many puzzles.)

Take as many as you can, I guess...

> (I am too involved in
> it to give any objective opinion.)

*That's* an objective opinion. Heh.

> Also, should I wait until the very last room and object is created?
> Or is it okay to have testers work on finished portions of the game
> while I build the next portion? (I am finishing the rooms in an order
> which allows me to play through many of the puzzles right now.)

I think it's *much* better to wait until it's all built. If you add more,
what if you accidentally introduce a bug at a point early in the game?
(This will happen anyway, as you fix bugs, but there's nothing you can do
about that.)

> Finally, if you are an author who has gone through the writing of a
> first game before, is there anything helpful you could tell me that
> immediately springs to mind?

You are not required to take every suggestion from every tester. If
someone says that a particular piece of text is unclear, you might change
it; but also consider the possibility that whatever you change it to will
be unclear to someone else.

--Z

--

"And Aholibamah bare Jeush, and Jaalam, and Korah: these were the
borogoves..."

Bob Adams

unread,
Sep 23, 1996, 3:00:00 AM9/23/96
to

In article <526r57$5...@news.dx.net>, Greg Falcon
<professo...@pnx.com> writes

>
>What I want to know is, where can I find good, knowledgable,
>trustworthy beta testers? How many will I need? (The game has [will
>have] fifty rooms or so, and many puzzles.) This is my first project,
>so I really have no gague as to how good it is. (I am too involved in

>it to give any objective opinion.)

There are actually some people who far happier testing adventures than
just 'playing' them - and thank heavens they exist. As the author you
are totally incapable of testing your own game. It is only when other
people attempt to do 'odd' things that you would never have attempted in
a million years, that the bugs get revealed.


>
>Also, should I wait until the very last room and object is created?

Yes. And also get it playtested again after every round of 'bug fixing'.
It is amazing how just a small alteration can cause all sorts of further
complications.

>
>Finally, if you are an author who has gone through the writing of a
>first game before, is there anything helpful you could tell me that
>immediately springs to mind?
>

Yep... don't get depressed and give up when the testers report on your
'perfect' game runs to four pages of fullscape listing all the errors.


--
Bob Adams
http://www.amster.demon.co.uk


Julian Arnold

unread,
Sep 24, 1996, 3:00:00 AM9/24/96
to

In article <526r57$5...@news.dx.net>, Greg Falcon
<URL:mailto:professo...@pnx.com> wrote:
>
> I have waited a while to ask this question. I have been working on
> Escape From Planet Thid for a good chunk of time now, and it is really
> starting to come together. What I want to know is, what is the proper
> way to test it?

The very best way is to enlist a Michael Kinyon. Sadly there is but
one, and I don't seem to have his address around here. I'm sure he'll
pop up himself though.

Failing that (well, as well as that really), post to r*i-f and ask for
beta-testers. Make sure you mention what your game is programmed in,
perhaps what the game basically is (length, perceived difficulty, genre,
etc.) and any little obsessions you have (some authors insist their
testers send a full transcript of every session). How many? Errm, 4-10?

> Also, should I wait until the very last room and object is created?

> Or is it okay to have testers work on finished portions of the game
> while I build the next portion? (I am finishing the rooms in an order
> which allows me to play through many of the puzzles right now.)

This is what happened when I tested "Gumshoe". OTOH "So Far" was very
much as it is today. It's up to you, just make sure your testers know
beforehand.

> Finally, if you are an author who has gone through the writing of a
> first game before, is there anything helpful you could tell me that
> immediately springs to mind?

I'm not an author, 'cos I have never finished a game. The moral? Make
sure you finish it.

Jools
--


Gareth Rees

unread,
Sep 27, 1996, 3:00:00 AM9/27/96
to

Greg Falcon <professo...@pnx.com> wrote:
> Should I wait until the very last room and object is created? Or is

> it okay to have testers work on finished portions of the game while I
> build the next portion?

Don't beta-test until you are *sure* that the game is in a final state.
If you start testing while you're still making major changes, then
you'll find you have to beta-test several times because the earlier
testers didn't have a chance to see the later changes. It just makes
more work for you.

> Where can I find good, knowledgable, trustworthy beta testers?

Advertise on rec.games.int-fiction for beta-testers. There are always
lots of people willing to test games. Some of them are very skilled at
finding obscure bugs. Ask for replies by e-mail only, and make sure
that your post has the header line

Followup-To: poster

so that rec.games.int-fiction does not fill up with "me too" posts.
Cancel your post when you have enough responses.

> How many will I need?

As many as you can cope with, but no more. There is a "diminishing
returns" effect as more and more people test the game, and you spend
time processing bugs several times. Perhaps four testers?

> Is there anything helpful you could tell me that immediately springs
> to mind?

Lots of things!

1. The beta-testing process will take a long time. Perhaps several
months. Don't be impatient to release the game; the longer you spend on
testing, the better the game will be.

2. A major danger is that you will introduce new bugs while you fix old
ones. One way to reduce the chances of this happening is to do a
regression test after each batch of bug fixes.

What you do is this: (i) write a command script that solves the game and
exercises all the puzzles, dead ends, special events, interesting
messages and so on; (ii) keep a transcript that results from running
this script; (iii) after making a batch of changes, but the script on
the new game. Compare the resulting transcript with the old transcript
using a difference program like `diff'. This will tell you what has
changed and will show up many simple blunders.

Note that you need some way of controlling the game's random events in
order to get meaningful results from this test. (Inform has the
"norandom" debugging command.)

3. You should consider exactly what you want from your beta-testers. Do
you just want bug reports, or are you interested in issues like
characterisation, plot consistency, spelling, writing style, and so on?
You should tell your testers what you want to get out of testing
(otherwise you'll get what they want to tell you).

4. Take care to tell your testers whether the game is going to be free,
shareware or commercial. You may find that some people are unwilling to
donate their time free of charge to someone intending to make money from
their efforts. Note that a good proofreader in the UK can charge 20
pounds an hour or more.

--
Gareth Rees

Reply all
Reply to author
Forward
0 new messages