Should Beta-Testers Be Given Requested Materials?
Al
given the requested items such
as walkthrus,how points are scored. list of puzzles without solutions,
etc in order to help speed
along the testing process?
Mike Snyder
news:1185755385.5...@z24g2000prh.googlegroups.com...
It depends. If you think the testers can proceed and meet goals without
those things, and if you don't want to make that information known, then no.
It's entirely up to you.
In professional development, yes. Testers would need to know everything
about everything, so they know what's working and what isn't. It's a little
different when you're having people you don't know -- people who don't
actually work for you -- test your stuff anonymously and for free. You just
have to decide how much you're willing to provide, to achieve your goals.
Walkthroughs and hints for testers aren't uncommon. Built-in (even if
temporary) cheats can also help.
---- Mike.
Al
> It depends. If you think the testers can proceed and meet goals without
> those things, and if you don't want to make that information known, then no.
> It's entirely up to you.
>
> In professional development, yes. Testers would need to know everything
> about everything, so they know what's working and what isn't. It's a little
> different when you're having people you don't know -- people who don't
> actually work for you -- test your stuff anonymously and for free. You just
> have to decide how much you're willing to provide, to achieve your goals.
>
> Walkthroughs and hints for testers aren't uncommon. Built-in (even if
> temporary) cheats can also help.
Thanks for your feedback. I'm testing a game right now and have asked
the author repeatedly
for the above materials: walkthru, how pts are scored, list of puzzles
without solutions and after
nearly 2 weeks got 1 out of 3. I explained to him that as a beta
tester I don't care about having
the game "spoiled" because after the game is posted to the group or
is put out for public consumption
or whatever is done with the game, when one hasn't played a game for a
long period of time it's s
if the person had never played it. You have to go back and retrace
just about everything.
Try not playing Zork for 5-10 years and see how well you do.
Khelwood
It's the author's choice. If he doesn't think you need those things,
or doesn't trust you, he's entitled to withhold them. Maybe he wants
to see how some of his testers do without extraneous help.
Jim Aikin
I like having a couple of testers experience the game exactly the way a
player will. When I read the transcripts they send, I often get
extremely useful insights by seeing what actions they may have tried,
even if they didn't flag the result (usually, a boring error message) as
a bug.
On the other hand, letting a tester "cheat" by jumping around in the
game to bang on stuff is also very useful, partly because it saves time.
(I speak from experience here; Al has been testing Lydia's Heart.)
Since testing is a voluntary activity, if you're not happy with what the
author is providing, just explain to them that you won't be doing any
more testing unless they can help you out.
--JA
Andrew Owen
for free with whatever they needed to do the work. I'd hold off on
walk-throughs in the early stages though as the tester may attempt to
do something which I hadn't thought of which might take the game in an
interesting new direction. If I supply a walkthrough too early it
might narrow their focus too much. But I would provide it once the
direction of the game had been sorted out.
Al
It is now 2 weeks into the testing process and I've asked the author
for a
walkthru. There "appears" to be no way to finish the game but I'm
still trying.
He's promised me a copy but I don't when he'll send it. We had a
discussion
and I told him that I was more interested in getting the maps drawn,
the bugs
removed and typos then actually "playing" the game so to speak. There
is
a diffeerrence between play testing and debugging IMHO.
Andrew Owen
> It is now 2 weeks into the testing process and I've asked the author
> for a walkthru. There "appears" to be no way to finish the game but I'm
> still trying.
More likely there's one specific way to finish the game that you
haven't discovered yet. It's not possible to cover every eventuality
but one reason for beta testing is so that beta testers can try novel
solutions to problems, which then get added in to the game.
> He's promised me a copy but I don't when he'll send it. We had a
> discussion and I told him that I was more interested in getting the
> maps drawn, the bugs removed and typos then actually "playing"
> the game so to speak.
Well if you were beta testing my game I'd have given you the maps from
the start. I think by the time it gets to beta testing most of the
bugs should be gone already (and in my case typos as well since I'm
working with another author).
> There is a diffeerrence between play testing and debugging IMHO.
I agree. But I wouldn't expect someone to beta test my game for free
unless they got some enjoyment out of it.
John H.
> It's the author's choice. If he doesn't think you need those things,
> or doesn't trust you, he's entitled to withhold them. Maybe he wants
> to see how some of his testers do without extraneous help.
And it's the tester's choice to walk away from the project, abandoning
it as a waste of time because the developer withholds information that
would allow him to do his job a lot faster.
Really good beta testers are thorough. They will go in and do every
likely problematic combination of circumstances in the game. This is
a task in itself, and can distract from trying to play the game
"normally." It's not enough for a tester to just win, he's gotta mess
around to see if he can crash it. Or make it unwinnable: that nearly
requires the tester be told exactly how to win!
I can understand the desire to find out of puzzles are too hard or if
the game can be figured out with no outside knowledge, but that
doesn't sound precisely like searching bugs to me.
- John H.
Emily Short
> I can understand the desire to find out of puzzles are too hard or if
> the game can be figured out with no outside knowledge, but that
> doesn't sound precisely like searching bugs to me.
>
> - John H.
In my experience, it's a good idea to recruit both testers who will
play a game "for real" (that is, as though they were ordinary players)
and those who will pound on everything. Bugs aren't the only thing a
beta-tester can tell you about; bad puzzle design, poor pacing, etc.,
also turn up at the testing phase.
Ryusui
author should have no qualms about dispensing clues when requested.
It's _important_ that testers be left to figure out as much as they
can on their own. A poorly-clued, underimplemented puzzle (of the
variety scattered liberally throughout "Moments Out Of Time: Adventure
Type") will not trigger the same warning bells if confronted with a
walkthrough: an author should treat every request for clues as a
suggested revision, unless their intention is to create a uniquely
opaque, difficult and frustrating game.
Zylon
"Ryusui" <TheR...@gmail.com> wrote in message
news:1185829609.3...@m37g2000prh.googlegroups.com...
>I believe testers shouldn't be given the keys to the kingdom, but an
> author should have no qualms about dispensing clues when requested.
I think you have to trust your testers. If you do, giving them everything
shouldn't matter. If you don't, why are you using that tester in the first
place? There's no valid reason to withhold anything at all if you trust the
person you are working with. You can distribute a walkthrough, for example,
but also say that you hope the tester will use that as a last resort because
part of what you're hoping to see is how people do without it. Withholding
it seems like treating a tester like a child. If someone did that to me (and
they have, although in the graphical games for AGS) I just stop testing. On
the other hand, if it's clear there's a reason, I'm more open. For example,
maybe you make it clear that half of your testers are going to have the
walkthrough and half aren't. Ask them which group they prefer to be a part
of.
Andrew Plotkin
>
> "Ryusui" <TheR...@gmail.com> wrote in message
> news:1185829609.3...@m37g2000prh.googlegroups.com...
> >I believe testers shouldn't be given the keys to the kingdom, but an
> > author should have no qualms about dispensing clues when requested.
>
> I think you have to trust your testers. If you do, giving them everything
> shouldn't matter.
Sure it matters. You can trust somebody's taste, analytical ability,
and judgement, but that doesn't mean they can guess what it's like to
have a first-time reaction. (Emotionally, puzzle-solvingly, etc.)
If I could block out my knowledge of my own work and encounter it "for
the first time", I wouldn't need beta-testers in the first place.
--Z
--
"And Aholibamah bare Jeush, and Jaalam, and Korah: these were the borogoves..."
*
If the Bush administration hasn't thrown you in military prison
without trial, it's for one reason: they don't feel like it. Not
because of the Fifth Amendment.
Zylon
news:f8llca$3ta$1...@reader2.panix.com...
> Sure it matters. You can trust somebody's taste, analytical ability,
> and judgement, but that doesn't mean they can guess what it's like to
> have a first-time reaction. (Emotionally, puzzle-solvingly, etc.)
You mean having a first-time reaction that wasn't anticipated. Using a
walkthrough still allows a first-time reaction to things but what it doesn't
allow is the "struggle" to get to certain points. There's no reason that I
can see to withhold such a thing from a tester that asks for it. By the same
logic you should withhold the hint system as well since if used enough, it
becomes a walkthrough. (Assuming the hint system is fully in place.)
When I say trust the tester, I mean you want somebody who already knows that
just reading a walkthrough may have them miss out on the experience of the
game. You're trusting them to test your game honestly and with the
realization that it is a game and part of the testing is experiencing the
game without relying exclusively on aids. And if your testers do resort to
this rather than trying to solve the game on their own, perhaps that also
tells you something about your game.
I just know that when people are doing me a favor (without pay, on their own
time) I tend not to impose too many restrictions at all, especially with
beta-testers which is different from just having casual users. Your emphasis
is on testing here; that's different from just regular use. If you wanted
beta-users, that's a different thing. People who focus on testing are not
the same kind of people who focus on using. When AGD released the remakes of
King's Quest I and King's Quest II, we had a clear separation of people who
were actually testing for us and those who were just acting as pre-release
users.
Rikard Peterson
"John H." <Joh...@gmail.com> wrote:
> I can understand the desire to find out of puzzles are too hard or if
> the game can be figured out with no outside knowledge, but that
> doesn't sound precisely like searching bugs to me.
Maybe it doesn't, but in my experience that's the most valuable part of
beta testing. As someone else said, testers finding mechanical bugs save
me time and effort (and are quite useful too) but that's still stuff I
could have done myself - it's for the design-level testing that other
people really are needed.
It may differ between game makers, or even between games, but that's the
way I look at it. This thread does remind of the need to communicate
clearly with our testers, though.
Rikard
Andrew Owen
getting a co-author. For IF, I'm convinced that two perspectives are
better than one. I guess it comes down to those two things I'm always
banging on about though... [I should just include this in every
post] ... good story and good writing.
Al
> In my experience, it's a good idea to recruit both testers who will
> play a game "for real" (that is, as though they were ordinary players)
> and those who will pound on everything. Bugs aren't the only thing a
> beta-tester can tell you about; bad puzzle design, poor pacing, etc.,
> also turn up at the testing phase.
I've tried to tell the author where he should make changes and I
ALWAYS
let the author do what she or he wants but it is frustating when said
author
is hard-headed and won't listen to reason.
Ryusui
context-sensitive hints into the game itself.
Adam Thornton
Ryusui <TheR...@gmail.com> wrote:
>Perhaps there should be some sort of compromise, like packaging
>context-sensitive hints into the game itself.
I don't know how much it matters.
My testers have access to my source code. They have an in-game hint
system as well.
As far as I can tell, they don't use #1, and I can't tell whether #2
just sucks, or whether they don't use it either.
Adam
Zylon
news:ru75o4...@quicksilver.fsf.net...
> My testers have access to my source code. They have an in-game hint
> system as well.
Interesting you mention that first point. When I had testers for my graphics
games, I also game them the source. Basically the scripts that drove the
various actions, such as what they could say or do. I found this to be the
most effective for my testers because those that did look often found little
things that I missed. Not so much in terms of bugs, but in terms of things I
didn't think to include at all such as a varying response to a certain
action that they didn't even think to try in the game but that did strike
them when looking at the source.
Blank
You're a volunteer. If it's stopped being enjoyable, or you feel that
you're wasting your time, then stop. It's not a question of either one
of you "being right", but purely a decision about how you feel you can
best spend your valuable leisure time.
jz