IF Beta-Testing Process
David Cornelson
beta-testers. What works and what doesn't?
There seems to be many different styles of testing. I usually get a
script with the testers comments mentioned in notations. I then run
through all of the notations, attempt to decipher their intent, and
make whatever fix I think should be made. Sometimes this is clear and
sometimes I'm clueless what the tester is asking me to do.
The reason I ask is that I may want to add a facility to do issue
tracking on the IF Library website and wondered if the current
free-form methods could ever be improved with such an application.
Talk amongst yourselves....
Jarb
David A. Cornelson
news:1f688a41.01121...@posting.google.com...
> I'd like people to respond with how they go about interacting with
> beta-testers. What works and what doesn't?
>
Jarb
Brian Payne
"David A. Cornelson" <dcorn...@placet.com> wrote in message
news:u1i9v6k...@corp.supernews.com...
I wouldn't say -useless-, but I'd be willing to bet that most
people either a) haven't given it much thought, or b) have other
things to worry about. ;)
As for me, well... since this is an IF newsgroup, and since my
only experience with beta-testers (and beta-testING) has been on the
software development end of things, I figured my opinions probably
aren't worth much.
But, if you insist... :)
It would seem to me that the most effective (if somewhat tedious)
way to go about beta-testing an IF game would be the one you
mentioned -- a transcript of their game with notes in the margin.
Firstly, this would give the author an exact path to follow to
recreate the problem (if there was one); second, there'd be no
confusion as to what part of the game is meant ("You know, right after
the bridge over the lava pit?" "-Which- lava pit?!").
Just my $0.02.
Brian Payne
sofaspud at sofaspud dot org
Adam Thornton
Mike Sousa
> I'd like people to respond with how they go about interacting with
> beta-testers. What works and what doesn't?
>
> <snip>
I find interaction that works is when a beta-tester actually provides
feedback. :)
As an author, I adapt to every style that presents itself, from script
files to free form rants. For me, all feedback *works*.
I do make it a point to reply to every issue they identify (where it
makes sense, obviously) so that they know 1. good find, I'll fix it, 2.
good point, I'll think about it, 3. okay, but I'm not doing anything
about it because ...
I do prefer the script file method, containing the beta-tester's
thoughts as well as pointing out grammar/typo issues, bugs, or missing
text. When I play the role of beta-tester, I supply this script file
method.
As for cycles, I try to minimize that as much as possible, unless of
course, a tester encounters a show-stopping bug. As a result, my
beta-testers usually get their original version plus one more (that
contains all of their feedback plus).
-- Mike
Magnus Olsson
No, it's a very interesting question. I've seen it discussed
before, probably here on the newsgroup, and there have been
articles about it in XYZZYnews, I think.
--
Magnus Olsson (m...@df.lth.se, m...@pobox.com)
------ http://www.pobox.com/~mol ------
J. D. Berry
> As an author, I adapt to every style that presents itself, from script
> files to free form rants. For me, all feedback *works*.
Why, Mike, what beta-tester would ever rant?
Jim
Lewis Raszewski
It depends a lot on what you want to test. I find that the annotated
transcript method works well for finding coding bugs, but a freeform
discussion works a lot better for finding bugs in the narrative; places
where the logic's shotty, the style is off, etc.
--
L. Ross Raszewski
The Johns Hopkins University
Wyman Park 407
"The sun has gone down and the moon has come up, and long ago somebody
left
with the cup. But he's striving and driving, and hugging the turns, and
thinking of someone for whom he still burns." -- Cake, The Distance
Raven Kayce
> I'd like people to respond with how they go about interacting with
> beta-testers. What works and what doesn't?
Strange, I thought I'd already answered this. I haven't yet released
one, but I've done testing on one.
> There seems to be many different styles of testing. I usually get a
> script with the testers comments mentioned in notations. I then run
> through all of the notations, attempt to decipher their intent, and
> make whatever fix I think should be made. Sometimes this is clear and
> sometimes I'm clueless what the tester is asking me to do.
I also went for the "script" route as I had only one tester and
didn't know how talkative that person would be. Having a script is
cool because you really see how they try to play the game over time.
You see where they run against a wall, and maybe you even get some
suggestions to alter the plot from things they try.
The tester had a pretty cool idea. Since I had hard-coded the game to
automatically turn on scripting at startup, he simply wrote his
comments at the game prompt. Of course the parser had complaints about
these comments, but they were immediately next to the game situation.
I liked this idea so much that I added a "remark" command to the game
so my tester didn't have to put up with error messages or
unnecessarily increase the turn cont.
Apart from that, I took apart the whole script and sent a reply
e-mail where I commented on most of the remarks, giving reasons for
what I would do or wouldn't and asked questions about remarks I didn't
understand. He found some pretty severe bugs in my game.
I think this is really a cool method because the player can play the
game pretty much like always and make comments instantaneously and
then forget about these issues again, which is much more fun to play
than having a separate window open for comments; and I have the
information where I immediately know what they refer to.
-- R.
David A. Cornelson
news:3C195ADB...@mediaone.net...
> David Cornelson wrote:
>
> > I'd like people to respond with how they go about interacting with
> > beta-testers. What works and what doesn't?
> >
> > <snip>
>
>
> I find interaction that works is when a beta-tester actually provides
> feedback. :)
>
> As an author, I adapt to every style that presents itself, from script
> files to free form rants. For me, all feedback *works*.
>
> I do make it a point to reply to every issue they identify (where it
> makes sense, obviously) so that they know 1. good find, I'll fix it, 2.
> good point, I'll think about it, 3. okay, but I'm not doing anything
> about it because ...
>
Here is what I was thinking about this subject....
We complain every year that many of the annual competition games are not
tested well enough. Clearly it's the responsibility of the author to have
their games tested and then to respond to those tests as thoroughly as
possible. A few iterations of this can clear up an enormous amount of
game-killing bugs.
On the other hand, as Ross has pointed out, a discussion of a game can also
lead to information on grammar, plot, story, and any number of things that
would not necessarily be addressed in a script file.
Anyway, my point isn't to dredge up what probably is a repeated thread from
days past.
I think beta-testing is a skill that not everyone has either inherently or
from experience. Certainly if you're not a programmer, you wouldn't know
about the detailed test plans that get created for the complex systems that
get developed. A writer undertands the editorial and rewriting process, but
do they really see that as the same thing as regression testing?
What I'm asking is it too much to build a testing application that allows
the author to create a test plan and for testers to either use it and/or add
to it?
In creating test plans for normal software development, we write enormously
tedious lists of actions, expected results, with reports of those expected
results either ocurring or something else happening.
In IF we have the dilemma of millions of potential intersections of
'actions' so a traditional test plan is likely out of the question. But
certainly there are 'main intersections' that could be test-planned and then
'scenes' or 'areas' that could be tested for cohesiveness.
So instead of a list of actions and results, the author writes:
: The following locations make up Scene A. Scene A has
: three major intersections in the game that revolve around
: x, y, and z.
So now the tester knows to pound on Scene A for the main intersections
(which are loosely described plot-lines or whatever).
Another tool that would be useful to the tester is a list of movable or
usable objects and defintions, much like a data model has a list of tables,
columns, and datatypes.
wrench:
original location: maintenance room
main uses: to break the glass in the toy shop
spoon:
original location: kitchen
main uses: eat soup, stick on nose to incite laughter, dig a hole very
slowly
It might even be nice to extend this sort of idea to a complete listing of
the game including all objects, what their purposes are, and other pertinent
information.
I wonder how hard it would be to write a library function in any of the main
languages that would spit out a listing of all locations, the objects that
originate there, and anything else we can think of. Sure, some things
programmtically appear due to some action and those types of things would
get left out of the list. But the list would then have a secondary portion
that showed all of the things that have no current location.
Anyway, I think testing is something that isn't done well overall in our
hobby and can be improved upon using the same methods we use for traditional
software development. I think we could probably build some things that would
help and I think we need some essays or documentation on how good testing of
an IF game is accomplished.
Jarb
Andrew Plotkin
> So instead of a list of actions and results, the author writes:
> : The following locations make up Scene A. Scene A has
> : three major intersections in the game that revolve around
> : x, y, and z.
> So now the tester knows to pound on Scene A for the main intersections
> (which are loosely described plot-lines or whatever).
I see a couple of problems:
(1) Beta-testers are not just testing code paths; they're testing
writing, pacing, and so on. A tester with this sort of plan has had
the story spoiled, and therefore can't give you much feedback on
whether the story worked. Nor on stuff like "Hey, I pounded on rooms q
and w for half an hour before I realized they weren't important."
(2) Someone who has a plan will test the plan, but this tells you
nothing about what players who *don't* have a plan will do. Real
players run into bugs in the commands they think of, not the commands
they were supposed to think of.
(3) 90% of the time -- okay, maybe 75% -- if I knew it needed testing,
I tested it. Beta-testers tell me what I *didn't* know needed testing.
--Z
"And Aholibamah bare Jeush, and Jaalam, and Korah: these were the borogoves..."
*
* Make your vote count. Get your vote counted.
David A. Cornelson
news:9vfnft$mgv$1...@news.panix.com...
> I see a couple of problems:
>
> (1) Beta-testers are not just testing code paths; they're testing
> writing, pacing, and so on. A tester with this sort of plan has had
> the story spoiled, and therefore can't give you much feedback on
> whether the story worked. Nor on stuff like "Hey, I pounded on rooms q
> and w for half an hour before I realized they weren't important."
>
> (2) Someone who has a plan will test the plan, but this tells you
> nothing about what players who *don't* have a plan will do. Real
> players run into bugs in the commands they think of, not the commands
> they were supposed to think of.
>
> (3) 90% of the time -- okay, maybe 75% -- if I knew it needed testing,
> I tested it. Beta-testers tell me what I *didn't* know needed testing.
>
I agree with this, but I'm not necessarily limiting whatever tools could be
made to the author/tester relationship. Maybe these tools would be for the
author themselves.
Jarb
Daniel Freas
<dcorn...@placet.com> wrote:
>I agree with this, but I'm not necessarily limiting whatever tools could be
>made to the author/tester relationship. Maybe these tools would be for the
>author themselves.
>
>Jarb
I lack the knowledge myself to do this but I don't imagine it would be
very hard to create a program that knew all of the acceptable commands
for a given game such as n, s, e, w, get, throw, etc. Such a program
could be extremely useful if it were told to run through every
possible combination and look for certain problems. For instance it
could be fitted with a spelling/grammar checker that scanned all
output text and made a list of errors. (That particular function would
be a *huge* help to me) It could also be told to log any game crashing
errors.
While I imagine any such program would not be a complete solution and
would also most likely take several hours to run through a given game
I think it would definately be worthwhile. Just the assurance that any
game I release won't crash would make me feel a lot better. It would
also make beta testing run a bit more smoothly as the testers could
focus on more subtle errors rather than reporting game crashes and
making long lists of spelling mistakes.
Of course puzzles present a bit of a problem since no program could be
expected to solve even the simplest of puzzles. Still in many cases it
isn't hard at all for the author of the game to automatically solve
all of the puzzles with a few lines of code (I did this when testing
pieces of my last game so I wouldn't have to go through all the work
of getting to the end of the game to test one tiny change). Or the
program could even be set up so that it would pause itself when it
finished all possible combinations and allow the author to take over
and play a few moves to solve the next puzzle and then hand control
back over to the program to run combinations on the game in its new
state.
Anyway I don't know how much work something like that would be and I
certainly don't know how to do it myself. I'd be very grateful if
someone did create it though.
---Daniel
David A. Cornelson
news:3c1b8783.2749233@news...
> On Sat, 15 Dec 2001 09:51:21 -0600, "David A. Cornelson"
> <dcorn...@placet.com> wrote:
>
> >I agree with this, but I'm not necessarily limiting whatever tools could
be
> >made to the author/tester relationship. Maybe these tools would be for
the
> >author themselves.
> >
> >Jarb
>
> I lack the knowledge myself to do this but I don't imagine it would be
> very hard to create a program that knew all of the acceptable commands
> for a given game such as n, s, e, w, get, throw, etc. Such a program
> could be extremely useful if it were told to run through every
> possible combination and look for certain problems. For instance it
> could be fitted with a spelling/grammar checker that scanned all
> output text and made a list of errors. (That particular function would
> be a *huge* help to me) It could also be told to log any game crashing
> errors.
>
the save/restore routine to complete the task throughout the game. It would
probably take forever to execute, but once it was functional, there may be
ways to optimize it.
Jarb
Gregg V. Carroll
> I lack the knowledge myself to do this but I don't imagine it would be
> very hard to create a program that knew all of the acceptable commands
> for a given game such as n, s, e, w, get, throw, etc. Such a program
> could be extremely useful if it were told to run through every
> possible combination and look for certain problems. For instance it
> could be fitted with a spelling/grammar checker that scanned all
> output text and made a list of errors. (That particular function would
> be a *huge* help to me) It could also be told to log any game crashing
> errors.
I would think that you're probably going to be stuck with using a word
processor for the last bit there. Seems to me that grammar checking would be
a fairly complex thing to program, and it's too widely available in
commercial software for someone to go to the trouble of integrating it with
a specialized program for a small audience. Maybe if you had MSWord on your
drive, such an app could access its preexisting libraries to perform such a
task, but even that might be difficult, given platform differences and
variations across versions of Word and so on.
I don't know what other people do, but I just write all my text in MSWord,
and give it a short, meaningful title, and save it all in one big document.
Then just perform the checks, and do the old copy-paste routine. Kind of
tedious, but not that bad. As long as you type something like:
FLASHLIGHT
It is a small, palm-sized flashlight yadda yadda yadda...
you can just do a search on "FLASHLIGHT" in the document to find the text,
so it doesn't matter if it's all in order, or if item and room descriptions
are all mixed up and so on. The search may hit the word "flashlight" early
on in another description, but if you use caps in the title, and make the
search case-sensitive, you can work around that.
As far as writing a program to test IF for problems, I don't know. I suppose
you could make something that tested the validity of the map, to make sure
that paths lead back to where they originated from, but you'd have to make
sure it wouldn't flag a room with a twisted pathway (ex. you leave from the
north, but in the next room you have to go southwest to get back to where
you started from), or got hung up on one-way passages, which you'd probably
have to manually override, since there's no way for a program to
differentiate between a one-way passage and a programming error. And then
there's the question of exits that are dependent on methods, which could
really complicate things.
It's just too complex, you need the human element, I think. As far as coming
up with a list of things for human beta-testers to check, that's more
practical. I've never beta-tested myself, but I would think it's a bit like
working for Consumer Reports. You get this really cool thing delivered to
you, and you're job is to first make sure it works on a technical level, and
then you try an break it. You take an NPC, and do the equivalent of taking a
laptop and putting it in a freezer, and see if it still works. :)
Gregg
Magnus Olsson
David A. Cornelson <dcorn...@placet.com> wrote:
>"Daniel Freas" <danie...@hotmail.com> wrote in message
>news:3c1b8783.2749233@news...
>> very hard to create a program that knew all of the acceptable commands
>> for a given game such as n, s, e, w, get, throw, etc. Such a program
>> could be extremely useful if it were told to run through every
>> possible combination and look for certain problems. For instance it
>> could be fitted with a spelling/grammar checker that scanned all
>> output text and made a list of errors. (That particular function would
>> be a *huge* help to me) It could also be told to log any game crashing
>> errors.
>>
>Interesting. I wonder if a library routine could be made to do this and use
>the save/restore routine to complete the task throughout the game. It would
>probably take forever to execute, but once it was functional, there may be
>ways to optimize it.
This would give you the mother of all combinatorial explosions.
How many possible commands are there in any given game state for a
typical game? Hundreds, I suppose (if you count each verb/noun
combinaion as one command). I fthere are just 100 possible commands,
this means that there are 100 possible one-move "games", 10000
two-move games, a million three-move games,
100000000000000000000000000000000000000 twenty-move games, and
so on.
So exhaustive testing is out of the question. Or did you mean
just exhaustive testing of all possible commands in a given game
state? That would be useful for finding bugs triggered by strange
commands such as "blow on cow", but in my experience most bugs don't
manifest themselves so easily.
Daniel Freas
<gr...@midcoast.com> wrote:
>
>I would think that you're probably going to be stuck with using a word
>processor for the last bit there. Seems to me that grammar checking would be
>a fairly complex thing to program, and it's too widely available in
>commercial software for someone to go to the trouble of integrating it with
>a specialized program for a small audience. Maybe if you had MSWord on your
>drive, such an app could access its preexisting libraries to perform such a
>task, but even that might be difficult, given platform differences and
>variations across versions of Word and so on.
Well, even if the program just wrote everything it did to a special
script file that contained only game generated output with none of the
command line interaction then that script file could be run through an
existing spell checker manually by the author. That's much better than
trying to go through a code file to find all the instances of text and
check them one by one.
>It's just too complex, you need the human element, I think.
It's only complex if you expect the program to find *all* errors. But
you could easily start with something simple and build up. For
instance make a program that crawls through a game map using a
combination of undo and restart to find every possible location and
just make a map and write a text file containing room descriptions.
You could then add to this by telling the program to type X OBJECT
replacing object with each word that shows up in the current room's
description. You could even optimize that by telling it to leave out
words like "the" "and" and the like. As the program became more
advanced it could start taking objects and carrying them around, using
verbs like open and break on every word it sees in descriptions etc.
Obviously it couldn't replace human beta testers but it would help
authors catch a lot of the more obvious bugs so that beta testing
wasn't quite so tedious. Also if it could be done well it would
virtually ensure that future comp games (or any other games for that
matter) wouldn't have any terrible game crashing bugs. They might
still have bugs where secret compartments open and "Inside the secret
compartment you see ." but that's better than having a game crash
when you try >TAKE BARS
>Gregg
>
---Daniel
Daniel Freas
>This would give you the mother of all combinatorial explosions.
Ok, well how about this: write the program to be language specific and
then instead of feeding it a compiled game give it the non-compiled
source code. It can run through all the combinations but only restart
when it detects a game state change. That would put a bit of a cap on
the conbinational explosion.
---Daniel
Adam Thornton
Daniel Freas <danie...@hotmail.com> wrote:
>I lack the knowledge myself to do this but I don't imagine it would be
>very hard to create a program that knew all of the acceptable commands
>for a given game such as n, s, e, w, get, throw, etc. Such a program
>could be extremely useful if it were told to run through every
>possible combination and look for certain problems. For instance it
>could be fitted with a spelling/grammar checker that scanned all
>output text and made a list of errors. (That particular function would
>be a *huge* help to me) It could also be told to log any game crashing
>errors.
Two words: combinatorial explosion.
>While I imagine any such program would not be a complete solution and
>would also most likely take several hours to run through a given game
s/hours/trillion millenia/;
Now, a stripped-down subset of this, exercising all exits from each
room, perhaps, might be doable, and possibly even useful.
Adam
ka...@plover.net
: Well, even if the program just wrote everything it did to a special
: script file that contained only game generated output with none of the
: command line interaction then that script file could be run through an
: existing spell checker manually by the author. That's much better than
: trying to go through a code file to find all the instances of text and
: check them one by one.
Even easier (if you're using Inform, anyway) is to use the Inform
compiler's '-r' flag, which outputs all text from the game file to
"gametext.txt". Much easier to run a spellchecker on.
I don't use TADS or Hugo, so I can't say whether they have anything
similar. I'm willing to assume someone will jump in and say so, tho. :)
katre
LoneCleric
>Even easier (if you're using Inform, anyway) is to use the Inform
>compiler's '-r' flag, which outputs all text from the game file to
>"gametext.txt". Much easier to run a spellchecker on.
>
With the Hugo compiler, the '-t' flag will copy that text into the
".lst" stats file.
>I don't use TADS or Hugo, so I can't say whether they have anything
>similar. I'm willing to assume someone will jump in and say so, tho. :)
>
So, was this a nice jump? :-)
LC
Daniel Freas
wrote:
>Two words: combinatorial explosion.
>
>>While I imagine any such program would not be a complete solution and
>>would also most likely take several hours to run through a given game
>
>s/hours/trillion millenia/;
>
>Now, a stripped-down subset of this, exercising all exits from each
>room, perhaps, might be doable, and possibly even useful.
Ok how about this then: solve all the problems involving game state
changes and combination explosions by simply making the program stay
in one room. The author of the game goes to a certain room and then
runs the program whereupon the program looks around the room and
starts examining throwing taking pushing pulling etc. This way the
author can go through the game solving puzzles and getting the game in
the proper state and then the program does the grunt work of trying
combinations. Since most game commands only involve a verb/noun
combination the available amount of combinations in any given room
isn't all that large.
Other programs could be written that do a mapping crawlthrough to test
exit validity or run through conversations with npcs or whatever. One
single program need not do everything.
---Daniel
David A. Cornelson
news:9vgfk6$n2b$1...@news.fsf.net...
>
> s/hours/trillion millenia/;
>
> Now, a stripped-down subset of this, exercising all exits from each
> room, perhaps, might be doable, and possibly even useful.
Well I wasn't thinking of trying all verb noun combinations...what I was
thinking was that the author could load the process with their own verbs and
sentences, limiting the regression to the most important tasks. I wasn't
thinking of some mapping AI program either. I was thinking simply of a
reporting process that spits out text and maybe in a format loadable into
Excel or something.
We have pretty powerful computers and granted, the millenia's worth of
combinations is not doable, but once the process is created it could be
refined to accomplish very useful tasks. Fine-tuning and such is probably
critical since we really don't know what our output would look like and how
we might even review the output.
So one program would first list all locations, objects, and available
grammar. A second program would use this to regression test throughout all
states of a game, but the author can remove items from that first list.
Jarb
T Raymond
> I wonder how hard it would be to write a library function in any
> of the main languages that would spit out a listing of all
> locations, the objects that originate there, and anything else we
> can think of. Sure, some things programmtically appear due to
> some action and those types of things would get left out of the
> list. But the list would then have a secondary portion that
> showed all of the things that have no current location.
FWIW, I use Informapper to do design/map layout and add in objects to
the room list. That list is printable, and helps me work things out.
The only thing it hasn't managed to help me do is finish something ;)
> Anyway, I think testing is something that isn't done well overall
> in our hobby and can be improved upon using the same methods we
> use for traditional software development. I think we could
> probably build some things that would help and I think we need
> some essays or documentation on how good testing of an IF game is
> accomplished.
Or we could maybe update or rebuild a tool that's already around to
make it more useful? Then again, I mentioned that in the thread for
mapping tools already.
Tom
--
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Tom Raymond af956 AT osfnDOTorg
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
-----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
-----== Over 80,000 Newsgroups - 16 Different Servers! =-----
ally
Mike Sousa
> very hard to create a program that knew all of the acceptable commands
> for a given game such as n, s, e, w, get, throw, etc. Such a program
> could be extremely useful if it were told to run through every
> possible combination and look for certain problems. For instance it
> could be fitted with a spelling/grammar checker that scanned all
> output text and made a list of errors. (That particular function would
> be a *huge* help to me) It could also be told to log any game crashing
> errors.
I use script files containing commands that I feed to the game. I keep
two sets of script files -- one that is the walkthrough and one that is
used to try everything. It's always a work in progress. That way when
I change the code, I have a quick way of knowing if I messed up the game
path. It obviously doesn't "run through every possible combination",
but it runs through every obvious combination.
For instance, let's say I add a sofa to a room. Once I code the item, I
switch windows to my script file and start:
>sit on sofa
>g
>get up
>x sofa
>look under sofa
>get sofa
>stand on sofa
>move sofa
>search sofa
etc...
(not to bore you to death, but I have taken it a step further by
defining macros in VSlick that create templates depending on what type
of object it is, but I digress...)
As you can imagine, the script files gets awfully big. I usually try to
break them up logically. After I execute these script files, I capture
the output and then run it through spell checker, followed by grammar
checker (the latter usually causes a memory overload on my PC).
This process tends to work for me. That's another reason why I like
script files from beta-testers -- I incorporate their game play into my
development script files.
Brain dump over...
Lewis Raszewski
>
> Well, even if the program just wrote everything it did to a special
> script file that contained only game generated output with none of the
> command line interaction then that script file could be run through an
> existing spell checker manually by the author. That's much better than
> trying to go through a code file to find all the instances of text and
> check them one by one.
>
However, as most if not all existing IF development systems have the
ability to extract the entire game text to a file, this hypothetical
project would be redundtant in that respect.
Also, if you tried to extract the text via those sorts of means, you'd
very likely get several billion pages of text, most of it redundant.
Now, what might be very nice would be if the text-extraction components
of, say, inform, could be asked to annotate the dump with file and line
numbers; God knows I've run gametext.txt through the spellchecker more
than a few times only to find that I;ve misspelled a word, with no idea
of where that word actually occurs.
--
L. Ross Raszewski
The Johns Hopkins University
Wyman Park 407
"You know if I leave you now, it doesn't mean that I love you any less;
it's
just the state I'm in, I can't be good to anyone else like this." --
Sarah
McLachlan, Fear
ka...@plover.net
: Now, what might be very nice would be if the text-extraction components
: of, say, inform, could be asked to annotate the dump with file and line
: numbers; God knows I've run gametext.txt through the spellchecker more
: than a few times only to find that I;ve misspelled a word, with no idea
: of where that word actually occurs.
Well, the good thing here is that you can just use your editor to search
the file for every occurance of the word 'teh'. Not like it's going to
be right once and wrong once. :)
katre
Jon Ingold
division into "Bugs" and "Comments". The quality is really in the tester -
the things that they try, the patience with which they deduce the game's
logic and then try to bang it right back at it.
[A good example: BLOW ON COW was found by a tester who deduced from another
context the way the game responded to Blow, and then went around trying it
on everything which might produce strange answers. Well - okay, I *assume*
that's how he found it.]
The problem with scripts, I find, is that in real beta-testing the tester is
*both* playing the game as a player might (and so coming across alternate
solutions, missing responses and such like) but they're also trying to
deliberately break it; and then needs a lot of UNDO's, RESTART's, and
general highly organised attack methods.
What's needed is just a nice long list of incredibly pedantic people with
lots of free time who like breaking things; as this seems to be the Ideal
Situation.
Jon
Stephen Granade
No need for that: the tads compiler will dump all strings to a file
with the -Fs <file> flag.
Stephen
--
Stephen Granade
sgra...@phy.duke.edu
Duke University, Physics Dept
Daniel Freas
wrote:
>No need for that: the tads compiler will dump all strings to a file
>with the -Fs <file> flag.
Thanks! I didn't know that....I'll definately be making use of that
feature from now on!!
---Daniel
Magnus Olsson
Jon Ingold <ji...@cam.ac.uk> wrote:
>[A good example: BLOW ON COW was found by a tester who deduced from another
>context the way the game responded to Blow, and then went around trying it
>on everything which might produce strange answers. Well - okay, I *assume*
>that's how he found it.]
WHat game was this in, and what actually happened when he blew on the
cow?
Kathleen
> The tester had a pretty cool idea. Since I had hard-coded the game to
> automatically turn on scripting at startup, he simply wrote his
> comments at the game prompt. Of course the parser had complaints about
> these comments, but they were immediately next to the game situation.
> I liked this idea so much that I added a "remark" command to the game
> so my tester didn't have to put up with error messages or
> unnecessarily increase the turn cont.
I've done the same (as a meta command) and I believe the command
remains even when debugging is off, so to this day in _Prized
Possession_ you can type 'C'/'COMMENT' at the prompt. I also find it
useful to print out a confirmation message, something like:
>LOOK
You sea a cow hear.
>C sea -> see, hear -> here
[ Noted. ]
This way, I can search the transcript for "[ Noted. ]"
Kathleen
-- Prized Possession
-- http://www.ifarchive.org/if-archive/games/competition2001/inform/possess
-- Masquerade - http://www.ifarchive.org/if-archive/games/inform/Mask.z5
-- The Cove - http://www.ifarchive.org/if-archive/games/inform/Cove.z5
-- Excuse me while I dance a little jig of despair
Kathleen
> The best testing I've ever had hasn't really had a format, exactly; a brief
> division into "Bugs" and "Comments". The quality is really in the tester -
> the things that they try, the patience with which they deduce the game's
> logic and then try to bang it right back at it.
>
> lots of free time who like breaking things; as this seems to be the Ideal
> Situation.
I posted on this about this time last year (2000-12-12). Thought I
would dust it off and trot it out again (fixing spelling error):
---
When beta testing, I discovered (somewhat the hard way) that there are
at least 5 distinct types of beta testers:
1. GRAMMAR GURUS: Know the difference between its and it's and have
the patience to pick out each instance. Can spot a spelling error
at a thousand yards.
2. PLOT PICKERS: Unresolved plot threads? Missing/unfulfilling
endings? These types can pick apart the best laid plot with
heartless efficiency.
3. PUZZLERS: These are sorts that rightly feel you should be able
to unscrew the control panel lid with your screwdriver (thus bypassing
the lock), pick up every item that isn't nailed down and try it out on
every other item in sight, etc.
4. UN-PUZZLERS: Have an uncanny ability to be confused by the
simplest task, are unable to locate the screwdriver (let alone realize
it should be used on the Big Red Screw). Easily felled by
Guess-The-Verb puzzles. These types will do wonders for your help
system.
5. COW BLOWERS: These are the sorts that try to eat furniture, sit
on the sofa, talk to the wall, take 1000 tissues from the tissue
box... etc. All games need at least ONE tester of this sort. The
weirder, the better.
The games with the fewest errors were probably the games whose
author/testers best filled those catagories and had enough time for
test/fix/retest cycles. :)
----
#5 are also (and more properly) known as PARROT SCRAPERS.
Kathleen (a #4)
Mary K. Kuhner
>Well, even if the program just wrote everything it did to a special
>script file that contained only game generated output with none of the
>command line interaction then that script file could be run through an
>existing spell checker manually by the author. That's much better than
>trying to go through a code file to find all the instances of text and
>check them one by one.
Hm. It seems to me that a Perl script to capture everything
in double quotes and dump it to a file would be
dead easy, and then you would have *all* the text, once each,
instead of some proportion of the text over and over, with the rest
missing. (This doesn't work for highly random-generated text,
but that's fairly rare in IF.)
I also think that I would not like to look at the results of
trying "x <every non-trivial word in the room description>". I
think the false alarm rate would be astonishing; in most room
descriptions there are about five non-trivial but non-object
words (mainly verbs, adverbs and adjectives) for each object, I'd
think.
"A red rock angles up from the burning sand, its shadow the only
relief from the noonday sun. All around you the desert spreads
to a mathematically precise horizon. Perhaps the gray smudge
far to the west might be hills, or clouds."
Okay, not as many as I thought: it's legit (in the game this
came from) to examine red, rock, sand, shadow, sun, desert,
horizon, smudge, hills, and clouds. Ten objects out of maybe
twenty-two non-trivial words. But I was trying to be rather
thorough.
On the other hand, there's no doubt that beta-testers get cranky
if given a really buggy game. Some automation would certainly
help, but I'd put more faith in the Perl script for spelling.
If MS Word is any indication, nothing automated will help much
with grammar. You could test for crashes, but most crashes don't
come from totally trivial actions (though there have been a
few memorable exceptions in the Comp)--they come from things
like my personal favorite, making a pants pocket that you can
put things into, and then putting the pants into their own
pocket....
Mary Kuhner mkku...@eskimo.com
Jon Ingold
Magnus Olsson <m...@df.lth.se> wrote in message
news:9vkqsh$9be$1...@news.lth.se...
> In article <9vihis$su0$1...@pegasus.csx.cam.ac.uk>,
> Jon Ingold <ji...@cam.ac.uk> wrote:
> >[A good example: BLOW ON COW was found by a tester who deduced from
another
> >context the way the game responded to Blow, and then went around
trying it
> >on everything which might produce strange answers. Well - okay, I
*assume*
> >that's how he found it.]
>
> WHat game was this in, and what actually happened when he blew on the
> cow?
The response was:
"I blow on the cow, as though imbuing it with luck."
I suspect the game is obvious. ;)
Jon
Kathleen M. Fischer
Mailgate, MailAndNews, or AOL. Wierd.
Kathleen (reposting yet again, feeling a tad foolish as it's really not that
important.)
Kathleen (a #4)
-- Prized Possession (Comp2001)
--
http://www.ifarchive.org/if-archive/games/competition2001/inform/possess
-- Masquerade (Comp2000, nominated for Best Story (XYZZY's))
-- http://www.ifarchive.org/if-archive/games/inform/Mask.z5
Bryan Miller
>>No need for that: the tads compiler will dump all strings to a file
>>with the -Fs <file> flag.
>
>Thanks! I didn't know that....I'll definately be making use of that
>feature from now on!!
>
>---Daniel
I'm new to TADS, and haven't even read a lot of the manual, so I guess I'm
kinda going out on a limb here, but I THINK the tads interpreter <tr32.exe>
can output the commands you use while testing to a file by using the -o
filename switch. I know you can also log game output that is produced by an
input script file by using the -l switch (I think... I'll be corrected if
wrong :)
Daniel Freas
How does one go about making an input script file? I've never even
heard of it before but I'm assuming it's something that runs a game
through a predetermined list of commands, correct? It sounds like a
great tool, if anyone knows of a webpage somewhere with good
instructions on how to go about creating such things I'd appreciate
the link.
---Daniel
Mike Sousa
Daniel Freas wrote:
> How does one go about making an input script file? I've never even
> heard of it before but I'm assuming it's something that runs a game
> through a predetermined list of commands, correct? It sounds like a
> great tool, if anyone knows of a webpage somewhere with good
> instructions on how to go about creating such things I'd appreciate
> the link.
>
> ---Daniel
It's actually pretty straight forward. Create an ASCII file that has a
">" in column 1 followed by a valid TADS command. (you can also
accomplish this by using the -O option described earlier in this thread)
For instance, your file will look like:
>LOOK
>GET KEY
>UNLOCK DOOR WITH KEY
When you want to regression test your game, use the -I option to 'feed'
your game the predetermined list of commands and the -L option to have
all subsequent output logged to another file.
For more information on those options consult the TADS manual or execute
the command line version of the TR app with no parms. I don't use the
workbench so I don't know if there are drop downs for this.
-- Mike
Mike Sousa
> Mike Sousa <mjs...@mediaone.net> wrote
>
>
>>As an author, I adapt to every style that presents itself, from script
>>files to free form rants. For me, all feedback *works*.
>>
>
> Why, Mike, what beta-tester would ever rant?
>
> Jim
>
I *still* haven't finished reading your rants, er, beta reports for No
Time To Squeal. I just hope you didn't warn me about something that I
never got around to implementing, like getting rid of that
restore/restart thing. :)
L. Ross Raszewski
>
>I've done the same (as a meta command) and I believe the command
>remains even when debugging is off, so to this day in _Prized
>Possession_ you can type 'C'/'COMMENT' at the prompt. I also find it
>useful to print out a confirmation message, something like:
>
>>LOOK
>You sea a cow hear.
>
>>C sea -> see, hear -> here
>[ Noted. ]
>
>This way, I can search the transcript for "[ Noted. ]"
>
That's neat. I went part of the way with moments, adding the command
'[', but I'm not even sure I ever set it as a metaverb.
Jon Ingold
> remains even when debugging is off, so to this day in _Prized
> Possession_ you can type 'C'/'COMMENT' at the prompt. I also find it
> useful to print out a confirmation message, something like:
I do this, but by catching a "*" at the start of a line in
BeforeParsing. Don't know why, really, a meta-verb would be far easier.
Jon
Jon Ingold
>
horses',
> IIRC.
I put that in deliberately as an in-joke, of course.
Jon