Tool to aid beta testing
Plugh!
the following was said & I think it's such a good idea that I'd like
to see more discussion of it.
Sure, it's a complicated idea & it'll never be 100%, but if it could
be properly specified (by this newfroup), then something could be
implemented...
> I'm mostly thinking about the biggest complaint I got when I gave my first
> game to a couple of beta testers. Missing objects and missing interactions.
> Somehow those need to be made more obvious (or even automated) in the way
> interactions are coded. I got complaints like the description of the room
> mentioned a rug but when the player types "x rug" it doesn't know the word
> "rug". Or when I created a pool of water but forgot descriptions of what
> happens for couple of possible interactions like "throw bread in water",
> which I never thought anyone would do.
"Cast your bread upon the waters, and it will return a thousand fold".
All well & good, but who wants a thousand loaves of soggy bread?
> I'd like some kind of format that would make such oversights obvious at a
> glance.
hmmm, the best way towards that is to be in the habit of never using
plain text words for descriptions. "There is a gorgeous " +
rug.adjective() + rug.noun() + "spread in front of the " +
fireplace.noun() ... might be easier to automate.
Otherwise, we're talking about a large dictionary which knows the
20,00 most common objects in the ?english? language (or maybe we could
get away with 1,k
or so).
Even there, it'd be tricky to automatically check that that you had
handled all possible fire interactions, since you mention a fireplace
not a fire. The program would have to know that fireplaces contain
fire. And what if you refer to it as a grate?
I still think that it's a desirable idea; I'm just pointing out
difficulties. It could be doable if authors would realise that it may
never catch 100%. However, there are some commonly understood
side-effects/pitfalls when dealing with fire/flammable obejcts, or
even scenery - "burn rug/ignite curtain (the flames spread throught
the library, consuming all in their path, including you)" and
liquids/containers "pour oil (for rusty door) onto fire"/"mix oil &
water" / "pour water onto fire" may extinhuish it, pitching the room
into darkness, etc, etc. If we could come up with some list of these,
a tool might be specified
> Or maybe what I want is a "dignostic scan" program to scan the source code
> and compile a table of all possible interactions of verbs with object and
> mark the ones that seem to be missing.
Now, that might be easier. Especially if it allows you mark some as
"ignore". You could whittle away at them until no unresolved cases
remain.
Gary Shannon
"Plugh!" <pl...@subdimension.com> wrote in message
news:98ef019f.02021...@posting.google.com...
<snip>
> > Or maybe what I want is a "dignostic scan" program to scan the source
code
> > and compile a table of all possible interactions of verbs with object
and
> > mark the ones that seem to be missing.
>
> Now, that might be easier. Especially if it allows you mark some as
> "ignore". You could whittle away at them until no unresolved cases
> remain.
Addressing just the easy part of this question, it should not be too
difficult to write some kind of a scan which, given the syntax of the
language, could produce a list of verbs classified as to whether they take
direct and indirect objects or not, and then produce a list of all the
possible interactions a player might enter.
Verbs which take no object can be disregarded since errors there will be
caught by the parser if the player tries to put an object with it: "n key".
Verbs that take a direct object only can be easily listed and checked
against programmed responses. It seems that the usual "I don't know how to
burn bread." covers the situation, though not in a particularly elegent
manner. Even so, it might be useful to see an enumeration of all the
possible responses given the defined d.o. verbs and objects.
Numerically, my first game, Dwenodon, has 26 objects the player can take and
a total of 67 objects including fixed items. I also use 47 verbs which take
a direct object. That gives me a total of 47 * 67 = 3149 possible two-word
utterances the game is capable of responding to. It would be trivial for a
scan program to generate this list, but either the author has to pour
through this entire list of 3149 generated responses or some set of criteria
has to be devised for judging the correctness of each reponse.
Now we look at the case where the verb takes an indirect object as well. In
my game I have 34 such verbs (even though about a third of these are defined
in adv.t and never actually used in my game) Now since I can use any of the
67 nouns in either place, including duplicates like "hit hammer with hammer"
there are 67 * 67 * 34 = 152,626 cases to consider. (That's 2775
single-spaced typwritten pages of responses!) Clearly just generating a list
is not acceptable. Some kind of judging criteria must be programmed to pare
down this list to managable size. In other words, the scan program would
have to have some common sense knowledge of how things work in the real
world to be able to recognize which responses made sense and which did not.
This could be a very difficult problem to solve. Is there a different way to
approach the problem?
--gary
Stuart Allen
> difficulties. It could be doable if authors would realise that it may
> never catch 100%. However, there are some commonly understood
> side-effects/pitfalls when dealing with fire/flammable obejcts, or
> even scenery - "burn rug/ignite curtain (the flames spread throught
> the library, consuming all in their path, including you)" and
> liquids/containers "pour oil (for rusty door) onto fire"/"mix oil &
> water" / "pour water onto fire" may extinhuish it, pitching the room
> into darkness, etc, etc. If we could come up with some list of these,
> a tool might be specified
Although I too think a beta testing tool may have its place,
particularly for regression testing of certain critical actions. I
think a lot of what you have mentioned above, however, should be
handled by the standard library. I personally feel that the time spent
making this debugging tool could be better used making the library
have a greater range of standard responses. The only way to achieve
this is to have a richer set of properties that can be given to
objects and subsequently tested for by all verbs. As you say, effects
like the above are commonly understood, so they should all happen
automatically without author doing anything more than giving the oil
LIQUID and FLAMMABLE, the fire place FLAMMABLE, BURNING and LUMINOUS,
and the water LIQUID. Of course when people fail to give objects all
the required attributes, you are right back where you started... ;)
Perhaps a library of common, well-coded objects is the answer. Then it
would beta tester's main job would be to see if the default action
sits well within the context of the game.
Stuart
Gary Shannon
"Stuart Allen" <stu...@jacl.animats.net> wrote in message
news:b2ce9981.02021...@posting.google.com...
> pl...@subdimension.com (Plugh!) wrote in message
<snip>
> Although I too think a beta testing tool may have its place,
> particularly for regression testing of certain critical actions. I
> think a lot of what you have mentioned above, however, should be
> handled by the standard library. I personally feel that the time spent
> making this debugging tool could be better used making the library
> have a greater range of standard responses. The only way to achieve
> this is to have a richer set of properties that can be given to
> objects and subsequently tested for by all verbs. As you say, effects
> like the above are commonly understood, so they should all happen
> automatically without author doing anything more than giving the oil
> LIQUID and FLAMMABLE, the fire place FLAMMABLE, BURNING and LUMINOUS,
> and the water LIQUID. Of course when people fail to give objects all
> the required attributes, you are right back where you started... ;)
> Perhaps a library of common, well-coded objects is the answer. Then it
> would beta tester's main job would be to see if the default action
> sits well within the context of the game.
>
> Stuart
There should, however, be a way to prune unnecessary items from the library
to prevent the finaly game file from being unnecesarily large. There are
twice as many verbs in the TADS library than I used in my first game, and I
can't help but wonder how much wasted space they occupy in my game file.
--gary
Jim Aikin
> This could be a very difficult problem to solve. Is there a different way to
> approach the problem?
The most direct way is for the author of the game to pay close attention
to what's in his/her room descriptions, and add scenery objects
accordingly. This is not hard to do, it's just time-consuming.
You can read a seven-line paragraph and understand its implications in
terms of your model world a heck of a lot faster than you can (a) run a
diagnostic tool that searches for nouns in specified text blocks and
then (b) sort through the long list it generates, referring back to your
room descriptions at every step to find out what sort of errors or
omissions it thinks it has spotted. That's my opinion, anyhow.
This whole line of inquiry strikes me as yet another of those threads
that pop up on raif from time to time, all of which can be lumped under
the general heading, "How can I get a computer tool that will make it
easy for me to write my dream adventure game without actually having to
think or do any hard work?"
Me, I'd settle for more informative compiler error messages and more
flexible verb syntax.
--Jim Aikin
**********************************
"Those instances of it which lack
the quality referred to as 'swing'
are meaningless." --Duke Ellington
**********************************
Gary Shannon
"Jim Aikin" <kill_spammers@kill_spammers.org> wrote in message
news:3C6C860E.1000206@kill_spammers.org...
> Gary Shannon wrote:
>
>
> > This could be a very difficult problem to solve. Is there a different
way to
> > approach the problem?
<snip>
> This whole line of inquiry strikes me as yet another of those threads
> that pop up on raif from time to time, all of which can be lumped under
> the general heading, "How can I get a computer tool that will make it
> easy for me to write my dream adventure game without actually having to
> think or do any hard work?"
>
> Me, I'd settle for more informative compiler error messages and more
> flexible verb syntax.
>
> --Jim Aikin
>
Well put. You've convinced me of the futility of pursuing the idea any
further. Besides, I'd rather be writing my second IF adventure anyway. ;-)
--gary
Plugh!
> Addressing just the easy part of this question, it should not be too
> difficult to write some kind of a scan which, given the syntax of the
> language, could produce a list of verbs classified as to whether they take
> direct and indirect objects or not, and then produce a list of all the
> possible interactions a player might enter.
I agree with the rest of your argument, that a brute force approach
quickly generates large numbers of permutations. Any ideas to get
around this? Is such a tool worthwhile anyway? Probably not.
And what about items described in the room description, but not
present in the game? The original poster said that beta-testers were
annoyed when the ldesc said that there was a rug in the room, but "x
rug" resulted in "I don't know what a rug is".
Plugh!
> to what's in his/her room descriptions, and add scenery objects
> accordingly. This is not hard to do, it's just time-consuming.
It's possibly the #1 complaint of things which can ruin a game, by
shaking the player out of the make believe world in which he has
immersed himself.
Personally, as I said before, I believe that the best way towards that
is to be in the habit of never using plain text words for
descriptions. "There is a gorgeous " + rug.adjective() + rug.noun() +
"spread in front of the " +
fireplace.noun() ... might be easier to automate and even without an
automated tool, you are then forced to declare objects for the rug &
the fire.
> You can read a seven-line paragraph and understand its implications in
> terms of your model world a heck of a lot faster than you can (a) run a
> diagnostic tool that searches for nouns in specified text blocks and
> then (b) sort through the long list it generates, referring back to your
> room descriptions at every step to find out what sort of errors or
> omissions it thinks it has spotted. That's my opinion, anyhow.
And you're right. Especially if the tool generates 100k interactions!
Now, can you apply your razor sharp logic to the problem & find a way
of simplifying such a tool? I'm not even sure that I am for it, but so
much gets overlooked (even by beta testers) that another weapon in the
author's arsenal can't be all bad.
> This whole line of inquiry strikes me as yet another of those threads
> that pop up on raif from time to time, all of which can be lumped under
> the general heading, "How can I get a computer tool that will make it
> easy for me to write my dream adventure game without actually having to
> think or do any hard work?"
Sure, there will always be lazy guys; just as there will always be
purists who advocate writing your own parser - in assembler. But
reasonable men generally take a middle way. The former group will
churn out trash, which will be ignored; the latter will never complete
their magnum opus. But anything which can help the reasonable man
ought to be considered (of course, I am biased, see
http://plugh.cjb.net).
> Me, I'd settle for more informative compiler error messages and more
> flexible verb syntax.
There's another thread at the moment "defining a new if programming
language" which is asking for details of such gripes. Maybe if you
deatail them there (as well as taking them up with the author of your
preferred i-f language), something might get done.
Thanks for the input.
Plugh!
> Although I too think a beta testing tool may have its place,
> particularly for regression testing of certain critical actions. I
> think a lot of what you have mentioned above, however, should be
> handled by the standard library.
> I personally feel that the time spent
> making this debugging tool could be better used making the library
> have a greater range of standard responses. The only way to achieve
> this is to have a richer set of properties that can be given to
> objects and subsequently tested for by all verbs. As you say, effects
> like the above are commonly understood, so they should all happen
> automatically without author doing anything more than giving the oil
> LIQUID and FLAMMABLE, the fire place FLAMMABLE, BURNING and LUMINOUS,
> and the water LIQUID.
There do exist soem add on libraries but, then again, some folks are
wary of using anything non-standard. I wonder if TADS 3 will bring
anything like this. A good idea, though.
> Of course when people fail to give objects all
> the required attributes, you are right back where you started... ;)
Well, that's one stumbling block we'll never get around.
> Perhaps a library of common, well-coded objects is the answer. Then it
> would beta tester's main job would be to see if the default action
> sits well within the context of the game.
t'would be nice if the game engine would support a run-time debug
command to give the class of an object.
Julian Fondren
> Jim Aikin <kill_spammers@kill_spammers.org> wrote in message news:<3C6C860E.1000206@kill_spammers.org>...
> > The most direct way is for the author of the game to pay close attention
> > to what's in his/her room descriptions, and add scenery objects
> > accordingly. This is not hard to do, it's just time-consuming.
> I totally agree. However, it doesn't always (or even often?) get done.
> It's possibly the #1 complaint of things which can ruin a game, by
> shaking the player out of the make believe world in which he has
> immersed himself.
This is the author's fault, and nothing else than attention to detail will
ever make it go away. I didn't want to reply to this thread, but...
1. Your idea, the very premise of your idea, and your arguments on its
pros are cons are idiotic.
2. I actually laughed when you pointed out that an English dictionary
might be needed for the tool.
3. Your apparent attempts to throw on all authors what I suspect to be
your own personal problem disgust me.
4. No, games often deal with objects referenced by the description, even
if only to say that they are unimportant, or to make them additional
names of the room.
5. The best way to deal with such objects is to deal with them when
writing the description, and then to put a gnat's effort into looking
at your own game -- if you really love it, you'll do this.
6. Pleading to the newsgroup to come up with a tool, or fully specify(!?)
a tool, and then to actually program(!?) a tool to DWYM is futile.
7. Everything I've said pretty well applies to object interactions. The
best tool here is a list of what verbs you have defined, and, again,
some intelligence and effort.
> Personally, as I said before, I believe that the best way towards that
> is to be in the habit of never using plain text words for
> descriptions. "There is a gorgeous " + rug.adjective() + rug.noun() +
> "spread in front of the " +
> fireplace.noun() ... might be easier to automate and even without an
> automated tool, you are then forced to declare objects for the rug &
> the fire.
No! This is stupid! You can say 'rug' more easily as you can 'rug.noun()',
and there is no benefit to the latter. What you'll end up with is confusing
descriptions, lessened ease in editing your own game, and useless bulk in
the result. Anyway, it appears to have failed to keep your own attention
to detail: what happens if the player refers to a 'gorgeous rug'?
> Now, can you apply your razor sharp logic to the problem & find a way
> of simplifying such a tool? I'm not even sure that I am for it, but so
I can, and very easily: I optimize it out of existence.
Jim Aikin
> Personally, as I said before, I believe that the best way towards that
> is to be in the habit of never using plain text words for
> descriptions. "There is a gorgeous " + rug.adjective() + rug.noun() +
> "spread in front of the " +
> fireplace.noun() ... might be easier to automate and even without an
> automated tool, you are then forced to declare objects for the rug &
> the fire.
And if you forget, the compiler will catch you, which is what I gather
you're hoping to achieve. There are two difficulties with this approach:
First, I'm a writer. (Yes, a professional, if it matters.) To have to
stop and insert that kind of code into my paragraphs is very
distracting. Second, it will only work if I code the scenery objects
*before* I write the room description. This is not a guaranteed or even
likely workflow for me. I will usually write the description first (with
my writer hat on) and then afterward add scenery objects based on what
I've envisioned for the scene. (If I remember to....)
> Now, can you apply your razor sharp logic to the problem & find a way
> of simplifying such a tool? I'm not even sure that I am for it, but so
> much gets overlooked (even by beta testers) that another weapon in the
> author's arsenal can't be all bad.
Well, in Inform you can give your rooms "names," which are never
printed. You can stuff any bunch of nouns you like into the room's name,
and the game will then say, "That's not something you need to refer to
in the course of this game." Which at least has the merit of not being rude.
But creating such a tool shouldn't be that difficult. If you were to
come up with a list of 10,000 nouns that were all *substantives* (tree,
rock, window, fireplace, dog, sky, etc.), it would be relatively easy to
search any text strings in the game and look for those substantives. You
could make the thing's lexicon user-extensible; that wouldn't be hard.
If your program could then report the object name of the object in which
it finds the noun string, you'd get a relatively simple printout that
would be easy to scan. Is that what you're looking for? It would still
be up to you to go through the list and say, "Hmm. Forgot the
fireplace." But the list would make an effective reminder -- you could
even check off items as you code them.
> Sure, there will always be lazy guys; just as there will always be
> purists who advocate writing your own parser - in assembler. But
> reasonable men generally take a middle way. The former group will
> churn out trash, which will be ignored; the latter will never complete
> their magnum opus. But anything which can help the reasonable man
> ought to be considered.
I fully agree!
> There's another thread at the moment "defining a new if programming
> language" which is asking for details of such gripes. Maybe if you
> deatail them there (as well as taking them up with the author of your
> preferred i-f language), something might get done.
I believe Mike Roberts is hard at work on TADS 3. I hate to bother him
with requests for tiny fixes to TADS 2.5.5.
--Jim Aikin
Plugh!
> 1. Your idea, the very premise of your idea, and your arguments on its
> pros are cons are idiotic.
The idea itself, maybe, but not the offering of it for discussion.
> 2. I actually laughed when you pointed out that an English dictionary
> might be needed for the tool.
Well, if they weren't available for free as plug in s/w modules, I
certainly wouldn't have suggetsed it & would have laughed with you at
anyone who did. I only mentioned it to show how complex such a tool
could become. In reality, it doesn't matter to the tool if it has a
list of 10 words or 100k; it just runs a little more slowly.
> 3. Your apparent attempts to throw on all authors what I suspect to be
> your own personal problem disgust me.
Not my problem at all, I rarely ever have such problems. However, I
notice many if-comp reviews & beta test posts to this group saying
that probably the #1 general problem is objects listed in room
descriptions which can't be examined.
> 4. No, games often deal with objects referenced by the description, even
> if only to say that they are unimportant, or to make them additional
> names of the room.
Yes, they. Yes, they should. Unfortunately, many often don't. As you
seem to be aware, it can be an exhaustive process.
> 5. The best way to deal with such objects is to deal with them when
> writing the description, and then to put a gnat's effort into looking
> at your own game -- if you really love it, you'll do this.
Agreed 100%.
> 6. Pleading to the newsgroup to come up with a tool, or fully specify(!?)
> a tool, and then to actually program(!?) a tool to DWYM is futile.
Yes, if probably is futile. However, I fear that you may have
misunderstood me. I don't expect anyone to write the tool for me.
Quite the reverse. If we can come to some agreemenmt about how the
tool should function, I'd be willing to try programming it myself.
> 7. Everything I've said pretty well applies to object interactions. The
> best tool here is a list of what verbs you have defined, and, again,
> some intelligence and effort.
Agreed. It helps if you have a checklist, whether mental or written,
of common problem ares - flammable items, liquids, etc.
>
> > Personally, as I said before, I believe that the best way towards that
> > is to be in the habit of never using plain text words for
> > descriptions. "There is a gorgeous " + rug.adjective() + rug.noun() +
> > "spread in front of the " +
> > fireplace.noun() ... might be easier to automate and even without an
> > automated tool, you are then forced to declare objects for the rug &
> > the fire.
>
> No! This is stupid!
Each to his own opinion. I agree that it requires extra effort, but it
does guarantee that I'll have to declare a rug object. I can do a
bunch of room descriptions like that, when I am feeling creative,
thenm come back later and compile, which immediately shows me which
objects I need to implement.
> You can say 'rug' more easily as you can 'rug.noun()',
Very true, but not so much quicker, to a reasonable typist.
> and there is no benefit to the latter. What you'll end up with is confusing
> descriptions, lessened ease in editing your own game, and useless bulk in
> the result.
Well, maybe it's just something that one can get used to. Everyone has
their own distinctive style.
> Anyway, it appears to have failed to keep your own attention
> to detail: what happens if the player refers to a 'gorgeous rug'?
I apologoize if my 'off the top of the head' spontaneous examples are
less than 100% perfect. Obviously, 'georgius' is rug.adjective() as is
'persian'.
Well, I seem to have riled you for some reason which I can't fathiom.
I wasn't trolling & I don't like flame wars. Can we agree to diifer?
If anyone can specify this tool, I'll have a crack at coding it &
giving something to the community. In the meantime, I am a proponent
of using tools to relieve tedium wherever possible, so you can expect
to see more such posts from me, hopefully not so ridiculous.
Yes, I agree with you that is probably not do-anle, but I didn't think
that it would hurt to kick the idea around a bit.
Thanks for posting.