Break My Parser, Win A Prize!
James Edward Gray II
plan to share it when it's ready.)
What I need is the biggest list I can find of all the commands
Inform/TADS/whatever understand. Naturally, I'm mainly interested in
the edge cases, but an exhaustive list is welcome. Does such a thing
exist?
Any pointers you can share would be great. Thanks for your time.
James
Stuart "Sslaxx" Moore
What's the parser for? Inform? TADS? Or another language?
Bye,
--
Stuart "Sslaxx" Moore.
http://www.livejournal.com/users/sslaxx/
Eric Eve
"James Edward Gray II" <te...@grayproductions.net> wrote in message
news:4b9e731c.04073...@posting.google.com...
Well, one way you could get at such a list (or something like it)
would be to look at the list of verbs defined in en_us.t in the adv3
library (the library files that come with TADS3). I expect there's
something similar for Inform, so if you looked at both lists you'd
have most of the common cases covered.
-- Eric
samwyse
> I'm trying to reality test my parser for my new IF library. (Yes, I
> plan to share it when it's ready.)
>
> What I need is the biggest list I can find of all the commands
> Inform/TADS/whatever understand. Naturally, I'm mainly interested in
> the edge cases, but an exhaustive list is welcome. Does such a thing
> exist?
I've got some edge cases for disambiguation on my laptop, which isn't
handy at the moment. Let me describe them. The player is in a room
with two locked boxes (say, silver and gold) and two keys (ditto).
There are 16 ways that those objects can be distributed between the
player and the location. For each permutation, parse the command
"unlock box with key" and make sure the parser doesn't get confused.
FYI: The current Inform parser fails in four of the cases. I hope to
fix it some day, but it's proving to be a tough nut to crack.
James Edward Gray II
> I've got some edge cases for disambiguation on my laptop, which isn't
> handy at the moment. Let me describe them. The player is in a room
> with two locked boxes (say, silver and gold) and two keys (ditto).
> There are 16 ways that those objects can be distributed between the
> player and the location. For each permutation, parse the command
> "unlock box with key" and make sure the parser doesn't get confused.
This sounds very promising.
Just so I understand, when you say "there are 16 ways those objects
can be distributed...", you mean with the player holding one or more
of the objects and the remained being in the room itself, right?
One more question. I'm not sure I know what the correct behavior is,
in ever case. Just as an example, what happens if the player is
holding just the silver key? What if the player is holding
everything?
Thanks for the idea.
James
samwyse
> samwyse <deja...@email.com> wrote in message news:
>>I've got some edge cases for disambiguation on my laptop, which isn't
>>handy at the moment. Let me describe them. The player is in a room
>>with two locked boxes (say, silver and gold) and two keys (ditto).
>>There are 16 ways that those objects can be distributed between the
>>player and the location. For each permutation, parse the command
>>"unlock box with key" and make sure the parser doesn't get confused.
>
> This sounds very promising.
>
> Just so I understand, when you say "there are 16 ways those objects
> can be distributed...", you mean with the player holding one or more
> of the objects and the remained being in the room itself, right?
Correct.
> One more question. I'm not sure I know what the correct behavior is,
> in ever case. Just as an example, what happens if the player is
> holding just the silver key? What if the player is holding
> everything?
Either of the messages, "You unlock the box" or "The key doesn't fit"
are OK. Asking which key, which box, or both, and then saying "I didn't
understand that sentence." is a failure.
I posted a slightly different example a while back at
http://tinyurl.com/62sln
(http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&c2coff=1&selm=3c863a7a.0404282108.14cae3d3%40posting.google.com)
The Kitchen
Just an ordinary kitchen.
You can see a key, a knife, a red box and a green box here.
>take all but the red box
key: Taken.
knife: Taken.
green box: Taken.
>unlock the box
What do you want to unlock the red box with?
>the key
I didn't understand that sentence.
>unlock the red box
What do you want to unlock the red box with?
>the key
You unlock the red box.
Sam Denton
> I've got some edge cases for disambiguation on my laptop, which isn't
> handy at the moment.
OK, the laptop's handy now, but I'm having to use Google Groups to
post. Here's a disambiguation testbed written in Inform. Test 5, 6,
9 and 10 give you the message "I didn't understand that sentence."
Tests 1, 2, 4, 7, 11, 13 and 14 have their quirks, but I regard them
as behaving correctly. My efforts so far to fix the parser have
sometimes resulted in the message "You can't see any such thing.",
which is also a failure.
Constant Story "TEST";
Constant Headline "^An Interactive Fiction Story^";
Constant DEBUG;
Include "parser";
Include "verblib";
Include "grammar";
Object Kitchen "The Kitchen"
with
description "Just an ordinary kitchen.",
has light;
Object WoodBox "wood box"
with
name 'wood' 'box',
description "It's just a wood box.",
with_key BrassKey,
has openable lockable locked;
Object IvoryBox "ivory box"
with
name 'ivory' 'box',
description "It's just an ivory box.",
with_key SilverKey,
has openable lockable locked;
Object BrassKey "brass key"
with
name 'brass' 'key',
description "It's just an old brass key.";
Object SilverKey "silver key"
with
name 'silver' 'key',
description "It's just an old silver key.";
Verb 'test'
* 'script' -> TestScript
* number -> TestNumber;
! I've tried to avoid hard-coding the number of objects (or
! values derived from that value) but in a few places I've
! used hex constants that assume that there are exactly four.
Constant NBR_OBJECTS=4;
Array thing --> WoodBox IvoryBox BrassKey SilverKey ;
[ TestNumberSub n x obj ;
x = 1;
for (n=0 : n<NBR_OBJECTS : n++) {
obj = thing-->n;
if (noun & x)
move obj to player;
else
move obj to Kitchen;
if (obj has openable) give obj ~open;
if (obj has lockable) give obj ~locked;
if (lookmode == 2)
print "Moved ", (the)obj, " to ",(the)parent(obj),".^";
x = 2 * x;
}
];
[ TestScriptSub n x ;
! figure out 2**NBR_OBJECTS
x = 1; for (n=0 : n<NBR_OBJECTS : n++) x = 2 * x;
! Remember, the goal isn't to lock a particular a box,
! it's to avoid getting any parser error messages.
print "* Commands to test ambiguity resolution^";
for (n=0 : n<x : n++) {
! set initial conditions
print "test ", n, "^";
! try to lock something
print "lock^";
! parser will assume any unique unheld object
if (~~(n == $7 or $b or $d or $e)) {
! expect parser to ask "lock what?"
print "the box^";
! if the boxes are in same place, expect a question
if ((n & $3) == $0 or $3) print "wood^";
}
! expect parser to ask "lock it with what?"
print "the key^";
! if the keys are in same place, expect a question
if ((n & $c) == $0 or $c) print "brass^";
}
print "* End of ambiguity resolution tests^";
];
[ Initialise;
location = Kitchen;
"This is a framework for testing how well the parser resolves
ambiguity in player input. To use it, first issue the command
'test script' and capture the resulting output to a file. Next,
issue the 'script' command to create a transcript file and issue
the 'replay' command to replay the commands generated earlier.
Finally, 'quit' the game and peruse the transcript for any
occurances of the message 'I didn't understand that sentence.'
or 'You can't see any such thing.' Hopefully, there won't be
any.^^";
];
Nikos Chantziaras
May I ask what that price is, and how I can break the parser if I don't have
access to it?
James Edward Gray II
> On or about 8/1/2004 10:10 AM, James Edward Gray II did proclaim:
> > in ever case. Just as an example, what happens if the player is
> > holding just the silver key? What if the player is holding
> > everything?
>
> Either of the messages, "You unlock the box" or "The key doesn't fit"
> are OK. Asking which key, which box, or both, and then saying "I didn't
> understand that sentence." is a failure.
Personally, I believe I would prefer the parser clarify the key, then
clarify the box, then perform the expected action. Any reason not to
go that way?
James
James Edward Gray II
> What's the parser for? Inform? TADS? Or another language?
C. I promise to share when it's ready.
James
James Edward Gray II
> The subject line says: "Break My Parser, Win A Prize!"
>
> May I ask what that price is, and how I can break the parser if I don't have
> access to it?
Don't worry it won't cost you a thing. Of course, if you didn't think
my subject was funny you probably won't like that joke either.
Perhaps you meant "prize".
I plan to share when it's ready.
James
samwyse
> samwyse <deja...@email.com> wrote in message news:<q%8Pc.148$b_....@newssvr23.news.prodigy.com>...
>>are OK. Asking which key, which box, or both, and then saying "I didn't
>>understand that sentence." is a failure.
>
> Personally, I believe I would prefer the parser clarify the key, then
> clarify the box, then perform the expected action. Any reason not to
> go that way?
As long as it avoids the "I didn't understand..." message, it could ask
my hair color. ;-) In other words, feel free to implement it anyway
you like, just watch out for pitfalls like the one I've described.
John Miles
te...@grayproductions.net says...
A couple of my favorite pathological cases:
"Light the light light blue light."
"Plant the pot plant in the plant pot."
-- jm
------------------------------------------------------
http://www.qsl.net/ke5fx
Note: My E-mail address has been altered to avoid spam
------------------------------------------------------
Richard Bos
> In article <4b9e731c.04073...@posting.google.com>,
> te...@grayproductions.net says...
> > I'm trying to reality test my parser for my new IF library. (Yes, I
> > plan to share it when it's ready.)
> >
> > What I need is the biggest list I can find of all the commands
> > Inform/TADS/whatever understand. Naturally, I'm mainly interested in
> > the edge cases, but an exhaustive list is welcome. Does such a thing
> > exist?
>
> A couple of my favorite pathological cases:
>
> "Light the light light blue light."
That truly _is_ pathological - you would never say it IRL. "Light the
light blue light", yes. But "the light light blue light" is not normal
language.
> "Plant the pot plant in the plant pot."
"...with the trowel". Yes. Not that Inform would get this right all by
itself, AFAICT - you'd need either an adjectives extension or a
judiciously written ChooseObjects to make it work. But it can be done.
Richard
Michael Roy
> John Miles <jmi...@pop.removethistomailme.net> wrote:
<snip>
>>"Plant the pot plant in the plant pot."
>
>
> "...with the trowel". Yes. Not that Inform would get this right all by
> itself, AFAICT - you'd need either an adjectives extension or a
> judiciously written ChooseObjects to make it work. But it can be done.
>
> Richard
Actually, I don't think an adjective extension would work--it still
wouldn't take word order into account so both "pot plant" and "plant
pot" would register as having one adjective and one noun, and we'd get
into an endless loop of
Which do you mean, the plant pot or the pot plant?
ChooseObjects could technically handle it, although there are so many
possible interactions that there'd probably always be a few bugs left in it.
I'd suggest using a parse_name property for this type of case, so you
could take word order into account. In both cases, a routine that
required the second word but allowed the first word to come *before* it
would probably work.
If we're under the Platypus library system, we could also use it's
method of returning both adjective and noun counts from a parse_name and
even get it to print messages like "(the pot plant)" in the correct
cases. And if we really want to get fancy we could allow them to be
referred to by the adjective only when only one is in scope.
(Returning to the original topic of this thread, something equivalent to
parse_name is a definite requirement of any parser. Can't find the
exact quotation, but I recall Mr. Nelson saying something along the
lines of "a parser must be general enough for every possible use," which
is definitely a mantra I'd display prominently if I were trying to write
one.)
Michael
John Miles
uitgeverij.nl says...
> John Miles <jmi...@pop.removethistomailme.net> wrote:
>
> > A couple of my favorite pathological cases:
> >
> > "Light the light light blue light."
>
> That truly _is_ pathological - you would never say it IRL. "Light the
> light blue light", yes. But "the light light blue light" is not normal
> language.
Sure it is. You have two light-blue lights: a heavy one, and a light
one. Which one do you want to light?
David Whyld
"John Miles" <jmi...@pop.removethistomailme.net> wrote in message
news:MPG.1b79bafd5...@news-central.giganews.com...
> >
> > That truly _is_ pathological - you would never say it IRL. "Light the
> > light blue light", yes. But "the light light blue light" is not normal
> > language.
>
> Sure it is. You have two light-blue lights: a heavy one, and a light
> one. Which one do you want to light?
>
Wouldn't you just type "light the heavy blue light" or "light the light blue
light"?
Mike Roberts
> "John Miles" <jmi...@pop.removethistomailme.net> wrote:
> > > light blue light", yes. But "the light light blue light" is not normal
> > > language.
> >
> > Sure it is. You have two light-blue lights: a heavy one, and a light
> > one. Which one do you want to light?
>
> Wouldn't you just type "light the heavy blue light" or "light the light
blue
> light"?
Of course not - you have to distinguish those from the heavy dark blue light
and the light dark blue light that are also present.
--Mike
mjr underscore at hotmail dot com
Florian
Then there's the light blue and green light, the light light blue and green
light, the bright light blue and green light, and of course the light bright
bright blue and bright green blacklight.
And later that night you might fight a feisty giant mite with a mighty bite.
You'll need your sight, so pick your lights right.
--
Which light bright bright blue and bright green blacklight you mean?
Sophie Fruehling
I guess I'd just type "quit". :)
--
Sophie Frühling
"El arte no viste pantalones."
-- Rubén Darío
Rexx Magnus
> Then there's the light blue and green light, the light light blue and
> green light, the bright light blue and green light, and of course the
> light bright bright blue and bright green blacklight.
>
> And later that night you might fight a feisty giant mite with a mighty
> bite. You'll need your sight, so pick your lights right.
You've been licking that frog, haven't you?
To email me, visit the site.
Mark J Musante
> "Plant the pot plant in the plant pot."
Yonks ago I wanted to find out how hard this was to do in TADS. As
it turns out, there's nothing special you need to do. I haven't tried
the light/heavy light/dark blue/red light yet, but I expect it's no
different.
-markm
-------------------------------------------------------------------------
#include <adv.t>
#include <std.t>
#pragma C+
startroom: room
sdesc = "Pottery Barn"
ldesc = "A potterer is you! "
;
pot_plant: item location = startroom
noun = 'plant'
adjective = 'pot'
sdesc = "pot plant"
verDoPlantIn(actor, iobj) = {}
;
plant_pot: item location = startroom
noun = 'pot'
adjective = 'plant'
sdesc = "plant pot"
verIoPlantIn(actor) = {}
ioPlantIn(actor, dobj) = {
"You plant <<dobj.thedesc>> in <<self.thedesc>>! ";
"\b\b* you have won *\b";
morePrompt();
quit();
}
;
plantVerb: deepverb
verb = 'plant'
sdesc = "plant"
ioAction(inPrep) = 'PlantIn'
;
-------------------------------------------------------------------------
Quintin Stone
> >Wouldn't you just type "light the heavy blue light" or "light the light
> >blue light"?
>
> I guess I'd just type "quit". :)
I'm with Sophie.
/====================================================================\
|| Quintin Stone O- > "You speak of necessary evil? One ||
|| Code Monkey < of those necessities is that if ||
|| Rebel Programmers Society > innocents must suffer, the guilty must ||
|| st...@rps.net < suffer more." -- Mackenzie Calhoun ||
|| http://www.rps.net/QS/ > "Once Burned" by Peter David ||
\====================================================================/
Matthew Russotto
John Miles <jmi...@pop.removethistomailme.net> wrote:
>In article <410fd642...@news.individual.net>, rlb@hoekstra-
>uitgeverij.nl says...
>> John Miles <jmi...@pop.removethistomailme.net> wrote:
>>
>> > A couple of my favorite pathological cases:
>> >
>> > "Light the light light blue light."
>>
>> That truly _is_ pathological - you would never say it IRL. "Light the
>> light blue light", yes. But "the light light blue light" is not normal
>> language.
>
>Sure it is. You have two light-blue lights: a heavy one, and a light
>one. Which one do you want to light?
The lighter.
Richard Bos
> Richard Bos wrote:
> > John Miles <jmi...@pop.removethistomailme.net> wrote:
> <snip>
> >>"Plant the pot plant in the plant pot."
> >
> >
> > "...with the trowel". Yes. Not that Inform would get this right all by
> > itself, AFAICT - you'd need either an adjectives extension or a
> > judiciously written ChooseObjects to make it work. But it can be done.
>
> Actually, I don't think an adjective extension would work--it still
> wouldn't take word order into account
Why not? See DM4 exercise 75, and note the description of
"calyx_adjectives.h" in the references just below.
Richard
Michael Roy
Ah yes, the version suggested in the DM4 would indeed work. I haven't
looked at the calyx_adjectives example and was basing my previous
comment on the handling of adjectives under the Platypus 4+ library,
which treats adjectives as "weak nouns" instead of as true adjectives
and thus maintains separate tallies of the two without requiring that
the adjectives come before the noun. In fact, the DM4 example is
basically implementing the force-adjectives-first idea that I suggested
for a parse_name routine on a global scale.
I'm guessing that Platypus probably allows adjectives both before and
after to better support translations to other languages, but as you
point out, such behavior is undesirable in the current case and the
version in ex. 75 would work admirably.
Michael
Gene Wirchenko
Do you mean the disposable lighter or the light light?
Sincerely,
Gene Wirchenko
Computerese Irregular Verb Conjugation:
I have preferences.
You have biases.
He/She has prejudices.
Asa Rossoff
So, you are saying the following actions would work:
Plant pot.
Plant pot in pot.
Plant pot in plant pot.
Pot pot. (If you have a pot verb.)
Pot plant.
Pot plant in pot.
Pot plant in plant pot.
I can see the spelled out "Plant pot plant in plant pot" is not so
difficult, but the above examples would be more difficult to parse (but
possible... in fact, they have only one appropriate "human"
interpretation and should be able to be parsed even without querying the
player."
Asa
Malcolm Ryan
> Mark J Musante wrote:
>
> > John Miles <jmi...@pop.removethistomailme.net> wrote:
> >
> >>"Plant the pot plant in the plant pot."
> >
> >
> > Yonks ago I wanted to find out how hard this was to do in TADS. As
> > it turns out, there's nothing special you need to do. I haven't tried
> > the light/heavy light/dark blue/red light yet, but I expect it's no
> > different.
>
> Plant pot.
> Plant pot in pot.
> Plant pot in plant pot.
> Pot pot. (If you have a pot verb.)
> Pot plant.
> Pot plant in pot.
> Pot plant in plant pot.
>
> I can see the spelled out "Plant pot plant in plant pot" is not so
> difficult, but the above examples would be more difficult to parse (but
> possible... in fact, they have only one appropriate "human"
> interpretation and should be able to be parsed even without querying the
> player."
Really? Suppose we're gardening. You ask me "what should I do now?"
and I answer "pot pot", I think your response would most likely be
"Sorry, what?".
Let's not put unrealistic expectations on our computers.
Malcolm
Asa Rossoff
True, using "pot" as a verb with pot would at least sound silly. I
don't actually know much about the Inform or Tads parsers; the verb word
might not actually be a significant issue.
However, this example is a clear case where with one object, the
adjective is more significant (pot plant) and with the other object, the
noun is more significant (plant pot), they both use the same words, and
they are both reasonable objects.
The parsers job in a simple sentence form like this still seems to come
down to just making correct logic decisions. Parsing a sentence like
"plant pot in pot" should be achievable. Without knowing what the
current parsers are doing already, a few approaches come to mind:
(1) Do what a person does: if there is more than one interpretation,
eliminate the unreasonable ones. (This may slow down the parser.)
(2) Have objects coded to indicate how they can be identified: Adj,
Noun, and/or Adj+Noun. No one would call a plant pot a "plant", so that
eliminates any ambiguity if the word "plant" is used alone (e.g. "plant
plant in pot"), but preference should /not/ be given to the word "plant"
in the case of the pot plant.
(3) Disambiguation questions can query the user in such a way that the
particular instance of a word in their sentence is specified: i.e., for
"plant pot in pot", "Which do you want to plant, the plant pot or the
pot plant?", followed by "Do you want to plant the pot plant in the pot
plant or in the plant pot?"
(4) Disambigation questions can indicate the possible complete
contextual interpretations of the sentence or complete phrase. Probably
less desirable, as this would require the player to either retype the
sentence/phrase, or select from a numbered list. This might be OK if
they were clickable(TADS..). E.g. (eliminating the least sensical
possibilities), "Do you want to plant _the pot plant in the plant pot_
or _the plant pot in the pot plant_?"
Asa
Alex Warren
> >>So, you are saying the following actions would work:
> >> Plant pot.
> >> Plant pot in pot.
> >> Plant pot in plant pot.
> >> Pot pot. (If you have a pot verb.)
> >> Pot plant.
> >> Pot plant in pot.
> >> Pot plant in plant pot.
> The parsers job in a simple sentence form like this still seems to come
> down to just making correct logic decisions. Parsing a sentence like
> "plant pot in pot" should be achievable. Without knowing what the
> current parsers are doing already, a few approaches come to mind:
> (1) Do what a person does: if there is more than one interpretation,
> eliminate the unreasonable ones. (This may slow down the parser.)
> (2) Have objects coded to indicate how they can be identified: Adj,
> Noun, and/or Adj+Noun. No one would call a plant pot a "plant", so that
> eliminates any ambiguity if the word "plant" is used alone (e.g. "plant
> plant in pot"), but preference should /not/ be given to the word "plant"
> in the case of the pot plant.
> (3) Disambiguation questions can query the user in such a way that the
> particular instance of a word in their sentence is specified: i.e., for
> "plant pot in pot", "Which do you want to plant, the plant pot or the
> pot plant?", followed by "Do you want to plant the pot plant in the pot
> plant or in the plant pot?"
> (4) Disambigation questions can indicate the possible complete
> contextual interpretations of the sentence or complete phrase. Probably
> less desirable, as this would require the player to either retype the
> sentence/phrase, or select from a numbered list. This might be OK if
> they were clickable(TADS..). E.g. (eliminating the least sensical
> possibilities), "Do you want to plant _the pot plant in the plant pot_
> or _the plant pot in the pot plant_?"
The above can all be parsed correctly by looking at the structure of the entire
command. There are four structures in the above set of commands:
Plant [something]
Plant [something] in [something]
Pot [something]
Pot [something] in [something]
It is easy to fit all the above commands to one of those structures by simple
string manipulation - it's pretty much just a matter of searching for the word
"in", which is a good word to use to discriminate as it's not an adjective, verb
or noun.
Once you've fitted the commands to that structure, then the "somethings" can be
filled in with "pot plant" or "plant" or "pot" and the command can be parsed
accurately without asking the user any further questions.
Alex
samwyse
> Asa Rossoff wrote:
> The above can all be parsed correctly by looking at the structure of the entire
> command. There are four structures in the above set of commands:
>
> Plant [something]
> Plant [something] in [something]
> Pot [something]
> Pot [something] in [something]
>
> It is easy to fit all the above commands to one of those structures by simple
> string manipulation - it's pretty much just a matter of searching for the word
> "in", which is a good word to use to discriminate as it's not an adjective, verb
> or noun.
>
> Once you've fitted the commands to that structure, then the "somethings" can be
> filled in with "pot plant" or "plant" or "pot" and the command can be parsed
> accurately without asking the user any further questions.
And in Inform, a good ChooseObject routine can help out by prefering
plants for the first "something" in each command, and a container for
the second.
Gene Wirchenko
[snip]
>The above can all be parsed correctly by looking at the structure of the entire
>command. There are four structures in the above set of commands:
>
>Plant [something]
>Plant [something] in [something]
>Pot [something]
>Pot [something] in [something]
>
>It is easy to fit all the above commands to one of those structures by simple
>string manipulation - it's pretty much just a matter of searching for the word
>"in", which is a good word to use to discriminate as it's not an adjective, verb
>or noun.
>join in group
The in group is busy studying English grammar. They do not even
notice you.
>join cabal
We have your dog.
>Once you've fitted the commands to that structure, then the "somethings" can be
>filled in with "pot plant" or "plant" or "pot" and the command can be parsed
>accurately without asking the user any further questions.
Sincerely,