What does a parser need to understand these days?

[Johann 'Myrkraverk' Oskarsson]

Hi all,

I've been playing with a prototype for my own game engine. Because
there are things I want to do which the current tools either don't or
will make it difficult to accomplish.

The annoying part is that requires me to build my own parser. Apart
from VERB NOUN1 NOUN2 commands such as PUT STONE ON TABLE and
NOUN-CLOUSE, VERB NOUN1 NOUN2 to give commands/suggestions or talk, what
kind of complexity does a modern parser need to handle?

That is, what kind of sentence structure do modern IF players expect to
work?


Johann

[Andrew Plotkin]

Here, Johann 'Myrkraverk' Oskarsson <myrkr...@yahoo.com> wrote:
>
> That is, what kind of sentence structure do modern IF players expect to
> work?

The "standard" vocabulary includes

VERB NOUN ("get thing")
VERB PREP NOUN ("get on thing")
VERB NOUN PREP NOUN ("throw thing at foo")
VERB NOUN NOUN ("give fox cheese")

Note that the last case is inherently ambiguous, since each NOUN can
be a multi-word phrase. Really that affects the previous cases too,
because a word might be acceptable as either a preposition or a noun
phrase. ("throw The Horror At Dunwich at the wall") So you see we've
already pulled in the possibility of backtracking and multiple
acceptable parsings.

I'd say it's more important to get enough abstraction in there. A
first-class IF system needs abstract actions ("get foo", "take foo",
"pick up foo" all map to the same action) and abstract objects (noun
synonyms, which may include single words and multi-word phrases with
adjectives piled in.) The author needs to be able to customize verb
phrasings, abstract actions, noun phrasings, and abstract objects as
four independent domains.

There's also the notion of scope. "take", "remember", and "say" all
have different possible ranges of nouns.

I'll repeat my usual warning: it takes a few days to build a basic IF
system, and two to four years to turn that into a high-quality one.

--Z

--
"And Aholibamah bare Jeush, and Jaalam, and Korah: these were the borogoves..."
*

[Johann 'Myrkraverk' Oskarsson]

Andrew Plotkin <erky...@eblong.com> writes:

> The "standard" vocabulary includes
>
> VERB NOUN ("get thing")
> VERB PREP NOUN ("get on thing")
> VERB NOUN PREP NOUN ("throw thing at foo")
> VERB NOUN NOUN ("give fox cheese")

Thank you. For the record, I am used to . separating sentences, so I
always expect "n.n.e." to work. Do people also expect "it", "her",
"him" to work too?

Other complex sentences people have come to expect since the 80s? Or
even then.

> Note that the last case is inherently ambiguous, since each NOUN can
> be a multi-word phrase. Really that affects the previous cases too,
> because a word might be acceptable as either a preposition or a noun
> phrase. ("throw The Horror At Dunwich at the wall") So you see we've
> already pulled in the possibility of backtracking and multiple
> acceptable parsings.

Yes, I have an idea to handle that - which might fail in practice but at
least an idea.

> I'd say it's more important to get enough abstraction in there. A
> first-class IF system needs abstract actions ("get foo", "take foo",
> "pick up foo" all map to the same action) and abstract objects (noun
> synonyms, which may include single words and multi-word phrases with
> adjectives piled in.) The author needs to be able to customize verb
> phrasings, abstract actions, noun phrasings, and abstract objects as
> four independent domains.

I'll keep that in mind, though for the record I already realized some of
that.

> There's also the notion of scope. "take", "remember", and "say" all
> have different possible ranges of nouns.

> I'll repeat my usual warning: it takes a few days to build a basic IF
> system, and two to four years to turn that into a high-quality one.

Noted.



Johann

[Adam Thornton]

In article <ujkcf1.x5ujkcf1...@asuka.myrkraverk.com>,

Johann 'Myrkraverk' Oskarsson <myrkr...@yahoo.com> wrote:

>Andrew Plotkin <erky...@eblong.com> writes:
>> I'll repeat my usual warning: it takes a few days to build a basic IF
>> system, and two to four years to turn that into a high-quality one.
>
>Noted.
>

I'm going to be more directly discouraging than Andrew:

It will ultimately be much less effort to adapt an existing tool to cope
with whatever extensions it is that you need, than it will be to write a
new system from scratch.

Adam

[Brian]

Johann 'Myrkraverk' Oskarsson <myrkr...@yahoo.com> wrote:

Out of interest what programming language are you using?

--
Regards Brian

[Johann 'Myrkraverk' Oskarsson]

Brian <bcl...@es.co.nz> writes:

> Johann 'Myrkraverk' Oskarsson <myrkr...@yahoo.com> wrote:

>> I've been playing with a prototype for my own game engine. Because
>> there are things I want to do which the current tools either don't or
>> will make it difficult to accomplish.
>

> Out of interest what programming language are you using?

Racket (Lisp) for now. I'm currently building a class hierarchy/system
for the world model.

I'm currently focusing my attention on stuff I don't expect to be easy
in, say, Inform 6 or 7.


Johann

[Brian]

I thought you might reply with C, C++ or some other popular language. I
don't know anything about Racket (Lisp) but might do some internet
research to find out more.

--
Regards Brian

[dklei...@gmail.com]

On Saturday, August 17, 2013 7:01:36 AM UTC-7, Andrew Plotkin wrote:

> The "standard" vocabulary includes
>
> VERB NOUN ("get thing")
> VERB PREP NOUN ("get on thing")
> VERB NOUN PREP NOUN ("throw thing at foo")
> VERB NOUN NOUN ("give fox cheese")

It is only by the sheerest accident I discovered this.

Analyzing the Inform 6 source code leads me to observe that the standard Inform library also contains

VERB "sing"
VERB PREP "stand up"
VERB PREP NOUN PREP NOUN "pry open the door with the crowbar"
VERB NOUN PREP PREP NOUN "pry the door open with the crowbar"

It seems one needs to be ready for up to two PREPs and any possible placement of NOUN clusters between them. Clusters means NOUN NOUN and, theoretically even more NOUNs. Once the cluster is isolated the code, as already mentioned, must somehow split it up.

"Pry" is the most complicated verb in the library. I didn't like that so I looked hard at ""with" and have decided that "with NOUN" should be treated separately as an instrumental. Introducing over instruments has ramifications through a lot of the code and, at the moment, remains just an idle speculation.

[Andrew Plotkin]

Here, dklei...@gmail.com wrote:
>
> It seems one needs to be ready for up to two PREPs and any possible placement of NOUN clusters between them. Clusters means NOUN NOUN and, theoretically even more NOUNs. Once the cluster is isolated the code, as already mentioned, must somehow split it up.

It's safer (more general) to assume that commands all look like

VERB PREP* NOUN PREP* NOUN PREP*

...where PREP* is a string of zero or more preposition words. If you
want to get a leg up on the Inform parser, permit more than two nouns.

Grammar *not* distinguished by an initial verb is handled by a
separate callout, in Inform. (This covers directions and other special
cases.)

[dklei...@gmail.com]

On Saturday, January 25, 2014 9:33:50 AM UTC-8, Andrew Plotkin wrote:
> Here, dklei...@gmail.com wrote:
>
> >
>
> > It seems one needs to be ready for up to two PREPs and any possible placement of NOUN clusters between them. Clusters means NOUN NOUN and, theoretically even more NOUNs. Once the cluster is isolated the code, as already mentioned, must somehow split it up.
>
>
>
> It's safer (more general) to assume that commands all look like
>
> VERB PREP* NOUN PREP* NOUN PREP*
>
> ...where PREP* is a string of zero or more preposition words. If you
> want to get a leg up on the Inform parser, permit more than two nouns.
>
> Grammar *not* distinguished by an initial verb is handled by a
> separate callout, in Inform. (This covers directions and other special
> cases.)

I was designing a Siri-like interface for an application and re-implementing the Inform grammar technique. I concluded a sentence string should be parsed by locating prepositions and then looking at the strings between them. This makes the parse like like

VERB (NOUN* PREP)* NOUN?

where the NOUN* is what I called a cluster.

In the library "pry" is the only verb where there are both two PREP's and a non-trivial final NOUN? In order to make "pry" more like everybody else, once I had found PREP I checked for "with" and if it was "with" I jumped out of the scan and assumed everything after "with" (including any the PREP's) was the instrumental NOUN?

I was already not simply scanning for PREP - rather for the appropriate short list of PREPs. That meant I already could have PREP's embedded in NOUN's and had to deal with them (in my application "of" was the only embedded PREP that was not an error).

I think this means I was reading

VERB (NOUN* PREP (NOUN PREP?)?)?

VERB "sing"
VERB PREP "stand up"

VERB NOUN PREP "pick the book up"
VERB PREP NOUN "pick up the book"
VERB PREP NOUN PREP "pry open the door with the crowbar" (using my "with" hack)
VERB NOUN PREP NOUN "throw the book at the mouse"
VERB NOUN PREP NOUN PREP "give the coal to the orc with the tongs"

This all worked quite well for me. There is an acknowledgement to Inform 6 in the comments to the application code. But I doubt that the application code, if it still survives, will ever be exposed to daylight.

[rmal...@gmail.com]

Em terça-feira, 20 de agosto de 2013 09h29min57s UTC-3, Johann 'Myrkraverk' Oskarsson escreveu:
> Brian writes:

>
>
>
> > Johann 'Myrkraverk' Oskarsson wrote:
>
>
>
> >> I've been playing with a prototype for my own game engine. Because
>
> >> there are things I want to do which the current tools either don't or
>
> >> will make it difficult to accomplish.
>
> >
>
> > Out of interest what programming language are you using?
>
>
>
> Racket (Lisp) for now. I'm currently building a class hierarchy/system
>
> for the world model.

racket is a cool Scheme implementation

haha, perhaps you'll could try implementing your interpreter here:

http://eblong.com/zarf/zweb/lists/

it's from Plotkin himself

[logican...@gmail.com]

Hey I'm writing my own text adventures from scratch, I'm using C# it has extensive built in string functionality, what are you using? I'm mostly working on text adventures to work on NPC AI, I'm new to IF myself so I am not building a typical parser, I read a lot on AI for newer games that deal with text input and chatbots. You don't necessarily need all the pronouns and different things, lots of things can be implied.

Stuff I read:
Matching sets of words instead of single words approximates meaning.
Discarding the need to match all the words in the input works well.

As I was saying before, C# has a lot of string functionality, you can take an input sentence and look to see if it contains keywords, or split it into an array of words and look for keywords individually, putting words into an array allows you to get word order which helps decipher meaning. You can pick out all your keywords then work on what it means based on their location relative to other words.

here is my game for anyone interested:
http://gamejolt.com/search/games?q=epic%20prose
https://www.facebook.com/epicProse/