focus on Q&A

[YKY (Yan King Yin, 甄景贤)]

My latest idea:  the AGI problem can be broken down into 2 parts:
A) knowledge maintenance

B) acting / planning

It may be possible to build and commercialize only part A as a question-answering service.  I think the problem decomposition is advantageous.

After the breakdown, our sub-problem is reduced to 4 algorithms:

1)  unification

-- this defines the logic

-- my latest idea is to use a graph-like logic similar to Opencog's Atomspace, and then unification becomes the graph-isomorphism problem

2)  Bayesian inference + fuzziness

-- mathematically completely formulated

-- Lisp code is about 95% finished

3)  inductive learning

-- right now we only have a very naive algorithm, but it may suffice for a minimal viable product

4)  distributive storage and clustering of the KB

-- key is to formulate a distance metric for clustering, and it's not too hard to find a heuristic for it

Also I have been chatting with William Taysom about the unification algorithm.  It seems not too hard to figure it out.

But still, the 4 tasks in total are rather overwhelming.  We may outsource #3 (the inductive learning problem) and just focus on creating logical standard, but then we won't have a commercializable product.

For #1 we can also use Marko Rodrigeuz's graphic logic library.  I have contacted him once but he hasn't replied =(

All in all, 1+2+3+4 doesn't look too hard, perhaps worth funding and developing.  I'm still thinking of how to reduce the total effort, but it looks almost minimal now... =P

--

KY

"The ultimate goal of mathematics is to eliminate any need for intelligent thought" -- Alfred North Whitehead

[Ivan Vodišek]

I've managed B. still looking for A.

https://sites.google.com/site/ivanvodisek/

2012/2/9 YKY (Yan King Yin, 甄景贤) <generic.in...@gmail.com>

[Matt Mahoney]

2012/2/9 YKY (Yan King Yin, 甄景贤) <generic.in...@gmail.com>:

> My latest idea:  the AGI problem can be broken down into 2 parts:
> A) knowledge maintenance
> B) acting / planning
>
> It may be possible to build and commercialize only part A as a
> question-answering service.  I think the problem decomposition is
> advantageous.

Peter Voss is starting an AI project that will focus on question
answering that will focus on text I/O only. Right now he is collecting
some sample questions, about 50 each in a dozen or so domains. I don't
know what approach he plans to use yet. He mentioned "redoing the
ontology" so I guess the old approach was a hand coded language model
that didn't work well. Anyone following it?
http://tech.groups.yahoo.com/group/projectAigo/

-- Matt Mahoney, mattma...@gmail.com

[YKY (Yan King Yin, 甄景贤)]

On Thu, Feb 9, 2012 at 11:54 PM, Ivan Vodišek <ivanv...@gmail.com> wrote:

I've managed B. still looking for A.

https://sites.google.com/site/ivanvodisek/

I still have not detailed idea as to how to integrate a planner with an inference engine.  I have some rather simple ideas in the book but they may be immature for implementation, and also not particularly interesting...  You're welcome to explore that question =)

It's good that you understand what an inference engine does (and you may not need to go into the details which I have not completely spelled out anyway), then you can think about how to hook up a planner with it to do more interesting things.

One caveat though is that there is a technique known as "deductive planning" that can reduce planning to deduction, and thus eliminate the need for a separate planner.  Actually I find that approach appealing.

Also, there are some aspects of an AGI (such as planning and acting) that I have not fully flexed out.  But I wish to implement the Q&A part first and to explore what kind of problems may arise.  As one of my friends suggested, it's hard to design an entire AGI on paper without any experimentation.

PS:  In your site I also noticed some stuff about unification grammars.  That idea seems to coincide with my idea of using logical unification to model natural concepts.  If I'm right, we may not need a separate unification grammar to handle natural language =)

KY

[Ivan Vodišek]

things about unification (?) are still under construction. The first thing i plan to do is swf slides that scratch the surface about KB. I was hoping that You, YKY would have an idea on how to implement learning stuff. If automated learner could be done for KB which is under construction, I believe we could have enough material for online-pdf-book with integrated swf-s on top of each chapter for easier understanding. I'll notify You when this slides would be done. Tomorrow, next week, next year, I don't know, but I'll do my best.

2012/2/23 YKY (Yan King Yin, 甄景贤) <generic.in...@gmail.com>

[Ivan Vodišek]

I just finished slides about KB: https://sites.google.com/site/ivanvodisek/knowledge-base-system

I was hoping for little help from You, YKY. Here are questions:

1. To form set of predicates that will be used as senses to outer world. Maybe console input would be enough.

2. To investigate rules that from input console build the knowledge.

Are You interested?

2012/2/23 Ivan Vodišek <ivanv...@gmail.com>

[YKY (Yan King Yin, 甄景贤)]

On Sat, Feb 25, 2012 at 4:35 PM, Ivan Vodišek <ivanv...@gmail.com> wrote:

I just finished slides about KB: https://sites.google.com/site/ivanvodisek/knowledge-base-system

I was hoping for little help from You, YKY. Here are questions:

1. To form set of predicates that will be used as senses to outer world. Maybe console input would be enough.

2. To investigate rules that from input console build the knowledge.

Are You interested?

Oh I see...  you have some figures that look like "unification grammars", like this one:

http://www-rohan.sdsu.edu/~ling354/unification.html 

(scroll down to the last figure).

But you were talking about something else -- nevermind.

Your terminology is also a bit different from mine.  What you call "variables" I'd call "terms".  And I doubt if strict type theory is suitable for natural-language concepts.  Sometimes the type hierarchy (or ontology) is inconsistent in human concepts.  For example, a "car chair" is a chair, a chair is a piece of furniture, yet most people would disagree that a car chair is a piece of furniture.  Also, sometimes humans would combine concepts in novel ways that may not obey a strict type discipline.  For example, "girl" is usually a noun but "girl movie" can refer to movies targeting girls.

So my idea is to allow free combinations of concepts as in untyped combinatory logic, which is equivalent to untyped lambda calculus.

If you agree with my ideas I'd be happy to include your videos in my book.  Otherwise we may need to discuss more to seek some consensus...

=)

KY

[Ivan Vodišek]

how would free combinations look like? can they be implemented using proposed KB? i believe that type theory can implement untyped lambda calculus also.

can some degree of untyped-ism be implemented by using only most general types?

2012/2/25 YKY (Yan King Yin, 甄景贤) <generic.in...@gmail.com>

[YKY (Yan King Yin, 甄景贤)]

On Sat, Feb 25, 2012 at 9:20 PM, Ivan Vodišek <ivanv...@gmail.com> wrote:

how would free combinations look like? can they be implemented using proposed KB? i believe that type theory can implement untyped lambda calculus also.

can some degree of untyped-ism be implemented by using only most general types?

A few points:

1.  Untyped λ calculus is actually more general than typed.  Type theory is a restriction of λ calculus, developed by Russell and Church etc, to deal with paradoxes.

2.  Type theory bans certain things, such as self-application.  Thus, type theory does not allow us to reason about paradoxes like humans do.  That's why I'm choosing untyped λ logic, as it allows totally free combinations of concepts.

It seems common for natural-language concepts to combine rather freely.  For example, "gender" is a noun, but in "gender studies" it is used as an adjective...

KY

[YKY (Yan King Yin, 甄景贤)]

PS:

On Sat, Feb 25, 2012 at 9:20 PM, Ivan Vodišek <ivanv...@gmail.com> wrote:

how would free combinations look like?

For example, "green idea" is:

    green ◦ idea

where the ◦ denotes "application".  Alternatively that can be written as:

    (green idea)

or

    green(idea).

They're all the same.

Then "colorless green idea" would be

    colorless ◦ (green ◦ idea)

etc...

can they be implemented using proposed KB?

Yes, the KB just stores logical formulas.  The syntax of untyped λ logic is just like Lisp expressions.

KY

[Ivan Vodišek]

I actually develop KB concepts for a while and i find zeroth order logic quite inspirative. So I thought: why not use the very ZOL for storing knowledge?

And here it goes - informations about two cars could be stored like following:

Car -> (Mark -> String & Color -> String)

(  (Mark -> "Mazda" & Color -> "Blue")  |

   (Mark -> "Suzuki" & Color -> "Turquoise")

) -> Car

It's not hard to imagine these informations shown in format used in slides.

But the problem with ZOL is that we can't use ordinary deduction and resolution rule to reach specific car.

We have this situation:

(Mazda | Suzuki) -> Car

Solution might be to use induction to reach specific car. I.e. if we say "Car" it is either "Mazda", either "Suzuki" either somthing else. Which of them would be the one, it depends what parameters we are looking for. Once one car is isolated, we can use ordinary deduction to reach unknown parameters (i.e. if we say "Color", from that follows resolvent "Blue" or "Turquoise").

It turned out that this form of notation can deal with polymorphism, so I write a plus for knowledge pyramid.

I wander are there knowledge structures that can't be dealt with this proposal KB? I'd be happy to extend it to usable form.

2012/2/25 YKY (Yan King Yin, 甄景贤) <generic.in...@gmail.com>

PS:

[YKY (Yan King Yin, 甄景贤)]

On Sun, Feb 26, 2012 at 2:09 AM, Ivan Vodišek <ivanv...@gmail.com> wrote:

I actually develop KB concepts for a while and i find zeroth order logic quite inspirative. So I thought: why not use the very ZOL for storing knowledge?

Looks like you have some misunderstanding of logic:

A proposition is, informally, a sentence or statement with a truth value.  "Car" alone seems not to be a proposition. 

Propositional logic is quite weak because it cannot go into the internals of sentences.  During Aristotle's time, all they knew was syllogistic logic, such as:

    Socrates is a man

    All man are mortal

    Socrates is mortal.

Predicate logic was invented so that a proposition can have internal structure.  A proposition is broken down into a predicate applied to one or more objects, for examples:

    tall(john)                  "John is tall"

    loves(john,mary)     "John loves Mary".

I'm going one step further by allowing all atomic concepts to freely combine to form propositions, for example:

    (slowly ◦ walks) ◦ john          "John walks slowly".

And here it goes - informations about two cars could be stored like following:

Car -> (Mark -> String & Color -> String)

You seem to be assigning key-attribute pairs.  This seems to be a rather limited knowledge representation.

How would you say:

    "That Mazda is missing a wheel"

    "That Suzuki runs noisily" ?

We have this situation:

(Mazda | Suzuki) -> Car

Solution might be to use induction to reach specific car. I.e. if we say "Car" it is either "Mazda", either "Suzuki" either somthing else. Which of them would be the one, it depends what parameters we are looking for. Once one car is isolated, we can use ordinary deduction to reach unknown parameters (i.e. if we say "Color", from that follows resolvent "Blue" or "Turquoise").

?

It turned out that this form of notation can deal with polymorphism, so I write a plus for knowledge pyramid.

I wander are there knowledge structures that can't be dealt with this proposal KB? I'd be happy to extend it to usable form.

I'm not sure what you're talking about here....

KY

[Ivan Vodišek]

Firstly, I want to thank You for interest, YKY :)

1.

"That Mazda is missing a wheel"

Missing -> (param1 -> Mazda & param2 -> Wheel)

2.

"That Suzuki runs noisily"

Runs -> (param1 -> Suzuki & param2 -> Noisily)

3.

"That Suzuki is missing a wheel"

Missing -> (param1 -> Suzuki & param2 -> Wheel)

With these three sentences plus induction upgrade to ZOL, if we pass following:

Missing -> (param2 -> Wheel)

we should get a following result:

Missing -> (param1 -> Mazda & param2 -> Wheel)

Missing -> (param1 -> Suzuki & param2 -> Wheel)

And controlled deduction with these two sentences and

Missing & param1

should give:

Mazda

Suzuki

just regular ZOL upgraded with induction.

2012/2/25 YKY (Yan King Yin, 甄景贤) <generic.in...@gmail.com>

On Sun, Feb 26, 2012 at 2:09 AM, Ivan Vodišek <ivanv...@gmail.com> wrote:

[Ivan Vodišek]

Actually, I'm looking for universal method to convert a stream of characters like:

"Bison that rounded a snake while grazing, met a stack of green hay. But the bison waited for the hay to turn yellow before it started to eat. Once it turns yellow, hay becomes very tasty."

to corresponding term (variable) combination. Stream of characters is just one of possible input methods. Basically I'm missing universal learning algorithm that would convert any (1-Dimensional, 2-Ddimensional, ?, ...) input stream to internal set of terms (a KB).

2012/2/25 Ivan Vodišek <ivanv...@gmail.com>

[YKY (Yan King Yin, 甄景贤)]

Ivan,

Thanks to your questions, I'm now thinking more about "types"...

In predicate logic, a predicate is a function that takes one or more objects and returns a truth value.  In type theory the type of truth values is usually denoted by Ω.

Examples where it is difficult to assign types

So "bad" is a predicate:

    bad : obj -> Ω.

For example:

    bad(john)                      "John is bad"

But "bad" is also an adjective:

    bad : obj -> obj

For example:

    bad(apple)                     the concept of bad apple

    eats(john,bad(apple))    "John eats a bad apple"

Now compare

    bad(people)        the concept of bad people, type obj

    bad(people)        "People are bad", type Ω.

Their syntactic forms are exactly the same.  And it is unsatisfactory to regard the 2 versions of "bad" as different merely because their types are different.

The concept "very" seems to take a predicate and returns another predicate:

    very : (obj -> Ω) -> (obj -> Ω)

    (very(bad)) (john)          "John is very bad".

Another example:

    animal(john)                  "John is an animal"

    animal(dog)                  "dogs are animals"

    eats(john, animal)         "John eats animals"

so "animal" can be of 2 types:

    animal : obj

    animal : obj -> Ω.

Another example:

    blind(love)                   "Love is blind"

    love(john, mary)          "John loves Mary"

where the types are:

    love : obj

    love : obj -> (obj -> Ω).

What should we do?

My current thinking is that a syntactic form such as:

    bad(people)

does not possess a type intrinsically.  Whether it has a truth value or not, depends on its relation to other items in the KB.  Usually TVs can be passed from one item to another through the implication operator, →.

For example, inductively one could learn:

    John is bad           <TV>

    Mary is bad           <TV>

    Peter is bad          <TV>

    etc...

    => people are bad         <TV>

The conclusion has a TV because the examples have TVs.

Or, one can do the deduction:

    everything is bad → people are bad

             <TV>                     <TV>

So we can learn the relations between concepts, and facts (with truth values) about concepts.

In other words, the syntactic structures that have truth values induce other syntactic structures to have truth values, in a dynamic way.

KY

[Ivan Vodišek]

imho, the same data stored internally can be expressed to outer world in numerous ways. i.e. the same "bad(john)" predicate can be expressed differently in different contexts like "john is bad" and "bad john ate an apple", depending on how much detail we want to provide. but how to always choose the same form for memorizing, and what that form would be i do not know. it would be awsome if different forms of the same input data could be ranked somehow by relevant basis.

2012/2/26 YKY (Yan King Yin, 甄景贤) <generic.in...@gmail.com>

[YKY (Yan King Yin, 甄景贤)]

On Sun, Feb 26, 2012 at 6:08 AM, Ivan Vodišek <ivanv...@gmail.com> wrote:

Firstly, I want to thank You for interest, YKY :)

1.

"That Mazda is missing a wheel"

Missing -> (param1 -> Mazda & param2 -> Wheel)

2.

"That Suzuki runs noisily"

Runs -> (param1 -> Suzuki & param2 -> Noisily)

1st sentence:

    you:    missing -> (mazda, wheel)

    me:     missing ◦   (mazda, wheel)

               = (missing ◦ wheel) ◦ mazda

2nd sentence:

    you:    runs -> (suzuki, noisily).

    me:     (noisily ◦ runs) ◦ suzuki

That's because I think "noisily" modifies "runs".

I'm using a technique known as Currying, eg:

    loves ◦ (john, mary) = (loves ◦ mary) ◦ john

If we compare your 2 examples:

    you:    missing -> (mazda, wheel)

    you:    runs      -> (suzuki, noisily).

You have treated the 2 sentences uniformly, but they are somewhat different:

    Mazda is missing a wheel  !→ Mazda is missing

    Suzuki runs noisily              → Suzuki runs

where → means "implies".

KY

[Ivan Vodišek]

i'm also of opinion that every predicate could have numerous named parameters. for example:

runs(what, how fast, how long, what acceleration, ...)

but there is a method for describing these with fixed parameter number predicates:

runs (what:x)

howfast (what :runs (what :x), amount :100km/h)

howlong (what :runs (what :x), amount :100min)
...

i'm a fan of named parameters. i like their determitivity.

2012/2/27 YKY (Yan King Yin, 甄景贤) <generic.in...@gmail.com>

[YKY (Yan King Yin, 甄景贤)]

Sorry, in the last post some of my examples are problematic.  I found out a more uniform way to represent things not using Currying.

"John walks":

    walks ◦ john

"John walks slowly":

    (slowly ◦ walks) ◦ john.

This is because "slowly" modifies "walk".

But this is also equivalent to:

    slowly ◦ (walks ◦ john)

because the order of application does not matter.  This property is known as associativity:

    (a ◦ b) ◦ c = a ◦ (b ◦ c).

Associativity is the most fundamental property of category theory:  the composition of arrows is associative.  So I guess it is a nice property to have in the logic.

The problem with Currying is that associativity breaks down:

    "John loves Mary" = (loves ◦ john) ◦ mary

                                 ≠ loves ◦ (john ◦ mary).

So instead of Currying I use:

"John loves Mary"

    = (mary ◦ loves) ◦ john

    = mary ◦ (loves ◦ john)

Notice that the order of atoms is completely reversed from the English order.  That is because:

    (loves ◦ john)   means "John loves [something]"

    (mary ◦ loves)  means "[someone's] loving of Mary".

Notice that this is completely analogous to:

    (walks ◦ john)   means "John walks"

    (slowly ◦ walks) means "[someone's] walking slowly".

Thus:

"The Mazda is missing a wheel":

    (wheel ◦ missing) ◦ mazda

"The Suzuki runs noisily":

    (noisily ◦ runs) ◦ suzuki.

Notice the order is again the reverse of English.

(wheel ◦ missing) means "[something]'s missing of a wheel", and when that is applied to "mazda" it means the Mazda is missing a wheel.

"the missing of a wheel" = wheel ◦ missing

"a wheel is missing"       = missing ◦ wheel

"___'s loving of John"     = john ◦ loves

"John loves ___"            = loves ◦ john

But I still cannot solve the problem:

    "John gives Mary the ball"

which seems to violate associativity no matter how I translate it:

    (mary ◦ (gives ◦ john)) ◦ ball

    = mary ◦ ((gives ◦ john) ◦ ball)

    = mary ◦ (gives ◦ (john ◦ ball)) ??

where (john ◦ ball) seems meaningless  :(

On Tue, Feb 28, 2012 at 2:31 AM, Ivan Vodišek <ivanv...@gmail.com> wrote:

i'm also of opinion that every predicate could have numerous named parameters. for example:

runs(what, how fast, how long, what acceleration, ...)

The number of parameters (or arguments) is called "arity".

but there is a method for describing these with fixed parameter number predicates:

runs (what:x)

howfast (what :runs (what :x), amount :100km/h)

howlong (what :runs (what :x), amount :100min)
...

i'm a fan of named parameters. i like their determitivity.

But you seems to be creating parameters in an ad hoc manner.

Also, how would you translate:

    "John gives Mary the ball"?

KY

[Ivan Vodišek]

i think we need distinction between function and its parameters.

function would be:

gives

and then by currying, parameters can be applyed in whatever order.

2012/2/28 YKY (Yan King Yin, 甄景贤) <generic.in...@gmail.com>

[YKY (Yan King Yin, 甄景贤)]

On Tue, Feb 28, 2012 at 5:39 PM, Ivan Vodišek <ivanv...@gmail.com> wrote:

i think we need distinction between function and its parameters.

function would be:

gives

and then by currying, parameters can be applyed in whatever order.

But you create named parameters such as "make", "color", and then sometimes you have unnamed parameters like #1 = mazda, #2 = noisily.  All these seem to lack mechanisms to determine.

My approach is to not give concepts internal parameters, because I think concepts should be defined via their external relations with each other.

KY

[Ivan Vodišek]

how about math equations? they can be expressed in numerous ways, i.e. explicit ant implicit forms. should they all share the same internal structure?

2012/2/28 YKY (Yan King Yin, 甄景贤) <generic.in...@gmail.com>

On Tue, Feb 28, 2012 at 5:39 PM, Ivan Vodišek <ivanv...@gmail.com> wrote:

[Ivan Vodišek]

about naming conventions:

i concluded that the same universal theory about knowledge has many faces, so type theory, higher order logic, lambda calculus and possibly other less-known systems are all shapes of the same theory that has to have following constituents:

1. generalized form of predicates (predicate definitions)

2. notation for specialized predicate applications

3. a query language to reach specific specialized applications by their parameters.

moreover, as this universal theory can be defined in itself, all mentioned systems can be self defined in each of them, even one in the other. so, it doesn't matter what version will be selected. any of them could be used for storing knowledge.

2012/2/28 Ivan Vodišek <ivanv...@gmail.com>

[YKY (Yan King Yin, 甄景贤)]

On Wed, Feb 29, 2012 at 3:14 AM, Ivan Vodišek <ivanv...@gmail.com> wrote:

about naming conventions:

i concluded that the same universal theory about knowledge has many faces, so type theory, higher order logic, lambda calculus and possibly other less-known systems are all shapes of the same theory that has to have following constituents:

1. generalized form of predicates (predicate definitions)

2. notation for specialized predicate applications

3. a query language to reach specific specialized applications by their parameters.

moreover, as this universal theory can be defined in itself, all mentioned systems can be self defined in each of them, even one in the other. so, it doesn't matter what version will be selected. any of them could be used for storing knowledge.

But we need to pick one format for implementation, otherwise this will lead to chaos and nothing gets done.

I've thought more about

    "John gives Mary the ball"

which may be rendered as:

    (mary, the ball) ◦ gives ◦ john.

The statement is equivalent to:

    mary ◦ gives ◦ john

&  the ball ◦ gives ◦ john.

This looks a bit awkward but seems to convey the meanings fine.

I will work out a set of real examples (they're already in my book, but were using the older version of logic).

What I'm trying to do is to design a general logic that can render NL most naturally and elegantly.  I believe it could save a lot of implementation efforts.

Humans should be able to translate NL into Geniform easily (ie, without any arbitrariness or indeterminacy).  But, Geniform still requires dynamic interpretation on the AGI's part -- it has to occur during inference.  For example, we can infer that:

    glass slippers are made of glass

    door knob is part of the door

    street prostitute works in the streets

however,

    glass slippers does not work in glass

    door knob is not made of door

    street prostitute is not part of the street.

In other words, dynamic interpretation (using KB knowledge) is a must, there is no escape of doing without it.

But the rendering of NL into logical form is deterministic and unambiguous;  at least that what my design strive to do...

KY

[Matt Mahoney]

2012/2/28 YKY (Yan King Yin, 甄景贤) <generic.in...@gmail.com>:

>     "John gives Mary the ball"

The goal is to answer questions like?

Who had the ball? (John).
Who has the ball? (Mary).
Did Mary give the ball to John? (no).
Did John give the ball to Mary? (yes).
Who took the ball? (Mary).
What did John give to Mary? (the ball).

How do humans solve the problem? What is the representation you formed
in your mind? What steps did you use to solve the problems? In other
words, how does the brain do it?

-- Matt Mahoney, mattma...@gmail.com

[james northrup]

all that remains to be seen is a test corpus, a kb, some code, and some useful results.

how hard can that be? :)

i find it interesting that no discussion is taking place as to how to fill in the gaps on large machine-readable kb's such as dbpedia and yago data, nor even consideration for the machinery required to deliver results on a useful scale or even the articulation of a proposal for those who might have such machinery to borrow for the task.

Matt proposes a prediction model and an open ended brew of stochastic network participants to follow attention and glean structure costing a digit followed by 12 zero dollars. 

YK is proposing a lisp or prolog invention that is more or less hand-coded perfection to account for all variance.

Ivan is saying it's all concepts and bifurcating types without yet really showing a bias beyond discussion terms.

i'm with Ivan, actually, because of the feasability of expressing types without assuming the perfection of types, using a c++ compiler, or an arsenal of compilers.  using c++ types allows one to use type classification in code, while not ruling out using c++ for procedures.  classification, in particular declarative classifications at compile time, can provide for machine coded perfection of type-based decisions, as perfect or as imperfect as those types may seem.  minor note, type-based-inference is part of c++.

Matt has in the past brought up the proposition that the complexity of structure must have a low lyapunov exponent, or in other words be extremely non-biased in permutation-land.  for this I propose that the perfect or imperfect type-based decisions of a c++ compiler could drive the generation of c++ for future compilation based on source code as a primitive with low lyapunov structure.  the profile of such segments of code can focus on relatively few facets of logic and knowledge, providing high streamability of IO and low overhead of RAM and CPU/GPU resources following the seams of a kb declaration.

at some level those who have invented corpi for scheme, lisp, and the like and call it X% accurate, such as the yago corpus from Max Plank Institute or to a much smaller degree cyc must intend for programmable logic to follow the assembly of assertions they have put together, and I beleive translating the assertions from scheme language intentions to c++ is a short gap considering polymorphism possed by both programming languages and the low lyapunov nature of predicate logic delivered as rdf and graph store data.

one thing i think is agreed upon by all is the need for scale, and at least prior to the efficiency of millions of runtime interpretters sharing cores and state -- i imagine there will need to be a pre-attention classifier layer in the form of object code selected from the most relevant or perfect type-based c++ decisions extended from the imperfect type primitives which drive the expression, and possess human oversight for relevance and correctness.  derived at least minimally from groomed kb's, and shaped further by human and statistical decisions.  Matt has stated in the past sometimes he winds up editing code without a clue why it improves something.  this is pretty reasonable on this scale.

this c++ compiled classifier collection would produce prediction input from raw input, and could do so with abstraction efficiencies at compile time unavailable to other languages to generally perform at or better than hand coded perfection for mildly imperfect type systems.

i think this may support Matt's idealism to accelerate classification, and therefore testable prediction models.  i do not quite think that brute force "promoting like" and "deleting dislike" is going to prove predictable against the reward biases presented and the cleverness of humans to steer capital to their own ends.

at the end of the day given c++'s language polymorphism, YK's inventions, ambiguously provable as perfect and correct, can be formed into the template language of the classifier layers(c++) using c++ renditions of lisp, prolog, or scheme, or self exported c++ renditions of same.

while the entirety of the above is assumed to misrepresent all facts and features to some degree, I think it's as good as it needs to be to propose to an entity with vast quantities of machines that the approach is worth experimentation, support, and research, using their resources. 

again, how hard can that be?

2012/2/28 YKY (Yan King Yin, 甄景贤) <generic.in...@gmail.com>

On Wed, Feb 29, 2012 at 3:14 AM, Ivan Vodišek <ivanv...@gmail.com> wrote:

[Matt Mahoney]

On Tue, Feb 28, 2012 at 4:42 PM, james northrup
<northru...@gmail.com> wrote:
> Matt proposes a prediction model and an open ended brew of stochastic
> network participants to follow attention and glean structure costing a digit
> followed by 12 zero dollars.

15, actually.

> i think this may support Matt's idealism to accelerate classification, and
> therefore testable prediction models.  i do not quite think that brute force
> "promoting like" and "deleting dislike" is going to prove predictable
> against the reward biases presented and the cleverness of humans to steer
> capital to their own ends.

Actually my proposal is explicitly about making money. But there are
still things we can do, short of automating the economy. But we won't
get anywhere without being more explicit about what that is.

So I would like to hear some suggestions. Exactly what are we going to
build? What are the inputs and what are the outputs? What tests must
be passed to declare success?

-- Matt Mahoney, mattma...@gmail.com

[james northrup]

to my knowledge a template metaprogramming c++ compiler doesn't yet exist, and without knowing exactly who's postulate to quote about complexity, turing completeness, etc. the evidence that raytracers exist at compile-time leads me to believe a c++ compiler is not too far off as pure template metaprogramming at compile-time.  the difference between a finished raytraced picture as object code and a c++ compiler as object code is that the latter can compile c++, and if in fact it is an optimizing compiler, stands to create a baseline for things to come in optimization and purity.

this would be one accomplishment measurable by fairly concrete terms.

the next would be to exploit the c++ language template type system as a set of rules which perform ontological mapping assertions in the simplest unit of compilation such that machine readable kb's which exist today could be sufficiently translated in part or in whole to compilation units and in fact the above mentioned c++ compiler could isolate only the parent ontologies and parent compiler featureset needed to compile a given compilation unit, when fully resolved.
the virtual memory req

[james northrup]

 random stray key hit sent.  lovely.  continuing.

 
the virtual memory requirements of a) a c++ compiler at compile-time or b) a multi-gig/terabyte kb at compile time are where the resource considerations seem to matter.

first, there probably are no compilers which could really hammer out executables based on gigs of pure template source code, none by the standards of compilers which receive attention for production optimized code in present tense.

so mixing the compile-time compilation with other ontologically modeled features or handwritten algorithms to break units of compilation down into the machinery we have on hand to process kb-sized ontological mappings to the c++ type system is a challenge which could be tested on kilobytes of tuples, megabytes of tuples, gigabytes of tuples and so forth with compile-time compilers

the goal of the proposal i posed a few posts up is for a classifier generation system from which assertions about input can be minted against vast IO loads and concurrency.  the net result would be more ontologically corresponding template code to feed into the c++ compilation lifecycle.  this does not scale depth-wise for concepts without a very determined upper-level ontology and i haven't seen one succeed to date, not that i've seen all of them yet.

so beyond "and so forth" the potential pool of classifier executables may wind up a fraction of the volume of output artifacts  from the previous generation of classifiers, plus or minus the procedural adjustments to the outcome initially specified by humans.

at this point we can debate the capabilities of c++ as a query facility.  can you write a c++ program to procedurally accomplish querying of prior data? sure, but why not let the compiler bake an executable specific to the once-data.  unused (orphaned) data nodes in a parent ontology are not sent to the assembler and dont occupy executable footprint nor resources at runtime.  so running the ontology nodes themselves as executables instead of data through a straw is the experiment that seems to break the mold, when considering replacing a database with a processing core array.

to say I'm an AI researcher might be a stretch, but if i were, I'd want a compiler bootstrapped n-levels deep into its own constructs, i'd want algorithmic guarantees based on c+++ templates, and i'd want the performance of compiled code that solves a trivial problem by using as much template-based fact assertion as can be afforded by its host organization, and as few c++ features to narrow down the pure logic dictating the procedural and functor composition that may be prescribed by such programs.

[Ivan Vodišek]

Here is translated paragraph:

"Bison that rounded a snake while grazing, met a stack of green hay. But the bison waited for the hay to turn yellow before it started to eat. Once it turns yellow, hay becomes very tasty."

While(Grazing, Rounded(Bison, Snake))

Met(Bison, Stack(Green(Hay)))

Before(Start(Eat), Waited(Bison, Turns(Hay, Yellow))

When(Turns(Hay, Yellow), Very(Tasty(Hay)))

It seems no problem to write a parser that would parse any paragraph to predefined predicates. But the real problem is to automatically build and learn new predicates from the stream of characters. I could use a little help there.

2012/2/29 james northrup <northru...@gmail.com>

[james northrup]

On 2/29/2012 2:09 AM, Ivan Vodišek wrote:

Here is translated paragraph:

"Bison that rounded a snake while grazing, met a stack of green hay. But the bison waited for the hay to turn yellow before it started to eat. Once it turns yellow, hay becomes very tasty."

While(Grazing, Rounded(Bison, Snake))

Met(Bison, Stack(Green(Hay)))

Before(Start(Eat), Waited(Bison, Turns(Hay, Yellow))

When(Turns(Hay, Yellow), Very(Tasty(Hay)))

It seems no problem to write a parser that would parse any paragraph to predefined predicates. But the real problem is to automatically build and learn new predicates from the stream of characters. I could use a little help there.

at face value, a collection of such statements could be hand coded, or competitive-learning construction based on this article, as it applies to the experiment I propose on other peoples processing

http://cpp-next.com/archive/2010/08/expressive-c-introduction/

building blocks of boost proto (and other TMP constructs) with a low lyapunov exponent would be the real-world embodiment that seems doable, beyond the human-maintained scale of detail.

[Matt Mahoney]

On Wed, Feb 29, 2012 at 5:09 AM, Ivan Vodišek <ivanv...@gmail.com> wrote:
> Here is translated paragraph:
>
> "Bison that rounded a snake while grazing, met a stack of green hay. But the
> bison waited for the hay to turn yellow before it started to eat. Once it
> turns yellow, hay becomes very tasty."
>
> While(Grazing, Rounded(Bison, Snake))
> Met(Bison, Stack(Green(Hay)))
> Before(Start(Eat), Waited(Bison, Turns(Hay, Yellow))
> When(Turns(Hay, Yellow), Very(Tasty(Hay)))
>
> It seems no problem to write a parser that would parse any paragraph to
> predefined predicates. But the real problem is to automatically build and
> learn new predicates from the stream of characters. I could use a little
> help there.

How do you parse the following?

I ate pizza with pepperoni.
I ate pizza with a fork.
I ate pizza with Bob.

The problem of parse ambiguity has been known since around 1960.
People can parse sentences because they know what the words mean.
People can figure out that "with pepperoni" modifies "pizza", "with a
fork" modifies "ate", and "with Bob" modifies "I". A machine could
figure it out too, based on co-location statistics in a large text
corpus. Words with related meanings are more likely to appear near
each other or in the same contexts.

But parsing is not the critical problem to be solved. The meaning of a
sentence depends mostly on the words it contains and much less on
their order. For example, if I say "with pizza pepperoni I ate", you
can figure out what I meant, and then put the words in the right
order.

Grammar is a set of rules that constrain word order. It's purpose is
to increase the reliability of communication in noisy environments. If
you can't hear half of the words, you can often figure out what was
said by applying the rules of grammar to narrow the set of
possibilities. In fact, that's what we do without thinking about it.
Nobody realized how noisy speech was until we started writing speech
recognition systems.

Returning to the original problem. You have some facts like "John gave
Mary the ball" (an exception to the rule that word order is usually
unimportant), and some questions like "who has the ball?" or "what did
John give to Mary?". You might notice that these questions can all be
answered simply by matching terms to the original fact and recalling
it verbatim, "John gave Mary the ball". It requires no deep
understanding of the sentence structure, either of the facts or the
queries. A thesaurus does help, so e.g. you can match "give" to
"gave".

Nevertheless, a model of grammar is useful, perhaps not so much for
understanding, as for error correction in noisy environments like
speech recognition, OCR, language translation, handwriting
recognition, and proofreading. This is a poorly understood problem.
Chomsky has tried to enumerate hundreds or thousands of rules of
English grammar, but it seems far from complete. We really don't know
how many rules there are, maybe millions. Why do we say "salt and
pepper" and not "pepper and salt"?

Humans apparently learn grammar by example. Perhaps it is a process we
can simulate. If I say "Blug gave the florgle the purg", you know
something about the grammatical role of the nonsense words. You don't
know what they mean, but you do know that they are nouns. You would
recognize that "the florgle has two purgs" is correct, even though you
have never before seen the word "purgs".

Here is a challenge. Given a bilingual dictionary (say Spanish to
Japanese), and 1 GB of random text in each language (not translations
of each other), write a translator. Do you think it is possible?

I think it is. But a direct application of the dictionary is
insufficient. For example, in Japanese, a sentence is
subject-object-verb instead of subject-verb-object. In Spanish,
adjectives come after nouns, not before. A direct translation would
put the words in the wrong order. But knowing the grammar rules, you
could rearrange them. Also, each word in one language would map
inexactly to several words in the other. Again, you could figure out
which is the correct meaning by context. So the challenge to you is to
develop an algorithm for learning a language model from a corpus. What
do you think?

-- Matt Mahoney, mattma...@gmail.com

[YKY (Yan King Yin, 甄景贤)]

On Wed, Feb 29, 2012 at 4:18 AM, Matt Mahoney <mattma...@gmail.com> wrote:

The goal is to answer questions like?

Who had the ball? (John).
Who has the ball? (Mary).
Did Mary give the ball to John? (no).
Did John give the ball to Mary? (yes).
Who took the ball? (Mary).
What did John give to Mary? (the ball).

How do humans solve the problem? What is the representation you formed
in your mind? What steps did you use to solve the problems? In other
words, how does the brain do it?

I don't care how the biological brain does it.  I believe Genifer can do it (in theory), and that's enough.

Genifer will store many pieces of knowledge about the act of "giving".  For example,

    "if X give Y Z then Z will be in the procession of Y afterwards"

Such rules can be expressed in logic, especially if the logic matches NL closely -- which is what I've been trying to do.

KY

[YKY (Yan King Yin, 甄景贤)]

On Wed, Feb 29, 2012 at 6:09 PM, Ivan Vodišek <ivanv...@gmail.com> wrote:

Here is translated paragraph:

"Bison that rounded a snake while grazing, met a stack of green hay. But the bison waited for the hay to turn yellow before it started to eat. Once it turns yellow, hay becomes very tasty."

While(Grazing, Rounded(Bison, Snake))

Met(Bison, Stack(Green(Hay)))

Before(Start(Eat), Waited(Bison, Turns(Hay, Yellow))

When(Turns(Hay, Yellow), Very(Tasty(Hay)))

Here is my version, omitting minor details such as tense, number, determiners, etc:

p1 := [ [(gazing ◦ while), snake] ◦ rounded), (green ◦ hay ◦ stack ◦ met) ] ◦ bison

This diagram may be easier to understand:

      

Each directed edge represents an "application", ◦.

Notice that any pair of application is a meaningful compound concept, even "while ◦ rounded", which means the concept of "[doing something] while rounded".

"p1" denotes proposition #1.  The general rule is that once a syntactic tree has a truth value, it should be regarded as one proposition, and further applications modifying that sentence should occur in a new proposition.

p2 := [(yellow ◦ turn ◦ to), (hay ◦ for)] ◦ waited ◦ bison

p3 := p4 ◦ but ◦ p1

p4 := p5 ◦ before ◦ p2

p5 := eat ◦ to ◦ started ◦ bison

p6 := yellow ◦ turns ◦ hay

p7 := p6 ◦ once ◦ p8

p8 := very ◦ tasty ◦ becomes ◦ hay

Notice that the above formulas are almost the same as English when read backwards!

It seems no problem to write a parser that would parse any paragraph to predefined predicates. But the real problem is to automatically build and learn new predicates from the stream of characters. I could use a little help there.

The predicates' names correspond to English words exactly.  There may be minor troubles arising from homonyms, such as "sun" and "son", or "lie" (to lie down) and "lie" (to be dishonest), but I'm quite satisfied with this scheme so far.  Notice that there would still be a mapping from English to logic, represented by logic formulas, but such a mapping may be relatively trivial because of the logic's close resemblance to English.

KY

[Ivan Vodišek]

> Notice that there would still be a mapping from English to logic, represented by logic formulas,

> but such a mapping may be relatively trivial because of the logic's close resemblance to English.

i'm of opinion that logic operators could be learned in practice, just like regular predicates. something like

generalization:

And (p1 :Bool,  p2 :Bool, result ;Bool)

specialization:

And (p1 :True,   p2 :True,  result :True)

And (p1 :True,   p2 :False, result :False)

And (p1 :False,  p2 :True,  result :False)

And (p1 :False,  p2 :False, result :False)

i beliave that if we put al logic operators on total induction, we could derive the very resolution rule also.

2012/3/1 YKY (Yan King Yin, 甄景贤) <generic.in...@gmail.com>

[YKY (Yan King Yin, 甄景贤)]

On Thu, Mar 1, 2012 at 8:30 PM, Ivan Vodišek <ivanv...@gmail.com> wrote:

i'm of opinion that logic operators could be learned in practice, just like regular predicates. something like

generalization:

And (p1 :Bool,  p2 :Bool, result ;Bool)

specialization:

And (p1 :True,   p2 :True,  result :True)

And (p1 :True,   p2 :False, result :False)

And (p1 :False,  p2 :True,  result :False)

And (p1 :False,  p2 :False, result :False)

i beliave that if we put al logic operators on total induction, we could derive the very resolution rule also.

Yes it seems possible, by stepping back to program space, where an AGI can search for logical forms.

You may want to check out the AIXI theory of Marcus Hutter:

http://vimeo.com/14888930

And then Tom Schaul proposes that AIXI can be biased by a domain language (such as a logic) to make the search more efficient (indeed, feasible at all):

http://vimeo.com/15817417

But such a choice must be imposed from externally to be efficient, which is why I would suggest to design the logic by hand rather than use AGI to search for the logic.

Note however that in Bayesian logic, the binary operators AND, and OR are generalized to conditional probability tables...

PS:  It would be nice to have a GUI that lets users input "graph English" like the diagram I made.  The user would have to specify how the graph is shaped.  But, this kind of parsing is optional because later, we would encode the translation rules of NL -> logic in the logic itself, and then NL-understanding would be performed by inference using the inference engine.

KY

[YKY (Yan King Yin, 甄景贤)]

On Thu, Mar 1, 2012 at 4:35 AM, Matt Mahoney <mattma...@gmail.com> wrote:

How do you parse the following?

I ate pizza with pepperoni.
I ate pizza with a fork.
I ate pizza with Bob.

[(pepperoni ◦ with), pizza] ◦ ate ◦ I

[(a ◦ fork     ◦ with), pizza] ◦ ate ◦ I

[(bob           ◦ with), pizza] ◦ ate ◦ I

So the syntactic forms are identical, which does not resolve the semantic disambiguation problem.  The word "with" here seems to be overloaded with several meanings but those meanings seem to have  a certain abstract unity, so it seems OK to group them under the same concept "with".

When asked questions such as:

"Did the pizza have pepperoni?"

"Did I ate bob?"

"Did I use a fork?"

etc, these questions can be answered by drawing additional knowledge from the KB -- those are the knowledge that establishes the semantics of concepts.

As an aside,

ate = (did ◦ eat)

which is the way I'd represent tenses.  Curiously, linguists believe that the English past tense actually evolved from saying "did" after verbs, eg: "round did" ==> "rounded".

But parsing is not the critical problem to be solved. The meaning of a
sentence depends mostly on the words it contains and much less on
their order. For example, if I say "with pizza pepperoni I ate", you
can figure out what I meant, and then put the words in the right
order.

Right, word order is of no essence to sentence meaning.  In my proposed logic, however, the order is fixed as application order.  So, "very ◦ bad" is the concept of "very" applied to "bad", and not the other way.  This makes the logic more well-defined than NL, making the inference algorithm simpler and more efficient.

Nevertheless, a model of grammar is useful, perhaps not so much for
understanding, as for error correction in noisy environments like
speech recognition, OCR, language translation, handwriting
recognition, and proofreading. This is a poorly understood problem.
Chomsky has tried to enumerate hundreds or thousands of rules of
English grammar, but it seems far from complete. We really don't know
how many rules there are, maybe millions. Why do we say "salt and
pepper" and not "pepper and salt"?

"Salt and pepper" is probably idiomatic, handed down to us from tradition.

In my logical form,

   (a, b) = (b, a)

   (a, b) ◦ c = (a◦c, b◦c)

so "salt and pepper" would be equivalent to "pepper and salt".  Some information (that seems to be called "prosody", eg, how we stress different parts of sentences, otherwise equivalent) would be lost in such a representation, and we may need to add extra tags (as predicates) to encode prosody.  This is not a significant problem.

Here is a challenge. Given a bilingual dictionary (say Spanish to
Japanese), and 1 GB of random text in each language (not translations
of each other), write a translator. Do you think it is possible?

I think it is. But a direct application of the dictionary is
insufficient. For example, in Japanese, a sentence is
subject-object-verb instead of subject-verb-object. In Spanish,
adjectives come after nouns, not before. A direct translation would
put the words in the wrong order. But knowing the grammar rules, you
could rearrange them. Also, each word in one language would map
inexactly to several words in the other. Again, you could figure out
which is the correct meaning by context. So the challenge to you is to
develop an algorithm for learning a language model from a corpus. What
do you think?

That would be an interesting application, but we'd be competing with existing services like Google Translate...

KY

[Matt Mahoney]

2012/3/1 YKY (Yan King Yin, 甄景贤) <generic.in...@gmail.com>:

> I don't care how the biological brain does it.  I believe Genifer can do it
> (in theory), and that's enough.
>
> Genifer will store many pieces of knowledge about the act of "giving".  For
> example,
>     "if X give Y Z then Z will be in the procession of Y afterwards"
> Such rules can be expressed in logic, especially if the logic matches NL
> closely -- which is what I've been trying to do.

I'm pretty sure this rule is already encoded in OpenCyc. What is your
estimate of the number of rules that need to be written to achieve
human level natural language understanding? How many person-years will
be needed?

-- Matt Mahoney, mattma...@gmail.com

[Matt Mahoney]

2012/3/1 YKY (Yan King Yin, 甄景贤) <generic.in...@gmail.com>:

> That would be an interesting application, but we'd be competing with
> existing services like Google Translate...

Maybe we could do it better. Or maybe there is another application
where there is no competition. Mailpool comes to mind.

Also, a language model has many uses besides translation. It could
also be used for speech recognition, OCR, handwriting recognition, and
text classification (spam detection). In each case, you can measure
progress using text compression.

-- Matt Mahoney, mattma...@gmail.com

[Ivan Vodišek]

what about senses other than text stream? how would agi look for sense of touch, vision, hearing, at least for non-text cognitions? is our kind of speech (subjects, predicates, ...) only kind of communication possibility? by the way, subjects seem to me as zero-parameter particles of the same kind as predicates with (one or two or three)-parameter particles. adjectives could be one-parameter particle of the same kind.

in fact, text stream represents one-dimensional values changed over time. vision is 2-d * time. space perception is 3-d * time. so the question is: is 1-d * time enough to reason about any more complex systems?

2012/3/1 Matt Mahoney <mattma...@gmail.com>

[Matt Mahoney]

On Thu, Mar 1, 2012 at 10:26 AM, Ivan Vodišek <ivanv...@gmail.com> wrote:
> what about senses other than text stream? how would agi look for sense of
> touch, vision, hearing, at least for non-text cognitions? is our kind of
> speech (subjects, predicates, ...) only kind of communication
> possibility? by the way, subjects seem to me as zero-parameter particles of
> the same kind as predicates with (one or two or three)-parameter particles.
> adjectives could be one-parameter particle of the same kind.

There are plenty of AI applications in image and video recognition.
How about an application that inputs a video and decides if it is
funny. Then you could set up a website with links to the funniest
ones. What algorithm would you use? There must be one, because humans
can do it.

-- Matt Mahoney, mattma...@gmail.com

[Ivan Vodišek]

here is algos:

if someone walks then hits the sun with head, it's funny wether it is john, mary or cindy :)

2012/3/1 Matt Mahoney <mattma...@gmail.com>

[Matt Mahoney]

On Thu, Mar 1, 2012 at 10:51 AM, Ivan Vodišek <ivanv...@gmail.com> wrote:
> here is algos:
> if someone walks then hits the sun with head, it's funny wether it is john,
> mary or cindy :)

Great. Write the program and let us know.

But seriously, how would you do it?

> 2012/3/1 Matt Mahoney <mattma...@gmail.com>
>>
>> On Thu, Mar 1, 2012 at 10:26 AM, Ivan Vodišek <ivanv...@gmail.com>
>> wrote:
>> > what about senses other than text stream? how would agi look for sense
>> > of
>> > touch, vision, hearing, at least for non-text cognitions? is our kind of
>> > speech (subjects, predicates, ...) only kind of communication
>> > possibility? by the way, subjects seem to me as zero-parameter particles
>> > of
>> > the same kind as predicates with (one or two or three)-parameter
>> > particles.
>> > adjectives could be one-parameter particle of the same kind.
>>
>> There are plenty of AI applications in image and video recognition.
>> How about an application that inputs a video and decides if it is
>> funny. Then you could set up a website with links to the funniest
>> ones. What algorithm would you use? There must be one, because humans
>> can do it.
>>
>>
>> -- Matt Mahoney, mattma...@gmail.com
>
>

--
-- Matt Mahoney, mattma...@gmail.com

[Ivan Vodišek]

what's funny? you got me. i guess it has some deal with rare coincidences. and it has to have taste. too banal and provocative humor can be horrible.

[YKY (Yan King Yin, 甄景贤)]

On Thu, Mar 1, 2012 at 11:57 PM, Matt Mahoney <mattma...@gmail.com> wrote:

But seriously, how would you do it?

From my observation, humor usually comes from irrational ideas with some kind of rare coincidence.

The last joke I've heard is during the AGI-2010 video, when the announcer said "good morning everybody, thanks for coming to church so early".

It's funny to regard the Singularity as a religion because most believers in hard-core technology and life-extension etc, are atheists.  Yet we also display some religious zeal... at least I do =)

KY

[YKY (Yan King Yin, 甄景贤)]

Correction:  i meant illogical, not exact irrational

[Matt Mahoney]

How about something simpler, then. Match images of faces to names. How
would we do it?

I am just brainstorming for ideas.

[Ivan Vodišek]

we have to isolate characteristic points that has every face.

so for eyes there have to be two circles in colors.

mouth? maybe pink ellipse?

and then measure angles, distances and sizes of and between all of them.

in real time? i would say dream on, but such applications really exist.

for ocr there is a method of reducing part of bitmap with letter to, say 10*10 px. then compare it with a sample. if it matches over, maybe 80%, then that's the letter. but it's about b/w bitmaps. matching colours would be more complex.

 

2012/3/1 Matt Mahoney <mattma...@gmail.com>

[YKY (Yan King Yin, 甄景贤)]

On Thu, Mar 1, 2012 at 11:26 PM, Ivan Vodišek <ivanv...@gmail.com> wrote:

what about senses other than text stream? how would agi look for sense of touch, vision, hearing, at least for non-text cognitions? is our kind of speech (subjects, predicates, ...) only kind of communication possibility? by the way, subjects seem to me as zero-parameter particles of the same kind as predicates with (one or two or three)-parameter particles. adjectives could be one-parameter particle of the same kind.

in fact, text stream represents one-dimensional values changed over time. vision is 2-d * time. space perception is 3-d * time. so the question is: is 1-d * time enough to reason about any more complex systems?

I have a chapter on vision in my book, that describes a logic-based approach -- it sounds a bit unusual to use logic on vision, but I've found it to be very effective (at least in theory, no experimental proof yet).

Vision requires a feature-extraction layer (ie, finding "edgels" from the image) where logic may not be most effective.  After that layer, any generic logic will do.  At the object-recognition level, the target concepts are NL concepts such as "a chair missing a leg", where Geniform would be advantageous.

I have tried to find someone to code the feature extraction layer before, but no one was interested so it was put on hold.  It requires some expertise in machine vision...

KY

[Ivan Vodišek]

i'm trying to find universal method for text, vision, sound... maybe it has to do with reconstruction ability. take letters for example. for letter A we draw three lines. now what's drawing? making lines with a pencil. by that we remembered the shape of letter A. maybe there can be some imaginary rotations and translations of dots we build just in mind by which we can recognize things. but for now, i'm concentrating on recognizing words and learning any provided language. 

2012/3/1 YKY (Yan King Yin, 甄景贤) <generic.in...@gmail.com>

On Thu, Mar 1, 2012 at 11:26 PM, Ivan Vodišek <ivanv...@gmail.com> wrote:

[YKY (Yan King Yin, 甄景贤)]

On Fri, Mar 2, 2012 at 12:03 AM, Matt Mahoney <mattma...@gmail.com> wrote:

How about something simpler, then. Match images of faces to names. How
would we do it?

I am just brainstorming for ideas.

The application I've been considering is question-answering, and I thought programming would be the first domain to focus on.  A friend advised that programmers tend to know a lot of places to get their info, because they're tech savvy, so it's better to target people who are more in need of informational help, perhaps medical?

But targeting programmers may also help us recruit more developers...

KY

[Matt Mahoney]

What test data can we collect.

For face recognition, we would need a large collection of tagged
photos (like facebook has).

For OCR, we need scanned text, which is not too hard to collect.
Ideally we should be able to read captchas.

For a programming assistant, perhaps we could collect some software
and introduce random bugs and let the system find them.

2012/3/1 YKY (Yan King Yin, 甄景贤) <generic.in...@gmail.com>:

--
-- Matt Mahoney, mattma...@gmail.com

[YKY (Yan King Yin, 甄景贤)]

On Fri, Mar 2, 2012 at 1:07 AM, Matt Mahoney <mattma...@gmail.com> wrote:

For a programming assistant, perhaps we could collect some software
and introduce random bugs and let the system find them.

Solving programming problems requires intelligent planning.  That may be difficult for our 1st prototype to do.

I was thinking simpler questions such as:

"What are the functions to read files in Lisp?"

for which simple retrieval of facts can answer.  So we would render a lot of texts about programming into logical form, and use logical queries to get answers.  Simple inference with not too much depth can be achieved (I guess).

KY

[Matt Mahoney]

2012/3/1 YKY (Yan King Yin, 甄景贤) <generic.in...@gmail.com>:

AdaptiveAI (Peter Voss) is doing something similar. His goal is a text
based question answering program. He started by collecting test sets
on about 20 different topics with 50 questions on each topic.

I'm no longer on their mailing list. They started a new list for
people actively working on the project, which I'm not.

I don't know what approach they plan to use. I did get a hint, some
mention of having to "redo the ontology". Their earlier work was on an
automated phone answering service a couple of years ago. I don't know
how well it actually worked.

[YKY (Yan King Yin, 甄景贤)]

Minimum requirements for a prototype to perform Q&A:

1.  equational unification

-- algorithm is standard but yet to be figured out

2.  Bayesian network / factor graph

-- algorithm known, code 90% complete in Lisp

3.  fuzzy-probabilistic calculus

-- algorithm known, relatively easy to code

4.  distributive inference via message-passing

-- algorithm known, non-distributive version already coded in Lisp

5.  distributive clustering

-- algorithm is standard but needs to be adopted to the distributive setting

6.  inductive learning

-- naive algorithm known, lots of room to be improved

7.  GUI

-- relatively straightforward

KY

[Matt Mahoney]

By "requirements" I meant like the ability to do the following with
human level (or some other level of) speed and accuracy:
- Answer questions in natural language.
- Predict or correct errors in text.
- Solve crossword puzzles.
- Given a bilingual dictionary and 1 GB of random text in each
language, learn to translate.
- Write, test, and debug programs.
- Identify people and common objects in images.
- Recognize speech and common sounds.
- Transcribe scanned text, including captchas.
- Match videos to their written descriptions.
- Recognize pornography (either to create it or to block it).
- Recognize humor.
- Recognize music by artist and genre.
- Given a robotic arm, pick up, place, throw, and catch objects on command.
- Given legs or wheels, navigate on roads or over rough terrain on command.
You can probably think of others.

Step 2 would be to collect test cases for the problems we have decided
to solve. For example, if we decide that we want to recognize when two
photos are of the same person, then we need a collection of tagged
images. We would also give the test to humans to obtain a baseline
score.

Step 3 is research. What progress have others made toward solving the problem?

Step 4 would be to design the system. The design would include a
budget for computing hardware, for software development, and for
collecting training data.

Step 5 would be to recruit people to solve the problem, if it turns
out we don't have the skills, time, or money to do it ourselves.

Step 6 would be to start building it. This would be the appropriate
time to choose a programming language.

2012/3/3 YKY (Yan King Yin, 甄景贤) <generic.in...@gmail.com>:

--
-- Matt Mahoney, mattma...@gmail.com

[Ivan Vodišek]

i think that a question answering machine would be a great experiment to learn from, even in mono-language support.

2012/3/3 Matt Mahoney <mattma...@gmail.com>

[Matt Mahoney]

On Sat, Mar 3, 2012 at 2:17 PM, Ivan Vodišek <ivanv...@gmail.com> wrote:
> i think that a question answering machine would be a great experiment to
> learn from, even in mono-language support.

Great. Here is a list of test questions I generated by clicking on the
random page feature of Wikipedia and then writing a question that is
answered in the first paragraph. To measure the state of the art, I
typed each question into Google just as I wrote it below. The number
in parenthesis is the rank of the original Wikipedia article. In most
cases when the article was not ranked first, the higher ranked pages
also answered the question.

1. (>10) What is another name for the red-pied lovebird?
2. (1) What is the collective name of the aliens on "Babylon 5"?
3. (2) Who is the Indian princess rescued by Phileas Fogg in Jules
Verne's "Around the World in 80 Days"?
4. (1) What is the original name of Grover, Utah?
5. (2) Who is Markandeya?
6. (1) Who is Liam Ó Buachalla?
7. (2) Who is Ida Bailey Allen?
8. (1) Where is the architect Richard Purcell from?
9. (>10) What is Mycetosoritis?
10.(1) Who was the Consul of Rome in 58 BC?
11.(1) Where were the 2003 Pan American Games?
12.(1) Who became head of the astronomy observatory at Lund
University in 1929?
13.(1) When was the York Cold War Bunker built?
14.(1) Who is Rohan Bail?
15.(1) What does the program "Beyond Compare" do?
16.(1) What actor began his career in the 1952 movie "Tia Loleng" by
LVN Pictures in the Philippines?
17.(3) Who directed the HBO series, Deadwood?
18.(2) How many species are there of diapriidae?
19.(2) In what year was the English writer Michael Chaplin born?
20.(2) What was the population of Crittenden County, Kentucky in 2000?

So steps 1 and 2 are done, unless anyone else would like to suggest
some other problems.

Note: I originally sent this list to the projectAlgo group when Peter
Voss was proposing a new AI project. But I think they were looking for
easier questions.

[Ivan Vodišek]

i would propose somehing like:

if i had 2 brothers and a sister, how much my parents have children?

2012/3/3 Matt Mahoney <mattma...@gmail.com>

[james northrup]

LarKC: the Large Knowledge Collider was onto something in terms of progress afaict.  without getting into specifics, they managed to get funding for a bunch of research into solving the problems similar to the cyclical discussions taking place here, without specifying a concrete end-result nor a discrete financial beneficiary.  Their premise seems to be the lack of preparation by enterprise to manage the entirety of what's predicted in terms of emergent data volumes in the wild. 

i would estimate that in order for general intelligence to exist stochastic subsets of intelligence need to emerge from experimentation.

[Matt Mahoney]

On Sat, Mar 3, 2012 at 4:04 PM, Ivan Vodišek <ivanv...@gmail.com> wrote:
> i would propose somehing like:
> if i had 2 brothers and a sister, how much my parents have children?

Good one. Google doesn't know.

-- Matt Mahoney, mattma...@gmail.com

[YKY (Yan King Yin, 甄景贤)]

I guess we should choose a very small domain -- it's bad to disappoint users if Genifer performs dumbly.

1. programming -- seems too hard initially unless we narrow it down

2. medical info -- also too hard unless we narrow it

The candidate should have a commercial need, and is very narrow.

So, let's brainstorm a bit? =)

KY

[Ivan Vodišek]

a math solving machine?

2012/3/4 YKY (Yan King Yin, 甄景贤) <generic.in...@gmail.com>

[YKY (Yan King Yin, 甄景贤)]

On Sun, Mar 4, 2012 at 8:28 PM, Ivan Vodišek <ivanv...@gmail.com> wrote:

a math solving machine?

We have Mathematica, Maple, MatLab, ..., and assistant theorem provers such as HOL Light.  It's a lot of work and is hard =)  Also, Genifer's logic's strength is in natural language...

KY

[YKY (Yan King Yin, 甄景贤)]

On Sun, Mar 4, 2012 at 8:28 PM, Ivan Vodišek <ivanv...@gmail.com> wrote:

a math solving machine?

Unless you're talking about word problems, like:

"John is twice Mary's age, and John is older than her by 10 years.  What is their age?"

But I'm not sure how often people need to ask those questions...  perhaps mostly primary school kids!

KY

[Matt Mahoney]

2012/3/4 YKY (Yan King Yin, 甄景贤) <generic.in...@gmail.com>:

Stuff like this was done in the 1960's using hand-coded parsers. It
would be interesting if you could use a machine language approach.
Could you train a language model to solve math problems like this just
by giving it lots of examples? It would require a lot more computing
power, something that wasn't available when the problem was first
solved.

[Ivan Vodišek]

about brainstorming:

text summarizer could be done. when we pass large text to gen, a few sentence sum about that text could be automatically returned. 

2012/3/5 Matt Mahoney <mattma...@gmail.com>

[YKY (Yan King Yin, 甄景贤)]

On Mon, Mar 5, 2012 at 4:15 PM, Ivan Vodišek <ivanv...@gmail.com> wrote:

about brainstorming:

text summarizer could be done. when we pass large text to gen, a few sentence sum about that text could be automatically returned. 

Summarizing is quite difficult because we need to learn a lot of relations between concepts, such as:

    shark Ì fish               "shark is a fish"

    whale ≈ fish                 "whale is similar to a fish"

    knob · part_of · door    "door knob is part of a door"

Then we can use these conceptual relations to summarize texts.  The fear is still that Gen may perform badly and disappoint users...

KY

[Matt Mahoney]

2012/3/5 YKY (Yan King Yin, 甄景贤) <generic.in...@gmail.com>:

One possible training set: Wikipedia. Given the article, predict the title.

[Ivan Vodišek]

this is what might be done in next few weaks:

an ai construction kit. that would be a parser for entering data and answering questions. custom output could be tuned up according to predicted input. custom built database could be uploaded to central internet base that can be reused in future projets, or be shared among users. i guess we could collect some donations too. does anyone have an opinion on that?

2012/3/5 Matt Mahoney <mattma...@gmail.com>

[Matt Mahoney]

On Mon, Mar 5, 2012 at 12:30 PM, Ivan Vodišek <ivanv...@gmail.com> wrote:
> this is what might be done in next few weaks:
> an ai construction kit. that would be a parser for entering data and
> answering questions. custom output could be tuned up according to predicted
> input. custom built database could be uploaded to central internet base that
> can be reused in future projets, or be shared among users. i guess we could
> collect some donations too. does anyone have an opinion on that?

What would be some use cases? Give some examples.

-- Matt Mahoney, mattma...@gmail.com

[Ivan Vodišek]

that would be a product for constructing an ai. lot of people are trying to make an ai. and they could use it for:

chatbot, chatbot serving some site about some products, ai guided knowledge inspector, instructor about ai, and my favourite: for a hobi!

could be put online if flash used...

2012/3/5 Matt Mahoney <mattma...@gmail.com>

[Matt Mahoney]

On Mon, Mar 5, 2012 at 3:03 PM, Ivan Vodišek <ivanv...@gmail.com> wrote:
> that would be a product for constructing an ai. lot of people are trying to
> make an ai. and they could use it for:
> chatbot, chatbot serving some site about some products, ai guided knowledge
> inspector, instructor about ai, and my favourite: for a hobi!
>
> could be put online if flash used...

That assumes we know how to construct AI. We don't. How would we
convince anyone that our tools would be useful if we can't use them
ourselves?

-- Matt Mahoney, mattma...@gmail.com

[Ivan Vodišek]

i gathered some knowledge about kb. what we know is: how knowledge looks like

(https://sites.google.com/site/ivanvodisek/knowledge-base-system).

mapping text to that kb structure is easy:

(http://www.youtube.com/watch?v=7Ba-tKOQDPw)

once we got parsed text, it should be easy to construct answers and reason about knowledge patterns (inductive learning).

there should be used some starting english kb, from which other knowledge would be entered. an action language can be constructed for behaviour of an ai, and there hobists pop in. that action language could control basic stuff (on question X print answer of X), and more complicated stuff based on bayesian calculations and other things, it's up to hobist. i predict 4 man-weeks of work for the core and infinite for entering knowledge and tuning behaviour. we could also host a site that gathers universal knowledge files that can be uploaded to user's private database.

 

2012/3/5 Matt Mahoney <mattma...@gmail.com>

[Matt Mahoney]

On Mon, Mar 5, 2012 at 3:19 PM, Ivan Vodišek <ivanv...@gmail.com> wrote:
> i gathered some knowledge about kb. what we know is: how knowledge looks
> like
> (https://sites.google.com/site/ivanvodisek/knowledge-base-system).
>
> mapping text to that kb structure is easy:
> (http://www.youtube.com/watch?v=7Ba-tKOQDPw)

Parsing English is a lot different than parsing a programming
language. For one, the order of parsing and semantic analysis is
reversed. For another, it requires a lot more computing power.
Programming languages are designed to be read by computers. They have
a precise specification. Natural language does not. Natural language
evolved to be learned by massively parallel neural networks.

-- Matt Mahoney, mattma...@gmail.com

[Ivan Vodišek]

here is great free book about english grammar: Angela Downing - Philip Locke: A University Course

2012/3/6 Matt Mahoney <mattma...@gmail.com>

[Matt Mahoney]

On Tue, Mar 6, 2012 at 7:38 AM, Ivan Vodišek <ivanv...@gmail.com> wrote:
> here is great free book about english grammar: Angela Downing - Philip
> Locke: A University Course

That's good for teaching humans, but not very good for teaching
computers unless the computer already as solved the AI problem so it
can understand the book.

Understanding natural language by computers remains an unsolved
problem. I run a benchmark aimed at testing natural language learning
and comprehension by measuring prediction text accuracy (by
compression ratio). If you have any good ideas, you are welcome to
submit them to http://mattmahoney.net/dc/text.html or
http://prize.hutter1.net/

-- Matt Mahoney, mattma...@gmail.com

[Ivan Vodišek]

i'll try to make a test with actionscript. i'll inform you about results.

2012/3/6 Matt Mahoney <mattma...@gmail.com>

[Ivan Vodišek]

milestone1: math done

here is math exp parser bundeled in classic html for download.

2012/3/6 Ivan Vodišek <ivanv...@gmail.com>

[Matt Mahoney]

On Thu, Mar 15, 2012 at 6:56 PM, Ivan Vodišek <ivanv...@gmail.com> wrote:
> milestone1: math done
>
> here is math exp parser bundeled in classic html for download.

That's interesting. Are you going to parse English next? That's a much
harder problem.

-- Matt Mahoney, mattma...@gmail.com

[Ivan Vodišek]

i'll try

2012/3/16 Matt Mahoney <mattma...@gmail.com>

[Mike Dougherty]

On Sat, Mar 3, 2012 at 4:04 PM, Ivan Vodišek <ivanv...@gmail.com> wrote:
> i would propose somehing like:
> if i had 2 brothers and a sister, how much my parents have children?

Error: mine or yours?

"how much my parents have children?" Are you asking what percentage
of your own parents have more than one child? I think the answer
would be 100% since you claim to have a sibling. If you are asking
how many children your parents have, then 4. There is also the
possibility that your own count of brothers and a sister are not the
complete set of children had by your parents, for example
half-siblings or step-siblings.

I propose the correct "answer" to this question is actually another
question/request for more information to remove these ambiguities.

I don't understand why AI is expected to output completely perfect
knowledge when the input (esp. from humans) is far far less than
perfect.

[Ivan Vodišek]

new milestone: BNF parser html

2012/3/16 Mike Dougherty <msd...@gmail.com>

[YKY (Yan King Yin, 甄景贤)]

On Sat, Mar 17, 2012 at 3:37 AM, Ivan Vodišek <ivanv...@gmail.com> wrote:

new milestone: BNF parser html

Sorry that I've been away meeting people etc.

So, it may be good to have a web-based parser that can generate Geniform, but I also had another idea which is to use Genifer's own logic to parse natural language (the idea is simple, forward-chaining in logic corresponds to bottom-up parsing in linguistics).

But yesterday I had another very interesting idea, after hearing a talk from a guy who bootstrapped his own Javascript-like language.  This trick is well-known in computer science -- a compiler can be written in its own language.  What I plan to do is to develop Genifer in Genifer's own logic...

KY

[rodney mulraney]

but if you define the system not as something that produces the
perfect answer, but as something that produces the perfect question, a
lot of the previous work and design in the original system gets
rendered void. This new question system might even be seen to have to
the side effect of producing the perfect answer as the final question
forinstance, in fact it might be seen later that the original design
was paradoxical and would never work out well, due to overlooking this
basic fact of communication between people...

however with simular reasoning and observation you might come to a
whole series of fundamentals in "AI" systems, where each observational
point renders any previously constructed system laughable as a serious
approach to "AGI"

I fear that we just dont have the tools or materials or basic
conceptual understanding of what we are trying to do still, where
these kinds of concepts are not part of any academic study or studies.

Once we can fully understand and prove why any kind of AGI is
impossible, we will be at the stage of thinking about and writting
actual AGI systems, until then its probably going to be more
progressive to churn out working practical systems to force running
into the problems we cant see to find them, and mapping out the AGI
landscape more...I think in general this is what is happening anyway.

[Ivan Vodišek]

my plan is pretty clear: to develop universal parser to provide AGI

programmers with a tool for controling answers and general output

in relation to entered text or general input.

any previous work done, like induction analysis, would have a great

impact on specific behaviour implementation.

2012/3/16 rodney mulraney <rjmul...@gmail.com>

[Mike Dougherty]

On Fri, Mar 16, 2012 at 4:01 PM, rodney mulraney <rjmul...@gmail.com> wrote:
> but if you define the system not as something that produces the
> perfect answer, but as something that produces the perfect question, a
> lot of the previous work and design in the original system gets
> rendered void. This new question system might even be seen to have to
> the side effect of producing the perfect answer as the final question
> forinstance, in fact it might be seen later that the original design
> was paradoxical and would never work out well, due to overlooking this
> basic fact of communication between people...

I don't propose to build a perfect anything. I was hoping to get some
dialog on the value of machines that also dialog. I think the
iterative process that gets two parties closer to a
mutually-agreed-upon "good enough" solution is a critical part of how
anything has ever been accomplished by humanity. I feel like that's a
pretty important part of HCI that is often overlooked.

I think a visualizer for so-called "big data" would be of enormous
commercial value. The machines doesn't have to know what is
interesting, just keep displaying different relationships in data
until an analyst "sees" what is interesting. That too will be a
interaction system. The intelligence will, of course, be distributed
between man and machine.

[rodney mulraney]

I think it goes without saying that pretty much everyone who makes a
statement is doing it with the implication of building/making/creating
"the perfect something"... otherwise you are admitting that when you
do things you purposely leave in imperfections, which as far as I am
aware is only a practice of some spiritual tribes (who believe if you
were to actually succeed in creating a perfect anything, your life is
complete and you will soon die), and electronics/other manufacturers
(for profit). Its much like when you hear someone say, "You always
think you are right"... well this too goes without saying or you're
admitting to lying...

I think if I advertised my new AGI system that was going to
revolutionize the world and I described it as not having any data, or
inputs to collect data, then I would be an embarrasment to the field
and universally considered a quack. I think its the same with non
dialog/interative systems, and the fact they are taken seriously shows
the entire field of AGI and AI as perceived and allowed to be
perceived by the public as quack science.

Lots of things are of enormous commercial value. I think an actual AGI
is one of them, and what the topic is here.

Ivan : "my plan is pretty clear: to develop universal parser to provide AGI

programmers with a tool for controling answers and general output
in relation to entered text or general input."

I wish you all the success on this project and I'm sure no would could
deny the value of such a tool. And I'm sure everyone else [in the
field] has already made the comment that: your "universal parser"
actually equates to an AGI system, the AGI programmers that use such a
system wouldnt be AGI programmers at all, but merely plumbers who
would write interfaces to your AGI for various real world
scenarios...using the term "universal parser" to describe your AGI
shows you need to do a lot of research in the area to better map out
what kind of problems are entailed, however writting a "universal
parser" is probably a great way learn some of these issues.

[YKY (Yan King Yin, 甄景贤)]

On Sat, Mar 17, 2012 at 4:35 AM, Ivan Vodišek <ivanv...@gmail.com> wrote:

my plan is pretty clear: to develop universal parser to provide AGI

programmers with a tool for controling answers and general output

in relation to entered text or general input.

In my latest design a parser may not be needed:  we can bootstrap Genifer using itself as the programming language.  At first we can use Lisp, which is a power functional language with practical impure features.  Then we enhance Lisp to become a logical-functional language with distributive message-passing that can also work within a single computer (multi-core).  Parsing is free as logical deduction / abduction.  This minimizes the code we have to write, and is very important for a large-scale and complex project.  (That said, using my simplified and abstracted design, the entire AGI code may be extremely short for a project of this complexity).  I hope that is the right direction...

KY

[Ivan Vodišek]

>And I'm sure everyone else [in the
>field] has already made the comment that: your "universal parser"
>actually equates to an AGI system, the AGI programmers that use such a
>system wouldnt be AGI programmers at all, but merely plumbers who
>would write interfaces to your AGI for various real world

i strongly disagree on plumbing comment. that's like you say that someone

who use any programming language is merely a plumber (not that i have

something against real plumbers, it's a good and useful job)

it might be a good experiment to write an english parser and a query language

that gives answers to posed questions, but the big deal is to  implement

automatic learner and behaviour control. if you try to answer the question

"what would computer do?", you'll see that it's not a trivial question.

my parser is merely equivalent to google translate part that parses text. the

real enigma is what would computer do on entered input. also,my input is

bounded to text for now. still i wander how to analyse more general input

from sensors like sound and vision.

[YKY (Yan King Yin, 甄景贤)]

Re building perfect systems and the need for dialogue:

It is actually feasible to build the perfect system without endless trial-and-error.  Well, "perfect" up to certain basic assumptions.

One foundation is algorithmic (aka Solomonoff) induction.  AIXI is an abstraction of intelligence into a single equation for optimization.  As long as we accept the basic assumptions involved in Solomonoff induction, such as the axioms of probability, Turing universality, etc, this is a very solid foundation.

My design has a fixed logic syntax, and a fixed calculus for computing numerical truth values.  But the formulas that constitute the KB would be machine-learned.  Thus, the logic apparatus can be viewed as an inductive bias within the AIXI framework.  Then we may use Solomonoff induction as a principle to score hypotheses during inductive learning.  IE, Genifer = AIXI + a special logic as inductive bias.

The naive algorithm in Genifer will answer queries with a single answer with the best truth value.  But, with a planner extension it can certainly engage in dialogue with the human user, and that behavior would be dictated by utility values either machine-learned or assigned by humans.

What I'm trying to say is, AGI really is a solved problem without any mystery.  Due to a few ideas I had lately, which I find very satisfactory, I'm now quite confident to start implementation... =)

KY

[Ivan Vodišek]

Some of You might want to take a brief look at a version of Type Theory: https://sites.google.com/site/ivanvodisek/knowledge-base-system

2012/3/20 YKY (Yan King Yin, 甄景贤) <generic.in...@gmail.com>

[YKY (Yan King Yin, 甄景贤)]

On Tue, Apr 24, 2012 at 3:36 AM, Ivan Vodišek <ivanv...@gmail.com> wrote:

Some of You might want to take a brief look at a version of Type Theory: https://sites.google.com/site/ivanvodisek/knowledge-base-system

What are your types?  How many types are required for natural language understanding?  Would it be correct to say any natural-language word can be a type in your system?

I think my theory can co-exist with a conventional type system -- just assign a special "typeless" type to all the terms in my logic.  So it is possible to use my logic with other typed logics or programming languages.

KY

[Ivan Vodišek]

Sentence -> (
    Subject -> {
       Noun;
       NounPhrase;
    },
    Predicate -> Verb,
    Object -> {
        Noun;
        NounPhrase
    }
)

2012/4/24 YKY (Yan King Yin, 甄景贤) <generic.in...@gmail.com>

[YKY (Yan King Yin, 甄景贤)]

On Tue, Apr 24, 2012 at 7:37 PM, Ivan Vodišek <ivanv...@gmail.com> wrote:

Sentence -> (
    Subject -> {
       Noun;
       NounPhrase;
    },
    Predicate -> Verb,
    Object -> {
        Noun;
        NounPhrase
    }
)

That doesn't seem to answer my questions...

Now you seem to have something that looks like a phrase structure grammar, which is a formal grammar.  The problem with formal grammars is that they tend to be able to parse >90% of natural language but there are always some exceptions that needs to be repaired.

Also, what is most special about your approach that makes you want to pursue it? There must be some cool idea behind it?  Maybe that will help me understand your reasoning better...

KY

[Ivan Vodišek]

There will always be exceptions that need to be handled by dirty programming tricks. Type theories exist for over decades now. What I found out is that:

1. all logics from zeroth to infinite order can be managed by zeroth order logic. In simplifyed sentence definition I wrote, round braces denote conjunction and curly braces denote disjunction. Cannonicaly yelding a sequence of types, filters set of results to final result as sequence:

"Sentence"

"Predicate"
yelds

"Verb"

or something that is applyed to Sentence.

2. polymorphic type theory is nothing more than BNF notation powered with inline type naming and some event control. Zeroth order logic is also just enpowered BNF notation. So all the time we are dealing with the same structures that have several shapes (BNF, logic 0, logic 00)

What I didn't realized yet is better fusion of multiple specifications from which a generalization follows. I wish I have it defined with some sleak neat rule that covers it all. Instead, I have zeroth order definition of relation general-specifics that is not inwired, but is just one of possible rules in infinite set of possible combinations.

 

2012/4/24 YKY (Yan King Yin, 甄景贤) <generic.in...@gmail.com>

On Tue, Apr 24, 2012 at 7:37 PM, Ivan Vodišek <ivanv...@gmail.com> wrote:

[Abram Demski]

On Tue, Apr 24, 2012 at 5:59 AM, Ivan Vodišek <ivanv...@gmail.com> wrote:

There will always be exceptions that need to be handled by dirty programming tricks. Type theories exist for over decades now. What I found out is that:

1. all logics from zeroth to infinite order can be managed by zeroth order logic. In simplifyed sentence definition I wrote, round braces denote conjunction and curly braces denote disjunction.

This is simply not true, so I wonder what you mean.

How would you represent the proposition (normally requiring 2nd-order logic):

A) X is an ancestor of Y if and only if there are some people such that X is one of those people, I am one of those people, and each person is either me or a parent of one of those people.

B) If George is my parent and Alice is George's parent, then Alice is my ancestor.

and how would you validate the inference from A to B?

 

Cannonicaly yelding a sequence of types, filters set of results to final result as sequence:

"Sentence"

"Predicate"
yelds

"Verb"

or something that is applyed to Sentence.

2. polymorphic type theory is nothing more than BNF notation powered with inline type naming and some event control. Zeroth order logic is also just enpowered BNF notation. So all the time we are dealing with the same structures that have several shapes (BNF, logic 0, logic 00)

What I didn't realized yet is better fusion of multiple specifications from which a generalization follows. I wish I have it defined with some sleak neat rule that covers it all. Instead, I have zeroth order definition of relation general-specifics that is not inwired, but is just one of possible rules in infinite set of possible combinations.

 

2012/4/24 YKY (Yan King Yin, 甄景贤) <generic.in...@gmail.com>

On Tue, Apr 24, 2012 at 7:37 PM, Ivan Vodišek <ivanv...@gmail.com> wrote:

Sentence -> (
    Subject -> {
       Noun;
       NounPhrase;
    },
    Predicate -> Verb,
    Object -> {
        Noun;
        NounPhrase
    }
)

That doesn't seem to answer my questions...

Now you seem to have something that looks like a phrase structure grammar, which is a formal grammar.  The problem with formal grammars is that they tend to be able to parse >90% of natural language but there are always some exceptions that needs to be repaired.

Also, what is most special about your approach that makes you want to pursue it? There must be some cool idea behind it?  Maybe that will help me understand your reasoning better...

KY

--
Abram Demski
http://lo-tho.blogspot.com/

[Abram Demski]

PS:

The common translation of (A) into first-order logic,

A') X is an ancestor of Y if and only if X is a parent of Y or X is a parent of an ancestor of Y

will do fine in most cases, but is not logically equivalent. It will have different implications in cases where there are an infinite number of entities (and so, we would expect it to behave differently for large numbers of entities in a realistic inference system). Specifically: it will correctly find all the positive cases of 'ancestor', but it will leave some negative cases undefined. (To prove that someone is not my ancestor will require proving that all their descendants are not my ancestors, which (A) can do in more cases than (A').)

--Abram

[Ivan Vodišek]

Well, you're right in a way that I forgot to note that zeroth order logic has to be enpowered by some event handling.
So, if we assume that this is a general form of a set (from Type Theory - i guess this is called infinite order logic):

Set -> Type ...

<   Elements

    ....

>

[   Events

    ....

]

then our B) would look like following:

Ancestor -> (X -> Person, Y -> Person)

<

   ... to be automaticaly filled with elements ...

>

[

   on Parent -> (%K, %L) && Parent -> (%L, %M)

      insert Ancestor (X (%K), Y(%M));

   ... remove on disambiguation goes here ...

];

now when we fill set of parents manually by 

Parent -> (Person, Person)

<

   Parent (George, Me);

   Parent (Alice, George);

>

we get a new ancestor lement.

Unfortunately, A) example is unclear to me. I'm not a machine, don't you know that? :o)

This is how Type Theory basically works. I just replaced types of set with conjunctions and disjunctions. I wander if I forgot something?

2012/4/24 Abram Demski <abram...@gmail.com>

[Abram Demski]

B is somewhat boring; A is the main one I wanted to see you encode. So if A is unclear, can you encode A'? Is that one more clear?

--Abram