update: Sept, 2012

[YKY (Yan King Yin, 甄景贤)]

My logic paper was rejected for AGI'12 -- seems that Ben has given it a -1 score, I recognized his writing -- but never mind, I could use another year to improve it...

Time is running short, but I'm still stuck...

My compositional logic may not be the best, but assume that we accept it as the substrate, then AGI boils down to machine learning, and the algorithm for machine learning is already known, so it also boils down to optimization (speedup).  Alas, speeding up logic inference (and thus machine learning) is a very hard and complex problem...  (it's a combinatorial problem which we want to speed up, whereas the usual combinatorial optimization problem is just to find a maximum value).

These days I'm looking into:

1.  The matrix technique -- I understand it better now, but it leaves us with a lot of points in high-dimensional space, and they require special techniques for indexing and querying (such as R-tree).  I'm looking for easier alternatives.

2.  Automata.  My logic is almost like a regular language, except with variables.  I'm wondering if I can translate logic formulas into automata, merge them into 1 big automaton, and perform inference using that?  People should have tried something like this already, no?  Anyway, this seems to me most promising right now...

3.  Genetic programming for speed up -- I considered it briefly, but seems that the problem's complexity is too high, and GP wouldn't be able to solve it unless we constrain the problem with additional structure.

On the maths side, I'd like to learn more about the following, but may not have the time:

1.  Rings and modules.  If my logic is a semi-ring, then module theory may be relevant to it?

2.  I still don't know what is homology and cohomology in topology.  Just curious...

KY

[Russell Wallace]

For what it's worth, I've ended up deciding to stick with first-order
logic for the moment, for several reasons, one perhaps not obvious one
being seeing just how much of the action these days is in SMT which
can be considered 0.5-order logic; I consider that strong evidence
that you can in practice get more done with first-order logic than I
had given it credit for.

You are absolutely right about genetic programming being far too slow
unless you can greatly constrain the search space.

I don't think automata are a promising direction. Off the top of my
head, they work well when you know the shape of the problem in
advance, which with logical inference you don't really.

[YKY (Yan King Yin, 甄景贤)]

On Mon, Sep 24, 2012 at 1:31 AM, Russell Wallace <russell...@gmail.com> wrote:

I don't think automata are a promising direction. Off the top of my
head, they work well when you know the shape of the problem in
advance, which with logical inference you don't really.

About automata, for example if we have a rule:

    R(X,a,b) <- P(X,a), Q(X,Y,b)

Then we can use an automaton to recognize each of the premises:

    P(X,a)

    Q(X,Y,b)

to see if the body can be satisfied by current facts.

The automata can be merged.  That way, we can merge the premises of all the rules in KB into 1 huge automaton.  Then, we feed the facts into this automaton to see which accepting states are found -- ie, which rule(s) would fire.  That might be very efficient?

KY

[Russell Wallace]

Perhaps you could come up with an efficient way to use an automaton
for such simple application of rules to data, but there would be no
particular benefit; that sort of thing is already fast enough. What
isn't fast enough is search problems like finding appropriate sets of
rules, checking for satisfiability, finding examples; and automata
won't help with those.

[Matt Mahoney]

What is the problem we are trying to solve? I know the ultimate goal
is to pass the Turing test, build humanoid robots that will do all our
jobs, become immortal, upload to virtual worlds, and launch a
singularity. But assuming we don't do that all at once, what is the
intermediate goal? Please don't tell me it is to implement some
design. I want to know what are the inputs and desired outputs. Give
me some test cases and scoring criteria.

--
-- Matt Mahoney, mattma...@gmail.com

[Russell Wallace]

I agree completely. For example, my intermediate goal is solving
logical problems with TPTP and SMTLIB as the test cases and scoring
criterion being the percentage of problems successfully solved.

YKY, are you still aiming for input in natural language? If so, this
would be a good time to try to define more precisely what kind of
questions or instructions you intend your system to accept and what
kind of outputs it should produce in response.

[Matt Mahoney]

BTW my goal is to improve natural language modeling as measured by
http://mattmahoney.net/dc/text.html
A better model of grammar learning might help. The best ones are very
primitive, unable to learn categories such as nouns and verbs. I don't
see how a model based on logic would help. A language model should be
able to learn the meaning of words like "and", "not", "if", "some",
"all", etc., but normally children learn these words only after
learning simple sentences with nouns, verbs, and adjectives.


On Sun, Sep 23, 2012 at 4:47 PM, Russell Wallace

[Ivan Vodišek]

i like genetic programming

2012/9/23 Matt Mahoney <mattma...@gmail.com>

[Matt Mahoney]

On Mon, Sep 24, 2012 at 1:44 PM, Ivan Vodišek <ivanv...@gmail.com> wrote:
> i like genetic programming

Genetic programming is too slow. The only known instance of human
level intelligence being produced this way took 3 billion years on a
planet sized, 160,000 terawatt molecular computer.


-- Matt Mahoney, mattma...@gmail.com

[YKY (Yan King Yin, 甄景贤)]

I'm considering some possible applications:

1.  programming assistant -- my 1st choice.  I want to be able to accept English input and produce code that solves simple problems, such as find-and-replace and other string and file processing tasks.

Advantages:  only a small subset of English is needed, but Genifer will develop a conceptual understanding of a small programming domain.  So it actually needs to perform *reasoning* in this domain.

1. biomedical informatics -- I'm not particularly familiar with this, though I've studied biochemistry (and neuroscience) before.

Disadvantages:  For low-level data, it seems that traditional statistical learning or data mining is sufficient.  For high-level thinking, the problems may be too complex (such as those currently requiring the intellects of human researchers).

3.  Machine translation -- it may help developing countries like China where the English language barrier is holding back a large population's modernization.

Disadvantages:  the "data" is too big, we need to spend a lot of time simply absorbing ontologies, similarity relations, etc, things that are not reasoning per se.  But if we can't build up the reasoning capacity in a short time, users may complain and think Genifer is dumb.

KY

[Russell Wallace]

For what it's worth, I'd recommend biomedical informatics if you're
going to do anything with natural language at all. One of the smartest
guys I've ever met is working in biomedical research, and he said the
one thing he'd like AI to do for him that could be feasible in the
foreseeable future is sift through the flood of publications for
information relevant to his work. Also, biomedical informatics has two
important properties: partial success would still be of value (unlike
the programming assistant) and there would be some possibility of
niche differentiation so you wouldn't have to compete head on with
Google on day one (unlike language translation).

[YKY (Yan King Yin, 甄景贤)]

Just a quick note:  Google translate for Chinese is very crappy.

This is what you wrote, translated to Chinese and back:

For what it's worth, I suggest that if you plan to do with the natural language in all of biomedical informatics.

Is one of the smartest people I've ever seen, in biomedical research, he said, he hoped that the AI for him to do one thing is feasible in the foreseeable future, by the flood of publications screening for his work.
Biomedical Informatics has two important attributes: the value of the partial success still different programming assistant, there will be some possibility of ecological niches, so you will not compete head-to-head with Google in one day (it is different language translation).

But, even in Chinese, the translation is already rather unintelligible.  You'd know if you're bilingual.  (Or maybe Google Translate does better for European languages...) 

KY

[YKY (Yan King Yin, 甄景贤)]

On Tue, Sep 25, 2012 at 6:29 PM, Russell Wallace <russell...@gmail.com> wrote:

For what it's worth, I'd recommend biomedical informatics if you're
going to do anything with natural language at all. One of the smartest
guys I've ever met is working in biomedical research, and he said the
one thing he'd like AI to do for him that could be feasible in the
foreseeable future is sift through the flood of publications for
information relevant to his work. Also, biomedical informatics has two
important properties: partial success would still be of value (unlike
the programming assistant) and there would be some possibility of
niche differentiation so you wouldn't have to compete head on with
Google on day one (unlike language translation).

What Genifer can offer is reasoning ability in a domain.  We're more familiar with programming so we can better constrain the domain to a small area such as string / file processing (like what Perl is used for, but we can incrementally build up the power).

The difficulty in other domains (such as translation and biomedics) is that we can't easily confine the vocabulary to a small area...  Too many things are inter-related and including one concept will include other related concepts.  Whereas in programming, that is more feasible because of the formal nature of programs...

If the programming task is small (such as:  "open a file and count the lines"), 100% success can be expected, and it would be useful for lazy programmers.

On the other hand, sifting through biomed papers is narrow AI and does not require any deep reasoning.  Genifer's goal is to develop reasoning ability...

KY

[james northrup]

On Tue, Sep 25, 2012 at 9:33 AM, YKY (Yan King Yin, 甄景贤) <generic.in...@gmail.com> wrote:

On Tue, Sep 25, 2012 at 6:29 PM, Russell Wallace <russell...@gmail.com> wrote:

For what it's worth, I'd recommend biomedical informatics if you're
going to do anything with natural language at all. One of the smartest
guys I've ever met is working in biomedical research, and he said the

one thing he'd like AI to do for him that could be feasible in the [..]

 

What Genifer can offer is reasoning ability in a domain.  We're more familiar with programming so we can better constrain the domain to a small area such as string / file processing (like what Perl is used for, but we can incrementally build up the power).

Towards Russel's proposed application, and your own, is simply to begin reasoning about AST's which form the bulk of machine readable data, and to ratchet up the relevance with experimentation.  narrow ai's produce machine-readable data using languages based on stable AST's.  increasing the closed loop analysis of a static AST can provide a huge benefit to users who may need suggestions superior to thier experience and familiarity.