John McCarthy R.I.P.

[RichD]

Anybody have any personal McCarthy stories to share?

--
Rich

[RichD]

On Nov 4, Don Geddis <d...@geddis.org> wrote:
> > Anybody have any personal McCarthy stories to share?
>

> He was on my thesis committee.  (I was quite honored.)
> He fell asleep in the middle of my presentation.
>
> Later, during the private question session, when his turn
> came to ask me questions, he related a five-minute anecdote
> of some other research he had recently come across.  At the
> end, he asked "is that related to your
> work?"  My reply: "no".  McCarthy had no more questions for me.
>
> (BTW: none of this is meant to be critical.  I loved having
> him there, and just thought the whole episode was funny.
> And I have enormous respect for his accomplishments
> during his career. )

wow

He was one of the founders of the field, along
with others of that generation at Stanford. But
they're mostly fertilizer now. What do you think
of the later generations? A comparable caliber?

Don Knuth is still active, still sharp, giving lectures,
you can find them if you poke around Stanford web
site.

McCarthy was also one of those responsible for the original
'thinking machines' hype; "we'll have electronic brains
any day now". And it succeeded - federal $$$ poured
in to the computer engineers at the big research
schools, as MIT, Stanford et al. became branches of
the Pentagon.


--
Rich

[casey]

On Nov 10, 8:17 am, RichD <r_delaney2...@yahoo.com> wrote:
>
> McCarthy was also one of those responsible for the original
> 'thinking machines' hype; "we'll have electronic brains
> any day now".

And the hype continues ...

JC

[Dmitry A. Kazakov]

Really? I seen no serious works in this field in recent years.

For long the focus has been shifted to studies of human brain (which of
course would not create AI any time soon), NN hype (ditto) and practical
problems (OCR etc) solvable without general intelligence.

--
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

[casey]

On Nov 10, 7:45 pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
wrote:

Well I stand corrected. It was just an impression with
things like,

http://en.wikipedia.org/wiki/Hierarchical_temporal_memory

http://en.wikipedia.org/wiki/Cyc

http://en.wikipedia.org/wiki/Watson_(computer)

There are those who say there will be a breakthrough and
suddenly we will have super ai machines with AGI and the
impending singularity.

http://bigthink.com/ideas/40822

jc

[Antti J Ylikoski]

In their book "Artificial Intelligence, A Modern Approach", 3rd Edition,
Russell-Norvig crudely estimate that the processing power of the human
brain as operations/sec is about 10 to the 17th power operations, and
the number of memory updates/sec is about 10 to the 14th power updates.
(Russell-Norvig, 2010, p.12).

I would like to add that there are some 10 to the 11th power neurons in
the human brain, and they are connected in the average to some thousand
other neurons.

So that is an estimate of the complexity that one must to my opinion
create to have a genuinely human-level Artificial Intelligence.

Another matter is that AI as an industry, as a branch of engineering and
as a science has been phenomenally successful. I would like to remark
that in order to build aeroplanes it has not been necessary to fly in
the same manner as birds.

yours, Antti J Ylikoski
Helsinki, Finland, the EU

PS. The creators of the Melissa virus are real-life super criminals.
One should send Batman to arrest them.

Idem

[Dmitry A. Kazakov]

On Thu, 10 Nov 2011 11:38:39 +0200, Antti J Ylikoski wrote:

> 10.11.2011 10:45, Dmitry A. Kazakov kirjoitti:
>> On Wed, 9 Nov 2011 14:04:22 -0800 (PST), casey wrote:
>>
>>> On Nov 10, 8:17 am, RichD<r_delaney2...@yahoo.com> wrote:
>>>>
>>>> McCarthy was also one of those responsible for the original
>>>> 'thinking machines' hype; "we'll have electronic brains
>>>> any day now".
>>>
>>> And the hype continues ...
>>
>> Really? I seen no serious works in this field in recent years.
>>
>> For long the focus has been shifted to studies of human brain (which of
>> course would not create AI any time soon), NN hype (ditto) and practical
>> problems (OCR etc) solvable without general intelligence.
>
> In their book "Artificial Intelligence, A Modern Approach", 3rd Edition,
> Russell-Norvig crudely estimate that the processing power of the human
> brain as operations/sec is about 10 to the 17th power operations, and
> the number of memory updates/sec is about 10 to the 14th power updates.
> (Russell-Norvig, 2010, p.12).

I love such estimations in the context that nobody knows what does general
intelligence mean algorithmically.

Continuing in that vein there is actually only one operation needed: the
operation "THINK". (:-))

> I would like to add that there are some 10 to the 11th power neurons in
> the human brain, and they are connected in the average to some thousand
> other neurons.

Without saying how exact a the digital model of a neuron [analogue thing]
must be [in order to do WHAT?] this number tells nothing.

And this is only the beginning. I could tell how much atoms of Si one
Pentium IV crystal possesses. That by no means would make Pentium out of 10
grams of river sand.

> So that is an estimate of the complexity that one must to my opinion
> create to have a genuinely human-level Artificial Intelligence.

The crucial point is computability of general intelligence. Without any
knowledge about what the intelligence actually does, all such comparisons
are totally meaningless. Let us consider the task of adding two numbers of
6 decimal places. For this task you need X neurons of human brain or a
calculator of paiir thousand transistors. Does it make 2K transistors
equivalent to human brain? The answer is YES, for this task, and NO for the
general intelligence task. Without specifying the TASK, comparisons are
rubbish.

> Another matter is that AI as an industry, as a branch of engineering and
> as a science has been phenomenally successful. I would like to remark
> that in order to build aeroplanes it has not been necessary to fly in
> the same manner as birds.

Absolutely. If AI will ever created, its design won't even resemble human
brain.

[Dmitry A. Kazakov]

On Thu, 10 Nov 2011 01:24:36 -0800 (PST), casey wrote:

> On Nov 10, 7:45�pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
> wrote:
>> On Wed, 9 Nov 2011 14:04:22 -0800 (PST), casey wrote:
>>> On Nov 10, 8:17�am, RichD <r_delaney2...@yahoo.com> wrote:
>>
>>>> McCarthy was also one of those responsible for the original
>>>> 'thinking machines' hype; "we'll have electronic brains
>>>> any day now".
>>
>>> And the hype continues ...
>>
>> Really? I seen no serious works in this field in recent years.
>>
>> For long the focus has been shifted to studies of human brain (which of
>> course would not create AI any time soon), NN hype (ditto) and practical
>> problems (OCR etc) solvable without general intelligence.
>>

> Well I stand corrected. It was just an impression with
> things like,
>
> http://en.wikipedia.org/wiki/Hierarchical_temporal_memory

Wow, the old good pyramid! I am pleased to see this 20 years old image
processing technique revamped. (Not that it would make anything more or
less intelligent that it already is...)

> There are those who say there will be a breakthrough and
> suddenly we will have super ai machines with AGI and the
> impending singularity.

I am rather sceptical about the theory, that we need not to understand
anything. Just mechanically putting many computing elements together won't
magically produce intelligence.

I think IF intelligence is computable (it could be sufficiently
incomputable), THEN as any other computational task it will be scalable.
I.e. it will be no matter how many resources you have, except for memory.
So i486 running this task will be intelligent in proportion to its power.
E.g. under real-time (=response time) and memory (=number of distinct
states) constraints it will be as intelligent as an ant or a frog or a
mouse etc. Without the real-time constraint (in simulation time), it will
be as intelligent as Einstein.

[Antti J Ylikoski]

10.11.2011 12:54, Dmitry A. Kazakov kirjoitti:
> On Thu, 10 Nov 2011 11:38:39 +0200, Antti J Ylikoski wrote:
>
>> 10.11.2011 10:45, Dmitry A. Kazakov kirjoitti:
>>> On Wed, 9 Nov 2011 14:04:22 -0800 (PST), casey wrote:
>>>
>>>> On Nov 10, 8:17 am, RichD<r_delaney2...@yahoo.com> wrote:
>>>>>
>>>>> McCarthy was also one of those responsible for the original
>>>>> 'thinking machines' hype; "we'll have electronic brains
>>>>> any day now".
>>>>
>>>> And the hype continues ...
>>>
>>> Really? I seen no serious works in this field in recent years.
>>>
>>> For long the focus has been shifted to studies of human brain (which of
>>> course would not create AI any time soon), NN hype (ditto) and practical
>>> problems (OCR etc) solvable without general intelligence.
>>
>> In their book "Artificial Intelligence, A Modern Approach", 3rd Edition,
>> Russell-Norvig crudely estimate that the processing power of the human
>> brain as operations/sec is about 10 to the 17th power operations, and
>> the number of memory updates/sec is about 10 to the 14th power updates.
>> (Russell-Norvig, 2010, p.12).
>
> I love such estimations in the context that nobody knows what does general
> intelligence mean algorithmically.

Think of comparisons of different cars. They can be compared by their
fuel consumption, highway top speed, weight, number of passengers, even
if the individuals who compare cars know nothing about the
thermodynamics of car engines.

>
> Continuing in that vein there is actually only one operation needed: the
> operation "THINK". (:-))

Why is that so? That is what is termed an allegation -- and it is
downright silly.

>
>> I would like to add that there are some 10 to the 11th power neurons in
>> the human brain, and they are connected in the average to some thousand
>> other neurons.
>
> Without saying how exact a the digital model of a neuron [analogue thing]
> must be [in order to do WHAT?] this number tells nothing.
>
> And this is only the beginning. I could tell how much atoms of Si one
> Pentium IV crystal possesses. That by no means would make Pentium out of 10
> grams of river sand.

That is totally silly. The information processing characteristics of
the human brain to my opinion give some kind of an estimate about how
complex a human-level AI must be.... That has nothing to do with the
quality of our digital neuron models, are they good or not.

Taking reductionism to the extreme is a well-known joke. One and a half
tons of iron, aluminium and mangesium could indeed not be driven like a
car at the speed of 100mph on a freeway -- yes indeed -- but that does
not imply that the car's fuel consumption or its number of passengers
tells nothing.

>
>> So that is an estimate of the complexity that one must to my opinion
>> create to have a genuinely human-level Artificial Intelligence.
>
> The crucial point is computability of general intelligence. Without any
> knowledge about what the intelligence actually does, all such comparisons
> are totally meaningless. Let us consider the task of adding two numbers of
> 6 decimal places. For this task you need X neurons of human brain or a
> calculator of paiir thousand transistors. Does it make 2K transistors
> equivalent to human brain? The answer is YES, for this task, and NO for the
> general intelligence task. Without specifying the TASK, comparisons are
> rubbish.
>

The human brain can quite easily specify and describe uncomputable
languages and functions. Full scale predicate logic (of unlimited
degree, I do not refer to FOPL) is uncomputable, but humans are using it
in mathematical proofs with great success. So I would claim that
genuinely human-level AI is -- uncomputable. Mirabile dictu, but so it
would seem to be.

Again, the comparisons of cars are meaningful independently of the
individuals in question knowing about the thermodynamics of car engines.

>> Another matter is that AI as an industry, as a branch of engineering and
>> as a science has been phenomenally successful. I would like to remark
>> that in order to build aeroplanes it has not been necessary to fly in
>> the same manner as birds.
>
> Absolutely. If AI will ever created, its design won't even resemble human
> brain.
>

Certain AI research has been quite successful emulating the human brain.
Also in other research than ANN's. Newell and Simon built their GPS
program after examining human experts solve various problems.


Kind regards V/R, Andy

[Dmitry A. Kazakov]

We know how car functions, that is why there exist objective features which
characterize a car. These features are used as criteria for comparison (for
the properties of interest). All this does not apply to intelligence.

>> Continuing in that vein there is actually only one operation needed: the
>> operation "THINK". (:-))
>
> Why is that so? That is what is termed an allegation -- and it is
> downright silly.

Less silly than counting ADD, SUB, MUL, MOV instructions. At least it is
known for sure that the instruction THINK does thinking, which cannot be
said about any existing combination of ADD, SUB, MUL, MOV...

>>> I would like to add that there are some 10 to the 11th power neurons in
>>> the human brain, and they are connected in the average to some thousand
>>> other neurons.
>>
>> Without saying how exact a the digital model of a neuron [analogue thing]
>> must be [in order to do WHAT?] this number tells nothing.
>>
>> And this is only the beginning. I could tell how much atoms of Si one
>> Pentium IV crystal possesses. That by no means would make Pentium out of 10
>> grams of river sand.
>
> That is totally silly. The information processing characteristics of
> the human brain to my opinion give some kind of an estimate about how
> complex a human-level AI must be....

Characteristics of what? Do you know how human brain process "information"?
Do you know *which* information it has to process in order to be
intelligent? Are you sure that intelligence <=, =, >= processing
information? Processing means if x is input, then f(y) is output.
Intelligence means an ability to solve problems of certain (so far unknown)
class. Formalization of intelligence in the form y=f(x) is a problem,
likely harder than AI itself.

> That has nothing to do with the
> quality of our digital neuron models, are they good or not.

> Taking reductionism to the extreme is a well-known joke. One and a half
> tons of iron, aluminium and mangesium could indeed not be driven like a
> car at the speed of 100mph on a freeway -- yes indeed -- but that does
> not imply that the car's fuel consumption or its number of passengers
> tells nothing.

1. It does not tell how to build a car. Which is exactly the problem with
AI.

2. Relevant (functional) car features like speed, fuel consumption, safety
etc are *measurable*. Relevant features of intelligence are unknown.

3. The features you are suggesting to compare (e.g. number of neurons) are
*irrelevant* so long it is unknown how intelligence works. Why not to use
brain tissue color? Cars are compared by colors, BTW.

> The human brain can quite easily specify and describe uncomputable
> languages and functions. Full scale predicate logic (of unlimited
> degree, I do not refer to FOPL) is uncomputable, but humans are using it
> in mathematical proofs with great success. So I would claim that
> genuinely human-level AI is -- uncomputable. Mirabile dictu, but so it
> would seem to be.

If you think so, then you should not worry about AI, there could be no such
thing on conventional computers in that case.

[Curt Welch]

"Dmitry A. Kazakov" <mai...@dmitry-kazakov.de> wrote:
> On Thu, 10 Nov 2011 11:38:39 +0200, Antti J Ylikoski wrote:
>
> > 10.11.2011 10:45, Dmitry A. Kazakov kirjoitti:
> >> On Wed, 9 Nov 2011 14:04:22 -0800 (PST), casey wrote:
> >>
> >>> On Nov 10, 8:17 am, RichD<r_delaney2...@yahoo.com> wrote:
> >>>>
> >>>> McCarthy was also one of those responsible for the original
> >>>> 'thinking machines' hype; "we'll have electronic brains
> >>>> any day now".
> >>>
> >>> And the hype continues ...
> >>
> >> Really? I seen no serious works in this field in recent years.
> >>
> >> For long the focus has been shifted to studies of human brain (which
> >> of course would not create AI any time soon), NN hype (ditto) and
> >> practical problems (OCR etc) solvable without general intelligence.
> >
> > In their book "Artificial Intelligence, A Modern Approach", 3rd
> > Edition, Russell-Norvig crudely estimate that the processing power of
> > the human brain as operations/sec is about 10 to the 17th power
> > operations, and the number of memory updates/sec is about 10 to the
> > 14th power updates.
> > (Russell-Norvig, 2010, p.12).
>
> I love such estimations in the context that nobody knows what does
> general intelligence mean algorithmically.

Yeah, and in the vain, when people post such large estimates, I like to
show how they might be irreverent, and how the amount of processing needed
could equally be amazingly small.

> Continuing in that vein there is actually only one operation needed: the
> operation "THINK". (:-))
>
> > I would like to add that there are some 10 to the 11th power neurons in
> > the human brain, and they are connected in the average to some thousand
> > other neurons.
>
> Without saying how exact a the digital model of a neuron [analogue thing]
> must be [in order to do WHAT?] this number tells nothing.

The number does tell us something valid, it's just that the number contains
a very large error factor.

> And this is only the beginning. I could tell how much atoms of Si one
> Pentium IV crystal possesses. That by no means would make Pentium out of
> 10 grams of river sand.

One thing I like to do, is reverse the question. I personally think
neurons and nerve fibers are about the most shitty processing devices you
can find. Only slightly better than marbles dropping though a gate. Lie
this:

http://www.youtube.com/watch?v=GcDshWmhF4A

Information propagates over nerves at less than 100 m/s and neurons can
output information in the 100 bits per second range - slow as shit.

So what happens when you try to use slow as shit hardware, to perform a
fairly information intensive calculation like visual pattern recognition?
You are forced to use highly parallel hardware to make up in the width what
you could not do in depth.

So yest the brain is big and complex (lots and lots of parts), but how
complex is the calculation it performs? This we do not know since no one
knows the calculation.

But we do know some other important things, like approximate information
bandwidth of the eyes, and touch, and of the control signals needed to move
our arms and legs. And these bandwidth based numbers show that the amount
of information the flows thought the brain, is trivial compared to our
modern data processing munchies. An iPhone has more bandwidth power than a
brain. A HD movie, had more a fare more information than the visual
information our eyes feed us for example, but an iPhone can suck that in
over a wireless connection, uncompresses it, and display it, in real time.

There's just as much reason to argue that a device like an iPhone, when it
displays a video with audio, is doing more computation, than the brain
does.

So back to my turn the question ground idea.

If neurons are such great information processors, lets look at another
question.

How many neurons, does it take to make an iPhone? Is it EVEN POSSIBLE?
(assuming you had needed senors and effectors to convert signals to never
impulses)?

How do you wire a network of neurons to perform an video uncompress and
audio uncompress algorithm? And how many neurons - which can only process
information at around 100 bits per second each, would it take?

I don't think that 100 billion neurons could do it.

Or lets try this one, how many neurons would take to duplicate the Google
search engine? That is, an "intelligence" good enough, to tell you the top
10 web URLs, for any short set of words you give it? And do it for
millions a people every second?

I think neurons suck at information processing, and the great complexity we
find in the brain, is there MOSTLY not due to how "smart" humans are, but
to how much evolution had to bend over backwards with massive parallelism
to get useful function out of one of the worse information processing
technologies on the planet.

> > So that is an estimate of the complexity that one must to my opinion
> > create to have a genuinely human-level Artificial Intelligence.
>
> The crucial point is computability of general intelligence. Without any
> knowledge about what the intelligence actually does, all such comparisons
> are totally meaningless.

Not totally meaningless. They give us some bounds. We can say the
computation needed to duplicate human intelligence is somewhere between a
small hand held battery operated processor, and a few warehouses full of
high end servers.

:)

> Let us consider the task of adding two numbers
> of 6 decimal places. For this task you need X neurons of human brain or a
> calculator of paiir thousand transistors. Does it make 2K transistors
> equivalent to human brain? The answer is YES, for this task, and NO for
> the general intelligence task. Without specifying the TASK, comparisons
> are rubbish.

Exactly. Without knowing the specifics of the computation, we can't
answer it yet. We can only give large bounds with some wild ass guesses at
probably distributions across that range.

I strongly believe, that most people believe AI is far harder, and far more
complex than it really is, and that belief causes them to make these
computation estimates WAY too high - they don't want to think of it as
easy, because 1) that shows how great their failures have been at trying to
understand AI and 2) it reduces what they are as a human, to something
insignificant - and most humans don't like to see themselves as something
insignificant. They believe, like in the movies, we are super-heroes with
magic powers ruling over the world! The idea that something like an iPhone
has enough power to equal human intelligence would freak a lot of people
out. (and I think human level AI, when it gets here, will freak a lot of
people out).

> > Another matter is that AI as an industry, as a branch of engineering
> > and as a science has been phenomenally successful. I would like to
> > remark that in order to build aeroplanes it has not been necessary to
> > fly in the same manner as birds.
>
> Absolutely. If AI will ever created, its design won't even resemble human
> brain.

Well, at some level of abstraction, it needs to match the brain, or else it
won't end up acting like a human. But at all other levels, there need be
no connections.

--
Curt Welch http://CurtWelch.Com/
cu...@kcwc.com http://NewsReader.Com/

[casey]

On Nov 11, 3:03 am, c...@kcwc.com (Curt Welch) wrote:
> "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de> wrote:

> > And this is only the beginning. I could tell how much atoms of Si one
> > Pentium IV crystal possesses. That by no means would make Pentium out of
> > 10 grams of river sand.
>
> One thing I like to do, is reverse the question.  I personally think
> neurons and nerve fibers are about the most shitty processing devices you

> can find.  Only slightly better than marbles dropping though a gate.  Like
> this:
>
> http://www.youtube.com/watch?v=GcDshWmhF4A

Nice little machine.

Your comments are spot on. Our electronics are much faster
and more reliable than neurons and thus we would not have
to match the brain in number of components and it is most
likely all about getting the right algorithms.

JC

[Dmitry A. Kazakov]

On 10 Nov 2011 16:03:00 GMT, Curt Welch wrote:

> One thing I like to do, is reverse the question. I personally think
> neurons and nerve fibers are about the most shitty processing devices you
> can find. Only slightly better than marbles dropping though a gate. Lie
> this:
>
> http://www.youtube.com/watch?v=GcDshWmhF4A

Cool, I remember a similar machine described in Scientific American a
quarter century ago. But the most fascinating was one built out of Covey's
Life gliders.

> But we do know some other important things, like approximate information
> bandwidth of the eyes, and touch, and of the control signals needed to move
> our arms and legs. And these bandwidth based numbers show that the amount
> of information the flows thought the brain, is trivial compared to our
> modern data processing munchies.

That is not the bandwidth of "information", it is one of the representation
of that information. The information space encoded by N bits has the
cardinality of 2**N. This is far beyond any possible computational power
when approached by brute force. Which means that there is more than enough
room for David to beat Goliath.

> An iPhone has more bandwidth power than a
> brain. A HD movie, had more a fare more information than the visual
> information our eyes feed us for example, but an iPhone can suck that in
> over a wireless connection, uncompresses it, and display it, in real time.
>
> There's just as much reason to argue that a device like an iPhone, when it
> displays a video with audio, is doing more computation, than the brain
> does.

Yes, but these computations are not equivalent. Comparing them requires
further premises, which might be satisfied or not.

> So back to my turn the question ground idea.
>
> If neurons are such great information processors, lets look at another
> question.
>
> How many neurons, does it take to make an iPhone? Is it EVEN POSSIBLE?
> (assuming you had needed senors and effectors to convert signals to never
> impulses)?
>
> How do you wire a network of neurons to perform an video uncompress and
> audio uncompress algorithm? And how many neurons - which can only process
> information at around 100 bits per second each, would it take?
>
> I don't think that 100 billion neurons could do it.
>
> Or lets try this one, how many neurons would take to duplicate the Google
> search engine? That is, an "intelligence" good enough, to tell you the top
> 10 web URLs, for any short set of words you give it? And do it for
> millions a people every second?

You could just ask to compute 100 digital places of sin(0.651)...

(However, the respondent might use his iPhone for that. iPhone could not do
so. Intelligence is when others do things for you! (:-))

> I think neurons suck at information processing, and the great complexity we
> find in the brain, is there MOSTLY not due to how "smart" humans are, but
> to how much evolution had to bend over backwards with massive parallelism
> to get useful function out of one of the worse information processing
> technologies on the planet.

Possible yes, but again it could be apples and oranges. We don't know how
much the computational substrate of neurons of brain vs. electronic
switches of CPU influences the complexity of the task. Your marble machine
is an example. Or consider a pendulum, as a machine. That machine solves
differential equation of pendulum. It does this quite good. How good is
iPhone in solving this equation? I bet that iPhone is much worse. Is iPhone
simpler or more complex than pendulum?

>>> So that is an estimate of the complexity that one must to my opinion
>>> create to have a genuinely human-level Artificial Intelligence.
>>
>> The crucial point is computability of general intelligence. Without any
>> knowledge about what the intelligence actually does, all such comparisons
>> are totally meaningless.
>
> Not totally meaningless. They give us some bounds. We can say the
> computation needed to duplicate human intelligence is somewhere between a
> small hand held battery operated processor, and a few warehouses full of
> high end servers.

The premise is that intelligence is computable, i.e. does not require
incomputable elements in a way that would prevent their encapsulation.
Though computers use such elements, they are encapsulated (e.g. real-time
clocks and hardware random generators).

> I strongly believe, that most people believe AI is far harder, and far more
> complex than it really is, and that belief causes them to make these
> computation estimates WAY too high - they don't want to think of it as
> easy, because 1) that shows how great their failures have been at trying to
> understand AI

BTW, there is a question if the power of general intelligence were
sufficient for understanding intelligence. This is not same as being
intelligent. In which relation both problems are is unknown.

> and 2) it reduces what they are as a human, to something
> insignificant - and most humans don't like to see themselves as something
> insignificant.

As well as the problem of identity. If any human being is exhaustively
described by its state that would be the worst nightmare. However
considering the way quantum physics works, I am rather sceptical about
that.

[James]

"casey" <jgkj...@yahoo.com.au> wrote in message
news:f10f9396-98f0-42b7...@u37g2000prh.googlegroups.com...

FWIW, check out what IBM is playing around with:

http://www.wired.com/wiredscience/2011/08/ibm-synapse-cognitive-computer

http://www.digitalnerds.net/ibm-neural-processor-118

[Curt Welch]

"Dmitry A. Kazakov" <mai...@dmitry-kazakov.de> wrote:

> On 10 Nov 2011 16:03:00 GMT, Curt Welch wrote:
>
> > One thing I like to do, is reverse the question. I personally think
> > neurons and nerve fibers are about the most shitty processing devices
> > you can find. Only slightly better than marbles dropping though a
> > gate. Lie this:
> >
> > http://www.youtube.com/watch?v=GcDshWmhF4A
>
> Cool, I remember a similar machine described in Scientific American a
> quarter century ago. But the most fascinating was one built out of
> Covey's Life gliders.
>
> > But we do know some other important things, like approximate
> > information bandwidth of the eyes, and touch, and of the control
> > signals needed to move our arms and legs. And these bandwidth based
> > numbers show that the amount of information the flows thought the
> > brain, is trivial compared to our modern data processing munchies.
>
> That is not the bandwidth of "information", it is one of the
> representation of that information. The information space encoded by N
> bits has the cardinality of 2**N. This is far beyond any possible
> computational power when approached by brute force. Which means that
> there is more than enough room for David to beat Goliath.

Not sure what point you think you are making there.

Are you saying that we can't push a MBit/sec through a computer because
it's information space is 2^^(MBit)?? You know that's not important right?

Bandwidth doesn't tell us how much computation has to be performed on the
data as it passes through, but it gives us a starting point.

> > An iPhone has more bandwidth power than a
> > brain. A HD movie, had more a fare more information than the visual
> > information our eyes feed us for example, but an iPhone can suck that
> > in over a wireless connection, uncompresses it, and display it, in real
> > time.
> >
> > There's just as much reason to argue that a device like an iPhone, when
> > it displays a video with audio, is doing more computation, than the
> > brain does.
>
> Yes, but these computations are not equivalent. Comparing them requires
> further premises, which might be satisfied or not.

Right. It's just a guess. We don't know the algorithm so it's all just a
guess.

That's mostly nonsense. The pendulum doesn't "solve the differential
equations" so the point is nonsense.

But, if were to try and replace the pendulum with computations, we might
have to implement the replacement with the computation of the differential
equations. And in that sense, the various analog properties of the brain
might require similar back-flips if we try to do replace it with digital
hardware. I don't believe such problems will arise, but until we have
solved that, we just don't know.

> >>> So that is an estimate of the complexity that one must to my opinion
> >>> create to have a genuinely human-level Artificial Intelligence.
> >>
> >> The crucial point is computability of general intelligence. Without
> >> any knowledge about what the intelligence actually does, all such
> >> comparisons are totally meaningless.
> >
> > Not totally meaningless. They give us some bounds. We can say the
> > computation needed to duplicate human intelligence is somewhere between
> > a small hand held battery operated processor, and a few warehouses full
> > of high end servers.
>
> The premise is that intelligence is computable, i.e. does not require
> incomputable elements in a way that would prevent their encapsulation.
> Though computers use such elements, they are encapsulated (e.g. real-time
> clocks and hardware random generators).
>
> > I strongly believe, that most people believe AI is far harder, and far
> > more complex than it really is, and that belief causes them to make
> > these computation estimates WAY too high - they don't want to think of
> > it as easy, because 1) that shows how great their failures have been at
> > trying to understand AI
>
> BTW, there is a question if the power of general intelligence were
> sufficient for understanding intelligence. This is not same as being
> intelligent. In which relation both problems are is unknown.

Yeah, it's a valid question, but I'm fairly sure I know what intelligence
is, and that question doesn't get in the way of the answer. It's because
intelligence is the emergent property of an optimization process - a
learning process. What we can "understand" is limited to what we can
learn, and the brain is too complex to understand itself in that way. But
the underlying process that creates the complexity is not too complex to
understand - which is how we can expect to get around the problem of
intelligence trying to understand itself. The answer is, we don't need to
understand ourselves, we only need to understand the learning process that
created our (adult) human intelligence.

> > and 2) it reduces what they are as a human, to something
> > insignificant - and most humans don't like to see themselves as
> > something insignificant.
>
> As well as the problem of identity. If any human being is exhaustively
> described by its state that would be the worst nightmare. However
> considering the way quantum physics works, I am rather sceptical about
> that.

--

[Dmitry A. Kazakov]

On 10 Nov 2011 21:07:03 GMT, Curt Welch wrote:

> "Dmitry A. Kazakov" <mai...@dmitry-kazakov.de> wrote:
>> On 10 Nov 2011 16:03:00 GMT, Curt Welch wrote:
>>
>>> But we do know some other important things, like approximate
>>> information bandwidth of the eyes, and touch, and of the control
>>> signals needed to move our arms and legs. And these bandwidth based
>>> numbers show that the amount of information the flows thought the
>>> brain, is trivial compared to our modern data processing munchies.
>>
>> That is not the bandwidth of "information", it is one of the
>> representation of that information. The information space encoded by N
>> bits has the cardinality of 2**N. This is far beyond any possible
>> computational power when approached by brute force. Which means that
>> there is more than enough room for David to beat Goliath.
>
> Not sure what point you think you are making there.
>
> Are you saying that we can't push a MBit/sec through a computer because
> it's information space is 2^^(MBit)??

Pushing implies data, not information. If you are talking about information
that would push + interpret. The complexity of interpretation depends on
the number of distinct states in the 2**N space.

> Bandwidth doesn't tell us how much computation has to be performed on the
> data as it passes through, but it gives us a starting point.

It gives the lower bound. 2**N is the upper bound.

>>> I think neurons suck at information processing, and the great
>>> complexity we find in the brain, is there MOSTLY not due to how "smart"
>>> humans are, but to how much evolution had to bend over backwards with
>>> massive parallelism to get useful function out of one of the worse
>>> information processing technologies on the planet.
>>
>> Possible yes, but again it could be apples and oranges. We don't know how
>> much the computational substrate of neurons of brain vs. electronic
>> switches of CPU influences the complexity of the task. Your marble
>> machine is an example. Or consider a pendulum, as a machine. That machine
>> solves differential equation of pendulum. It does this quite good. How
>> good is iPhone in solving this equation? I bet that iPhone is much worse.
>> Is iPhone simpler or more complex than pendulum?
>
> That's mostly nonsense. The pendulum doesn't "solve the differential
> equations" so the point is nonsense.

There is no magic difference between the CPU and pendulum, both are mere
physical systems. Any semantics of what a system "does" is our
interpretation.

>>> I strongly believe, that most people believe AI is far harder, and far
>>> more complex than it really is, and that belief causes them to make
>>> these computation estimates WAY too high - they don't want to think of
>>> it as easy, because 1) that shows how great their failures have been at
>>> trying to understand AI
>>
>> BTW, there is a question if the power of general intelligence were
>> sufficient for understanding intelligence. This is not same as being
>> intelligent. In which relation both problems are is unknown.
>
> Yeah, it's a valid question, but I'm fairly sure I know what intelligence
> is, and that question doesn't get in the way of the answer. It's because
> intelligence is the emergent property of an optimization process - a
> learning process. What we can "understand" is limited to what we can
> learn, and the brain is too complex to understand itself in that way. But
> the underlying process that creates the complexity is not too complex to
> understand - which is how we can expect to get around the problem of
> intelligence trying to understand itself. The answer is, we don't need to
> understand ourselves, we only need to understand the learning process that
> created our (adult) human intelligence.

It does not simplify the problem. I mean it could simplify it if the system
being taught could be considered as a black box, i.e. as a "hardware". It
is indeed so with human pupils. But the idea of AI is not only about the
process of learning (though 16 years of learning is a not what we would
expect from an industrial AI system), but it is also about the "hardware".
This hardware need to be built and has to be understood. It is a big
question if the architecture:

PC hardware (low-level)
|
Software system with an ability to learn (higher-level)
|
Training process

would simplify things. Certainly it would not understanding.

It is hoped that training magically produces intelligence. But again, in
order to *know* this a few things must be shown:

1. the completeness of the software-hardware system, i.e. that there are
states of the system corresponding to "intelligence".

2. That the training process converges to one of these states.

The scenario described by Asimov in his novels: systems exposing
intelligence, while nobody actually understand how do they work, is what
many are putting their hopes into. Does not this vividly resemble searching
for the philosopher's stone?

[Curt Welch]

"Dmitry A. Kazakov" <mai...@dmitry-kazakov.de> wrote:
> On 10 Nov 2011 21:07:03 GMT, Curt Welch wrote:
>
> > "Dmitry A. Kazakov" <mai...@dmitry-kazakov.de> wrote:
> >> On 10 Nov 2011 16:03:00 GMT, Curt Welch wrote:
> >>
> >>> But we do know some other important things, like approximate
> >>> information bandwidth of the eyes, and touch, and of the control
> >>> signals needed to move our arms and legs. And these bandwidth based
> >>> numbers show that the amount of information the flows thought the
> >>> brain, is trivial compared to our modern data processing munchies.
> >>
> >> That is not the bandwidth of "information", it is one of the
> >> representation of that information. The information space encoded by N
> >> bits has the cardinality of 2**N. This is far beyond any possible
> >> computational power when approached by brute force. Which means that
> >> there is more than enough room for David to beat Goliath.
> >
> > Not sure what point you think you are making there.
> >
> > Are you saying that we can't push a MBit/sec through a computer because
> > it's information space is 2^^(MBit)??
>
> Pushing implies data, not information. If you are talking about
> information that would push + interpret. The complexity of interpretation
> depends on the number of distinct states in the 2**N space.

Well, it does and it doesn't. The potential MAX amount of computation
could be said to expand at that rate, but it's just silly to talk about it.

> > Bandwidth doesn't tell us how much computation has to be performed on
> > the data as it passes through, but it gives us a starting point.
>
> It gives the lower bound. 2**N is the upper bound.

Yeah, well, at 1 Mbit per second, the 2**N state space quickly expands to
the point of being too large to explore even if all the matter in the
universe were turned into a giant computer. So that's a stupid upper
limit to even bring up. It's by definition not the "real" upper limit
needed for anything the brain is doing.

But your point is certainly valid that if we don't know how much
computation we need to do, then just talking about the bandwidth doesn't
tell us much of anything. The real question hinges on how much computation
is needed per bit "pushed" and since that is ultimately still unknown, we
really just don't have the facts we need to make a reasonable estimate.

> >>> I think neurons suck at information processing, and the great
> >>> complexity we find in the brain, is there MOSTLY not due to how
> >>> "smart" humans are, but to how much evolution had to bend over
> >>> backwards with massive parallelism to get useful function out of one
> >>> of the worse information processing technologies on the planet.
> >>
> >> Possible yes, but again it could be apples and oranges. We don't know
> >> how much the computational substrate of neurons of brain vs.
> >> electronic switches of CPU influences the complexity of the task. Your
> >> marble machine is an example. Or consider a pendulum, as a machine.
> >> That machine solves differential equation of pendulum. It does this
> >> quite good. How good is iPhone in solving this equation? I bet that
> >> iPhone is much worse. Is iPhone simpler or more complex than pendulum?
> >
> > That's mostly nonsense. The pendulum doesn't "solve the differential
> > equations" so the point is nonsense.
>
> There is no magic difference between the CPU and pendulum, both are mere
> physical systems. Any semantics of what a system "does" is our
> interpretation.

Sure they are just physical systems. But we have words that talk about
characteristics of physical systems, such as "wheel", and, "clock", and
"digital computer". You can't call a lake a "wheel" just because both are
physical system.

You used the word "solve the differential equations", which like "wheel" is
a very specific type of behavior we find in physical systems, and one which
I'm sorry, but a pendulum doesn't do.

If you want to predict the motion of the pendulum using the LANGUAGE OF
MATHEMATICS, you would need to solve the differential equations to make
such a prediction. But the pendulum is not a language processing machine
which is busy at work solving differential equations in order to predict
the motion of some other pendulum (or of itself). To suggest it is is as
ABSURD as trying to argue that a lake is the same thing as a planet.

Yes, it is very much our interpretation of what the words "solve the
differential equations" means about physical systems, but by social
conventions, we do have a fairly clear definition of what that is and it's
not what a pendulum is able to do.

> >>> I strongly believe, that most people believe AI is far harder, and
> >>> far more complex than it really is, and that belief causes them to
> >>> make these computation estimates WAY too high - they don't want to
> >>> think of it as easy, because 1) that shows how great their failures
> >>> have been at trying to understand AI
> >>
> >> BTW, there is a question if the power of general intelligence were
> >> sufficient for understanding intelligence. This is not same as being
> >> intelligent. In which relation both problems are is unknown.
> >
> > Yeah, it's a valid question, but I'm fairly sure I know what
> > intelligence is, and that question doesn't get in the way of the
> > answer. It's because intelligence is the emergent property of an
> > optimization process - a learning process. What we can "understand" is
> > limited to what we can learn, and the brain is too complex to
> > understand itself in that way. But the underlying process that creates
> > the complexity is not too complex to understand - which is how we can
> > expect to get around the problem of intelligence trying to understand
> > itself. The answer is, we don't need to understand ourselves, we only
> > need to understand the learning process that created our (adult) human
> > intelligence.
>
> It does not simplify the problem. I mean it could simplify it if the
> system being taught could be considered as a black box, i.e. as a
> "hardware".

It must be hardware. There is nothing else that exists in this universe.
Not sure what you mean by "is a hardware".

> It is indeed so with human pupils. But the idea of AI is not
> only about the process of learning (though 16 years of learning is a not
> what we would expect from an industrial AI system), but it is also about
> the "hardware". This hardware need to be built and has to be understood.

No it doesn't. That's why you are failing to grasp how learning algorithms
work.

TD-Gammon used neural networks combined with reinforcement learning to
learn how to play the game of backgammon. The guy that wrote it,
understood exactly how and why such a system would work. But after he let
it play itself a few million games, it was able to play as well as the best
human players.

What it ended up doing, in effect, was evolving a function that could
produce an estimated value for any game board position. It was in effect,
the learning machines estimation of the probability of winning the game
from any game board position.

The function created by the learning system, was a neural network which
used a few hundred weights (each weight was just a floating point number).

This program calculated optimal values for those weights - which created
the definition of the function that mapped a game board to a 0 to 1
probability.

Using that evaluation function to guide the selection of moves, the program
plays as well as the best humans. SO the function the learning algorithm
created, proved to be roughly equal to the function the brain creates, when
a human goes about learning how to play backgammon.

Now, instead of using a learning algorithm to create the function, the
author could have attempted to set the values of the weights by hand - to
hand-program the evaluation function.

He in fact, had tried to do just that, in past Backgammon programs he had
written. But the function created by the learning algorithm was better
than anything he had ever created. And not only was it better, the author
had no clue how it worked - or why those values, made it a "better
function" than anything he had tried to created by hand.

Setting those weights, is how the program was "programmed" to play
backgammon. But yet, the setting of those weights, is beyond the
comprehension of any human. There is NO HUMAN that could hand-program that
function. The game of backgammon is too complex for a human to understand
at the level needed to hand program a solution like that by setting the
values of a few hundred weights of a neural network.

The author of the program actually patented the weights - aka the "program"
created by his learning algorithm.

Learning algorithms program computers for us, so we don't have to. And
they do it by nothing more than calculating statistics for us, which is
beyond our ability to calculate (too many calculations for us to do by hand
in many life times).

By analyzing all the numbers of a million games of backgammon, they create
an "understanding" that goes beyond what any human can understand. But
yet, we, as humans, can understand why the learning algorithm works.

There are learning algorithms that are in our reach of "understanding"
which can, create solutions, that our beyond our reach of understanding.

And that's one of the beauties of learning algorithms. They create a level
of complexity, from their own simplicity. They are something simple, that
gives rise to something more complex than itself.

> It is a big question if the architecture:
>
> PC hardware (low-level)
> |
> Software system with an ability to learn (higher-level)
> |
> Training process
>
> would simplify things. Certainly it would not understanding.

It doesn't "simply things". When it "learns" it is actually building a
machine - a machine that is normally more complex than the machine that is
doing the "learning".

And I think these sorts of systems certainly do "understand". But we would
have to get into a debate about what "understanding" is. Which I'm willing
to do if you want to.

> It is hoped that training magically produces intelligence. But again, in
> order to *know* this a few things must be shown:

Well, we have to define what intelligence is. I define it in a way that
removes all the "magic". So I don't have to "hope" that it "might" produce
"intelligence". I know for fact it DOES produce intelligence per my
definition of it.

The only think I have to hope for is that my definition of intelligence is
correct. I do hope for that, but that is being answered by my attempts to
create better learning systems. It will be answered IFF this path yields
machines that people generally agree are intelligent That is just a wait
and see problem.

> 1. the completeness of the software-hardware system, i.e. that there are
> states of the system corresponding to "intelligence".

Yes.

> 2. That the training process converges to one of these states.

Yes.

> The scenario described by Asimov in his novels: systems exposing
> intelligence, while nobody actually understand how do they work, is what
> many are putting their hopes into. Does not this vividly resemble
> searching for the philosopher's stone?

There really isn't anything magical about human intelligence. Skinner and
the other behaviorists figured all that out 70 years ago. Anyone that
doesn't get that is just confused (and there are a LOT of people still very
confused about that). The only thing waiting to be resolved, is solving
the engineering problem of how to build a practical reinforcement learning
algorithm that operates in the high dimension domain that humans and
animals operate in. That engineering problem has proved to be a very
tricky one (no one has figured it out even tough lots of people have been
trying over the past 70 years), but good progress has been made, and we are
getting much closer to the solution every day.

[casey]

On Nov 13, 9:38 am, c...@kcwc.com (Curt Welch) wrote:
> [...]

> TD-Gammon used neural networks combined with reinforcement
> learning to learn how to play the game of backgammon.
> The guy that wrote it, understood exactly how and why such
> a system would work. But after he let it play itself a
> few million games, it was able to play as well as the best
> human players.

But the human backgammon players do not have to play a few
million games, so they are learning in a different way.

The ANN is a fancy way of viewing a statistical number
crunching program which is what computers are good at and
yes they can find solutions we can't find because we aren't
doing that kind of number crunching.

The ANN used in TD-Gammon is your cherry picked example
which proves nothing except there is a findable set of
weights, using the brute force of a computer, which can
value a backgammon state.

It says that out of all the possible weight combinations
there are enough working combinations for this problem to
find a working set in a few million trials unlike say a
game of chess.

> Well, we have to define what intelligence is. I define
> it in a way that removes all the "magic". So I don't
> have to "hope" that it "might" produce "intelligence".
>
> I know for fact it DOES produce intelligence per my
> definition of it.

Your definition is not your definition it is nothing but
Darwinian evolution.

However not all things can evolve and you don't know what
is required for a brain or man made system to learn only
that it involves a feedback system that reinforces some
connections and/or weights and not others.

> The only think I have to hope for is that my definition
> of intelligence is correct.

What you call "intelligence" we call "learning". Of course
all intelligent behavior is the result of learning either
by the species or in real time by the individual.


JC

[Dmitry A. Kazakov]

On 12 Nov 2011 22:38:38 GMT, Curt Welch wrote:

> "Dmitry A. Kazakov" <mai...@dmitry-kazakov.de> wrote:
>> On 10 Nov 2011 21:07:03 GMT, Curt Welch wrote:
>>
>>> Bandwidth doesn't tell us how much computation has to be performed on
>>> the data as it passes through, but it gives us a starting point.
>>
>> It gives the lower bound. 2**N is the upper bound.
>
> Yeah, well, at 1 Mbit per second, the 2**N state space quickly expands to
> the point of being too large to explore even if all the matter in the
> universe were turned into a giant computer. So that's a stupid upper
> limit to even bring up. It's by definition not the "real" upper limit
> needed for anything the brain is doing.
>
> But your point is certainly valid that if we don't know how much
> computation we need to do, then just talking about the bandwidth doesn't
> tell us much of anything. The real question hinges on how much computation
> is needed per bit "pushed" and since that is ultimately still unknown, we
> really just don't have the facts we need to make a reasonable estimate.

The term "amount of computation" is too frequently used out of the context
of computing environment. The upper bound shows that for the worst case
scenario intelligence might be technically incomputable (even being
formally computable) in the environment of switching computers.

The bitter pill for the optimists of AI might be that the environment of
neurons could be capable to solve intelligence, notwithstanding that the
same brain sucks in most problems trivial for switching computers.

And this can well have place even if intelligence is formally computable,
which is usually taken for granted, but might be not the case.

In my humble view the attack at the problem of computability should start
at least intelligent beings like insects. We should try to model their
behavior using AI and parallel to model the structure of their nervous and
sensor systems - compare design - understand - repeat. [I remember some
interesting publications on flies etc]

>>>>> I think neurons suck at information processing, and the great
>>>>> complexity we find in the brain, is there MOSTLY not due to how
>>>>> "smart" humans are, but to how much evolution had to bend over
>>>>> backwards with massive parallelism to get useful function out of one
>>>>> of the worse information processing technologies on the planet.
>>>>
>>>> Possible yes, but again it could be apples and oranges. We don't know
>>>> how much the computational substrate of neurons of brain vs.
>>>> electronic switches of CPU influences the complexity of the task. Your
>>>> marble machine is an example. Or consider a pendulum, as a machine.
>>>> That machine solves differential equation of pendulum. It does this
>>>> quite good. How good is iPhone in solving this equation? I bet that
>>>> iPhone is much worse. Is iPhone simpler or more complex than pendulum?
>>>
>>> That's mostly nonsense. The pendulum doesn't "solve the differential
>>> equations" so the point is nonsense.
>>
>> There is no magic difference between the CPU and pendulum, both are mere
>> physical systems. Any semantics of what a system "does" is our
>> interpretation.
>
> Sure they are just physical systems. But we have words that talk about
> characteristics of physical systems, such as "wheel", and, "clock", and
> "digital computer". You can't call a lake a "wheel" just because both are
> physical system.

Computing is using some system to solve certain problem. This is how we
abstract things in order to understand them. If clock and wheel solve the
same problem, they are same in this context. You certainly can use wheel as
clock and some clocks as wheels...

> If you want to predict the motion of the pendulum using the LANGUAGE OF
> MATHEMATICS, you would need to solve the differential equations to make
> such a prediction.

The language of mathematics is comprised of various computing systems, e.g.
systems of axioms with inference rules.

> But the pendulum is not a language processing machine
> which is busy at work solving differential equations in order to predict
> the motion of some other pendulum (or of itself).

Why not? Again, it is at our full discretion to think out the purpose of
pendulum's behavior.

> To suggest it is is as
> ABSURD as trying to argue that a lake is the same thing as a planet.

I have no doubts that a lake could be used as an alphabet...

>>>>> I strongly believe, that most people believe AI is far harder, and
>>>>> far more complex than it really is, and that belief causes them to
>>>>> make these computation estimates WAY too high - they don't want to
>>>>> think of it as easy, because 1) that shows how great their failures
>>>>> have been at trying to understand AI
>>>>
>>>> BTW, there is a question if the power of general intelligence were
>>>> sufficient for understanding intelligence. This is not same as being
>>>> intelligent. In which relation both problems are is unknown.
>>>
>>> Yeah, it's a valid question, but I'm fairly sure I know what
>>> intelligence is, and that question doesn't get in the way of the
>>> answer. It's because intelligence is the emergent property of an
>>> optimization process - a learning process. What we can "understand" is
>>> limited to what we can learn, and the brain is too complex to
>>> understand itself in that way. But the underlying process that creates
>>> the complexity is not too complex to understand - which is how we can
>>> expect to get around the problem of intelligence trying to understand
>>> itself. The answer is, we don't need to understand ourselves, we only
>>> need to understand the learning process that created our (adult) human
>>> intelligence.
>>
>> It does not simplify the problem. I mean it could simplify it if the
>> system being taught could be considered as a black box, i.e. as a
>> "hardware".
>
> It must be hardware. There is nothing else that exists in this universe.
> Not sure what you mean by "is a hardware".

Hardware is something given for the software. More the hardware does less
software is needed and conversely. Returning to the example of pendulum, it
very much "hardware" and almost no "software" (the weight and the length).

>> It is indeed so with human pupils. But the idea of AI is not
>> only about the process of learning (though 16 years of learning is a not
>> what we would expect from an industrial AI system), but it is also about
>> the "hardware". This hardware need to be built and has to be understood.
>
> No it doesn't. That's why you are failing to grasp how learning algorithms
> work.
>
> TD-Gammon used neural networks combined with reinforcement learning to
> learn how to play the game of backgammon. The guy that wrote it,
> understood exactly how and why such a system would work. But after he let
> it play itself a few million games, it was able to play as well as the best
> human players.

How does this disprove my point? The hardware (NN) was built. It is well
understood how it does work. There initially was some gambling, because it
was a-priory unknown if NN is capable to solve the problem, but that
belongs to the rules of the game.

> He in fact, had tried to do just that, in past Backgammon programs he had
> written. But the function created by the learning algorithm was better
> than anything he had ever created. And not only was it better, the author
> had no clue how it worked - or why those values, made it a "better
> function" than anything he had tried to created by hand.
>
> Setting those weights, is how the program was "programmed" to play
> backgammon. But yet, the setting of those weights, is beyond the
> comprehension of any human. There is NO HUMAN that could hand-program that
> function. The game of backgammon is too complex for a human to understand
> at the level needed to hand program a solution like that by setting the
> values of a few hundred weights of a neural network.

If he tried to adjust iteration steps of the Newton's method while finding
zeros of some function, he would also fail to win. So what?

> The author of the program actually patented the weights - aka the "program"
> created by his learning algorithm.

He could also patent first 100 digital places of tg(23.4)/sqrt(4.5). There
is no limits to the stupidity of the patent law.

> Learning algorithms program computers for us, so we don't have to. And
> they do it by nothing more than calculating statistics for us, which is
> beyond our ability to calculate (too many calculations for us to do by hand
> in many life times).

No, they don't program anything. Learning program is itself a program (per
definition), which acts as it was programmed. It does NOTHING what it was
not programmed for before the button "go" was pressed.

Any differences between direct and iterative methods of solving problems do
not change anything here. It would be silly to believe that the Newton's
method programs anything.

> By analyzing all the numbers of a million games of backgammon, they create
> an "understanding" that goes beyond what any human can understand. But
> yet, we, as humans, can understand why the learning algorithm works.

Understanding here comes from the *analysis* of the numbers. The program
has understood nothing.

> And that's one of the beauties of learning algorithms. They create a level
> of complexity, from their own simplicity. They are something simple, that
> gives rise to something more complex than itself.

Simplicity and complexity are conditionals. Yes there are very simple
looking things exposing an extremely complex behavior. E.g. the Mandelbrot
set or the Covey's Life. But complexity is only our perception.

Furthermore, there is no reason to suggest that what a complex system does
is what you actually need. In fact, it suggests exactly the opposite.
Complex the system are harder to validate.

BTW, one possible definition of "intelligent" could be "too complex to
understand." There is a famous quote by Artur C. Clarke: "Any sufficiently
advanced technology is indistinguishable from magic."

>> It is a big question if the architecture:
>>
>> PC hardware (low-level)
>> |
>> Software system with an ability to learn (higher-level)
>> |
>> Training process
>>
>> would simplify things. Certainly it would not understanding.
>
> It doesn't "simply things". When it "learns" it is actually building a
> machine - a machine that is normally more complex than the machine that is
> doing the "learning".

Nope, learning *always* creates a simpler system than it was before. It is
asserted by noting that the effect of any learning is in reduction of the
number of possible states the system enter. A structure of constraints is
all what learning brings.

> And I think these sorts of systems certainly do "understand". But we would
> have to get into a debate about what "understanding" is. Which I'm willing
> to do if you want to.

I think that understanding is a function of intelligence. I am not sure if
it were possible to somehow formalize "understanding" without knowing
"intelligence." It likely presumes building a model of the understood thing
in the computing environment of the intelligent agent.

>> It is hoped that training magically produces intelligence. But again, in
>> order to *know* this a few things must be shown:
>
> Well, we have to define what intelligence is. I define it in a way that
> removes all the "magic". So I don't have to "hope" that it "might" produce
> "intelligence". I know for fact it DOES produce intelligence per my
> definition of it.

If you define intelligence per construction, you have a problem showing
equivalence of different constructs. You can declare a bottle of Coca-Cola
intelligent. The problem is to show that this "intelligence" is also the
intelligence exposed by humans.

> The only think I have to hope for is that my definition of intelligence is
> correct. I do hope for that, but that is being answered by my attempts to
> create better learning systems. It will be answered IFF this path yields
> machines that people generally agree are intelligent That is just a wait
> and see problem.

Yep, Turing test etc...

No, I don't think that is a fruitful way. For all, because Turing test is
based on an *inability* for an intelligent system to disclose another
system as unintelligent.

It is a logical fallacy: in inability to prove A, does not prove not-A. It
actually proves nothing.

>> The scenario described by Asimov in his novels: systems exposing
>> intelligence, while nobody actually understand how do they work, is what
>> many are putting their hopes into. Does not this vividly resemble
>> searching for the philosopher's stone?
>
> There really isn't anything magical about human intelligence. Skinner and
> the other behaviorists figured all that out 70 years ago. Anyone that
> doesn't get that is just confused (and there are a LOT of people still very
> confused about that). The only thing waiting to be resolved, is solving
> the engineering problem of how to build a practical reinforcement learning
> algorithm that operates in the high dimension domain that humans and
> animals operate in. That engineering problem has proved to be a very
> tricky one (no one has figured it out even tough lots of people have been
> trying over the past 70 years), but good progress has been made, and we are
> getting much closer to the solution every day.

There is nothing magical in the philosopher's stone either. Chemical
elements indeed can be converted into another. The problem is in attempts
to build a nuclear transmutation device without a slightest knowledge of
kernel physics. And even if they per chance found the stone, that still
would not give them that knowledge. After all an intelligent system can be
built by two heterosexual adults in just 9 months...

[Robert Maas, http://tinyurl.com/uh3t]

Shit, he was one of only four still-living people listed in
NNDB.Com whom I ever met in real-life, the other three being Whit
Diffie, Richard Stallman and Donald Knuth, of which only JMC and
Whit would likely still remember me, now only Whit. (I also met
Eugene McCarthy, but he died before NNDB.Com existed. I wish Bill
Gosper or Hans Moravec or Kent Pitman were listed in NNDB.Com,
because they would each remember me too. I'm hanging by a thread
from NNDB.Com, sigh.)
This newsgroup thread is the first I learned of JMC's death.
(Oops, two different meanings of "thread" in same paragraph. Oh well.)

> From: RichD <r_delaney2...@yahoo.com>
> Anybody have any personal McCarthy stories to share?

In late 1970 I was hanging around the A.I. lab, doing my own
software research using SAIL (based on Algol 60) to analyze
random-number generators using a graphic tree representation of
short-growing-context statistics, when he approached me to throw me
out of the lab because I wasn't formally affiliated with Stanford.
But he told me that because I had placed top five in the Putnam
contest, he'd give me a second chance: If I did research for him to
achieve good data compression, he might hire me. So I was allowed
to continue to use the computer, more directed towards
data-compression. The next spring he finally hired me, for two
months.

Later I met his daughter Sarah, who became the 23rd gal I ever
kissed. I wonder what she's up to lately.

P.s. it was Greta Torp, the 3rd gal I ever kissed, who told me
about folk dancing at Stanford, and it was there at Stanford folk
dancing where I heard about the Chess-playing program at the A.I.
lab, causing me to go visit there and start hanging around there,
allowing me to meet Arthur Samuel and later John McCarthy.

Google-groups-search-key: imtrgfdi

[Robert Maas, http://tinyurl.com/uh3t]

> From: "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>

> We know how car functions, that is why there exist objective
> features which characterize a car. These features are used as
> criteria for comparison (for the properties of interest). All
> this does not apply to intelligence.

For a car we know in detail at multiple levels from overall system
function down to chemistry and mechanics, how a car functions. We
don't yet know so well how a brain functions, but in recent years
it's become apparent that the brain is a loose collection of
special-purpose processors, to perform routine data-processing
functions such as visual-feature extraction and muscle-servo, and
specialized but complex problem-solving functions such as building
a model of what's in the visual field, fitted into a longer-term
model of the entire local geography, and figuring out how to
perform navigation and hand-arm manipulation actions upon external
objects. Accordingly I believe it would be appropriate to define
classes of tasks performed by the various processing centers of the
brain, and then to try to devise computer systems to perform each
of these tasks.

Several data-processing tasks have already been successfully
automated: OCR, voice-input, parameterization of graphical input,
musical-tune recognition, crawling/walking/hopping. One high-level
almost-real-world problem-solving task has been demonstrated:
Solving some of the kinds of natural-language riddles/puzzles per
the "Jeopardy!" TV show. See below for my suggestions for
additional kinds of special-domain almost-real-world
problem-solving tasks.

> >> Continuing in that vein there is actually only one operation needed: the
> >> operation "THINK". (:-))
> > Why is that so? That is what is termed an allegation -- and it is
> > downright silly.
> Less silly than counting ADD, SUB, MUL, MOV instructions. At
> least it is known for sure that the instruction THINK does
> thinking, which cannot be said about any existing combination of
> ADD, SUB, MUL, MOV...

I disagree. Defining a name as equivalent to some natural-language
phrase, without any idea what the phrase *really* means, doesn't
contribute to the discussion.

> 2. Relevant (functional) car features like speed, fuel
> consumption, safety etc are *measurable*. Relevant features of
> intelligence are unknown.

I disagree. "Jeopardy!" is already a feature that can be measured,
and in fact has been measured, where the computer did quite well.
Task-test-sets exists for other kinds of problem-solving. For
example, mathematical problem solving can be tested by questions
from any of the High School math contest (University of Santa
Clara), Putnam math contest (nationwide), or American Mathematical
Monthly "Elementary Problems" (similar to high-end of high-school
questions or low-end of Putnam questions. I propose that A.I.
engineers who have not previously taken these tests, and in
particular haven't seen the archives of past questions, but who are
competant at high-school-graduate mathematics, should be shown the
first set (one exam, i.e. one year of high school or Putnam, or one
issue of Elementary Problems) of one of these sequences of prolem
sets, and asked to devise a smart system that can solve these
questions and "any similar". Then after verifying the smart system
can indeed solve that one set of problems, feed it the second of
the particular sequence and see how it does, probably not well at
all. Then have engineeers upgrade the smart system until it is
flexible enough to do a good job with both of those first two
problem sets. Then test it on the third, whereupon it probably
fails again. Iterate adding capability of handling one more single
set then testing on the next one after it. If at some point it
starts being able to solve problems it hasn't yet seen, from that
same sequence, we'll consider that a success, otherwise if it needs
upgrade for each new set of problems, then that's a failure.

Google and Bing have been working on another test set, users'
queries in their search engines, to try to present what the user
really is asking about before the other keyword matches. Last I saw
they haven't done a good job. But at least we have some sort of
test set by which to measure their progress by user satisfaction
vs. frustration, not a well-defined measure, but at least a crude
way to tell if they are doing a good job or not. If they ever get
to the point where I can ask a direct question to Google or Bing
and get back as first response a direct and correct answer to my
question, I'll consider that they have started to succeed.

Question: Who invented LISP?

Google:
Lisp was originally created as a practical mathematical notation for
computer programs, influenced by the notation of Alonzo Church's lambda
calculus. It quickly ...
en.wikipedia.org/wiki/Lisp_(programming_language)
[Text excerpt doesn't answer question, but the linked WikiPedia
article has enough information to construct the answer, as I've
done later below.]

Bing:
Scheme is a statically scoped and properly tail-recursive dialect
of the Lisp programming language invented by Guy Lewis Steele Jr.
and Gerald Jay Sussman.
en.wikipedia.org/wiki/Lisp_(programming_language)
[Text excerpt isn't even on the correct topic, namely the
original invention of LISP, even though it links to same
WikiPedia article as Google did.]
Who invented the word LISP? ChaCha Answer: The word lisp has its
origin in Old English and is taken from the word '-wlyspian'.
www.chacha.com/question/who-invented-the-word-lisp
[Seriously off-topic.]
.. several answers later:
John McCarthy, the researcher who invented LISP and coined the ...
reddit: the front page of the internet ... use the following search
parameters to narrow your results: reddit:{name}
www.reddit.com/r/technology/comments/loa5e/john_mccarthy_the_resear
cher_who_invented_lisp
[Half correct answer, then irrelevant reddit navigation text.]

Correct answer:
John McCarthy invented the concept of LISP as an adaption from
Alonzo Church's lambda calculus, and then Steve Russell converted
a modified form of that concept into a working implementation.

Google-groups-search-key: imtrgfdi

[Dmitry A. Kazakov]

On Sun, 13 Nov 2011 18:00:06 -0800, Robert Maas, http://tinyurl.com/uh3t
wrote:

>> From: "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
>> We know how car functions, that is why there exist objective
>> features which characterize a car. These features are used as
>> criteria for comparison (for the properties of interest). All
>> this does not apply to intelligence.
>
> For a car we know in detail at multiple levels from overall system
> function down to chemistry and mechanics, how a car functions. We
> don't yet know so well how a brain functions, but in recent years
> it's become apparent that the brain is a loose collection of
> special-purpose processors, to perform routine data-processing
> functions such as visual-feature extraction and muscle-servo, and
> specialized but complex problem-solving functions such as building
> a model of what's in the visual field, fitted into a longer-term
> model of the entire local geography, and figuring out how to
> perform navigation and hand-arm manipulation actions upon external
> objects. Accordingly I believe it would be appropriate to define
> classes of tasks performed by the various processing centers of the
> brain, and then to try to devise computer systems to perform each
> of these tasks.

Yes. A "bottom up" approach is more productive. At least it produces
solutions for some problems.

However, we still don't know the architecture of intelligence. The tasks we
had identified reflect our understanding of intelligence, incomplete and
likely wrong. We don't know in which relation these tasks are to general
intelligence.

Furthermore, considering these tasks in the context of intelligence, it
seems not so important to have a solution, rather the method used. For a
long time it was thought, for example, that playing chess requires
intelligence. Then the task was solved using a stupid machine. Arguably,
solutions of this kind combined will never emerge into intelligence.

This is the key issue of AI: if all subtasks were solved by whatever means,
would that result in intelligence? Or is intelligence rather a method than
any concrete task at hand.

>>>> Continuing in that vein there is actually only one operation needed: the
>>>> operation "THINK". (:-))
>>> Why is that so? That is what is termed an allegation -- and it is
>>> downright silly.
>> Less silly than counting ADD, SUB, MUL, MOV instructions. At
>> least it is known for sure that the instruction THINK does
>> thinking, which cannot be said about any existing combination of
>> ADD, SUB, MUL, MOV...
>
> I disagree. Defining a name as equivalent to some natural-language
> phrase, without any idea what the phrase *really* means, doesn't
> contribute to the discussion.

The point was that since it is unknown what intelligence means, then
absence or presence of any meaning for the given mix of primitive
operations is irrelevant. You can count ADD instructions, neurons, beer
bottles with the same [meaningless] result.

>> 2. Relevant (functional) car features like speed, fuel
>> consumption, safety etc are *measurable*. Relevant features of
>> intelligence are unknown.
>
> I disagree. "Jeopardy!" is already a feature that can be measured,
> and in fact has been measured, where the computer did quite well.

But is it a *relevant* feature? I think that sweeping floor is a much
harder problem, much closer to general intelligence than indexing massives
of texts.

> Google and Bing have been working on another test set, users'
> queries in their search engines, to try to present what the user
> really is asking about before the other keyword matches. Last I saw
> they haven't done a good job.

Actually they become worse each year. As with an AI for sweeping floor they
show unintelligence by total inability to classify content between
"valuable" and "junk."

> But at least we have some sort of
> test set by which to measure their progress by user satisfaction
> vs. frustration, not a well-defined measure, but at least a crude
> way to tell if they are doing a good job or not.

I don't think this could serve as an intelligence test.

> Question: Who invented LISP?"

Question: "Images of inventor of LISP with one cat and another person at
the same picture wearing red T-shirt"

[Curt Welch]

casey <jgkj...@yahoo.com.au> wrote:

> On Nov 13, 9:38=A0am, c...@kcwc.com (Curt Welch) wrote:
> > [...]
>
> > TD-Gammon used neural networks combined with reinforcement
> > learning to learn how to play the game of backgammon.
> > The guy that wrote it, understood exactly how and why such
> > a system would work. But after he let it play itself a
> > few million games, it was able to play as well as the best
> > human players.
>
> But the human backgammon players do not have to play a few
> million games, so they are learning in a different way.

That argument is not valid. They can be learning exactly the same way, but
simply be learning faster, or they can be learning in exactly the same way,
and be applying their lessons in life outside of backgammon, to the game of
backgammon.

The fact that a human doesn't need to play a million games tells us NOTHING
about "what way" a human learns John.

> The ANN is a fancy way of viewing a statistical number
> crunching program which is what computers are good at and
> yes they can find solutions we can't find because we aren't
> doing that kind of number crunching.

The brain is doing the same sort of thing John. One way or another it must
be doing the same sort of thing. It can't be learning by operant
conditioning if it were not. Our brains are just statistical number
crunchers that take the lessons of life and uses them to adjust its
synaptic weights and wiring to make it act differently in the future. It's
working in a more analog domain instead of digital as it updates the
weights, but it's doing the same sort of iterative learning process that
works in TD-Gammon.

> The ANN used in TD-Gammon is your cherry picked example
> which proves nothing except there is a findable set of
> weights, using the brute force of a computer, which can
> value a backgammon state.

It proves EXACTLY what I said it proved. I bring it up because it's
existence proves my point - that the approach can work.

> It says that out of all the possible weight combinations
> there are enough working combinations for this problem to
> find a working set in a few million trials unlike say a
> game of chess.
>
> > Well, we have to define what intelligence is. I define
> > it in a way that removes all the "magic". So I don't
> > have to "hope" that it "might" produce "intelligence".
> >
> > I know for fact it DOES produce intelligence per my
> > definition of it.
>
> Your definition is not your definition it is nothing but
> Darwinian evolution.

The way I use the word is what makes it MY DEFINITION you fool. It's not
because I created something new.

> However not all things can evolve and you don't know what
> is required for a brain or man made system to learn

I know a lot about learning John. The fact that I can not make a human
level AI only shows I don't know _everything_ I need to know yet.

> only
> that it involves a feedback system that reinforces some
> connections and/or weights and not others.

I know a lot more than that even if you don't. The field of machine
learning has actually made some very good progress in the past 50 years and
it's not just "update weights from feedback" anymore like it was in the
1950s'.

> > The only think I have to hope for is that my definition
> > of intelligence is correct.
>
> What you call "intelligence" we call "learning". Of course
> all intelligent behavior is the result of learning either
> by the species or in real time by the individual.

Glad to see you are catching on.

[Dmitry A. Kazakov]

On 14 Nov 2011 15:28:16 GMT, Curt Welch wrote:

> casey <jgkj...@yahoo.com.au> wrote:

>> The ANN is a fancy way of viewing a statistical number
>> crunching program which is what computers are good at and
>> yes they can find solutions we can't find because we aren't
>> doing that kind of number crunching.
>
> The brain is doing the same sort of thing John. One way or another it must
> be doing the same sort of thing.

The point was that brain probably does it in *another* way, not by brute
force, using very little explicit training. I think this is crucial to
intelligence and really differentiates that from "stupid" machine learning.
An intelligent system needs coverage of training lesser in many magnitudes.

> Our brains are just statistical number
> crunchers that take the lessons of life and uses them to adjust its
> synaptic weights and wiring to make it act differently in the future.

It is worth to note that insects have practically no memory, learn almost
nothing, yet are exposing a behavior more intelligent, complex and adaptive
than computers armed with TBytes of memory. They do not crunch numbers.
Even if they did, that would be useless because of lack of memory.

Surely, one could argue that training happened per evolution, but I bet
that if we took the very first insect from a drop of amber and revived it,
it would behave no less complex than modern insects.

[casey]

On Nov 15, 2:28 am, c...@kcwc.com (Curt Welch) wrote:

> casey <jgkjca...@yahoo.com.au> wrote:
> On Nov 13, 9:38=A0am, c...@kcwc.com (Curt Welch) wrote:
> > [...]

>> > TD-Gammon used neural networks combined with reinforcement
>> > learning to learn how to play the game of backgammon.
>> > The guy that wrote it, understood exactly how and why such
>> > a system would work. But after he let it play itself a
>> > few million games, it was able to play as well as the best
>> > human players.
>>
>>
>> But the human backgammon players do not have to play a few
>> million games, so they are learning in a different way.
>
> That argument is not valid. They can be learning exactly
> the same way, but simply be learning faster, or they can
> be learning in exactly the same way, and be applying their
> lessons in life outside of backgammon, to the game of
> backgammon.

But note that the td-backgammon program cannot use its
lessons outside of backgammon. Are you suggesting that if
we give TD-Gammon other problems, say chess or checker
game states to evaluate, the ANN will learn to evaluate
backgammon states much faster?

TD-Gammon required hand crafted features and about 1 million
games of experience. And unlike the human brain TD-Gammon's
success hasn't translated well for other implementations or
other games. The human ability to transfer what is learned
in one situation to another situation is not found in the
TD-Gammon program. It has limits and its success owes much
to the nature of the game as to any inherent general purpose
learning abilities of its ANN.

The ANN of TD-Gammon only evaluates game states. A human
can also talk about their moves, they aren't limited to
saying "the result of making that move looks good" which
is all a talking TD-Gammon could talk about. It could not
use reasoning to evaluate WHY a move was good.

>> Your definition is not your definition it is nothing
>> but Darwinian evolution.
>
>

> The way I use the word is what makes it my definition.

You used the word "intelligence" the way others use the
word "learning". You confuse the process with the result.

> I know a lot about learning John.

I also know a lot about learning including animal learning
outside a Skinner box.


JC

[Curt Welch]

"Dmitry A. Kazakov" <mai...@dmitry-kazakov.de> wrote:

> On 14 Nov 2011 15:28:16 GMT, Curt Welch wrote:
>
> > casey <jgkj...@yahoo.com.au> wrote:
>
> >> The ANN is a fancy way of viewing a statistical number
> >> crunching program which is what computers are good at and
> >> yes they can find solutions we can't find because we aren't
> >> doing that kind of number crunching.
> >
> > The brain is doing the same sort of thing John. One way or another it
> > must be doing the same sort of thing.
>
> The point was that brain probably does it in *another* way, not by brute
> force, using very little explicit training.

I don't believe that' true for general human intelligent behavior. I
believe it's mostly trained.

> I think this is crucial to
> intelligence and really differentiates that from "stupid" machine
> learning. An intelligent system needs coverage of training lesser in many
> magnitudes.

I don't know what "coverage of training lesser in many magnitudes" means.

Are you saying it needs less training in order to act intelligent than say
stupid machine learning"?

If humans can learn faster, than also also be justified simply as a better
learning algorithm. We aren't required to assume the brain has innate
ability beyond just better learning.

> > Our brains are just statistical number
> > crunchers that take the lessons of life and uses them to adjust its
> > synaptic weights and wiring to make it act differently in the future.
>
> It is worth to note that insects have practically no memory, learn almost
> nothing, yet are exposing a behavior more intelligent, complex and
> adaptive than computers armed with TBytes of memory. They do not crunch
> numbers. Even if they did, that would be useless because of lack of
> memory.
>
> Surely, one could argue that training happened per evolution, but I bet
> that if we took the very first insect from a drop of amber and revived
> it, it would behave no less complex than modern insects.

Yes, evolution is a learning process. It evolves the design of life to
optimize it's fit to the environment for the purpose of survival. Insects
are made to act "intelligent" by the long slow process of evolution.

Humans have been built by that same long slow process so any number of our
skills can, at a first guess, be thought of as created by that process.

However, much of what an adult humans does, could not have been created by
evolution, because the behavior is too new relative to evolutionary history
to have been hard-wired by evolution. Evolution didn't hard wire us with
car driving skills.

But it could have given us various underlying ability, which we apply to
car driving, so it's difficult to know how and where that line between
nature and nature might be drawn. It's an endless debate.

However, I personally don't buy that there is much of anything humans are
able to do, which we tend to call "intelligent" that was not learned. So I
strongly believe, that the source of our real intelligence, is not skills
wired into us by the long slow learning process of evolution, but by our
ability to learn after birth. The important thing evolution gave us was
the strong generic learning hardware.

[Dmitry A. Kazakov]

On 17 Nov 2011 22:19:05 GMT, Curt Welch wrote:

> "Dmitry A. Kazakov" <mai...@dmitry-kazakov.de> wrote:

>> I think this is crucial to
>> intelligence and really differentiates that from "stupid" machine
>> learning. An intelligent system needs coverage of training lesser in many
>> magnitudes.
>
> I don't know what "coverage of training lesser in many magnitudes" means.

Coverage =

cardinality of the features space / number of unique training samples

With coverage 100% you can trivially memorize samples and repeat them back.
Due to the dimensionality of the inputs human brain has and the big
latencies it has (milliseconds), the coverage of full life long learning is
extremely low.

> Are you saying it needs less training in order to act intelligent than say
> stupid machine learning"?

Yes.

> If humans can learn faster, than also also be justified simply as a better
> learning algorithm.

Certainly yes.

> We aren't required to assume the brain has innate
> ability beyond just better learning.

Better learning might mean using a different feature space. Any learning
algorithm can be considered as a least distance approximation of the
training samples in some metric space by some class of classifying
functions. Potential problems why "stupid learning" could neve be able to
compete:

1. The class of classifying functions. For example, linear functions cannot
separate some patterns. We have no idea which class of functions human
brain applies.

2. The choice of the distance in the features space. For example, there
exist good techniques for least squares optimizations and only poor ones
for C-norms etc.

3. Handling uncertainties (randomness, fuzziness), which in any form
requires massively parallel computing to track down all relevant
alternatives until the point of decision making. Modern computer
architectures are absolutely unsuitable for such methods.

4. The features used for training and classification. No idea which ones
are used by brain at all its multiple layers. Some are possibly genetically
preset, others evidently undergo some selection (using unknown to us
methods).

>>> Our brains are just statistical number
>>> crunchers that take the lessons of life and uses them to adjust its
>>> synaptic weights and wiring to make it act differently in the future.
>>
>> It is worth to note that insects have practically no memory, learn almost
>> nothing, yet are exposing a behavior more intelligent, complex and
>> adaptive than computers armed with TBytes of memory. They do not crunch
>> numbers. Even if they did, that would be useless because of lack of
>> memory.
>>
>> Surely, one could argue that training happened per evolution, but I bet
>> that if we took the very first insect from a drop of amber and revived
>> it, it would behave no less complex than modern insects.
>
> Yes, evolution is a learning process. It evolves the design of life to
> optimize it's fit to the environment for the purpose of survival. Insects
> are made to act "intelligent" by the long slow process of evolution.

That is a belief I don't share. I think when we learn a bit more about it,
we will discover to our dismay, that same as with the DNA, there is just
one design of "intelligence" shared by all living beings, which apparently
was selected before first bacteria appeared...

> However, much of what an adult humans does, could not have been created by
> evolution, because the behavior is too new relative to evolutionary history
> to have been hard-wired by evolution. Evolution didn't hard wire us with
> car driving skills.

Did it the skill of learning how to drive cars?

> However, I personally don't buy that there is much of anything humans are
> able to do, which we tend to call "intelligent" that was not learned.

Ability to learn?

[RichD]

On Nov 9, RichD <r_delaney2...@yahoo.com> wrote:
> McCarthy was also one of those responsible for the original
> 'thinking machines' hype; "we'll have electronic brains

> any day now".  And it succeeded - federal $$$ poured
> in to the computer engineers at the big research
> schools, as MIT, Stanford et al. became branches of
> the Pentagon.

And, with beautiful timing:

http://www.nytimes.com/2011/12/06/science/creating-artificial-intelligence-based-on-the-real-thing.html

"IBM, Cornell, Columbia, the University of Wisconsin,
and U. of California, Merced...
The project has been encouraging enough that in
August it won a $21 million round of government
financing from DARPA, bringing the total to $41 million"

Can I call em, or can I call em?

"But since 2008, the project itself has evolved, becoming more
focused, if not scaled back... These days at the
IBM. Almaden Research Center, there is not a lot of
talk of reverse-engineering the brain.
Wide-ranging ambitions that narrow over time,
Dr. Modha explained, are part of research and discovery,
even if his earlier rhetoric was inflated or misunderstood"

hahahahahahaaa!

But the project is already a sucess, to the
tune of $41 million - that's not a narrowing,
but an expansion, from $20 million -

Old french proverb: plus ca change, plus ca
reste le meme.

--
Rich

[Robert Maas, http://tinyurl.com/uh3t]

DAK> We know how car functions, that is why there exist objective
DAK> features which characterize a car. These features are used as
DAK> criteria for comparison (for the properties of interest). All
DAK> this does not apply to intelligence.

REM>... in recent years it's become apparent that the brain is a
REM>... loose collection of special-purpose processors, to perform
REM>... routine data-processing functions such as visual-feature
REM>... extraction and muscle-servo, and specialized but complex
REM>... problem-solving functions such as building a model of
REM>... what's in the visual field, fitted into a longer-term model
REM>... of the entire local geography, and figuring out how to
REM>... perform navigation and hand-arm manipulation actions upon
REM>... external objects. Accordingly I believe it would be
REM>... appropriate to define classes of tasks performed by the
REM>... various processing centers of the brain, and then to try to
REM>... devise computer systems to perform each of these tasks.

> From: "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
> Yes. A "bottom up" approach is more productive. At least it
> produces solutions for some problems. However, we still don't
> know the architecture of intelligence. The tasks we had
> identified reflect our understanding of intelligence, incomplete
> and likely wrong. We don't know in which relation these tasks are
> to general intelligence.

I'm leaning toward the view that there's no such thing as "general
intelligence", that what we appreciate in humans (and to a lesser
degree in other great apes, cetaceans, cephalopod, and some birds),
and what we thus try to measure in IQ tests, isn't a single
"general intelligence", but rather a hodge podge of specialized
types of cognitive skills. We are blind to skills that we humans
don't have, thus blind to whatever other capabilities a true
"general intelligence" would have if it existed, thus unable to
distinguish between our own hodge podge and a hypothetical "general
intelligence", because all we see of either is the intersection
between what we can see and what's actually there, namely our hodge
podge in both cases. It's analagous between humans not being able
to distinguish a 3-color photo of a flower and a true all-spectrai
view (including UV) of a flower, because we can't see what's
different, namely the presence or absence of the UV. (Insects
however *can* see the UV, so they would not confuse the two.)

> Furthermore, considering these tasks in the context of
> intelligence, it seems not so important to have a solution,
> rather the method used. For a long time it was thought, for
> example, that playing chess requires intelligence. Then the task
> was solved using a stupid machine. Arguably, solutions of this
> kind combined will never emerge into intelligence.

Humans who play a high level of Chess use a more general mental
capacity that *is* one of the components of the hodge podge of
natural human cognitive skills. But skill at playing Chess is such a
narrow specialized skill that no such general method is needed,
a "stupid machine" is sufficient.

If we knew the full capabilities of that particular skill, we might
devise a test that is much more general than just Chess, perhaps
the ability to learn any new kind of board game, and then to
self-train at such a game and get better, and also the ability to
recognize game-theoretical ideas within the natural world and human
society, such as the "war between the sexes" and diplomacy between
nations, respectively. If we could then build a computer system
that did well at this full range of game-theoretic rules-learning
and strategy-optimization, that might turn out to be a component of
A.I. rather than just a "stupid machine" algorithm.

> This is the key issue of AI: if all subtasks were solved by
> whatever means, would that result in intelligence?

If the full range of subtasks were solved by tools that were
specialized only enough to match the capabilities of the various
processing centers of the human brain, maybe yes. But if we break
down the skillset to overly-specialized skills, we'd need billions
of different programs, one per too-specific skill, and every time
we discover a new overly-specialized skill that humans can solve,
we'd have to start from scratch writing yet another program to
solve that one new skill, and it'd be a "mad queen race" between
people inventing new skills that humans can solve but the computer
can't yet solve, and people inventing special programs for the
computer to solve those new skills. Thus I agree with you if we
devise algorithms to solve too-specific skills such as Chess, but
if we instead define wide-span skill-types that match parts of the
human brain that might indeed "result in intelligence" to match
human capabilities.

> Or is intelligence rather a method than any concrete task at hand.

It's a combination of different methods, each of which can solve
one rather general kind of task, if my leaning is correct.

> ... since it is unknown what intelligence means,

The word "intelligence" means anything we define it to mean. In
some context we might define it simply to mean the ability to
operate an android with sufficient skill to pass as a human through
all the ordinary natural and social situations of a normal human
life, including moving about and socializing with humans, engaging
in intimate relationships, learning useful job skills from a
human-oriented train program and consequently performing useful
work to "earn a living", etc. In another context, when robotics
aren't yet developed well enough to make a functional android, we
might use a lesser definition whereby the "A.I." system can browse
newsgroups and compose replies that make more sense than Xah Lee.

REM> I disagree. "Jeopardy!" is already a feature that can be measured,
REM> and in fact has been measured, where the computer did quite well.

> But is it a *relevant* feature?

IMO it's a broader-span skill than expertise at Chess. IMO it
doesn't closely match one of the built-in skills of the human
brain, but I don't know whether it's a broad-enough skill to be a
component in an A.I. Maybe the underlying technology of Watson
(with minor generalization to avoid the trick
answer-before-question facade of Jeopardy, to answer several other
formats of free-language fact-lookup questions) is broad enough to
serve as a component of an A.I. that is rather non-human, or maybe
it's too narrow at all and needs a new research breakthrough to
widen it enough.

> I think that sweeping floor is a much harder problem,

I think that sweeping floor is a totally different type of skill,
not comparable to playing Jeopardy, like apples and not oranges and
not even earthworms but thermophilic sulfur bacteria. Maybe the
underlying Watson technology can be generalized to one component of
the hodge podge, and sweeping floor can be generalized to another,
and twenty more equally generalized components will suffice to
match human cognitive ability.

By the way, I expect Japanese robotics to be able to build a
floor-sweeping robot within the next ten years. But given that
automated vacuum cleaners are a better way to clean tiny debris off
the floor, and in fact automated vacuum cleaners are already nearly
consumer ready, there'll be no economic incentive to actually
produce a floor-sweeping robot, except if some major company such
as Google offers an "X prize" for it. (Aside: I learned just within
the past week that Google has offered an "X prize" for any
non-government whatever to send a robot rover to Luna.)

> much closer to general intelligence than indexing massives of texts.

Which is closer to having a complete automobile, one wheel, or one
cylinder of an engine? In fact you need to generalize the cylinder
to a complete engine, and generalize the wheel to a complete
wheel+tire+axel+driveshaft system, and also add the chassis+frame
and several other systems before you have a working car.

Indexing massives of texts may turn out to be one key part of just
one component of human intelligence.

REM> Google and Bing have been working on another test set, users'
REM> queries in their search engines, to try to present what the user
REM> really is asking about before the other keyword matches. Last I saw
REM> they haven't done a good job.

> Actually they become worse each year.

I don't have an objective test to determine if your suspicion is
correct, but I have a vague feeling you might be correct, except
the effective spelling checker built into Google's search engine
hasn't gotten noticeably worse lately, and might be getting
slightly better. The one area I've noticed is utter crap and not
getting any better is disambiguating search terms such as people's
names. I've been designing a cybernetic (mix of human and computer
components) system that will organize the various Google search
results according to the meaning of the term. For example, if you
search for "Heather Thompson" it would disambiguate that into the
several hundred people by that name, perhaps organized into a
hierarchy or a sub-search engine, and then once you have picked
*which* individual person you are looking for (the one who was
beaten by her husband, or the math professor, or the one who drove
wrecklessly on a country road, or the one who works at a bank,
etc.), it'd show you the Google search results *only* for that one
person. Watch TinyURL.Com/RLlink which will include disambiguation
and identification of individual people as the first component.

> As with an AI for sweeping floor they show unintelligence by
> total inability to classify content between "valuable" and "junk."

That's yet another dimension, after relevance to what you really
wanted to know about. But whereas relevance (which "Heather
Thompson" are you asking about) is a matter of fact, and
truth-value is also a matter of fact (for which TinyURL.Com/TruFut
will be a useful cybernetic aid), value contributing toward your
current task-goal is a matter of opinion, *your* opinion ("you"
being the person requesting the search), because nobody else knows
what you are aiming for. You are in a bit of a dilemma. If you
publicize your entire research programme, so that Google can in
principle "read your mind" as to what relevant to your research
needs, somebody else can steal your programme and publish before
you do and thus deprive you of all your work. But if you keep the
purpose of your query confidential, Google can't even in principle
"read your mind" to know what will really help you and what will be
useless to your current need. I suppose you can play a game where
you give Google just enough clues to be able to find what's
relevant to your needs, but not enough information to be able to
steal your research programme. There several types of
back-and-forth interactive query-disambiguation systems that might
be used to guide Google towards increased relevance:
- Salton's original idea of the user scoring each search result per
relevance and then the ISR system using a sort of Bayesian
fitting of clues with relevance to add weight to some clues and
discount others.
- Wikipedia's current system of disambiguation pages.
- User manually adding additional search terms.

I suspect an A.I. system might operate both ends of the Google/User
relationship, playing role of user to suggest queries that would be
of use, playing role of Google to retrieve some information,
playing role of user to evaluate those results and thus do one of
these:
- formally rate the search results, per Salton's system;
- select a sub-category of search results, per Wikipedia's method;
- modify the query for the next round.

One thing IMDB.Com does is to assign unique terms to each person or
each movie/TVprogram/episode per disambiguation of the original
free-form search terms. Such a technique would allow an intermix of
disambiguation of terms and additional terms to refine the search.

Google-groups-search-key: imtrgfdi

[Dmitry A. Kazakov]

On Tue, 27 Mar 2012 12:25:47 -0700, Robert Maas, http://tinyurl.com/uh3t
wrote:

>> From: "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
>> Yes. A "bottom up" approach is more productive. At least it
>> produces solutions for some problems. However, we still don't
>> know the architecture of intelligence. The tasks we had
>> identified reflect our understanding of intelligence, incomplete
>> and likely wrong. We don't know in which relation these tasks are
>> to general intelligence.
>
> I'm leaning toward the view that there's no such thing as "general
> intelligence", that what we appreciate in humans (and to a lesser
> degree in other great apes, cetaceans, cephalopod, and some birds),
> and what we thus try to measure in IQ tests, isn't a single
> "general intelligence", but rather a hodge podge of specialized
> types of cognitive skills.

OK, call it "cognitive skills," what changed? There is a minimal set of
skills required to be intelligent. That tells something about the way
intelligence is built (a disparate set of skills?), but nothing about how
these skills function and what is required for them to work.

> It's analagous between humans not being able
> to distinguish a 3-color photo of a flower and a true all-spectrai
> view (including UV) of a flower, because we can't see what's
> different, namely the presence or absence of the UV. (Insects
> however *can* see the UV, so they would not confuse the two.)

This is a wrong analogy. An intelligent being can create a UV detector and
thus gain the required ability. Presumably there may exist things which
cannot be understood directly or indirectly in any way in any time. Is this
what you meant?

>> I think that sweeping floor is a much harder problem,

[...]

> By the way, I expect Japanese robotics to be able to build a
> floor-sweeping robot within the next ten years. But given that
> automated vacuum cleaners are a better way to clean tiny debris off
> the floor, and in fact automated vacuum cleaners are already nearly
> consumer ready, there'll be no economic incentive to actually
> produce a floor-sweeping robot,

Mechanics is the least problem. The actual issue is classification between
things to remove and things to stay. A vacuum cleaner does it by
considering dirt everything it can suck in. An intelligent system considers
stain rather as a subject for more careful cleaning.

>> As with an AI for sweeping floor they show unintelligence by
>> total inability to classify content between "valuable" and "junk."
>
> That's yet another dimension, after relevance to what you really
> wanted to know about.

An intelligent system is able to maintain a model of the world in which
things like relevance (as well as many other things) get defined.
Unintelligent systems are bound to a method to measure relevance. An
intelligent system does not need that, it already knows what is relevant,
it is itself a measurement instrument.

[Curt Welch]

"Dmitry A. Kazakov" <mai...@dmitry-kazakov.de> wrote:

> On Tue, 27 Mar 2012 12:25:47 -0700, Robert Maas, http://tinyurl.com/uh3t
> wrote:
>
> >> From: "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>

> >> As with an AI for sweeping floor they show unintelligence by
> >> total inability to classify content between "valuable" and "junk."
> >
> > That's yet another dimension, after relevance to what you really
> > wanted to know about.

"what you want" is the foundation of all relevance. You have neither
extended it or added greater insight to the problem by talking about what
you want to "know" vs what you want to "clean". It's still all becomes the
same question of relevance which all translates back to action selection.
The job of the brain is to select which behavior to produce at every
instant of a human's life. That selection problem is the foundational
problem of relevance that the brain solves. It's the definition of
relevance - or should be. It's the answer to the question "what do I do
now?".

> An intelligent system is able to maintain a model of the world in which
> things like relevance (as well as many other things) get defined.
> Unintelligent systems are bound to a method to measure relevance. An
> intelligent system does not need that, it already knows what is relevant,
> it is itself a measurement instrument.

Unintelligent systems are bound to a method of measure relevance? But we
are not? That makes no sense to me at all. The brain is very much bound
by genetics to it's innate measures of relevance. I don't get to choose
whether eating is relevant or not. My brain is bound by it's innate design
to FORCE me to want to eat. I don't get to choose whether I want to keep
my skin from being burned. I'm bound by my innate genetics to not want my
skin to be damaged.

I structure my ENTIRE LIFE around making sure I have food to eat, and that
my skin is protected. That was not a choice I got to make. It was one made
for me by how the structure of my brain evolved.

My brain is a reinforcement learning machine. It is very much bound to
the reward signals it is wired to maximize. It uses those reward signals
to help it calculate which behaviors are the most likely to lead to higher
rewards at any instant in time. The output of that low level statistically
driven behavior selection process is what we call "human intelligent
behavior". And we are VERY MUCH bound to the internal systems that measure
that relevance for us. We are very much bound to the fact that heat
applied to the skin will creates a highly negative measure of relevance,
and we use all our "intelligence" to plot highly complex ways to reduce the
odds of that happening to us in how our "intelligent" brain selects our
behaviors.

In order to do a good job at selecting behaviors for the purpose of
maximizing future rewards, the brain must become a very good reward
predictor. It must build very complex models of the environment, for the
sole purpose of being a better reward predictor. It must build an estimate
of how likely any given action, at any second in time, is going to effect
future rewards. Because of this, a prime secondary purpose of our brain,
is the assignment of "value" to everything it can sense. We see value, (or
lack of value aka negative value) in everything. We can't do a job like
sweeping a floor and not at the same time, assign value to everything we
sense. The dirt has low value. Getting it off the floor and into the
trash, has high value - which is why we do the sweeping in the first place.
The stain has low value - unless it happens to look like Elvis, or Jesus,
Homer Simpson, at which times we take pictures of it and post it on
Facebook. Everything we do and sense comes automatically with a value
judgement attached to it, because that's what the brain is - a value
estimating machine that uses those value estimations to pick our actions.

But all those estimations, are derived from our innately built in reward
hardware created and tuned by millions of years of evolution for its
usefulness in driving our learning brain into producing actions that tend
to work well at keeping us alive. We are 100% slaves, to our innate low
level reward hardware. without that innate value defining hardware, we
would have no sense of value at all.

All our derived, aka learned values, are extensions of our value measuring
hardware that we are slaves to. Dirt only has negative value, because it
leads to increased odds of our skin getting hurt, our of not being able to
eat.

[Dmitry A. Kazakov]

On 30 Mar 2012 15:56:05 GMT, Curt Welch wrote:

> "Dmitry A. Kazakov" <mai...@dmitry-kazakov.de> wrote:
>>
>> An intelligent system is able to maintain a model of the world in which
>> things like relevance (as well as many other things) get defined.
>> Unintelligent systems are bound to a method to measure relevance. An
>> intelligent system does not need that, it already knows what is relevant,
>> it is itself a measurement instrument.
>
> Unintelligent systems are bound to a method of measure relevance? But we
> are not? That makes no sense to me at all. The brain is very much bound
> by genetics to it's innate measures of relevance.

That is a theory, or rather a philosophical concept, which tells nothing
about how intelligence works. Arguably, when (if) understood intelligence
would become just another method.

One way to define intelligence is as a way to solve problems, for which
there is no "unintelligent" method known. Once method is found the problem
is no more considered as requiring intelligence.

> My brain is a reinforcement learning machine.

It could be thought as such within certain model of intelligence. Alas,
this model is as unproductive (weak? inadequate? wrong?) as any other.

It is widely believed in that intelligence evolves "magically" per
learning, once the structure of "brain" is set right, a damaging idea IMO.
It is similar to the alchemists attempts to create gold.

It is also interesting to observe that people who believe that intelligence
is mechanical, is a kind of clockwork, also honestly believe that there is
no better way to put the cogwheels in place than to kick the mechanism when
it gives wrong answers. That would certainly rearrange the cogwheels
properly...

[Daniel Pitts]

On 3/30/12 9:32 AM, Dmitry A. Kazakov wrote:
> On 30 Mar 2012 15:56:05 GMT, Curt Welch wrote:
>
>> "Dmitry A. Kazakov"<mai...@dmitry-kazakov.de> wrote:
>>>
>>> An intelligent system is able to maintain a model of the world in which
>>> things like relevance (as well as many other things) get defined.
>>> Unintelligent systems are bound to a method to measure relevance. An
>>> intelligent system does not need that, it already knows what is relevant,
>>> it is itself a measurement instrument.
>>
>> Unintelligent systems are bound to a method of measure relevance? But we
>> are not? That makes no sense to me at all. The brain is very much bound
>> by genetics to it's innate measures of relevance.
>
> That is a theory, or rather a philosophical concept, which tells nothing
> about how intelligence works. Arguably, when (if) understood intelligence
> would become just another method.
>
> One way to define intelligence is as a way to solve problems, for which
> there is no "unintelligent" method known. Once method is found the problem
> is no more considered as requiring intelligence.

In other words: The academic field of Artificial Intelligence is a field
where once a phenomena is understood, it is no longer a part of that
field ;-)

[Patricia Shanahan]

On 3/30/2012 2:20 PM, Daniel Pitts wrote:
...

> In other words: The academic field of Artificial Intelligence is a field
> where once a phenomena is understood, it is no longer a part of that
> field ;-)

Indeed. When I first studied AI checkers playing was an AI topic. Now,
playing chess at the grand master level is not AI.

Patricia

[Gary Forbis]

On Mar 30, 9:32 am, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
wrote:

> It is widely believed in that intelligence evolves "magically" per
> learning, once the structure of "brain" is set right, a damaging idea IMO.
> It is similar to the alchemists attempts to create gold.

Hmmm... Self-assembly is a proven method. Do you suppose the lattice
structures of crystals can only be formed by the hand of God? It
really is
turtles all the way down.

> It is also interesting to observe that people who believe that intelligence
> is mechanical, is a kind of clockwork, also honestly believe that there is
> no better way to put the cogwheels in place than to kick the mechanism when
> it gives wrong answers. That would certainly rearrange the cogwheels
> properly...

That there might be a better way doesn't make that way more likely.
A sufficient way that is more likely is still more likely than an
efficient way
that is less likely.

[Dmitry A. Kazakov]

On Mon, 2 Apr 2012 19:46:46 -0700 (PDT), Gary Forbis wrote:

> On Mar 30, 9:32 am, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
> wrote:
>
>> It is widely believed in that intelligence evolves "magically" per
>> learning, once the structure of "brain" is set right, a damaging idea IMO.
>> It is similar to the alchemists attempts to create gold.
>
> Hmmm... Self-assembly is a proven method.

In engineering?

> Do you suppose the lattice
> structures of crystals can only be formed by the hand of God?

The word "intelligent" has slipped away. But if you brought it back, that
would be exactly my question. Why the solution of the problem is sought by
the hand of God (learning) rather than in understanding? I don't mind
iterative/adaptive methods. I do have a problem, when they are applied
without specifying what is to find, if and where the process converges, how
exact is the result is.

>> It is also interesting to observe that people who believe that intelligence
>> is mechanical, is a kind of clockwork, also honestly believe that there is
>> no better way to put the cogwheels in place than to kick the mechanism when
>> it gives wrong answers. That would certainly rearrange the cogwheels
>> properly...
>
> That there might be a better way doesn't make that way more likely.

Likelihood of what? What is the upper bound of the probability of happy
thing to happen?

[Gary Forbis]

On Apr 3, 12:51 am, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>

wrote:
> On Mon, 2 Apr 2012 19:46:46 -0700 (PDT), Gary Forbis wrote:
> > On Mar 30, 9:32 am, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
> > wrote:
>
> >> It is widely believed in that intelligence evolves "magically" per
> >> learning, once the structure of "brain" is set right, a damaging idea IMO.
> >> It is similar to the alchemists attempts to create gold.
>
> > Hmmm... Self-assembly is a proven method.
>
> In engineering?

Do you suppose humans, our exemplar, is engineerd?

In the past I have referred to Artifactual Intelligence. It was
to empahsize that the intelligence was real but created by man.

If you notice, the interior edge of a nut is ever so slightly
indented.
This is to help the nut and bolt align when we get them close enough
together. Most word processors have spell checkers. SSIS builds
massive SQL queries just by connecting two boxes together on the
screen. Most computers today are built from standard products in
modified configurations.

The arc of engineering is towards less and less human involvement
between conceptualization and implementation.

> > Do you suppose the lattice
> > structures of crystals can only be formed by the hand of God?
>
> The word "intelligent" has slipped away. But if you brought it back, that
> would be exactly my question. Why the solution of the problem is sought by
> the hand of God (learning) rather than in understanding? I don't mind
> iterative/adaptive methods. I do have a problem, when they are applied
> without specifying what is to find, if and where the process converges, how
> exact is the result is.

I program via a mix of top down, bottom up, and middle out programming
styles. I see no reason to waste my time on the easy stuff if I can't
do the
hard stuff needed to complete the project. If I fully qualify the
specification
of the product prior to starting the development I am likely to fail
because I
presume the answer to a systemic problem not fully understood. The
nature
of the problem and its habitat guides me.

> >> It is also interesting to observe that people who believe that intelligence
> >> is mechanical, is a kind of clockwork, also honestly believe that there is
> >> no better way to put the cogwheels in place than to kick the mechanism when
> >> it gives wrong answers. That would certainly rearrange the cogwheels
> >> properly...
>
> > That there might be a better way doesn't make that way more likely.
>
> Likelihood of what? What is the upper bound of the probability of happy
> thing to happen?

100%. But I fear that the upper bound isn't your real question.

Some nuts have several threads. If one puts the nut on the bolt cross
threaded
you will likely strip the nut or bolt, whichever is softer. When one
detects a
cross treaded nut the solution is to back the nut off until one feels
it settle.

Human engineered soltuions tend to be torn apart by nature. On the
other hand
naturally developed solution have achieved static or homeostatic
configurations
that last very long times. One needn't know absolute probabilities to
know
relative probabilities. The methods that settle into natural
configurations are
more likely than those which must fight against them.

[Dmitry A. Kazakov]

On Tue, 3 Apr 2012 05:22:16 -0700 (PDT), Gary Forbis wrote:

> On Apr 3, 12:51 am, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
> wrote:
>> On Mon, 2 Apr 2012 19:46:46 -0700 (PDT), Gary Forbis wrote:
>>> On Mar 30, 9:32 am, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
>>> wrote:
>>
>>>> It is widely believed in that intelligence evolves "magically" per
>>>> learning, once the structure of "brain" is set right, a damaging idea IMO.
>>>> It is similar to the alchemists attempts to create gold.
>>
>>> Hmmm... Self-assembly is a proven method.
>>
>> In engineering?
>
> Do you suppose humans, our exemplar, is engineerd?

I have no idea. There is no any *scientific* theory describing human
intelligence and its evolution, much of silly hand waving and
ideological/religious debates though. But the point was about the AI.

> In the past I have referred to Artifactual Intelligence. It was
> to empahsize that the intelligence was real but created by man.

Intelligence is not an artifact, a thing. It is an ability, a property of a
computing system. In order to be able to say whether given computing system
is intelligent we need a constructive (in mathematical sense) definition of
intelligence.

> If you notice, the interior edge of a nut is ever so slightly indented.
> This is to help the nut and bolt align when we get them close enough
> together. Most word processors have spell checkers. SSIS builds
> massive SQL queries just by connecting two boxes together on the
> screen. Most computers today are built from standard products in
> modified configurations.

Do you suppose that this would eventually evolve into intelligence?

[Spellchecking is like chess, a perfect example how a problem attributed to
intelligence was solved by brute force. The key question of AI is: whether
all problems of intelligence could be. In short, whether intelligence is
computable on a FSM. The delusion of many is that bigger the FSM is more
intelligent it becomes. It is like if you had a program bug to fix, you
would run the code on a fatter server. OK, some customers could indeed be
fooled that way (:-))]

> The arc of engineering is towards less and less human involvement
> between conceptualization and implementation.

I don't see that. It is true that some parts get automated. But that by no
means excludes human intelligence, not even reduces its role. Yes, we
become able to solve the problems we could not solve before. But the way
these are solved is still same. We have an intelligent human agent using
unintelligent tools. For example, in programming it is irrelevant if you
program in machine code, or in a higher level programming language. The
code generator is not intelligent. Only productivity and quality differ.

>>> Do you suppose the lattice
>>> structures of crystals can only be formed by the hand of God?
>>
>> The word "intelligent" has slipped away. But if you brought it back, that
>> would be exactly my question. Why the solution of the problem is sought by
>> the hand of God (learning) rather than in understanding? I don't mind
>> iterative/adaptive methods. I do have a problem, when they are applied
>> without specifying what is to find, if and where the process converges, how
>> exact is the result is.
>
> I program via a mix of top down, bottom up, and middle out programming
> styles. I see no reason to waste my time on the easy stuff if I can't
> do the hard stuff needed to complete the project.

Right, you must know in advance if the hard stuff (intelligence) can be
done.

> If I fully qualify the
> specification of the product prior to starting the development I am likely to fail
> because I presume the answer to a systemic problem not fully understood.

Sorry, but AI as a problem is light years distant from being
over-specified: no specifications there, whatsoever.

> Human engineered soltuions tend to be torn apart by nature. On the other hand
> naturally developed solution have achieved static or homeostatic configurations
> that last very long times.

Really? A CPU chip may exist for millions of years.

Engineered solution are far superior to the biological ones in all respects
if the designs are comparable. None of really successful and widely used
solutions can be found in the living nature: car, jet, computer, kettle,
nuclear powerplant, TV, an endless list...

It is wrong to compare massively layered hierarchical designs with the
monolithic ones, especially where the latter do not work.

When our engineering techniques will become able to deploy nano-technology
we will beat the nature on that field too. Consider medicine, you could
fight bacteria or cancer cells by micro robots destroying them physically
rather than chemically. No chances to adapt.

There exist severe limitation in how "evolution" is supposed to work. There
exist solutions (local optimums) which cannot be achieved by gradual steps.
Stanislav Lem considered an example: the wheel.

[Curt Welch]

"Dmitry A. Kazakov" <mai...@dmitry-kazakov.de> wrote:

> On Tue, 3 Apr 2012 05:22:16 -0700 (PDT), Gary Forbis wrote:
>
> > On Apr 3, 12:51 am, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
> > wrote:
> >> On Mon, 2 Apr 2012 19:46:46 -0700 (PDT), Gary Forbis wrote:
> >>> On Mar 30, 9:32 am, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
> >>> wrote:
> >>
> >>>> It is widely believed in that intelligence evolves "magically" per
> >>>> learning, once the structure of "brain" is set right, a damaging
> >>>> idea IMO. It is similar to the alchemists attempts to create gold.
> >>
> >>> Hmmm... Self-assembly is a proven method.
> >>
> >> In engineering?
> >
> > Do you suppose humans, our exemplar, is engineerd?
>
> I have no idea. There is no any *scientific* theory describing human
> intelligence and its evolution, much of silly hand waving and
> ideological/religious debates though. But the point was about the AI.
>
> > In the past I have referred to Artifactual Intelligence. It was
> > to empahsize that the intelligence was real but created by man.
>
> Intelligence is not an artifact, a thing. It is an ability, a property of
> a computing system. In order to be able to say whether given computing
> system is intelligent we need a constructive (in mathematical sense)
> definition of intelligence.

AIXI is one such definition that generally works. Why do you talk as if it
doesn't exist?

> > If you notice, the interior edge of a nut is ever so slightly indented.
> > This is to help the nut and bolt align when we get them close enough
> > together. Most word processors have spell checkers. SSIS builds
> > massive SQL queries just by connecting two boxes together on the
> > screen. Most computers today are built from standard products in
> > modified configurations.
>
> Do you suppose that this would eventually evolve into intelligence?
>
> [Spellchecking is like chess, a perfect example how a problem attributed
> to intelligence was solved by brute force. The key question of AI is:
> whether all problems of intelligence could be. In short, whether
> intelligence is computable on a FSM. The delusion of many is that bigger
> the FSM is more intelligent it becomes. It is like if you had a program
> bug to fix, you would run the code on a fatter server. OK, some customers
> could indeed be fooled that way (:-))]
>
> > The arc of engineering is towards less and less human involvement
> > between conceptualization and implementation.
>
> I don't see that. It is true that some parts get automated. But that by
> no means excludes human intelligence, not even reduces its role. Yes, we
> become able to solve the problems we could not solve before. But the way
> these are solved is still same. We have an intelligent human agent using
> unintelligent tools. For example, in programming it is irrelevant if you
> program in machine code, or in a higher level programming language. The
> code generator is not intelligent. Only productivity and quality differ.

Learning machines are in fact intelligent and are able to replace the human
intelligence in the creation of new software. If it's not a learning
machine, then yes, there is no intelligence in there other than what the
human engineering copied from his own brain (in effect).

But if it's a learning algorithm, then it has it's own intelligence, and
does not need the intelligence to come from the engineer.

We have endless examples of how learning machines have demonstrated
themselves to be more intelligent than the human the created them (in
limited domains).

The only advantage humans still hold over these intelligent optimization
processes is that our hardware is still more generic than any of the
machines we have built. The brain's behavior optimizing system is more
generic and applicable to a wider range of applications. Learning systems
however are advancing and we won't be able to hold the title of "best
learning machine on the planet" for much longer.

> >>> Do you suppose the lattice
> >>> structures of crystals can only be formed by the hand of God?
> >>
> >> The word "intelligent" has slipped away. But if you brought it back,
> >> that would be exactly my question. Why the solution of the problem is
> >> sought by the hand of God (learning) rather than in understanding? I
> >> don't mind iterative/adaptive methods. I do have a problem, when they
> >> are applied without specifying what is to find, if and where the
> >> process converges, how exact is the result is.
> >
> > I program via a mix of top down, bottom up, and middle out programming
> > styles. I see no reason to waste my time on the easy stuff if I can't
> > do the hard stuff needed to complete the project.
>
> Right, you must know in advance if the hard stuff (intelligence) can be
> done.
>
> > If I fully qualify the
> > specification of the product prior to starting the development I am
> > likely to fail because I presume the answer to a systemic problem not
> > fully understood.
>
> Sorry, but AI as a problem is light years distant from being
> over-specified: no specifications there, whatsoever.

Your lack of understanding is not proof they are not there.

> > Human engineered soltuions tend to be torn apart by nature. On the
> > other hand naturally developed solution have achieved static or
> > homeostatic configurations that last very long times.
>
> Really? A CPU chip may exist for millions of years.
>
> Engineered solution are far superior to the biological ones in all
> respects if the designs are comparable. None of really successful and
> widely used solutions can be found in the living nature: car, jet,
> computer, kettle, nuclear powerplant, TV, an endless list...
>
> It is wrong to compare massively layered hierarchical designs with the
> monolithic ones, especially where the latter do not work.
>
> When our engineering techniques will become able to deploy
> nano-technology we will beat the nature on that field too. Consider
> medicine, you could fight bacteria or cancer cells by micro robots
> destroying them physically rather than chemically. No chances to adapt.
>
> There exist severe limitation in how "evolution" is supposed to work.
> There exist solutions (local optimums) which cannot be achieved by
> gradual steps. Stanislav Lem considered an example: the wheel.

Well, that's just silly, because the Earth is full of wheels, and they all
got here by evolution. It's trivial to prove that evolution in fact can
create wheels - just look at the nearest car.

But if you meant "created by the evolution of DNA machines", then yeah,
it's hard to create rotating wheels when the body parts must be connected
together with blood vessels in order for them to stay alive and grow.

Many other things are hard for DNA machines to grow as well, like jet
engines, and micro chips. But instead of growing everything, evolution
discovered it could make even more interesting things by growing a crude
reinstatement learning controller and allowing it to control arms and
hands, which then could build wheels out of non-living matter! So
evolution did solve the problem of how to build wheels.

Only a human would be arrogant enough to believe "he" had done it alone.
That's the same arrogance that allowed men to believe the earth was the
center of the universe for so long! :)

[Curt Welch]

It is also interesting to see how much ignorance there is in the world
about machine learning technology! It has nothing to do with "kicking the
machine when it gives the wrong answer". RL programs are continuous
optimization processes that learn with or without being "kicked". Anyone
that thinks "reinforcement learning" just means "kicking the machine when
it gives the wrong answer" shows no understanding of the many subtle
complexities of the mathematics.

[Dmitry A. Kazakov]

On 05 Apr 2012 00:24:32 GMT, Curt Welch wrote:

> "Dmitry A. Kazakov" <mai...@dmitry-kazakov.de> wrote:

>> Intelligence is not an artifact, a thing. It is an ability, a property of
>> a computing system. In order to be able to say whether given computing
>> system is intelligent we need a constructive (in mathematical sense)
>> definition of intelligence.
>
> AIXI is one such definition that generally works.

I don't see that. Much hand waving, and a sort of brute-force approach
behind, which, I suppose, serves as a warranty to prevent anybody from
testing it in real life.

> Learning machines are in fact intelligent and are able to replace the human
> intelligence in the creation of new software.

Care to provide an example of a software project written by such a machine?

> We have endless examples of how learning machines have demonstrated
> themselves to be more intelligent than the human the created them (in
> limited domains).

More intelligent? Measured how? Claiming that a nail in the wall is more
intelligent than you because it can support a picture hanging on it for a
much longer time than you can? Without a working definition it is
impossible to tell. It just boils down to the Turing test: X seems
intelligent to Y, because Y does not know how X does Z.

> The only advantage humans still hold over these intelligent optimization
> processes is that our hardware is still more generic than any of the
> machines we have built.

Ditto. "More generic" measured how? You are making your argument to the
observed effect rather than the architectural/programming differences
between brain and computer.

> The brain's behavior optimizing system is more
> generic and applicable to a wider range of applications.

It is applicable to intelligent applications, while the machines you refer
to are not. It is not a range, it is a qualitative difference, so long no
quantitative measure provided.

>>> If I fully qualify the
>>> specification of the product prior to starting the development I am
>>> likely to fail because I presume the answer to a systemic problem not
>>> fully understood.
>>
>> Sorry, but AI as a problem is light years distant from being
>> over-specified: no specifications there, whatsoever.
>
> Your lack of understanding is not proof they are not there.

The claim was that something [a software project] might not work if overly
specified upfront. Did AI suffer this peril? The fact that AIs do not write
compilers, DBMS, computer games etc is because of that?

>>> Human engineered soltuions tend to be torn apart by nature. On the
>>> other hand naturally developed solution have achieved static or
>>> homeostatic configurations that last very long times.
>>
>> Really? A CPU chip may exist for millions of years.
>>
>> Engineered solution are far superior to the biological ones in all
>> respects if the designs are comparable. None of really successful and
>> widely used solutions can be found in the living nature: car, jet,
>> computer, kettle, nuclear powerplant, TV, an endless list...
>>
>> It is wrong to compare massively layered hierarchical designs with the
>> monolithic ones, especially where the latter do not work.
>>
>> When our engineering techniques will become able to deploy
>> nano-technology we will beat the nature on that field too. Consider
>> medicine, you could fight bacteria or cancer cells by micro robots
>> destroying them physically rather than chemically. No chances to adapt.
>>
>> There exist severe limitation in how "evolution" is supposed to work.
>> There exist solutions (local optimums) which cannot be achieved by
>> gradual steps. Stanislav Lem considered an example: the wheel.
>
> Well, that's just silly, because the Earth is full of wheels, and they all
> got here by evolution. It's trivial to prove that evolution in fact can
> create wheels - just look at the nearest car.

In that case the whole argument becomes bogus. Gary tried to make a
distinction between human engineered and "naturally" developed solutions.

> But if you meant "created by the evolution of DNA machines", then yeah,
> it's hard to create rotating wheels when the body parts must be connected
> together with blood vessels in order for them to stay alive and grow.

Lots of body parts have no blood vessels. A wheel, so the Lem's argument,
cannot evolve because there is nothing functional to precede it. Any organ
or limb was something functionally useful before it reached its present
form. A wheel is just a jump too large for an incremental process.

> So evolution did solve the problem of how to build wheels.

In that case there is no any AI problem. Evolution solved it 1.0E6 years
ago. Give nine months of time to a man and a woman and they will create you
a new intelligent system, or even two if you are lucky...

[Robert Maas, http://tinyurl.com/uh3t]

REM> I'm leaning toward the view that there's no such thing as
REM> "general intelligence", that what we appreciate in humans (and
REM> to a lesser degree in other great apes, cetaceans,
REM> cephalopods, and some birds), and what we thus try to measure
REM> in IQ tests, isn't a single "general intelligence", but rather
REM> a hodge podge of specialized types of cognitive skills.

> From: "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
> OK, call it "cognitive skills," what changed?

We can try to list most/all of those particular kinds of skills,
and then use them to define what we will call "intelligence" during
the rest of this discussion, instead of begging the question by
saying "I can't define intelligence but nevertheless I can decide
who has it and who doesn't just from my intuition, and everyone
else had better agree what is and what is not intelligence or I'll
act like any such disagreer is stupid".

> There is a minimal set of skills required to be intelligent.

SInce "intelligent" hasn't been defined up to this point, I presume
this is a skeleton of a definition of "intelligent"?

> That tells something about the way intelligence is built (a
> disparate set of skills?),

Most likely a set of low-level skills plus a set of skills that
coordinate the low-level skills. Perhaps a heirarchy that has more
than of two bottom-up levels.

> but nothing about how these skills function and what is required
> for them to work.

Right. First we state what we mean by "intelligent" or
"intelligence", i.e. how we would test whether a being is or is not
intelligent, whether a being has or has not intelligence. Then we
can try to construct algorithms that pass the test and test them
for complience with the requirements to learn whether the
algorithms have each of the required low-level skills and higher-up
coordination skills. It's analagous to specifying an Internet
protocol such as SMTP or TELNET, and then trying to write code that
implements the protocol. Or for this purpose, a set of 'Turing'
tests which an intelligent being is supposed to pass, and then the
algorithms to achieve that result. Software requirements, and then
code to implement the requirements, in that sequence.

REM> We are blind to skills that we humans don't have, thus blind
REM> to whatever other capabilities a true "general intelligence"
REM> would have if it existed, thus unable to distinguish between
REM> our own hodge podge and a hypothetical "general intelligence",
REM> because all we see of either is the intersection between what
REM> we can see and what's actually there, namely our hodge podge
REM> in both cases. It's analagous between humans not being able to
REM> distinguish a 3-color photo of a flower and a true
REM> all-spectrai view (including UV) of a flower, because we can't
REM> see what's different, namely the presence or absence of the
REM> UV. (Insects however *can* see the UV, so they would not
REM> confuse the two.)

> This is a wrong analogy. An intelligent being can create a UV
> detector and thus gain the required ability.

Are you claiming that because we can't create a
general-intelligence detector, we will forever be restricted to the
intersection of the hodge-podge-of-skills intelligence we have
ourselves and whatever a "A.I." device might have, thus forever
unable to tell whether the "A.I." device goes beyond what we have
or not?

If that's your claim, then I disagree. As we study various animals,
we will discover types of intelligence some of them have which we
do not ourselves have. Thus we will be able to devise tests for
types of intelligence that go beyond our own. If at some point we
find an algorithm for a device to "self-teach" to bootstrap a type
of intelligence that includes all of our skills and all the skills
of each of the animals we've studied, and also lots of skills that
no animal on Earth can do, with all the parts fully integrated, all
together in a single device, we may have that "Eureka!" moment when
we realize that we've invented a true "general intelligence". We
may then also develop an algorithm for generating problems to
solve, some of which humans can solve, some of which only other
animals can solve, and some of which *only* our
general-intelligence A.I. device can solve. This problem-generator
together with a rig for asking the human or animal or A.I. device
to solve each problem, could then serve as a true test of general
intelligence.

Note that to date the only skills we've tested in animals are those
which are similar to skills humans have, but I expect in the next
20 years animal-behaiour scientists will start to test "outside the
box", such as learning how whales and birds navigate thousands of
miles, how eusocial insects maintain "law and order" within the
hive, etc., and thus start to find types of intelligence we don't
yet appreciate because as snobs we aren't willing to admit that
other animals are smarter or more intelligent than we are for some
kinds of problems that stump us but which they do solve.

> Presumably there may exist things which cannot be understood
> directly or indirectly in any way in any time. Is this what you
> meant?

No, just that present we aren't thinking outside the
human-intelligence "box" so we are blind (at present) to other
forms of intelligence.

> >> I think that sweeping floor is a much harder problem,

REM> By the way, I expect Japanese robotics to be able to build a
REM> floor-sweeping robot within the next ten years. But given that
REM> automated vacuum cleaners are a better way to clean tiny
REM> debris off the floor, and in fact automated vacuum cleaners
REM> are already nearly consumer ready, there'll be no economic
REM> incentive to actually produce a floor-sweeping robot,

> Mechanics is the least problem. The actual issue is
> classification between things to remove and things to stay. A
> vacuum cleaner does it by considering dirt everything it can suck
> in. An intelligent system considers stain rather as a subject for
> more careful cleaning.

Good point. I've never seen an automated cleaning device except on
TV, mostly science programs from NHK, so I can only guess as to its
true capailities and limitations. If you have used one of them in
RL, please tell us your observations, else you're not just guessing
too. My guess is that so-far they've managed to construct an
internal model of the geography of the local environment (the
building to be cleaned) and use that model to keep track of which
regions of the floor still need to be cleaned, and navigate not
just to avoid obstacles but to make sure that each portion of the
floor gets cleaned at least once during each cleaning session.

The kind of smarter system you requested is somewhat like what I
have proposed for cleaning litter from sidewalks and gutters and
lawns and parking lots etc. There are typically lots of leaves
fallen from trees, some loose and some blown into clumps. There are
also odd bits of litter such as empty beverage cups, metal cans,
plastic bottles, etc. Ideally EVERYTHING that doesn't belong in
place would be collected, but then the various types of litter
would be separated, such as leaf litter sent to composter, aluminum
cans and plastic bottles sent to recyclers, other plastic and paper
etc. sent to incinerator, etc.

A few years ago I posted an alternate idea, that birds can be
trained to collect litter and separate the various materials, which
might be faster to develop than an A.I. system.

> >> As with an AI for sweeping floor they show unintelligence by
> >> total inability to classify content between "valuable" and "junk."
> > That's yet another dimension, after relevance to what you really
> > wanted to know about.
> An intelligent system is able to maintain a model of the world in
> which things like relevance (as well as many other things) get
> defined. Unintelligent systems are bound to a method to measure
> relevance. An intelligent system does not need that, it already
> knows what is relevant, it is itself a measurement instrument.

That remark smells like a circular definition.

Google-groups-search-key: imtrgfdi

[Dmitry A. Kazakov]

On Tue, 17 Apr 2012 21:57:35 -0700, Robert Maas, http://tinyurl.com/uh3t
wrote:

> REM> I'm leaning toward the view that there's no such thing as
> REM> "general intelligence", that what we appreciate in humans (and
> REM> to a lesser degree in other great apes, cetaceans,
> REM> cephalopods, and some birds), and what we thus try to measure
> REM> in IQ tests, isn't a single "general intelligence", but rather
> REM> a hodge podge of specialized types of cognitive skills.
>
>> From: "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
>> OK, call it "cognitive skills," what changed?
>
> We can try to list most/all of those particular kinds of skills,
> and then use them to define what we will call "intelligence" during
> the rest of this discussion,

The obvious problem is that possibly these would not constitute
intelligence or else would require intelligence to implement. So far
"intelligence" resisted all decomposition attempts.

>> That tells something about the way intelligence is built (a
>> disparate set of skills?),
>
> Most likely a set of low-level skills plus a set of skills that
> coordinate the low-level skills. Perhaps a heirarchy that has more
> than of two bottom-up levels.

and a huge gap somewhere between these levels, which we have not been able
to bridge.

>> but nothing about how these skills function and what is required
>> for them to work.
>
> Right. First we state what we mean by "intelligent" or
> "intelligence", i.e. how we would test whether a being is or is not
> intelligent, whether a being has or has not intelligence.

Well. The task AI1 is to build an intelligent system. The task AI2 is to
build a system capable to determine if AI1 is intelligent.

My bet is that the task AI2 is likely far more complex than AI1. We know
this from software engineering, proving correctness, testing, debugging,
maintenance tasks are more difficult than mere solving the end problem.

> REM> We are blind to skills that we humans don't have, thus blind
> REM> to whatever other capabilities a true "general intelligence"
> REM> would have if it existed, thus unable to distinguish between
> REM> our own hodge podge and a hypothetical "general intelligence",
> REM> because all we see of either is the intersection between what
> REM> we can see and what's actually there, namely our hodge podge
> REM> in both cases. It's analagous between humans not being able to
> REM> distinguish a 3-color photo of a flower and a true
> REM> all-spectrai view (including UV) of a flower, because we can't
> REM> see what's different, namely the presence or absence of the
> REM> UV. (Insects however *can* see the UV, so they would not
> REM> confuse the two.)
>
>> This is a wrong analogy. An intelligent being can create a UV
>> detector and thus gain the required ability.
>
> Are you claiming that because we can't create a
> general-intelligence detector, we will forever be restricted to the
> intersection of the hodge-podge-of-skills intelligence we have
> ourselves and whatever a "A.I." device might have, thus forever
> unable to tell whether the "A.I." device goes beyond what we have
> or not?

Rather the reverse. If we cannot solve AI1, we will not do AI2.

> If that's your claim, then I disagree. As we study various animals,
> we will discover types of intelligence some of them have which we
> do not ourselves have.

Intelligence /= ability. Though I agree that it is important to design
systems of lesser intelligence, certainly more important than full-scale
human intelligence.

>> Presumably there may exist things which cannot be understood
>> directly or indirectly in any way in any time. Is this what you
>> meant?
>
> No, just that present we aren't thinking outside the
> human-intelligence "box" so we are blind (at present) to other
> forms of intelligence.

Other forms in the sense that they don't imply human intelligence? In that
case they are irrelevant to the task of building human AI.

> A few years ago I posted an alternate idea, that birds can be
> trained to collect litter and separate the various materials, which
> might be faster to develop than an A.I. system.

Huh. The uncomfortable fact is, that not only birds, just ants are far more
intelligent than anything we were able to build so far. This might be a
perception problem or a real one. I hope that studies simulating nervous
system of insects and whole insects (brain + sensors + actuators) will shed
some light.

>>>> As with an AI for sweeping floor they show unintelligence by
>>>> total inability to classify content between "valuable" and "junk."
>>> That's yet another dimension, after relevance to what you really
>>> wanted to know about.
>> An intelligent system is able to maintain a model of the world in
>> which things like relevance (as well as many other things) get
>> defined. Unintelligent systems are bound to a method to measure
>> relevance. An intelligent system does not need that, it already
>> knows what is relevant, it is itself a measurement instrument.
>
> That remark smells like a circular definition.

Yes, as well as the Turing test itself, it is not a definition. It is how
intelligence is perceived when not properly defined. Note also that if
incomputable, then there cannot be a definition without some incomputable
reference system. Computability of intelligence and its constructive
definition are close issues.

[Robert Maas, http://tinyurl.com/uh3t]

> > >> As with an AI for sweeping floor they show unintelligence by
> > >> total inability to classify content between "valuable" and "junk."
> > > That's yet another dimension, after relevance to what you really
> > > wanted to know about.

> From: c...@kcwc.com (Curt Welch)

> "what you want" is the foundation of all relevance. You have
> neither extended it or added greater insight to the problem by
> talking about what you want to "know" vs what you want to
> "clean". It's still all becomes the same question of relevance
> which all translates back to action selection. The job of the
> brain is to select which behavior to produce at every instant of
> a human's life. That selection problem is the foundational
> problem of relevance that the brain solves. It's the definition
> of relevance - or should be. It's the answer to the question
> "what do I do now?".

I think I agree. Clarifying what you said a little: Relevance of
some piece of knowledge means to what degree obtaining that
knowledge is useful for aiding a decision what to do next. OK?

So the robot needs to estimate what information would be worth
getting, to aid the what-next decision, as opposed to what
information is of no immediate use. But a longer-term optimization
might include gathering additional information that is of no
immediate use but which is likely to be useful later and is much
cheaper to get now while "here" rather than needing to come back
here to get it later when it's urgently needed.

The rest of what you said sounds good and I have nothing to add.

[Robert Maas, http://tinyurl.com/uh3t]

> > In other words: The academic field of Artificial Intelligence is a field
> > where once a phenomena is understood, it is no longer a part of that
> > field ;-)

> From: Patricia Shanahan <p...@acm.org>

> Indeed. When I first studied AI checkers playing was an AI topic.
> Now, playing chess at the grand master level is not AI.

Because the solution found was a specific algorithm for a narrow
skill, rather than a general-purpose problem-solving algorithm.
Thus it was shown that such a single "needs a bright human" skill
was misunderstood, that in reality all it needed was a special
algorithm. If and when we find a way that a fixed computer
algorithm (self-training without any further major programming
changes) can learn not just the set of skills it was designed for
but a whole range of totally new skills, just about every kind of
problem-to-solve that we throw at it, then we would have A.I. that
remains A.I. rather than being demoted to "just another specialized
algorithm".

And furthermore, playing checkers, or playing chess, etc. are each
very very too specific to even be a component in my hodge-podge
model of intelligence. If, however, we build an algorithm that can
learn *every* board game just by reading the rules in mathematical
form (or reading the rules in natural language accompanied by some
coaching towards understanding what the rules really mean
mathematically), followed by ordinary practice and study just like
the way humans learn, that would be one step closer to the level of
generality required as a single component of my hodge-podge model
of intelligence. Yet another level of generalization would be an
algorithm that is able to learn not just board games but virtually
every contrived form of human competition, including Poker,
athletic games (individual and team), mating rituals, warfare,
economics, etc. That level of generality might actually match one
of the actual built-in human hodge-podge components.

So anybody want to guess when step 1 (all board games) and step 2
(all contrived forms of competition) will be achieved? I'll make a
wild guess: Another 50 years for each successive step, i.e. 2062
and 2112 respectively. Then an additional 100 years (2212) to learn
all the other components and algorithimize each, then another 50
years (2262) to put them all together into a true A.I. device.

Or maybe all we need to do is devise a good genetic algorithm for
breeding algorithms that accomplish each of the steps I outlined
above, and let the fastest "cloud" computing system in the world
run this genetic algorithm during all otherwise idle moments,
whereupon it might achieve these A.I. research results faster than
humans could. Perhaps the "Watson" program to sift through a huge
database of facts to find anything relevant to a "teaser" and then
evaluate relevance to the "teaser" could be modified to extract
from the collection of all online human knowledge anything that
might be relevant to this research project, thereby proposing
experimental algorithms to test against the goals I outlined above,
thus creating the pieces of "genome" to mix-and-match in the
genetic algorithm. And then every time somebody comes up with a new
idea for A.I. and posts the idea on the Internet, that idea is
immediately captured by Watson-2 and incorporated as another piece
of "genome". Once enough pieces are available to allow board games
to be automatically learned, perhaps those "genome" pieces will be
sufficient that no further human suggestions will be needed, i.e.
pure Darwinian evolution will be sufficient to finish the entire
R&D program, and voila: Colossus: The Forbin Project

[casey]

On Apr 22, 8:53 am, seeWebInst...@rem.intarweb.org (Robert Maas,

http://tinyurl.com/uh3t) wrote:


> Or maybe all we need to do is devise a good genetic
> algorithm for breeding algorithms that accomplish
> each of the steps I outlined above, and let the
> fastest "cloud" computing system in the world run
> this genetic algorithm during all otherwise idle
> moments, whereupon it might achieve these A.I.
> research results faster than humans could.

The problem here is one of generating variants that
are worth selecting and the making of a selective
mechanism. We are the product of natural selection
whereas our evolutionary networks are only subject
to artificial selection.

I would direct you to the "The Blind Watchmaker"
chapter 3 and the issue of natural selection vs. the
artificial selection requiring a human in the loop.

I would point out that not all systems can evolve
and exactly how we could make one that would evolve
given our current environment is unknown.

[Chris Uppal]

Robert Maas, http://tinyurl.com/uh3t wrote:

> Or maybe all we need to do is devise a good genetic algorithm for
> breeding algorithms that accomplish each of the steps I outlined
> above, and let the fastest "cloud" computing system in the world
> run this genetic algorithm during all otherwise idle moments,
> whereupon it might achieve these A.I. research results faster than
> humans could.

People following this thread, who are not already aware of it, might be
interested in Jürgen Schmidhuber's work:
http://www.idsia.ch/~juergen/

In particular, his concept of "Gödel machines":
http://www.idsia.ch/~juergen/goedelmachine.html

From that page: "Goedel machines are self-referential universal problem solvers
making provably optimal self-improvements."

-- chris

[Don Stockbauer]

On Apr 22, 4:03 am, "Chris Uppal" <chris.up...@metagnostic.REMOVE-
THIS.org> wrote:
> Robert Maas,http://tinyurl.com/uh3twrote:

But doesn't PI trump AI?

[Robert Maas, http://tinyurl.com/uh3t]

> From: "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>

> So far "intelligence" resisted all decomposition attempts.

Yes, the job is nowhere near done, but neurobiologists *have* made
some progress in that direction, using NMR imaging to watch which
parts of the brain are active when performing which kinds of
cognitive tasks. I we can identify which cognitive tasks are
performed by each part of the brain, then we can lump those
same-part skills into a better research program where each group of
researchers concentrates on trying to understand just that small
group of skills instead of trying to do everything at one time.
Thus both the decomposition per se and the detailed studying of
each component looks feasible as this kind of research progresses.

> > Most likely a set of low-level skills plus a set of skills that
> > coordinate the low-level skills. Perhaps a heirarchy that has more
> > than of two bottom-up levels.
> and a huge gap somewhere between these levels, which we have not
> been able to bridge.

Not yet, but someday. In 1800 we did not yet understand how
earthquakes are caused, but now we basically have it figured out
(tektonic plates grinding against each other in various
configurations, and magma pressing+melting+flowing upward toward
the surface). We might likewise make similar progress in
understanding natural intelligence over the next 200 years, and
thus be better able to emulate it in A.I..

> The task AI1 is to build an intelligent system. The task AI2 is
> to build a system capable to determine if AI1 is intelligent.

> My bet is that the task AI2 is likely far more complex than AI1.
> We know this from software engineering, proving correctness,
> testing, debugging, maintenance tasks are more difficult than
> mere solving the end problem.

I side with extreme TDD (Test-Driven Development), whereby you
write one requirement the software must satisfy, then write the
simplest software capable of satisfying, then add another
requirement, then modify the software to satisfy both, then add a
third requirement, then modify the software to satisfy all three,
etc. AI2 (both the specification and the test/validation rig) are
easier than AI1 at each step of adding requirements. If the
requirements include solving "trap door" problems, then it's *much*
easier to write the specification and test rig than to design the
algorithm to solve the problem.

Although it's possible to manually calculate the result from each
input, and simply write a program that compares each input for
exact match against a known input and simply writes out the
corresponding canned answer, a better approach would be to write an
algorithm that really does compute the answers, thus given several
test examples is able to satisfy not just those cases but several
additional cases of the same type not yet presented to it.

Or a specification for a class of input could be given, and a
specification for a way to test whether the algorithm under
construction produces a correct answer, and then it's the job of
the software team to write a minimal program that works with *any*
input within that class to produce output that always passes the
specified test. That would make it impossible for a simple lookup
table to pass validation.

Here's an example of a task and validator, not closely related to
A.I., but something like what I worked on for a while in 1992:
Given a bit-raster, which contains mostly off bits, but some
sequences of on bits that form (curved/bet) lines, what are called
lineaments, parameterize each such linement, i.e. put all the
on-bits within each lineament into a sequence from one end to the
other end. The test rig takes the output from the program under
development, verifies that each list of points is indeed locally
connected, then turns on all the bits in a raster per the
lineaments, and does a bitwize compare between the original and
reconstructed rasters to see whether the lineaments were indeed the
correct ones from the original image. To generate a set of test
data, all we need is a random-walk program that avoids turning too
tightly and avoids closing back to itself. (A more advanced task
would involve multiple lineaments that are allowed to cross each
other and also allowed to form closed loops, whereby the graph must
be cut anywhere two lieaments cross, and closed loops that don't
include any crossing or branching points may be "started" at any
arbitrary point around the loop. An even more advanced task would
also allow lineaments that were more than one pixel wide, varying
in width along the path, which is actually what I worked on in
1992. However I didn't know about TDD back then, so we checked the
results manually/visually rather than writing an automatic test
rig.)

Back to AI: I think we might easily devise test rigs to produce
test data and verify correct output from the AI program under
development. For example, we can specify that there's a table with
an object on it, and the AI program is supposed to use its "eyes"
to locate the object and then use its "arm" to reach out and grab
the object and lift it up and drop it into a hopper. It should be
easy to devise an automated test rig that tosses any of several
objects onto the table, bouncing off walls, ending an a sort of
random final position, and then to verify by pressure sensor when
the object hits the bottom of the hopper. To make the test rig
easiest of all, the bottom of the hopper could be attached to a
high-pressure air hose and explosive-style pump, so that a single
object could be repeatedly tossed onto the table (by the test rig)
and then "fetched" (by the program under development) without human
intervention. Every so often, the test rig could empty the current
obeject from the hopper and replace it with a different object.
Maybe a device similar to a music-CD "carousel" would suffice for
that task.

In summary, there are a whole bunch of AI tasks that are easy to
set up the test conditions and test whether the program does the
task correctly, but not so easy to devise an algorithm to actually
do the task under test. Thus AI2 is much easier than AI1.

> If we cannot solve AI1, we will not do AI2.

I disagree, see previous paragraphs.

> > If that's your claim, then I disagree. As we study various animals,
> > we will discover types of intelligence some of them have which we
> > do not ourselves have.

> Intelligence /= ability.

We might be nitpicking as to the difference between "intelligent"
(ability to solve new kinds of problems) and "smart" (ability to
solve a specific class of problems). In fact most humans aren't
intelligent in the general sense. They can't even solve mathematics
problems except if they are almost exactly of a type they've been
taught how to solve in a class. Very few humans can solve the kinds
of mathematics problems typical of the University of Santa Clara
high-school math contest, or "Elementary Problems" in the American
Mathematical Monthly, or problems typical of the William Lowell
Putnam mathematics competition. At best, most humans are "smart" in
that they can solve algebraic problems and a few limiited kinds of
"word problems" treduce to algebraic problems, and at worst most
humans can't even do elementary algebra to save their lives.

Most humans at best can solve the kinds of problems (math, or
social situations, or economic situations, or political situations,
or housekeeping situations) where they've already been taught the
basics of that kind of situation and at most have to stretch a
teensy bit from what they taught to solve a slightly novel problem,
such as how to prepare a recipe if you're missing one of the
ingrediants, or how to clean a stain on the rug if simple detergent
and scrub-brush isn't sufficient, or how to vote if the polling
place moved, etc. Even learning how to use the next version of
MicroSoft Office can be a daunting task, even a stumper, for many
workers in offices.

For the most part, in both humans and other animals, the limit of
intelligence is being "smart" at what one has already learned by
explicit classroom instruction and/or on-the-job training, plus
being able to self-extend that smartness just a teensy bit further.
If we can build smart computer systems that are able to self-extend
to solve slightly novel problems, I think that will be entirely
sufficient to achieve Artificial run-of-the-mill Intelligence. If
we could make an A.I. device that demonstrated greater intelligence
than 90% of humans, that would be "good enough" for this century.
No need to set our sights too high, to Artificial Genius.

> >> Presumably there may exist things which cannot be understood
> >> directly or indirectly in any way in any time. Is this what you
> >> meant?
> > No, just that present we aren't thinking outside the
> > human-intelligence "box" so we are blind (at present) to other
> > forms of intelligence.
> Other forms in the sense that they don't imply human
> intelligence? In that case they are irrelevant to the task of
> building human AI.

It wasn't until we studied other planets, that we finally had the
ability to truly appreciate how Earth works. Likewise, until we
study other animals, we won't truly appreciate how Human
intelligence works. In A.I. as well as in mathematics and Earth
science, doing the general is more productive than doing the
specific. Understanding the specific comes from understanding some
of the general first. We need to study the specific in more cases
than our target, then generalize, then finally finish our specific
target.

I believe we'll build (non-human) animal models first, then have
enough examples to understand the general pattern across all
animals including human, before we can finally actually get to work
on Artifical true human run-of-the-mill Intelligence.

> > A few years ago I posted an alternate idea, that birds can be
> > trained to collect litter and separate the various materials, which
> > might be faster to develop than an A.I. system.

> Huh. The uncomfortable fact is, that not only birds, just ants
> are far more intelligent than anything we were able to build so
> far. This might be a perception problem or a real one. I hope
> that studies simulating nervous system of insects and whole
> insects (brain + sensors + actuators) will shed some light.

I agree, we're nowhere close to emulating even a single ant,
although specific aspects of an ant such as locomotion are close to
practical use, such as a 6-legged rover for Luna or Mars.

> >> An intelligent system is able to maintain a model of the world in
> >> which things like relevance (as well as many other things) get
> >> defined. Unintelligent systems are bound to a method to measure
> >> relevance. An intelligent system does not need that, it already
> >> knows what is relevant, it is itself a measurement instrument.
> > That remark smells like a circular definition.
> Yes, as well as the Turing test itself, it is not a definition.

The Turing test itself is not a precise specification that could be
implemented in an automated test rig, but it *could* be implemented
by "crowdsourcing", whereby lots of random people on the Internet
are asked to judge whether they are talking with a human or a
computer program, and various humans and computer programs are thus
rated per the score they get.

There are some Web sites such as "Bing" which make pretense that
they answer user's questions, but my experience is that every last
one of them is complete crap, worse than a regular search engine
such as Google because of the pretense that builds up false hope
only to be dashed by the fact that they aren't even as good as
Google.

I have long proposed a simple FAQ-lookup-engine, which would be
much better than anything Bing has offered. It would match
user-submitted questions against the questions listed in FAQs, and
upon confirmation (by the user) of a correct match, will then tell
the answer to the user. I have developed some technology, such as
SegMat, which ought to be useful for such a question-matcher, and
have several times asked if anyone would like me to go ahead and
write the application, but nobody has offered to pay me for my
time, and nobody has even offered emotional support, such as being
willing to try whatever I build and give me feedback as to how well
it performs, so I've had better things to do with my time.

But if and when anyone contracts for me to write the FAQ-answering
application, that can be the first benchmark towards an AI
question-answering service. My first benchmark would define the Q&A
interface, and then anyone claiming to have an AI
question-answering system could ask me to link theirs with mine,
whereby if the AI system thinks it understands the question well
enough to construct an appropriate answer based on information it
finds via traditional or Watson-style searching on the Internet,
then the AI system submits the Q&A pair to my system as if it were
from an existing FAQ list, and my system then treats it as such,
and if the AI-constructed Q is one of the closest five or ten
matches to the user's question, then the user has a chance to
select it, and if selected it's then evaluated by the user for
correctness, and thus the AI program is rated as to whether it's
just bluffing or really is answering questions appropriately. Any
number of competing AI question-answering programs can thus be
included within this basic framework. If the AI programs are
"honest", they will decline to answer (in the Watson/Jeopardy
model, they will decline to "buzz in") if they don't have
confidence in any answer. With tens of AI programs all playing the
game, only a few honest programs will "buzz in" for any partiular
question. Something like TinyURL.Com/TruFut can be used to evaluate
the appropriateness and correctness of each individual AI answer,
and also to "predict" whether a particular AI system is bluffing or
actually answering. A combination of the TruFut value of a given AI
program and the nearness-of-match of Q from a regular FAQ, can be
used to rate answers as to likelihood of correctly answering the
user's question.

> It is how intelligence is perceived when not properly defined.

Agreed. We are nowhere near having a mathematically precise
definition of intelligence that can be implemented as a test rig.
Crowdsourced Turing-style testing is still our best way to
generally judge intelligence, although specific "smart" tasks can
be test-rigged already.

The sorta-vague definition of intelligence (more like genius
actually), which I gave above (ability to solve new kinds of
problems somewhat unlike any seen before), is sorta well defined,
but nowhere near mathematically precise, because we at present have
no measure of how different various problems are, hence no way to
set a threshold whereby solving problems more than a threshold away
from any previous problems would count as intelligence while
solving problems within threshold of old problems would only count
as "run of the mill" self-extending a smart system. We have an
intuitive "feeling" how to measure how novel a problem is, hence
whether intelligence is needed to solve it, but at best we can
engage in political debate whether a particular problem is
sufficiently novel to be used as an intelligence test.

In addition to the question-answering service, another obvious way
to test intelligence (or at least evolved smarts) is a test of
survival. Set up a physical or mathematical arena where a "robot"
is supposed to try to survive against either randomly-generated
environmental challenges or a "live" enemy (another AI robot).
Unlike the "Robot Wars" TV series, where the "robots" are merely
tele-operated devices, controlled in real-time by humans, but in a
similar physical arena, the robots would be totally
self-AI-controlled, with no input from humans allowed once the bout
has started. If a given robot pushes its "enemy" off the edge of
the platform first, it wins. If after ten minutes neither robot has
been pushed off-platform, then BOTH robots lose the bout.

> Note also that if incomputable, then there cannot be a definition
> without some incomputable reference system. Computability of
> intelligence and its constructive definition are close issues.

Agree, it's currently an open question whether we can define a
class of problems (mathematical, or physical) to solve, and define
a distance function (a "metric") between any two problems, such
that we can then define a measure (not an absolute yes/no decision)
of intelligence as to how wide a metric/distance gap the AI program
can bridge without further human input. Well, yeah, it's trivial to
define a class and metric that is totolly worthless for our actual
purpose, but what I mean is defining a class of problems where
human intelligence can bridge a distance somewhat proportional to
I.Q. of that person with a specified time (3 hours, i.e. one
session of a Putnam math contest, seems reasoanble). So we need to
define a class of mathematical problems that covers virtually every
Elementary Problem or Putnam problem to date, or a class of
physical problems that covers all the ordinary life tasks that we
witness various animals performing (finding food or mate or place
to sleep, etc.), and captures the difficulty of the problems as
well as the distance between any two of the problems, and
correlates the distance with how high the IQ is needed to cross
that distance in a single 3-hour contemplative+scientificExperiment
step without any outside coaching, such that the distance is highly
coreated with the IQ needed to cross that distance (i.e. most of
the variance of the IQ needed is captured by the distance between
known and new problem). Has anybody ever even begun to tackle that
problem-of-definition??

Virtually every Elementary Problem and Putnam problem is quite
strongly a Word Problem, seldom anything that can be directly
translated to an algebraic equation as with "word problems" typical
of high-school algebra classes. Whereas those simple algebraic
"word problems" might be automatically translated to algebraic
equations without too much trouble, the kinds of strong Word
Problems I'm talking about would require somewhat true AI just to
understand the question, nevermind solving it. Most of these
problems are of a general nature, so evidence of understanding the
problem could consist of nothing more than generating a few
examples of what the question is talking about. Perhaps we could
use such evidence of understanding the question as a good test of
true AI? If so, again we must define the class of problems, and a
way to test whether the AI program has generated sufficient
examples to show understanding of the problem (nevermind solving
the problem), then define a metric (distance) between different
problems, etc. as in previous paragraph.

For the physical (robot-wars combat arena) test of AI, the question
is very simple, can you push the other robot off the platform or
not, so we'll stick to the actual task of doing that, and not
bother testing whether the robot understands the task or not.

Slight variation on the robot-wars test of survival: Multi-robot
contest, where the task is to join forces with other robots to gang
up against one of the other robots and push it off the edge of the
platform. Thus social smartness is as important as brute combat
ability. The bout ends as soon as one robots goes off-edge, with
all the remaining robots scoring a "win". Of course any one robot
that's very much stronger than any other robot can ignore
the cooperation theme and just immediately push one of the other robots
off the edge to share in a win. With randomized combinations of
robots in each bout, the super-strong robot will accumulate more
wins than any other robot. But as more and more of the weaker
robots are eliminated from the contest, only the strongest robots
will remain, and none of them will be able to push any of the
others off-platform so easily, and even attempting such a push may
make the pushing robot vulnerable to a rear/side attack by a third
robot, so then ganging up will be the only way to win on a regular
basis.

Google-groups-search-key: imtrgfdi

[Walter Banks]

"Robert Maas, http://tinyurl.com/uh3t" wrote:

> > From: "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
> > So far "intelligence" resisted all decomposition attempts.
>
> Yes, the job is nowhere near done, but neurobiologists *have* made
> some progress in that direction, using NMR imaging to watch which
> parts of the brain are active when performing which kinds of
> cognitive tasks. I we can identify which cognitive tasks are
> performed by each part of the brain, then we can lump those
> same-part skills into a better research program where each group of
> researchers concentrates on trying to understand just that small
> group of skills instead of trying to do everything at one time.
> Thus both the decomposition per se and the detailed studying of
> each component looks feasible as this kind of research progresses.
>

Two things. Your tiny url is broken or it may be as intended
and I don't have enough intelligence or information to
know the difference.

I have done quite a bit of AI over the years. The most
important comment anyone has ever made to me about
AI is. We have spent so much effort parsing external
image sources (text, speech, image) and so little
effort in the extraction of information.

Some of the current AI successes have been brute force.
There is a project at the University of Waterloo on speech
response systems that basically does a lot of brute force
pattern matching in parallel and decides from degree of
matches and context the most likely meaning and responds
appropriately.

Walter..

[Curt Welch]

seeWeb...@rem.intarweb.org (Robert Maas, http://tinyurl.com/uh3t)
wrote:

> > From: "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
> > So far "intelligence" resisted all decomposition attempts.

Skinner decomposed it just fine 60 years ago.

The decomposition was easy. Implementing it is what has turned out to be
hard.

> Yes, the job is nowhere near done, but neurobiologists *have* made
> some progress in that direction, using NMR imaging to watch which
> parts of the brain are active when performing which kinds of
> cognitive tasks. I we can identify which cognitive tasks are
> performed by each part of the brain, then we can lump those
> same-part skills into a better research program where each group of
> researchers concentrates on trying to understand just that small
> group of skills instead of trying to do everything at one time.
> Thus both the decomposition per se and the detailed studying of
> each component looks feasible as this kind of research progresses.
>
> > > Most likely a set of low-level skills plus a set of skills that
> > > coordinate the low-level skills. Perhaps a heirarchy that has more
> > > than of two bottom-up levels.
> > and a huge gap somewhere between these levels, which we have not
> > been able to bridge.

Filling the gap is not hard. Recursive definitions do that very quickly.
Finding and implementing the right recursive definition to create the
implementation is what has proved hard.

> Not yet, but someday. In 1800 we did not yet understand how
> earthquakes are caused, but now we basically have it figured out
> (tektonic plates grinding against each other in various
> configurations, and magma pressing+melting+flowing upward toward
> the surface). We might likewise make similar progress in
> understanding natural intelligence over the next 200 years, and
> thus be better able to emulate it in A.I..

More like 20 years. People studying the brain get themselves lost in the
great complexity of details and are typically unable to see the overall
simplicity of the system that makes it all work.

> > The task AI1 is to build an intelligent system. The task AI2 is
> > to build a system capable to determine if AI1 is intelligent.
>
> > My bet is that the task AI2 is likely far more complex than AI1.
> > We know this from software engineering, proving correctness,
> > testing, debugging, maintenance tasks are more difficult than
> > mere solving the end problem.
>
> I side with extreme TDD (Test-Driven Development), whereby you
> write one requirement the software must satisfy, then write the
> simplest software capable of satisfying, then add another
> requirement, then modify the software to satisfy both, then add a
> third requirement, then modify the software to satisfy all three,
> etc. AI2 (both the specification and the test/validation rig) are
> easier than AI1 at each step of adding requirements. If the
> requirements include solving "trap door" problems, then it's *much*
> easier to write the specification and test rig than to design the
> algorithm to solve the problem.

The requirements were written long ago by Skinner and the other
behaviorists. No one has figured out ANY way to implement them. No other
requirements are needed. You just need to find a way to implement the ones
already written.

The requirements were written again, in a slightly different form, by
Hutter (AIXI). Again, no one has found a way to implement them yet.

Writing the requirements turns out to be the easy part of this puzzle.

You could also simulate that entire environment in software and never have
to waste any time building hardware.

Yes, AI is in fact simple. It's why the specifications written that full
explain what it is, are all so simple. It's just learning the application
of abstractions. Finding a way to implement this to solve problems of the
scale the brain can solve, is what has stumped everyone for 60 years.

Many people choose to believe that since these specification were so hard
to implement that they must be wrong. But being hard to implement, does
not make the specification wrong. These people have searched endlessly for
"easy" specification to implement. It's pointless. If you don't implement
the real specification, you don't get "real" AI.

> > >> Presumably there may exist things which cannot be understood
> > >> directly or indirectly in any way in any time. Is this what you
> > >> meant?
> > > No, just that present we aren't thinking outside the
> > > human-intelligence "box" so we are blind (at present) to other
> > > forms of intelligence.
> > Other forms in the sense that they don't imply human
> > intelligence? In that case they are irrelevant to the task of
> > building human AI.
>
> It wasn't until we studied other planets, that we finally had the
> ability to truly appreciate how Earth works. Likewise, until we
> study other animals, we won't truly appreciate how Human
> intelligence works.

Yes, that's why studying pigeons 80 years ago turned out to be so useful.

> In A.I. as well as in mathematics and Earth
> science, doing the general is more productive than doing the
> specific. Understanding the specific comes from understanding some
> of the general first. We need to study the specific in more cases
> than our target, then generalize, then finally finish our specific
> target.

Yes, yes. That was all done 60 years ago. If you want to build AI, you
have to implement a high dimension version of operant and classical
conditioning. That is the full specification of what AI is. In 60 years,
no one has been able to build a machine to match the specification.

> I believe we'll build (non-human) animal models first, then have
> enough examples to understand the general pattern across all
> animals including human, before we can finally actually get to work
> on Artifical true human run-of-the-mill Intelligence.

The animals use the same technology humans use. Both are generic learning
machines that learn by operant and classical conditioning. Rats and mouse
are just as hard to copy as humans are. No one has yet figured out how to
build intelligence equal to a mouse, because the specification for making
mouse-intelligence is the same specification for making human intelligence
and no has figured out how to code it yet.

> > > A few years ago I posted an alternate idea, that birds can be
> > > trained to collect litter and separate the various materials, which
> > > might be faster to develop than an A.I. system.
>
> > Huh. The uncomfortable fact is, that not only birds, just ants
> > are far more intelligent than anything we were able to build so
> > far. This might be a perception problem or a real one. I hope
> > that studies simulating nervous system of insects and whole
> > insects (brain + sensors + actuators) will shed some light.
>
> I agree, we're nowhere close to emulating even a single ant,

Ant's don't implement operant and classical conditioning (or so little of
it that they are not really worthy subjects). They are one of many of the
animals that have almost all their behaviors hard-coded by evolution.

We know how to hard code behavior into our machines. It's how all our
machines currently work. What we don't know how to do, is implement strong
generic learning in a high dimension sensor and behavior space. Until we
implement that strong generic learning, we will have not human or mouse
like intelligence.

Google works pretty well just because it's searching all the FAQs and
answer sites on the web. Don't really know how you think your specialized
system would be any different or better.

> But if and when anyone contracts for me to write the FAQ-answering
> application, that can be the first benchmark towards an AI
> question-answering service.

Google is already a far better benchmark. People often don't understand
how intelligent Google really is. It's core technology is a far closer
parallel to the general AI problem that most people realize.

It's often better not to get one answer, but instead, to see 10 different
answers from different people.

It's called "the field of AI". They have been trying to define what AI is
for 60 years now.

> Virtually every Elementary Problem and Putnam problem is quite
> strongly a Word Problem, seldom anything that can be directly
> translated to an algebraic equation as with "word problems" typical
> of high-school algebra classes. Whereas those simple algebraic
> "word problems" might be automatically translated to algebraic
> equations without too much trouble, the kinds of strong Word
> Problems I'm talking about would require somewhat true AI just to
> understand the question, nevermind solving it. Most of these
> problems are of a general nature, so evidence of understanding the
> problem could consist of nothing more than generating a few
> examples of what the question is talking about. Perhaps we could
> use such evidence of understanding the question as a good test of
> true AI? If so, again we must define the class of problems, and a
> way to test whether the AI program has generated sufficient
> examples to show understanding of the problem (nevermind solving
> the problem), then define a metric (distance) between different
> problems, etc. as in previous paragraph.
>
> For the physical (robot-wars combat arena) test of AI, the question
> is very simple, can you push the other robot off the platform or
> not, so we'll stick to the actual task of doing that, and not
> bother testing whether the robot understands the task or not.

Doing a specific task like that is not AI. The "task" of AI is "learning".
To test for intelligence, you must test the machine's ability to learn in a
high dimension sensory/action environment.

> Slight variation on the robot-wars test of survival: Multi-robot
> contest, where the task is to join forces with other robots to gang
> up against one of the other robots and push it off the edge of the
> platform. Thus social smartness is as important as brute combat
> ability. The bout ends as soon as one robots goes off-edge, with
> all the remaining robots scoring a "win". Of course any one robot
> that's very much stronger than any other robot can ignore
> the cooperation theme and just immediately push one of the other robots
> off the edge to share in a win. With randomized combinations of
> robots in each bout, the super-strong robot will accumulate more
> wins than any other robot. But as more and more of the weaker
> robots are eliminated from the contest, only the strongest robots
> will remain, and none of them will be able to push any of the
> others off-platform so easily, and even attempting such a push may
> make the pushing robot vulnerable to a rear/side attack by a third
> robot, so then ganging up will be the only way to win on a regular
> basis.
>
> Google-groups-search-key: imtrgfdi

[Dmitry A. Kazakov]

On Wed, 30 May 2012 00:38:45 -0700, Robert Maas, http://tinyurl.com/uh3t
wrote:

>> From: "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
>> So far "intelligence" resisted all decomposition attempts.
>
> Yes, the job is nowhere near done, but neurobiologists *have* made
> some progress in that direction, using NMR imaging to watch which
> parts of the brain are active when performing which kinds of
> cognitive tasks.

Like measuring heat spots in a CPU in order to understand how it works? As
Rutherford said once: "stamp collecting."

>> The task AI1 is to build an intelligent system. The task AI2 is
>> to build a system capable to determine if AI1 is intelligent.
>
>> My bet is that the task AI2 is likely far more complex than AI1.
>> We know this from software engineering, proving correctness,
>> testing, debugging, maintenance tasks are more difficult than
>> mere solving the end problem.
>
> I side with extreme TDD (Test-Driven Development), whereby you
> write one requirement the software must satisfy, then write the
> simplest software capable of satisfying, then add another
> requirement, then modify the software to satisfy both, then add a
> third requirement, then modify the software to satisfy all three,
> etc.

Actually TDD claims that no requirements are needed, so the "design" is
driven by chaotic test cases rather than by upfront analysis (where
requirements should come from).

As with genetic machine learning approaches there are [very strong]
premises to satisfy for this approach to work. Since nobody gives a damn to
verify if these are, otherwise they would know that these are not, this
does not qualify as decent engineering.

[...]

> In summary, there are a whole bunch of AI tasks that are easy to
> set up the test conditions and test whether the program does the
> task correctly, but not so easy to devise an algorithm to actually
> do the task under test. Thus AI2 is much easier than AI1.

The tasks you describe do not require intelligence. And the first question
you have to answer is about the probabilities of false positive and false
negative your tests have. Let me bet, they are 0.5 each?

>>>> An intelligent system is able to maintain a model of the world in
>>>> which things like relevance (as well as many other things) get
>>>> defined. Unintelligent systems are bound to a method to measure
>>>> relevance. An intelligent system does not need that, it already
>>>> knows what is relevant, it is itself a measurement instrument.
>>> That remark smells like a circular definition.
>> Yes, as well as the Turing test itself, it is not a definition.
>
> The Turing test itself is not a precise specification that could be
> implemented in an automated test rig, but it *could* be implemented
> by "crowdsourcing", whereby lots of random people on the Internet
> are asked to judge whether they are talking with a human or a
> computer program, and various humans and computer programs are thus
> rated per the score they get.

Rating of hit records would not help you in producing another one. It just
does not work this way. Would you suggest VW or Ford to design new
combustion engines based on Web-ratings? This is "stamp collecting" in its
utter form.

> I have long proposed a simple FAQ-lookup-engine, which would be
> much better than anything Bing has offered. It would match
> user-submitted questions against the questions listed in FAQs, and
> upon confirmation (by the user) of a correct match, will then tell
> the answer to the user.

User feedback is the major problem. People in which opinion you would be
interested in most would not care to respond. Jerks would.

More detailed answer you require less likely anybody would respond and more
variance in the responses you get. Already "like" vs. "not like" is both
too much and too fuzzy.

I think that feedback should be subconscious, a kind of polygrpah built-in
in the mouse, keyboard, seat, camera monitoring eyes motion etc. That could
indeed help, but I doubt anybody would allow such stuff outside China or
Iran.

> In addition to the question-answering service, another obvious way
> to test intelligence (or at least evolved smarts) is a test of
> survival.

That is the same old generic learning stuff, which simply cannot work, and
does not too. Cockroaches surpass humans in the art of suvival. The problem
of natural selection is selection. Once you know what you are looking for
in details sufficient for safe selection, you instantly find that it is
much simpler to construct that thing rather than to wait it emerging per
chance.

[...]

> For the physical (robot-wars combat arena) test of AI, the question
> is very simple, can you push the other robot off the platform or
> not, so we'll stick to the actual task of doing that, and not
> bother testing whether the robot understands the task or not.

Let them in. Then sit and wait until they become enough intelligent to
solve AI. Ask politely. Write down the answer. Head to the Norwegian Nobel
Committee... (:-))

BTW, see excellent "Callahan and the Wheelies" by Stephen Barr, published
around 1960. That makes 52 years ago!

[Robert Maas, http://tinyurl.com/uh3t]

> From: Walter Banks <wal...@bytecraft.com>

> Your tiny url is broken or it may be as intended and I don't have
> enough intelligence or information to know the difference.

I use lots of tinyURLs. Which one is giving you trouble?

> I have done quite a bit of AI over the years. The most
> important comment anyone has ever made to me about
> AI is. We have spent so much effort parsing external
> image sources (text, speech, image) and so little
> effort in the extraction of information.

I think I agree with you, but it would be helpful (to this
discussion) if you give 3-5 examples of a situation as follows:
- Overall general situation
- Specific data being observed with respect to that situation
- How far the A.I. work has gone in parsing the data
- What next step is missing, what *should* be done next, what
actual information should be gleaned from the parse-tree that
was already computed.

> Some of the current AI successes have been brute force.
> There is a project at the University of Waterloo on speech
> response systems that basically does a lot of brute force
> pattern matching in parallel and decides from degree of
> matches and context the most likely meaning and responds
> appropriately.

That sounds vaguely like the high level part of the methodology
used by the moderately-successful "Watson" system used to play
"Jeopardy" (TV show), with different low-level pattern matching
tools due to different type of data being analyzed. Based on what
I've learned about recent research in natural intelligence, such as
reported on the "Brain" series of "Charlie Rose" and various
reports on other science programs, I'm leaning toward believing
that part of natural intelligence actually works that way, with
various neurons competing for attention, with their signal
amplitudes proportional to their confidences in their respective
proposals, such that most confident answer wins the debate and is
passed on to the next level of decision making.

Google-groups-search-key: imtrgfdi

[casey]

On Jun 19, 4:08 am, seeWebInst...@rem.intarweb.org (Robert Maas,

Sounds like the pandemonium architecture? Selfridge 1958.

[Walter Banks]

"Robert Maas, http://tinyurl.com/uh3t" wrote:

> > From: Walter Banks <wal...@bytecraft.com>
> > Your tiny url is broken or it may be as intended and I don't have
> > enough intelligence or information to know the difference.
>
> I use lots of tinyURLs. Which one is giving you trouble?

http://tinyurl.com/uh3t

w..

[Robert Maas, http://tinyurl.com/uh3t]

> > > Your tiny url is broken or it may be as intended and I don't have
> > > enough intelligence or information to know the difference.
> > I use lots of tinyURLs. Which one is giving you trouble?

> From: Walter Banks <wal...@bytecraft.com>
> http://tinyurl.com/uh3t

That URL works fine for me. Here's what I see on my screen:
<<< uh3t home page
Seeking employment
WAP login form
Contact me: e-mail Twitter LinkedIn
How to remember my tiny URL
Services I do/will/might provide, for: desktop/laptop cell-phone
Good services others provide
The first line with the <<< is the WebPage title. The rest is the
rendered contents of the Web page. (This is how it appears in lynx,
the text-mode browser for VT100 terms on Unix.)

How does that differ from what you see on your screen?

It's possible your Web browser isn't following HTTP redirection.
If that's what your problem is, try these two ideas:

- Use this URL instead: http://preview.tinyurl.com/uh3t
That gives you an explicit Web page with link to actual Web site,
instead sending your browser a HTTP redirection header.

The preview page has some boilerplate, then this preview section:
Preview of TinyURL.com/uh3t
This TinyURL redirects to:
http://www.rawbw.com/~rem/WAP.html
Proceed to this site.
:then more boilerplate.

After you get that page, you click on the link to proceed to my Web site.

- Try using the direct URL: http://www.rawbw.com/~rem/WAP.html

I know tinyurl is issuing the correct redirect header to my Web
page, which is then sending valid HTML, because the The W3C Markup
Validation Service at http://validator.w3.org/ obeys the tinyurl
redirection and then tells me:
This document was successfully checked as HTML 3.2!
Result: Passed
Address: http://www.rawbw.com/~rem/WAP.html________________
Encoding: us-ascii [(detect automatically)___________]
Doctype: HTML 3.2 [(detect automatically)________________]
Root Element: HTML

I'm curious whether either of the tricks listed above solves your
problem.

Google-groups-search-key: imtrgfdi

[Walter Banks]

"Robert Maas, http://tinyurl.com/uh3t" wrote:

It works just fine now. The message I got when I posted the message was something to
the effect that it was an invalid tiny url. I am not sure what happened. I tried a
couple times as well as using another tinyurl around the same time.

Case closed

w..