Are neural networks worth the pain.
Sanny
Addition.
How will I teach neural network the Addition?
1. 5+5 -> 10
2. 99+5 -> 104
3. 15+5 -> 20
4. 15+15 -> 30
5. 52+16-> 68
6. 11+5 -> 16
How long should I continue will the Nural network learn the Addition.
How will I provide trhe input and what will be the structure of Nural
Network to learn Addition.
I want that It learn giving 90% correct value. It it possible?
Bye
Sanny
Pascal Bourguignon
Your best bet would be to teach it addition like you were taught.
First start by teaching it to count from 0 to 9
0 -> 1
1 -> 2
2 -> 3
...
8 -> 9
Then using that neural network, teach it to count from 0 up to the
infinite. Well, instead of the infinite, you may assume a short-term
memory of 9 digits, and teach it to increment a number stored in that
short term memory.
Then add a new NN and teach it the addition table, [0..9]x[0..9].
Then another NN that would be able to add two numbers with the carry.
Trying to build a single neural network to do something like an
addition (over what range of numbers?) would be silly. Neural network
work better with fuzzy data. To implement a Von Neumann computer over
neural network, it's best to do like in the brain, have several
different neural networks interconnected, like units in a processor.
--
__Pascal Bourguignon__ http://www.informatimago.com/
WARNING: This product attracts every other piece of matter in the
universe, including the products of other manufacturers, with a
force proportional to the product of the masses and inversely
proportional to the distance between them.
Wolf K.
> Sanny <soft...@hotmail.com> writes:
>
>> If I have a problem say I want to input 2 numbers and Result is say
>> Addition.
>>
>> How will I teach neural network the Addition?
>>
>> 1. 5+5 -> 10
>> 2. 99+5 -> 104
>> 3. 15+5 -> 20
>> 4. 15+15 -> 30
>> 5. 52+16-> 68
>> 6. 11+5 -> 16
>>
>> How long should I continue will the Nural network learn the Addition.
>>
>> How will I provide trhe input and what will be the structure of Nural
>> Network to learn Addition.
>>
>> I want that It learn giving 90% correct value. It it possible?
>
> Your best bet would be to teach it addition like you were taught.
And exactly how was that accomplished?
[snip hand waving]
--
wolf k.
J.A. Legris
> Pascal Bourguignon wrote:
You just reinforce successively closer approximations to the correct
answers. If it doesn't work, blame uncontrolled variables. If it does
work, suppress the urge to look surprised.
Wolf K.
Nice one. ;-)
So where do those behaviours that are "successive approximations" come
from? You know perfectly well that they have to exist before you can
train the ANN (or human child) to count. IOW, it has to do something
that approximates counting to start with. Just what behaviours in humans
are shaped when we teach a child to count? Can we start with different
behaviours in different humans and still end up with counting in all of
them? (I think you can, BTW.)
Pascal obviously thinks it's easy to train a NN. It doesn't seem to
occur to him that training a system to count requires a pretty large
repertoire of prior behaviours. In fact, an ANN can't do it. You need a
system of interconnected ANNs. And AFAICT, there hasn't been enough
work done on systems of ANNs to have even the faintest idea of how to
arrange one so that it can learn to count, let alone that it can learn
to add, subtract, and so forth.
Possibly related comment:
I just read an advert for a e-mail client that "learns" your
preferences, and will for example automatically file e-mails based on
your filing habits for similar e-mails. It's obvious I think that the
program applies spam detection algorithms to different uses. To say that
it learns your preferences is a metaphor. But it seems to me that a
system of ANNs that detects spam (etc) could be said to learn. It also
seems obvious to me that it would have to be capable of some prior
behaviours before it could learn to detect and deal with spam, but just
what those prior behaviours might be is not obvious.
Cheers! ;-)
--
wolf k.
J.A. Legris
Here's an introduction to neural networks that includes an example of
training one to add binary numbers:
http://genome.tugraz.at/MedicalInformatics2/Neural-networks.pdf
Here's an advanced article that proves neural networks can be
programmed to do many computations with only a few layers, but it does
not discuss training:
KAI-YEUNG SIU AND JEHOSHUA BRUCK. 1990. Neural Computation of
Arithmetic
Functions. PROCEEDINGS OF THE IEEE, VOL. 78, NO. IO, OCTOBER 1990
Pascal Bourguignon
Yes, that's exactly what I tried to explain.
> And AFAICT, there hasn't been enough
> work done on systems of ANNs to have even the faintest idea of how to
> arrange one so that it can learn to count, let alone that it can learn
> to add, subtract, and so forth.
You teach each ANN in turn. Essentially, you're building components
in isolation and connect them when they're working individually.
--
__Pascal Bourguignon__ http://www.informatimago.com/
PUBLIC NOTICE AS REQUIRED BY LAW: Any use of this product, in any
manner whatsoever, will increase the amount of disorder in the
universe. Although no liability is implied herein, the consumer is
warned that this process will ultimately lead to the heat death of
the universe.
Tim Tyler
> I want that It learn giving 90% correct value. It it possible?
``Solving arithmetic problems using feed-forward neural networks''
- http://citeseer.ist.psu.edu/franco98solving.html
--
__________
|im |yler http://timtyler.org/ t...@tt1lock.org Remove lock to reply.
J.A. Legris
> Sanny wrote:
> > I want that It learn giving 90% correct value. It it possible?
>
> ``Solving arithmetic problems using feed-forward neural networks''
>
This is a very nice paper, but the authors do not discuss learning
techniques in any detail - rather, they obtain exact solutions
analytically. I am wondering if the O.P really needs his N.N. to learn
addition, as opposed to just doing addition.
--
Joe
Wolf K.
Ok, I misread your level of understanding. However, your explanation is
too high level (abstract) to be helpful. It doesn't even help in
understanding how humans learn arithmetic. I've taught arithmetic to
children: I can tell you that it's not nearly as simple and
straightforward as the AI community appears to think it is, and as your
sequence of symbolic representations suggests. Based on my experience,
there is no simple psychological connection between counting and
addition, for example. "Counting" shapes "pointing" and "describing",
BTW (among other things.) "Addition" shapes "collecting stuff" (among
other things.) "Sorting" is implicated in both behaviours (and may be
the link between them.) Then there is the behaviour of writing
arithmetic. Writing an addition is not the same as doing the addition.
(This was Montessori's insight. Study Montessori. The method is not the
be-all and end-all that its devotees claim, but it provides very good
clues to the complexities of shaping apparently simple behaviours.) Etc.
>> And AFAICT, there hasn't been enough
>> work done on systems of ANNs to have even the faintest idea of how to
>> arrange one so that it can learn to count, let alone that it can learn
>> to add, subtract, and so forth.
>
> You teach each ANN in turn. Essentially, you're building components
> in isolation and connect them when they're working individually.
Yes, that's a common belief in the AI community. But that "connect them"
is mere hand waving. Also, the behaviours offered as "simple" ones are
almost all complex behaviours. Eg, "recognise A" presupposes sensors
that differentiates between different orientations of edges, and ANNs
that can be trained to respond to edges/lines of different orientations.
The outputs of these simpler ANNs can be used to train letter
recognition, which requires differentiating between combinations of
edges. Lines and edges aren't the same, either, since all edges are
lines but not all lines are edges. Etc.
And there's the assumption that the components' behaviour in isolation
will be maintained when they are connected. Why should that be? After
all, connected components communicate with each other. Components that
interact may not behave as they do when operating in isolation.
A system's behaviour is not simply a sum of individual behaviours. One
of the reasons for this is that behaviours themselves act as
discriminants (stimuli for behaviours). That implies feedback loops, and
loops of feed-back loops, etc, that quickly add up to tangles of
daunting complexity. IOW, just because each ANN has been trained to do
something doesn't mean that when they are connected the result will be a
more complex task. The architecture for serial discrimination (as in 1,
2, 3...) may be simpler than multiple, but I see no a priori reason to
suppose so.
--
wolf k.
Wolf K.
True, just doing arithmetic is easier, but I think learning how to do it
is a demonstration AI problem. Humans learn addition. It seems to me
that figuring out how to build an AI system whose behaviours can be
shaped so that it can add/subtract/etc would be very instructive.
So I think OP is onto something: ANN systems that can learn to do what
other systems can be programmed to do are a test of both ANNs and AI.
--
wolf k.
Sanny
> that figuring out how to build anAIsystem whose behaviours can be
> shaped so that it can add/subtract/etc would be very instructive.
>
> So I think OP is onto something: ANN systems that can learn to do what
> other systems can be programmed to do are a test of both ANNs andAI.
I want to know what type of Nural Network can perform Addition/
Subtraction/ Multiplication with 90% acuracy after feeding them with
1000s of data.
This will be easy to test whether the Nural network is working
correctly or not.
Still my Question is Unsolved.
Bye
Sanny
Wolf K.
The short answer is, none.
Your mistake is thinking "neural network" rather than "AI system." A
system that can learn to add/subtract/etc will consist of many ANNs,
plus assorted hard-wired modules. A few minutes thought should convince
you that the topology of such a system will be, uh, quite complicated.
Even if you cut out most of and just focus on the subsystem that learn
arithmetic, that subsystem will not be simple.
HTH
--
wolf k.
Glen M. Sizemore
"Wolf K." <wol...@sympatico.ca> wrote in message
news:47ffef9f$0$10985$9a6e...@news.newshosting.com...
> Just what behaviours in humans are shaped when we teach a child to count?
There are 2 or three separate broad repertoires that must be acquired, and
they are acquired somewhat separately. Once the conglomeration of responses
is reinforced in total, it may become a single response class. The first
thing that is established is an echoic. The child first learns to say "1, 2,
3, etc." There is nothing different about this then the way that children
acquire much of their vocabulary; they acquire a generalized operant
response class whereby they can imitate any word they hear. The second thing
that must be trained, and this occurs largely through generalized imitation,
is behavior of pointing to one and only one object of a group of objects as
one engages in the echoic. [The "echoic" need not be echoic anymore, and
eventually is not, of course. That is, "1,2,3,4..." simply becomes an
intraverbal under stimulus control of the verbal "count" and later, "how
many are there?"]. The child must also learn to remember which objects have
been pointed to, since the last things a child must do is to stop speaking
when they have pointed to the last object, and announce the last number as
the answer to the question (if the behavior was occasioned by the mand: "How
many are there?"). Pointing to each object only once is, of course,
facilitated by arranging the things to be counted in some systematic way. In
a line, of course, is the typical way, and it is the fool that tries to
teach a child to count objects laying on the floor in some random fashion.
So, in fact, in a way, the child does not have to remember which objects
were pointed to, it must simply move its hand down the line as it initiates
each component of the intraverbal. Later, adults are able to count objects
more randomly distributed in space, but this corresponds to learning the
sorts of responses that cognitive "scientists" refer to as "short-term
memory." Older humans can, but young children cannot, count objects without
touching them. Older humans acquire the behavior of visually fixating on
each object as they utter (now, often privately) each component of the
intraverbal. I like counting because it is an example of behavior that
starts out as easily observable but becomes ALMOST private. The only
observable behavior that remains is the movement of the eyes. In a way, it
is a more compelling demonstration of the public/private issue than when the
once publicly-observable behavior becomes totally unobservable by any one
else but the behaving individual.
J.A. Legris
Start with 1-digit decimal addition: sum = X + Y + carry
Input layer: 10 units for X (0 to 9), 10 units for Y (0 to 9), 1 unit
for the carry input.
Hidden layer: 20 units for the sum (0-19)
Output layer: 1 unit for the carry output and 10 units for the 1-digit
sum.
Train 1 network using backpropagation. When it works satisfactorily,
construct and chain together N-1 identical copies for N-digit
addition.
--
Joe
Wolf K.
So what you get an ALU -- in fact, as I understand it, you've just
translated an ALU into a an ANN. That's not what I think of as a device
that "learns to do arithmetic."
OTOH, what you've described is not learning, but the acquisition or
development of a brand-new behaviour (1). It seems to me that your
device may be a good model of development, in that the ANN's electronics
may be seen as its genetics, the input as the environmental stimulus,
and backpropagation as the genetic program that restructures the ANN.
That is, if you accept development as the action of genes in response to
environmental control (external input).
(1) I find it odd that so many AI workers seem to believe that
reinforcement learning (conditioning) is exemplified in blank-slate
machines. There is no blank slate. Thus a machine that "learns
arithmetic" must be one that has some pre-existing behaviours which are
shaped to produce the behaviours we call "doing arithmetic." The trick
is to figure out what pre-existing behaviours could be thus shaped. I
think Glen's exposition is a start.
Fact is that many animals can distinguish between large and small groups
of objects. This behaviour can be refined in some of them (eg, parrots).
NB that in these cases, we have a pre-existing behaviour that is shaped
by the trainer (which could of course be the habitat of the animal.) It
may be that this is a prerequisite for learning to add. I don't know.
--
wolf k.
feedbackdroid
Curt Welch
> (1) I find it odd that so many AI workers seem to believe that
> reinforcement learning (conditioning) is exemplified in blank-slate
> machines. There is no blank slate. Thus a machine that "learns
> arithmetic" must be one that has some pre-existing behaviours which are
> shaped to produce the behaviours we call "doing arithmetic."
I'm surprised you think a "blank slate" doesn't have innate behaviors.
There is no blank slate? That's a good one. :) I sure have seen a lot of
blank slates in school. So what do you think a blank slate is? A area of
empty space with nothing in it? I've never seen anyone imply that.
Last time I looked at a real blank slate, it was a physical chunk of slate
which has a collection of innate behaviors such as the default ability to
be uniformly dark, and the ability to change its light reflectivity when it
was marked with chalk and it comes with other features, like a felt eraser.
The slate, the chalk, and the felt eraser are all parts of the device we
call the "blank slate". The starting condition of the slate is "blank"
only because it's initial behavior (all dark) is said to hold no useful
information.
It's absurd to suggest that anything someone refers to as a "blank slate
learning machine" is empty space with no behavior - which is the only way
your comment above would make any sense. Why does everyone forget about
the slate?
Blank slate learning machines are machines which, just like a real blank
slate, start of in a condition with all the memory it uses to hold what it
learns being "blank", or being "all the same value". It has no a priori
knowledge about the environment it must learn in.
In the case of reinforcement learning, what the system learns is the value
of a behavior context association. What the blank slate learning machine
starts out with is a value system where all behaviors in all contexts are
evaluated as being equally valued. A program like TD-Gammon starts with
the innate understanding that all moves in the game are equally good. That
is its blank slate starting condition. It has no innate understanding of
game strategy. But as it learns, this "blank state" staring condition is
transformed into one where all the moves take on different values, just
like what happens when you start to write all over a real blank slate.
"blank slate learning machine" NEVER MEANS "no behaviors". It means "no
knowledge about the environment in which the innate behaviors are to be
used". All learning machines have innate behaviors. A robot with wheels
has the innate behavior of turning the wheels. An ANN had the innate
behavior of producing an output value as a function of its input values.
My blank slate pulse sorting network has the innate behavior of sorting
pulses. What they don't know when they start, is which behavior is better
than another in the environment they will be placed into. It means no a
priori knowledge of the problem it is to solve before it starts learning.
It's knowledge about how to use its innate behaviors is what is blank, not
it's ability to produce behavior.
Blank slate learning machines never learn new behaviors. They simply learn
how to best use their innate behaviors. We like to talk about a behavior
as a "new behavior" when in fact it never is. All that happens, is that
the probability of producing a sequence of innate behaviors changes as it
learns. A reinforcement learning machine could have innate behaviors of
hitting keys on a typewriter. When it starts, the assumption is that any
key is as good as any other key to hit. So it will produce what seems to
be random behavior. Any sequence of behaviors will happen, if you just
give it enough time. Of course, to type the works of Shakespeare would
take billions of years. But with conditioning, you can shape the
probability of the system's behavior to produce it in far less time. That's
how reinforcement learning works. It doesn't "create" new behaviors. It
simply shapes the probability of the innate behaviors that already exist in
the system. But yet, we call that "blank slate" learning because of the
fact that the probabilities, like the real blank slate, are all the same
value when it starts.
Blank slate learning in AI never means "blank behaviors". It means no a
priori knowledge about how to use it's innate behaviors.
In the case of reinforcement learning, there's yet another aspect people
get confused about. They see human behavior as having innate drives or
needs and as such, we must not be "blank". Again, in the case of
reinforcement learning, this is not part of the learning system, but
instead, it's in the critic which is logically seen as part of the
environment, and not part of the blank slate learning machine. However, in
all real world machines, like humans, the critic is implemented as hardware
in us, so it is part of the human, but not part of their "blank slate
learning hardware module".
It's very unclear just how "blank" the brain really is and how much a
priori knowledge might be genetically built in. But, the majority of our
behavior is blank slate learning by the fact that at birth, we have almost
no knowledge about how to use our innate behaviors (arm, leg, foot, hand,
finger, head, eye movements). Other than a collection of very minor (but
important) behaviors like sucking, the value of almost all our adult
behaviors must be learned from experience. Compared to the large and
complex behavior value system we have as adults, a new born human baby is
for all intent and purposes, blank.
--
Curt Welch http://CurtWelch.Com/
cu...@kcwc.com http://NewsReader.Com/
Tim Tyler
> Compared to the large and
> complex behavior value system we have as adults, a new born human baby is
> for all intent and purposes, blank.
Many emphasize the opposite perspective - e.g. see:
http://en.wikipedia.org/wiki/The_Blank_Slate
http://en.wikipedia.org/wiki/The_Language_Instinct
It was once thought that humans had relatively little
instinctual knowledge. However, that view is now
widely regarded as being incorrect.
Sure, the adult brain is more complex than the brain-
construction recipe in the genome. But it doesn't
follow that the brain-recipe is simple - it isn't.
Alpha
> Curt Welch wrote:
> > Compared to the large and
> > complex behavior value system we have as adults, a new born human baby is
> > for all intent and purposes, blank.
>
> Many emphasize the opposite perspective - e.g. see:
>
>
> It was once thought that humans had relatively little
> instinctual knowledge. However, that view is now
> widely regarded as being incorrect.
>
> Sure, the adult brain is more complex than the brain-
> construction recipe in the genome. But it doesn't
> follow that the brain-recipe is simple - it isn't.
Well, Curt did say that such was compared to adult brains, so there is
a world-o-difference.
But to your point, infant brains are definately not blank slate; huge
functional areas are already mapped out, esp. subcortical regions and
sensory regions. Plasticity enables other areas to take over from
recipe-induced organizations/area-focus.
Tim Tyler
> Many emphasize the opposite perspective - e.g. see:
>
> http://en.wikipedia.org/wiki/The_Blank_Slate
> http://en.wikipedia.org/wiki/The_Language_Instinct
>
> It was once thought that humans had relatively little
> instinctual knowledge. However, that view is now
> widely regarded as being incorrect.
``The world is too complicated to cope with if we have
to learn everything from scratch. Indeed, learning
itself cannot get off the ground without inbuilt
competences. We humans have more instincts than
other species, not fewer. As Stephen Pinker puts it,
"complexity in the mind is not caused by learning;
learning is caused by complexity in the mind" [...].''
- Susan Blackmore, TMM, p.112.
feedbackdroid
> Tim Tyler wrote:
> > Many emphasize the opposite perspective - e.g. see:
>
> >http://en.wikipedia.org/wiki/The_Blank_Slate
> >http://en.wikipedia.org/wiki/The_Language_Instinct
>
> > It was once thought that humans had relatively little
> > instinctual knowledge. However, that view is now
> > widely regarded as being incorrect.
>
> ``The world is too complicated to cope with if we have
> to learn everything from scratch. Indeed, learning
> itself cannot get off the ground without inbuilt
> competences. We humans have more instincts than
> other species, not fewer. As Stephen Pinker puts it,
> "complexity in the mind is not caused by learning;
> learning is caused by complexity in the mind" [...].''
>
> - Susan Blackmore, TMM, p.112.
> --
This is not an uncommon observation among neuroscientists, besides
just the evo-pysch people, even though Susan Blackmore is a science
"writer" rather than a scientist per se.
Also, if one considers that human brains and rat brains have similar
architecture, including significant and similar modularization of the
cortex, and since the human brain is so much larger that just one
cortical area is about as large as the entire rat brain, then one
shan't be surprised in regards finding a lot of additional functional
areas.
I am surprised that Curt still keeps talking about blank slates.
Learning doesn't work in a vacuum, it works atop a very highly-evolved
underlying infrastructure.
Curt Welch
> Curt Welch wrote:
>
> > Compared to the large and
> > complex behavior value system we have as adults, a new born human baby
> > is for all intent and purposes, blank.
>
> Many emphasize the opposite perspective - e.g. see:
>
> http://en.wikipedia.org/wiki/The_Blank_Slate
I've had that book on my stack for a few years but haven't gotten around to
reading it.
> http://en.wikipedia.org/wiki/The_Language_Instinct
> It was once thought that humans had relatively little
> instinctual knowledge. However, that view is now
> widely regarded as being incorrect.
All the arguments I've seen make the argument that since we have an innate
ability to learn, we must not be blank slates. A real blank slate has the
innate ability to hold information but yet we still call it blank. the
question isn't about our innate ability to learn, the question is about
what we know at birth what we don't know. We don't know English at birth.
WE don't know _any_ language at birth. We instead, have an innate ability
to _learn_ language.
Why is it so hard for people to grasp this distinction?
The whole concept of learning seems to be too hard for most people, even
very smart and well educated people, to correctly grasp. Blank slates have
an innate ability to learn. The fact that they an learn is not proof
against the blank slate view.
> Sure, the adult brain is more complex than the brain-
> construction recipe in the genome. But it doesn't
> follow that the brain-recipe is simple - it isn't.
Sure, first of all, a large chunk of the brain by volume is not a learning
system at all. So when I talk about AI being a blank slate problem, I'm
never talking about the entire brain and all the complex functions it
performs. I'm only talking about the part which allows us to learn complex
environments in response to our environment. The rest is just hard-wired
stuff evolution gave us to keep us alive and of no particular significance
to the hard problem of AI. That stuff us just standard engineering which we
already know how to do and which we will our machines with already.
In addition, all blank slate learning systems have many tunable learning
parameters which allow it to be optimized to the class of problem it can
solve. They have limits in their resolution, and speed, and amount of
information they can learn. The brain has been highly tuned in regards to
how much of this basic blank slate learning technology has been allocated
to different sensory and behavior modalities (how much is allocated to the
visual cortex and how much is allocated to language learning etc). All
this tuning of the hardware and the size of the pathways between sections
is specified by genetics which gives us an innate profile of what we can
learn and what we can't learn.
But, innate learning features, does not stop it from being a blank slate
learning system.
The blank slate idea that Pinker seems to argue against is the one from
social science, not the one in computer science. The one in social science
seems to ignore the fact that the power to learn must be innate, and that
all powers to learn can not be equal. When you fail to realize that, you
have taken the blank slate idea too far. And that is what Pinker seems to
argue against.
But in computer science, the innate features of the learning system are
part of the "slate" and not part of the "blank".
And for human intelligence, I consider what we know, and can do, at birth,
because it is built into us an innate feature, is our part of the human
"slate", and everything we learn after birth, is what is written on the
"blank" part.
Now, I do tend to argue for a stronger and more generic learning than most
do, but I don't believe the brain is a totally homogeneous learning system
by any stretch of the imagination.
I do believe however that the big missing piece of the hard problem of AI
is strong generic blank slate learning. We won't solve AI until we first
create stronger generic blank slate learning systems, but once we do create
them, we will paste them together in all sorts of interesting ways to give
our AIs just the right levels of learning skills, and just the right
combinations of motivations, for whatever job we want the machine to
perform for us.
Curt Welch
> Tim Tyler wrote:
>
> > Many emphasize the opposite perspective - e.g. see:
> >
> > http://en.wikipedia.org/wiki/The_Blank_Slate
> > http://en.wikipedia.org/wiki/The_Language_Instinct
> >
> > It was once thought that humans had relatively little
> > instinctual knowledge. However, that view is now
> > widely regarded as being incorrect.
>
> ``The world is too complicated to cope with if we have
> to learn everything from scratch. Indeed, learning
> itself cannot get off the ground without inbuilt
> competences. We humans have more instincts than
> other species, not fewer. As Stephen Pinker puts it,
> "complexity in the mind is not caused by learning;
> learning is caused by complexity in the mind" [...].''
>
> - Susan Blackmore, TMM, p.112.
That's exactly the type of thinking I argue against.
It comes from the fact that we don't have the strong learning systems
working yet and because we don't have them, people like Susan think it's
not possible to build them. It's not an illogical argument, but it's one I
just don't put any faith in. I believe the world is not too complicated to
cope with when learning everything from scratch. People like Susan (and
maybe Pinker) just don't understand what's possible. Learning is not as
hard as they think it is - it's only hard to figure out how to make it
easier. :)
As Dan has posted recently, it took Hinton 17 years to find a better
network learning algorithm, but now he's got one which learns faster, and
scales better than past ones. He's created something that many people
would argue was impossible to do just like Susan is trying to argue that
it's impossible to learn what we learn from scratch. Creating stronger
learning algorithms is no simple task. But there's a lot of evidence to
suggest they exist, with the biggest piece of evidence being how much a
human changes between birth and death and in how much the environment
effects how we change.
They are underestimating the power of blank slate learning in my view.
Tim Tyler
> The question isn't about our innate ability to learn, the question is about
> what we know at birth what we don't know. We don't know English at birth.
> WE don't know _any_ language at birth. We instead, have an innate ability
> to _learn_ language.
>
> Why is it so hard for people to grasp this distinction?
The argument is about language elements which are apparently
not learned - but are built in:
``Much of the book refers to Chomsky's concept of a
universal grammar, a meta-grammar into which all human
languages fit. Pinker explains that a universal grammar
represents specific structures in the human brain that
recognize the general rules of other humans' speech, such
as whether the local language places adjectives before or
after nouns, and begin a specialized and very rapid
learning process not explainable as reasoning from first
principles or pure logic.''
- http://en.wikipedia.org/wiki/The_Language_Instinct
Curt Welch
> On Apr 19, 10:24=A0am, Tim Tyler <seemy...@googlemail.com> wrote:
> I am surprised that Curt still keeps talking about blank slates.
> Learning doesn't work in a vacuum, it works atop a very highly-evolved
> underlying infrastructure.
Would you call Hinton's new network an example of "blank slate" learning?
I would, because when it's put to work learning to recognize images of hand
written digits it's not given any a priori knowledge of what it's looking
for. It's simply feed raw images and the network on its own, learns to
classify the images based on the statistical properties which exist in the
images. With no outside "teacher" helping it, it is still able to
correctly classify the images into sets which match our understanding of
the images (as digits). All the work is done by extracting, and using, the
statistical information in the data.
But yet, his network learning system has "highly-evolved" (took 17 years
from the last step in the evolution :)) underlying structure. But the
evolved structure there is to make the learning better, not make it "less
blank".
Same thing with TD-Gammon. When he wrote that program, he _removed_ all
the strategy and innate game playing knowledge he had put into his previous
programs and instead, built a blank slate learning system which knew
nothing about how to play the game. And this machine, the one with the
least amount of innate game playing knowledge, but the strongest learning
ability, was the one that played the game the best of all his programs.
The underlying structure we have to build into the machine, is strong blank
slate learning, and nothing else. The more generic the learning becomes,
the closer it gets to true blank slate learning, the more adaptive it is
(it can learn anything) and the more intelligent it becomes.
Curt Welch
Well again, I think they are talking about powers of the underlying blank
slate learning systems without understanding that is what they are talking
about.
This debate won't be resolved until we develop machines that can duplicate
human language skills. If that machines turns out to be a generic strong
learning system, the same generic strong learning system that allows us to
learn to walk and do all the other intelligent things we learn to do, my
point will be made. If on the other hand, we have to use a very different
type of learning system to learn to walk, then the one we use to learn to
talk, their point will be made. Until one of us has that working machine,
neither of us will be able to prove our point.
Tim Tyler
> Tim Tyler <seem...@googlemail.com> wrote:
>> ``The world is too complicated to cope with if we have
>> to learn everything from scratch. Indeed, learning
>> itself cannot get off the ground without inbuilt
>> competences. We humans have more instincts than
>> other species, not fewer. As Stephen Pinker puts it,
>> "complexity in the mind is not caused by learning;
>> learning is caused by complexity in the mind" [...].''
>>
>> - Susan Blackmore, TMM, p.112.
>
> That's exactly the type of thinking I argue against.
>
> It comes from the fact that we don't have the strong learning systems
> working yet and because we don't have them, people like Susan think it's
> not possible to build them. [...]
Those folk are mostly talking about how the human brain
actually works.
It /may/ be possible to get a simpler AI system off the ground -
if the system was prepared to put in more learning time than
human babies have available to them - and endure the cost of
making more mistakes during their infancy.
...but our proof-of-concept learning machine is not
itself particularly simple - and /very/ simple learning
machines do not seem to get very far - or AI would be
done by now. So, some of that complexity is needed
in order to "take off" - and the issue is: how much.
Tim Tyler
> Tim Tyler <seem...@googlemail.com> wrote:
>> The argument is about language elements which are apparently
>> not learned - but are built in:
>>
>> ``Much of the book refers to Chomsky's concept of a
>> universal grammar, a meta-grammar into which all human
>> languages fit. Pinker explains that a universal grammar
>> represents specific structures in the human brain that
>> recognize the general rules of other humans' speech, such
>> as whether the local language places adjectives before or
>> after nouns, and begin a specialized and very rapid
>> learning process not explainable as reasoning from first
>> principles or pure logic.''
>>
>> - http://en.wikipedia.org/wiki/The_Language_Instinct
>
> Well again, I think they are talking about powers of the underlying blank
> slate learning systems without understanding that is what they are talking
> about.
They are talking about powers of the underlying learning system.
They are claiming it already knows a whole bunch of stuff at
birth - and so is hardly well-described as being "blank".
> This debate won't be resolved until we develop machines that can duplicate
> human language skills. If that machines turns out to be a generic strong
> learning system, the same generic strong learning system that allows us to
> learn to walk and do all the other intelligent things we learn to do, my
> point will be made. If on the other hand, we have to use a very different
> type of learning system to learn to walk, then the one we use to learn to
> talk, their point will be made. Until one of us has that working machine,
> neither of us will be able to prove our point.
Trying to duplicate language learning in a machine
is not the only way to investigate these issues
scientifically - you can also investigate how
quickly languages are learned, look at of "creole
languages" - and see what impact deviations from
the proposed "universal grammar" have:
http://en.wikipedia.org/wiki/Universal_grammar
feedbackdroid
> feedbackdroid <feedbackdr...@yahoo.com> wrote:
> > On Apr 19, 10:24=A0am, Tim Tyler <seemy...@googlemail.com> wrote:
> > I am surprised that Curt still keeps talking about blank slates.
> > Learning doesn't work in a vacuum, it works atop a very highly-evolved
> > underlying infrastructure.
>
> Would you call Hinton's new network an example of "blank slate" learning?
That's kind of irrelevant, since Hintons NN is not the brain.
But you do also notice his net does have a specific structure, plus
[the key ingredient] top-down arrangement, that was likely chosen by
him. In contrast, the classical BP net of old was very unstructured.
Just 3 layers, fully-connected. That was more the blank slate
architecture, and also one shown to hit-the-wall, in terms of poor
scalability to non-toy problems.
> I would, because when it's put to work learning to recognize images of hand
> written digits it's not given any a priori knowledge of what it's looking
> for. It's simply feed raw images and the network on its own, learns to
> classify the images based on the statistical properties which exist in the
> images. With no outside "teacher" helping it, it is still able to
> correctly classify the images into sets which match our understanding of
> the images (as digits). All the work is done by extracting, and using, the
> statistical information in the data.
>
> But yet, his network learning system has "highly-evolved" (took 17 years
> from the last step in the evolution :)) underlying structure. But the
> evolved structure there is to make the learning better, not make it "less
> blank".
>
I suspect he didn;t evolve the structure, but took 17 years to
discover what worked.
>
> Same thing with TD-Gammon. When he wrote that program, he _removed_ all
> the strategy and innate game playing knowledge he had put into his previous
> programs and instead, built a blank slate learning system which knew
> nothing about how to play the game. And this machine, the one with the
> least amount of innate game playing knowledge, but the strongest learning
> ability, was the one that played the game the best of all his programs.
>
> The underlying structure we have to build into the machine, is strong blank
> slate learning, and nothing else. The more generic the learning becomes,
> the closer it gets to true blank slate learning, the more adaptive it is
> (it can learn anything) and the more intelligent it becomes.
>
As mentioned, the old BP nets were more the blank slate, homogenous
architecture, and also severely limited in what htey could do.
> --
> Curt Welch http://CurtWelch.Com/
feedbackdroid
It might have been better for SB to have stated that, if every newly-
born organism in the wild had to learn every thing from a blank slate,
beta-0 scratch, they'd likely not survive their first hour. In fact,
that's what likely happened to any organisms that had evolved along
that path.
>
> As Dan has posted recently, it took Hinton 17 years to find a better
> network learning algorithm, but now he's got one which learns faster, and
> scales better than past ones. He's created something that many people
> would argue was impossible to do just like Susan is trying to argue that
> it's impossible to learn what we learn from scratch. Creating stronger
> learning algorithms is no simple task. But there's a lot of evidence to
> suggest they exist, with the biggest piece of evidence being how much a
> human changes between birth and death and in how much the environment
> effects how we change.
>
> They are underestimating the power of blank slate learning in my view.
>
I think you'll not properly acknowledging the underlying structure
that "is" in Hinton's net. That which makes successful learning
possible, as opposed to being just another old-type loose-architecture
BP net.
> --
> Curt Welch http://CurtWelch.Com/
> c...@kcwc.com http://NewsReader.Com/- Hide quoted text -
>
> - Show quoted text -
feedbackdroid
> Tim Tyler <seemy...@googlemail.com> wrote:
> > Curt Welch wrote:
>
> > > Compared to the large and
> > > complex behavior value system we have as adults, a new born human baby
> > > is for all intent and purposes, blank.
>
> > Many emphasize the opposite perspective - e.g. see:
>
> >http://en.wikipedia.org/wiki/The_Blank_Slate
>
> I've had that book on my stack for a few years but haven't gotten around to
> reading it.
>
> >http://en.wikipedia.org/wiki/The_Language_Instinct
> > It was once thought that humans had relatively little
> > instinctual knowledge. However, that view is now
> > widely regarded as being incorrect.
>
> All the arguments I've seen make the argument that since we have an innate
> ability to learn, we must not be blank slates. A real blank slate has the
> innate ability to hold information but yet we still call it blank. the
> question isn't about our innate ability to learn, the question is about
> what we know at birth what we don't know. We don't know English at birth.
> WE don't know _any_ language at birth. We instead, have an innate ability
> to _learn_ language.
>
This is exactly it. Genetics gives us the underlying infrastructure to
learn and use any language, rather than just building in a specific
language. We don't use the language/speech centers of the brain for
vision/etc.
Likewise, bipedal walking has to be "learned" by each newborn, but all
the underlying and necessary structure is already bequeathed by
genetics ... legs, muscles, joints, feet, motor control centers,
cerebellum, motor cortex, proprioceptive feedback from the joints,
balance organs in the inner ear, plus "built-in" crawling reflexes, on
and on.
By putting these all together ... along with months and months of
practice ... we eventualy learn to walk upright. Eliminate almost ANY
part of the instrastructure, eg the balance organs or the pressure
sensors on the bottoms of the feet, and we'll likely never learn to
walk upright.
feedbackdroid
> Tim Tyler <seemy...@googlemail.com> wrote:
> > Curt Welch wrote:
>
> > > The question isn't about our innate ability to learn, the question is
> > > about what we know at birth what we don't know. We don't know English
> > > at birth. WE don't know _any_ language at birth. We instead, have an
> > > innate ability to _learn_ language.
>
> > > Why is it so hard for people to grasp this distinction?
>
> > The argument is about language elements which are apparently
> > not learned - but are built in:
>
> > ``Much of the book refers to Chomsky's concept of a
> > universal grammar, a meta-grammar into which all human
> > languages fit. Pinker explains that a universal grammar
> > represents specific structures in the human brain that
> > recognize the general rules of other humans' speech, such
> > as whether the local language places adjectives before or
> > after nouns, and begin a specialized and very rapid
> > learning process not explainable as reasoning from first
> > principles or pure logic.''
>
>
> Well again, I think they are talking about powers of the underlying blank
> slate learning systems without understanding that is what they are talking
> about.
>
> This debate won't be resolved until we develop machines that can duplicate
> human language skills. If that machines turns out to be a generic strong
> learning system, the same generic strong learning system that allows us to
> learn to walk and do all the other intelligent things we learn to do, my
> point will be made. If on the other hand, we have to use a very different
> type of learning system to learn to walk, then the one we use to learn to
> talk, their point will be made. Until one of us has that working machine,
> neither of us will be able to prove our point.
>
The "blank slate" idea was developed back when they thought the cortex
had thin peripheral cortical areas involved in sensory input and motor
control, but where about 95% or the volume was labelled as
"association cortex". Nowadays we know that 40% or so of the cortex is
involved just in vision alone, and there are 30+ areas with more or
less specific and different identifiable function.
> --
> Curt Welch http://CurtWelch.Com/
Traveler
I think this debate can be resolved now and it shows that the
Chomskyan concept of a special language organ (i.e., Broca's area) is
hogwash. It is true that Broca's area is involved in language
production/learning but it is also involved in such things as humming
or whistling a tune. This is only because Broca's area sends and
receives signals directly to and from the moto-cortical areas that
control the mouth and throat muscles. Deaf people learn to use sign
language, not with their Broca's areas but with other parts of the
cortex that control arm and hand muscles. It follows that the
"specialness" that Chomsky's followers love to talk about is a myth.
Language acquisition does not involve a special pre-programmed
sensori-motor organ any more than grasping, swimming or walking. IOW,
there is nothing inherently different between learning how to walk and
learning how to talk. There is no specialness and there has never been
any.
So yes, as far as learning language skills are concerned, it is all
blank slate learning. Chomsky is out to lunch. Has been for some time.
Louis Savain
Rebel Science News:
http://rebelscience.blogspot.com/
Wolf K.
[...]
> As Dan has posted recently, it took Hinton 17 years to find a better
> network learning algorithm, but now he's got one which learns faster, and
> scales better than past ones. He's created something that many people
> would argue was impossible to do just like Susan is trying to argue that
> it's impossible to learn what we learn from scratch. Creating stronger
> learning algorithms is no simple task. But there's a lot of evidence to
> suggest they exist, with the biggest piece of evidence being how much a
> human changes between birth and death and in how much the environment
> effects how we change.
>
> They are underestimating the power of blank slate learning in my view.
>
Any system that has a learning algorithm built in is not a blank slate.
Obviously.
--
wolf k.
Curt Welch
> On Apr 19, 11:46=A0am, c...@kcwc.com (Curt Welch) wrote:
> > But yet, his network learning system has "highly-evolved" (took 17
> > years from the last step in the evolution :)) underlying structure.
> > =A0But the evolved structure there is to make the learning better, not
> > make it "less blank".
> >
>
> I suspect he didn;t evolve the structure, but took 17 years to
> discover what worked.
Yeah, that's what I meant by "evolved". He no doubt tried many different
variations, keeping features that seemed to be useful and throwing out
features that seemed useless until he came on a combination of features
that worked better. To me, that's evolution. :) I didn't mean he did
something like use A GA approach to create it.
--
Curt Welch http://CurtWelch.Com/
Curt Welch
> On Apr 19, 11:30=A0am, c...@kcwc.com (Curt Welch) wrote:
> I think you'll not properly acknowledging the underlying structure
> that "is" in Hinton's net. That which makes successful learning
> possible, as opposed to being just another old-type loose-architecture
> BP net.
What underlying structure are you making reference to? As far as I can
tell from the video, it's just a fully connected multilayer feed forward
network.
--
Curt Welch http://CurtWelch.Com/
Curt Welch
Well, in that case, you are just using the term "blank slate" very
differently from the way other people in AI use it which would explain why
you see it odd that they use it.
Wolf K.
Well, this a forum on the philosophy of AI, so I can say whatever I
want, eh? Within reason, that is. As I perceive reason, anyhow. ;-)
I'm using "blank slate" as I understand Locke (the originator of the
term and concept, about 200 years ago) to have used it. He claimed that
humans were "blank slates on which experience wrote its information", or
words to that effect. It never occurred to him that "experience" was
inside the human, not outside: that the experience of being a bat would
necessarily be different from that of being a human, and so the bat and
the human would learn different things. (I'm borrowing and drastically
summarising someone's refutation of Locke, who was one of those people
who though that having a powerful intellect was sufficient guarantee of
being right. He had no sens of humour, ie, he was unable to criticise
himself. I apparently have a deficient sense of humor, as some of my
friends have been kind enough to point out to me, but I do hope I can
criticise my own ideas. Sometimes. ;-))
Hinton's algorithm (HA) does not learn from scratch. Its learning is
utterly dependent on its inputs. Also, it can learn only those patterns
(for want of a better word) that its architecture permits. I'm quite
willing to accept that HA's architecture allows it to learn a much wider
range of patterns than its predecessors, but that merely means it's less
of a blank slate than its predecessors, not more. What's more, those
inputs must be presented in a way/mode that HA can process. So there
must be devices that translate real-world inputs into those modes, or HA
can't learn from them. And so on. In short, HA is and must be part of a
system that has some built-in capabilities in order for HA to do its
thing. It should be perfectly clear that a learning system must have
built in capabilities of many kinds, just as Susan Blackmore points out.
My stance does not of course diminish Hinton's achievement, but it does,
I hope, place it in proper perspective. Specifically, I think it
clarifies that HA or something like it must exist in any learning
system. It's necessary, but not sufficient. So more power to Hinton.
Just don't think that scaling it up will somehow produce a learning system.
My fundamental axiom of learning, if you wish to call it that, is: A
system will learn only that which it is able to learn. IOW, there is no
universal learning device or machine or whatever. There are only
devices/machines/systems that will learn specific things. More or less
easily and well. And depending on its learning history. Etc. All that
should be obvious.
HTH
--
wolf k.
forbi...@msn.com
> On 19 Apr 2008 17:52:10 GMT, c...@kcwc.com (Curt Welch) wrote:
>
>
>
>
>
> >Tim Tyler <seemy...@googlemail.com> wrote:
> >> Curt Welch wrote:
>
> >> > The question isn't about our innate ability to learn, the question is
> >> > about what we know at birth what we don't know. We don't know English
> >> > at birth. WE don't know _any_ language at birth. We instead, have an
> >> > innate ability to _learn_ language.
>
> >> > Why is it so hard for people to grasp this distinction?
>
> >> The argument is about language elements which are apparently
> >> not learned - but are built in:
>
> >> ``Much of the book refers to Chomsky's concept of a
> >> universal grammar, a meta-grammar into which all human
> >> languages fit. Pinker explains that a universal grammar
> >> represents specific structures in the human brain that
> >> recognize the general rules of other humans' speech, such
> >> as whether the local language places adjectives before or
> >> after nouns, and begin a specialized and very rapid
> >> learning process not explainable as reasoning from first
> >> principles or pure logic.''
>
>
> >Well again, I think they are talking about powers of the underlying blank
> >slate learning systems without understanding that is what they are talking
> >about.
>
> >This debate won't be resolved until we develop machines that can duplicate
> >human language skills.
>
> I think this debate can be resolved now and it shows that the
> Chomskyan concept of a special language organ (i.e., Broca's area) is
> hogwash. It is true that Broca's area is involved in language
> production/learning but it is also involved in such things as humming
> or whistling a tune. This is only because Broca's area sends and
> receives signals directly to and from the moto-cortical areas that
> control the mouth and throat muscles. Deaf people learn to use sign
> language, not with their Broca's areas but with other parts of the
> cortex that control arm and hand muscles. It follows that the
> "specialness" that Chomsky's followers love to talk about is a myth.
> Language acquisition does not involve a special pre-programmed
> sensori-motor organ any more than grasping, swimming or walking. IOW,
> there is nothing inherently different between learning how to walk and
> learning how to talk. There is no specialness and there has never been
> any.
>
> So yes, as far as learning language skills are concerned, it is all
> blank slate learning. Chomsky is out to lunch. Has been for some time.
Maybe I'm misunderstanding or you're overstating. I can't tell which.
Many animals are up and walking within hours of birth and yet they
never learn to talk. If the brain is a blank slate then why is this
so?
Traveler
This is rather telling, don't you think? It follows from this
observation that the brain of an animal is much less of a tabula rasa
than the brain of a human being. One can also conclude that the animal
brain is not only pre-wired to a large extent but also that it has a
much lower capacity for general learning. I have excellent reasons to
believe that the general learning capacity of the human brain is
biologically impossible and that something else is at play besides
neurons and synapses. This is especially evident with regard to the
prodigious memory of certain autistic savants. This is not to say that
animals cannot learn but that their knowledge space is severely
constrained within domains dictated by their genetic programming.
feedbackdroid
The other alternative [and most likely one, it seems to me] is, since
the human brain has so many more neurons than other brains, that it
has BOTH much more specific structure, in terms of areas evolved for
dealing with specific problems [as the evo-psych guys say], and also
has much more generic learning space, for dealing with general
unstructured problems. Just going from chimps to humans increases
#neurons by about 10X or so.
Traveler
I seriously doubt that brain size has much to do with it since
historical and medical accounts of humans who are born with brains
much smaller than those of dogs and chimps show that the humans have
no trouble learning language. Of course, if one insists on seeing
everything through Darwinian glasses, one runs the risk of missing
important aspects of nature.
The general learning capacity of humans has nothing to do with
something as nebulous as "general learning space". There is really not
much anatomical difference between a human brain and a simian brain or
even a dog's brain. Many mammalian brains have more than enough neural
processing power to play chess, learn a sophisticated language and
send rockets to the moon. They can't do it because they lack the
ability to make real time temporal associations between events in
distant areas of the brain, except where the connections are
genetically pre-wired. Apparently the human brain can make these
associations instantly without a physical connection. Said
associations are eventually consolidated (via axonic and synaptic
growth) during sleep. If not, they are forgotten although some
austitic savants retain their memories forever.
feedbackdroid
I already said this. The major "outward" difference is in #neurons,
where human neuron count is 10X to 100X greater than your examples.
That alone means you can have many more specific cortical areas having
evolved.
And this isn't even to mention the possibility that the 10X larger
system has added a lot of additional complexity in terms of cross-
connections between regions.
>
> Many mammalian brains have more than enough neural
> processing power to play chess, learn a sophisticated language and
> send rockets to the moon. They can't do it because they lack the
> ability to make real time temporal associations between events in
> distant areas of the brain,
>
Well, I agree wholehaeartedly with this. Eg, see books and papers
by ...
http://www.google.com/custom?q=phil+lieberman
He [like you] loves the BG connections.
>
> except where the connections are
> genetically pre-wired. Apparently the human brain can make these
> associations instantly without a physical connection. Said
> associations are eventually consolidated (via axonic and synaptic
> growth) during sleep. If not, they are forgotten although some
> austitic savants retain their memories forever.
>
> Louis Savain
>
> Rebel Science News:http://rebelscience.blogspot.com/- Hide quoted text -
forbi...@msn.com
That's my take on it too. Generality comes from the variability
of the habitat enhabited and hardware solutions are in place where
generic learning isn't needed. Hardware solutions are usually,
if not always, more efficient but at the cost of robustness.
Vend
<snip>
> The underlying structure we have to build into the machine, is strong blank
> slate learning, and nothing else. The more generic the learning becomes,
> the closer it gets to true blank slate learning, the more adaptive it is
> (it can learn anything) and the more intelligent it becomes.
A system that can learn anything, that is, it performs well on any
arbitrary learning problem, is a mathematical impossibility.
The question is whether it's possible to create a "generic and
simple" (made without using problem-specific information) learning
system that performs well on any "real-life" problem (those you might
want an intelligent system to practically solve).
In order for this to be possible, it must be the case that all "real-
life" learning problems share a common sub-structure that can be
exploited by a learning algorithm.
As far as I know, no such sub-structure is known. 50 years of research
in AI in general, and learning algorithms in particular failed to find
it, and neurological evidence suggests that the human brain doesn't
work that way, but rather it has a large number of modules, each one
specialized in a particular class of problems.
Hence I'm not particularly confident that a generic learning system
can exist, unless it's made by combining a large number of problem-
specific learning systems, like the human brain.
Tim Tyler
> On 19 Apr, 19:46, c...@kcwc.com (Curt Welch) wrote:
>> The underlying structure we have to build into the machine, is strong blank
>> slate learning, and nothing else. The more generic the learning becomes,
>> the closer it gets to true blank slate learning, the more adaptive it is
>> (it can learn anything) and the more intelligent it becomes.
>
> A system that can learn anything, that is, it performs well on any
> arbitrary learning problem, is a mathematical impossibility.
Hmm. The meaning of "arbitrary learning problem" would
seem to depend on how you choose to enumerate learning
problems - and so that statement is not well defined :-|
> The question is whether it's possible to create a "generic and
> simple" (made without using problem-specific information) learning
> system that performs well on any "real-life" problem (those you might
> want an intelligent system to practically solve).
>
> In order for this to be possible, it must be the case that all "real-
> life" learning problems share a common sub-structure that can be
> exploited by a learning algorithm.
>
> As far as I know, no such sub-structure is known. [...]
What, and so there is No Free Lunch? IMO, that's not
correct. The class of survival problems which organisms'
brains face is far from arbitrary. There are all kinds
of regularity in the problem space - as a result of the
physics of how actuators and sensors function, the
regularities of developmental programs, the uniformity
of nature - and so on.
Check the performance of random search in engineering
problems. It is pretty poor. So, the problems humans
face are rather regular in structure. They have not
been chosen at random - but rather cooked up by nature.
If there were no such regularities, evolution would not
be effective at finding solutions.
So: free Lunch issues are not very relevant in practice.
Curt Welch
> On 19 Apr, 19:46, c...@kcwc.com (Curt Welch) wrote:
> <snip>
> > The underlying structure we have to build into the machine, is strong
> > blank slate learning, and nothing else. The more generic the learning
> > becomes, the closer it gets to true blank slate learning, the more
> > adaptive it is (it can learn anything) and the more intelligent it
> > becomes.
>
> A system that can learn anything, that is, it performs well on any
> arbitrary learning problem, is a mathematical impossibility.
Many things that seem impossible aren't because the assumptions made in the
analysis was wrong.
> The question is whether it's possible to create a "generic and
> simple" (made without using problem-specific information) learning
> system that performs well on any "real-life" problem (those you might
> want an intelligent system to practically solve).
Well, it doesn't have to solve _any_ problem, just the ones we want it to
solve, like learning to walk and talk. :)
> In order for this to be possible, it must be the case that all "real-
> life" learning problems share a common sub-structure that can be
> exploited by a learning algorithm.
That's true.
> As far as I know, no such sub-structure is known.
I think I know what it is. Mostly.
> 50 years of research
> in AI in general, and learning algorithms in particular failed to find
> it, and neurological evidence suggests that the human brain doesn't
> work that way, but rather it has a large number of modules, each one
> specialized in a particular class of problems.
>
> Hence I'm not particularly confident that a generic learning system
> can exist, unless it's made by combining a large number of problem-
> specific learning systems, like the human brain.
I really do have to finish this work and show a few people what they failed
to understand. :)
Traveler
wrote:
>In order for this to be possible, it must be the case that all "real-
>life" learning problems share a common sub-structure that can be
>exploited by a learning algorithm.
A common structure must be based on a general principle. In turn, a
general principle must be deduced from a fundamental charateristic
that is common to all types of sensory information. It turns out that
the only universal characteristic of sensory signals is their
temporality. From this we can infer a general principle: sensory
signals can be either concurrent or sequential. A common structure
must therefore use the temporality principle as the basis of
everything such as learning, reacting, memorizing, adapting, etc...
In conclusion, let me say that intelligence is inherently and
ultimately temporal in nature. I've been saying this for a long time.
AI scientists wasted half a century chasing the symbol manipulation
red herring. It's good to see that a few searchers (e.g., Hawkins) in
the AI field are beginning to take temporality seriously. It is the
future of AI.
Tim Tyler
> In conclusion, let me say that intelligence is inherently and
> ultimately temporal in nature. I've been saying this for a long time.
> AI scientists wasted half a century chasing the symbol manipulation
> red herring. It's good to see that a few searchers (e.g., Hawkins) in
> the AI field are beginning to take temporality seriously. It is the
> future of AI.
In practice, all AI problems have temporal constraints,
symbol manipulation or no. Without temporal constraints
it is not possible to sensibly allocate temporal resources
to a problem - and problems tend to never get solved.
Vend
> Vend wrote:
> > On 19 Apr, 19:46, c...@kcwc.com (Curt Welch) wrote:
> >> The underlying structure we have to build into the machine, is strong blank
> >> slate learning, and nothing else. The more generic the learning becomes,
> >> the closer it gets to true blank slate learning, the more adaptive it is
> >> (it can learn anything) and the more intelligent it becomes.
>
> > A system that can learn anything, that is, it performs well on any
> > arbitrary learning problem, is a mathematical impossibility.
>
> Hmm. The meaning of "arbitrary learning problem" would
> seem to depend on how you choose to enumerate learning
> problems - and so that statement is not well defined :-|
Why do you think so?
> > The question is whether it's possible to create a "generic and
> > simple" (made without using problem-specific information) learning
> > system that performs well on any "real-life" problem (those you might
> > want an intelligent system to practically solve).
>
> > In order for this to be possible, it must be the case that all "real-
> > life" learning problems share a common sub-structure that can be
> > exploited by a learning algorithm.
>
> > As far as I know, no such sub-structure is known. [...]
>
> What, and so there is No Free Lunch? IMO, that's not
> correct. The class of survival problems which organisms'
> brains face is far from arbitrary. There are all kinds
> of regularity in the problem space - as a result of the
> physics of how actuators and sensors function, the
> regularities of developmental programs, the uniformity
> of nature - and so on.
>
> Check the performance of random search in engineering
> problems. It is pretty poor. So, the problems humans
> face are rather regular in structure. They have not
> been chosen at random - but rather cooked up by nature.
Yes, there are regularities, but they may not be enough to allow the
existence of a generic learning system which obtains human-level
performance on real-life problems.
> If there were no such regularities, evolution would not
> be effective at finding solutions.
Biological evolution exploits such regularities (a single mutation
often doesn't change too much the fitness of an organism, and
offspring usually have a fitness similar to those of their parents).
But it's also worth noting that biological evolution is target-less:
it solved some problems, but if you were to restart it from the
scratch, it might fail to solve them and solve other problems instead.
Vend
<snip>
> > As far as I know, no such sub-structure is known.
>
> I think I know what it is. Mostly.
May I ask what do you think it is?
> > 50 years of research
> > in AI in general, and learning algorithms in particular failed to find
> > it, and neurological evidence suggests that the human brain doesn't
> > work that way, but rather it has a large number of modules, each one
> > specialized in a particular class of problems.
>
> > Hence I'm not particularly confident that a generic learning system
> > can exist, unless it's made by combining a large number of problem-
> > specific learning systems, like the human brain.
>
> I really do have to finish this work and show a few people what they failed
> to understand. :)
I hope you do, but I'm not confident you can.
Curt Welch
> Yes, there are regularities, but they may not be enough to allow the
> existence of a generic learning system which obtains human-level
> performance on real-life problems.
That's the interesting question here. I have no doubt that we can create
far stronger generic learning machines than we currently have. But how
close to human level behavior with these machines can you get, and how much
does the design have to optimized and adapted to duplicate full human level
behavior? Until we get the stronger generic learning system to work with,
we won't be able to answer that. However, no matter how much the system
has to be customized and configured just right to duplicate human like
behavior in a machine, I think the foundation of what we are missing today
is strong generic learning technology. Without it, no one is ever going to
come close to solving AI. It's the missing link that has held up AI for
the past 50 years.
Tim Tyler
> On 22 Apr, 00:41, Tim Tyler <seemy...@googlemail.com> wrote:
>> Vend wrote:
>>> On 19 Apr, 19:46, c...@kcwc.com (Curt Welch) wrote:
>>>> The underlying structure we have to build into the machine, is strong blank
>>>> slate learning, and nothing else. The more generic the learning becomes,
>>>> the closer it gets to true blank slate learning, the more adaptive it is
>>>> (it can learn anything) and the more intelligent it becomes.
>>> A system that can learn anything, that is, it performs well on any
>>> arbitrary learning problem, is a mathematical impossibility.
>> Hmm. The meaning of "arbitrary learning problem" would
>> seem to depend on how you choose to enumerate learning
>> problems - and so that statement is not well defined :-|
>
> Why do you think so?
Because its an infinite set - you cannot pick
"arbitrary" problems (they would be enormous)
and there are plenty of ways of enumerating
the set so that if you pick a small index,
you tend to get a problems which is soluble
using a simple algorithm.
> Biological evolution exploits such regularities (a single mutation
> often doesn't change too much the fitness of an organism, and
> offspring usually have a fitness similar to those of their parents).
> But it's also worth noting that biological evolution is target-less:
> it solved some problems, but if you were to restart it from the
> scratch, it might fail to solve them and solve other problems instead.
Convergent evolution is important, though. See:
Life's Solution: Inevitable Humans in a Lonely Universe
by Simon Conway Morris
http://www.amazon.co.uk/Lifes-Solution-Inevitable-Humans-Universe/dp/0521603250/
...or my:
http://originoflife.net/direction/
The problem life solves is essentially the one all dissipative
structures face: how to turn energy gradients into
self-perpetuating order and exhaust rapidly. There is not much
opportunity for picking a different problem.
rs...@nycap.rr.com
> Vend <ven...@virgilio.it> wrote:
> > Yes, there are regularities, but they may not be enough to allow the
> > existence of a generic learning system which obtains human-level
> > performance on real-life problems.
>
> That's the interesting question here. I have no doubt that we can create
> far stronger generic learning machines than we currently have.
> the past 50 years.
The blank slate (tabula rasa) was a brain concept put forth by people
who were singularly ill-equipped to discuss the brain--philosophers,
mathematicians, psychologists, and such. The brain is constructed and
connected by the genome to respond to the environment as it appeared
in the past. V1 (area 17) is precisely connected to recognize any line
segment appearing in the visual field. It can do this at birth. It
need not learn.
The species need learn the location of food and water sources, a
secure resting place, and to recognize a mate. The rest is froth.
ray
feedbackdroid
> Vend wrote:
>
> > Biological evolution exploits such regularities (a single mutation
> > often doesn't change too much the fitness of an organism, and
> > offspring usually have a fitness similar to those of their parents).
> > But it's also worth noting that biological evolution is target-less:
> > it solved some problems, but if you were to restart it from the
> > scratch, it might fail to solve them and solve other problems instead.
>
> Convergent evolution is important, though. See:
>
"Inevitability", that's a good one. Human beings, being egotistical
beings and mostly religious in the bottom core, "would" believe in the
inevitability of human beings.
Stephen Gould, however, agrees with Vend above. If the dice were
rethrown a 2nd time, it's hardly likely we'd end up with us a 2nd
time. If the dinos hadn't been wiped out by an asteroid [some believe
this], it's likely they'd have eaten anything mammalian larger than a
rat. Just look at the videos of gazelle herds trying to swim across
croc-infested rivers to see survival of the fittest in action. 20
start across, 0 get across.
>
> Life's Solution: Inevitable Humans in a Lonely Universe
> by Simon Conway Morris
>
> http://www.amazon.co.uk/Lifes-Solution-Inevitable-Humans-Universe/dp/...
Tim Tyler
>>> Biological evolution exploits such regularities (a single mutation
>>> often doesn't change too much the fitness of an organism, and
>>> offspring usually have a fitness similar to those of their parents).
>>> But it's also worth noting that biological evolution is target-less:
>>> it solved some problems, but if you were to restart it from the
>>> scratch, it might fail to solve them and solve other problems instead.
>> Convergent evolution is important, though. See: [snip SCM ref]
>
> "Inevitability", that's a good one. Human beings, being egotistical
> beings and mostly religious in the bottom core, "would" believe in the
> inevitability of human beings.
>
> Stephen Gould, however, agrees with Vend above. If the dice were
> rethrown a 2nd time, it's hardly likely we'd end up with us a 2nd
> time. [...]
...Right. But Stephen Gould thought all kinds of stuff:
On Stephen Jay Gould's "Darwinian Fundamentalism" and "Evolution: The
Pleasures of Pluralism"
http://cogweb.ucla.edu/Debate/CEP_Gould.html
Homo Deceptus - Never trust Stephen Jay Gould.
http://www.slate.com/default.aspx?id=2016
"Darwinian Fundamentalism": An Exchange
http://www.stephenjaygould.org/reviews/dennett_exchange.html
THE ACCIDENTAL CREATIONIST
http://www.nonzero.org/newyorker.htm
...so his emphasising the role of chance in evoultion
doesn't mean much of any significance.
Vend
> Vend wrote:
> > On 22 Apr, 00:41, Tim Tyler <seemy...@googlemail.com> wrote:
> >> Vend wrote:
> >>> On 19 Apr, 19:46, c...@kcwc.com (Curt Welch) wrote:
> >>>> The underlying structure we have to build into the machine, is strong blank
> >>>> slate learning, and nothing else. The more generic the learning becomes,
> >>>> the closer it gets to true blank slate learning, the more adaptive it is
> >>>> (it can learn anything) and the more intelligent it becomes.
> >>> A system that can learn anything, that is, it performs well on any
> >>> arbitrary learning problem, is a mathematical impossibility.
> >> Hmm. The meaning of "arbitrary learning problem" would
> >> seem to depend on how you choose to enumerate learning
> >> problems - and so that statement is not well defined :-|
>
> > Why do you think so?
>
> Because its an infinite set - you cannot pick
> "arbitrary" problems (they would be enormous)
> and there are plenty of ways of enumerating
> the set so that if you pick a small index,
> you tend to get a problems which is soluble
> using a simple algorithm.
There are various ways of defining a size measure. The performance of
any algorithm averaged on the problem space of any given size (under
most definitions of size) is the same.
> > Biological evolution exploits such regularities (a single mutation
> > often doesn't change too much the fitness of an organism, and
> > offspring usually have a fitness similar to those of their parents).
> > But it's also worth noting that biological evolution is target-less:
> > it solved some problems, but if you were to restart it from the
> > scratch, it might fail to solve them and solve other problems instead.
>
> Convergent evolution is important, though. See:
>
> Life's Solution: Inevitable Humans in a Lonely Universe
> by Simon Conway Morris
>
> http://www.amazon.co.uk/Lifes-Solution-Inevitable-Humans-Universe/dp/...
>
> ...or my:
>
> http://originoflife.net/direction/
Humans as the top of the great chain of beings? Doesn't seem very
informative.
> The problem life solves is essentially the one all dissipative
> structures face: how to turn energy gradients into
> self-perpetuating order and exhaust rapidly.
What does it mean?
feedbackdroid
> feedbackdroid wrote:
> > On Apr 22, 11:56 am, Tim Tyler <seemy...@googlemail.com> wrote:
> >> Vend wrote:
> >>> Biological evolution exploits such regularities (a single mutation
> >>> often doesn't change too much the fitness of an organism, and
> >>> offspring usually have a fitness similar to those of their parents).
> >>> But it's also worth noting that biological evolution is target-less:
> >>> it solved some problems, but if you were to restart it from the
> >>> scratch, it might fail to solve them and solve other problems instead.
> >> Convergent evolution is important, though. See: [snip SCM ref]
>
> > "Inevitability", that's a good one. Human beings, being egotistical
> > beings and mostly religious in the bottom core, "would" believe in the
> > inevitability of human beings.
>
> > Stephen Gould, however, agrees with Vend above. If the dice were
> > rethrown a 2nd time, it's hardly likely we'd end up with us a 2nd
> > time. [...]
>
> ...Right. But Stephen Gould thought all kinds of stuff:
>
So, you think the originoflife.net guy is ever gonna get his
"inevitability of life" ideas published in refereed journals? Now
here's a brilliant aphorism ... "Since weather forecasters are part of
nature, it seems clear that nature contains elements capable of
predicting the future.". DOH.
And how many of Gould's academic "critics" believe in the
inevitability of human lifeforms?
> On Stephen Jay Gould's "Darwinian Fundamentalism" and "Evolution: The
> Pleasures of Pluralism"http://cogweb.ucla.edu/Debate/CEP_Gould.html
>
> Homo Deceptus - Never trust Stephen Jay Gould.http://www.slate.com/default.aspx?id=2016
>
> "Darwinian Fundamentalism": An Exchangehttp://www.stephenjaygould.org/reviews/dennett_exchange.html
>
> THE ACCIDENTAL CREATIONISThttp://www.nonzero.org/newyorker.htm
>
> ...so his emphasising the role of chance in evoultion
> doesn't mean much of any significance.
> --
> __________
> |im |yler http://timtyler.org/ t...@tt1lock.org Remove lock to reply.- Hide quoted text -
Tim Tyler
> So, you think the originoflife.net guy is ever gonna get his
> "inevitability of life" ideas published in refereed journals?
That's my web site.
Plenty of material on this subject has been published - some
of it in refereed journals. Some recent pubs on the subject
include:
New Findings Confirm Darwin's Theory: Evolution Not Random
http://www.sciencedaily.com/releases/2008/01/080118134531.htm
Evolution Is Deterministic, Not Random, Biologists Conclude From
Multi-species Study
http://www.sciencedaily.com/releases/2007/11/071119123929.htm
Tim Tyler
> On 22 Apr, 19:56, Tim Tyler <seemy...@googlemail.com> wrote:
>> Vend wrote:
>>> On 22 Apr, 00:41, Tim Tyler <seemy...@googlemail.com> wrote:
>>>> Vend wrote:
>>>>> A system that can learn anything, that is, it performs well on any
>>>>> arbitrary learning problem, is a mathematical impossibility.
>>>> Hmm. The meaning of "arbitrary learning problem" would
>>>> seem to depend on how you choose to enumerate learning
>>>> problems - and so that statement is not well defined :-|
>>> Why do you think so?
>> Because its an infinite set - you cannot pick
>> "arbitrary" problems (they would be enormous)
>> and there are plenty of ways of enumerating
>> the set so that if you pick a small index,
>> you tend to get a problems which is soluble
>> using a simple algorithm.
>
> There are various ways of defining a size measure. The performance of
> any algorithm averaged on the problem space of any given size (under
> most definitions of size) is the same.
Trivial counter-example - on the space of halting problems:
Algorithm 1. Print "halts".
Algorithm 2. Count to a million. Print "does not halt".
>> The problem life solves is essentially the one all dissipative
>> structures face: how to turn energy gradients into
>> self-perpetuating order and exhaust rapidly.
>
> What does it mean?
Living systems rapidly transform resources into genomes and
waste - such as heat.
That is what living systems do. They cannot just pick some
other problem to solve - if they want to survive. God's
Utility Function is quite specific.
feedbackdroid
> feedbackdroid wrote:
> > So, you think the originoflife.net guy is ever gonna get his
> > "inevitability of life" ideas published in refereed journals?
>
> That's my web site.
>
The question still stands. In triplicate.
>
> Plenty of material on this subject has been published - some
> of it in refereed journals. Some recent pubs on the subject
> include:
>
> New Findings Confirm Darwin's Theory: Evolution Not Random
>
> http://www.sciencedaily.com/releases/2008/01/080118134531.htm
>
And you still seem to be batting 100% when it comes to finding
bizarre, off-worldy references via google.
Reading the abstract makes it seem like the authors don't know the
difference between the mutation and the selection parts of evolution.
Mutation is random, selection is not. There's a reason the lion
usually eats the zebra, rather than the other way around.
However, some interesting commentary here [although there seems to be
no "official" Dawkins on the "official" page]....
feedbackdroid
>
> New Findings Confirm Darwin's Theory: Evolution Not Random
>
BTW, instead of latching onto every new HEADLINE that comes down the
pike as "truth, whooole truth, and nothing but", you should find a
proper forum to vett it in. Eg, see what these guys say.
Wolf Kirchmeir
>> Evolution Is Deterministic, Not Random, Biologists Conclude From
>> Multi-species Study
>>
>> http://www.sciencedaily.com/releases/2007/11/071119123929.htm
>> --
>> __________
>> |im |yler http://timtyler.org/ t...@tt1lock.org Remove lock to reply.
>
Now why should "random" be contrary to "deterministic"?
--
wolf k.
Tim Tyler
> On Apr 22, 3:50 pm, Tim Tyler <seemy...@googlemail.com> wrote:
>> feedbackdroid wrote:
>>> So, you think the originoflife.net guy is ever gonna get his
>>> "inevitability of life" ideas published in refereed journals?
>> That's my web site.
>
> The question still stands. In triplicate.
Probably not. I tend to put my work into the public domain -
and refereed journals often require material to be copyrighted,
so they can use it to obtain money from readers to pay the
wages of the referees.
>> Plenty of material on this subject has been published - some
>> of it in refereed journals. Some recent pubs on the subject
>> include:
>>
>> New Findings Confirm Darwin's Theory: Evolution Not Random
>>
>> http://www.sciencedaily.com/releases/2008/01/080118134531.htm
>
> And you still seem to be batting 100% when it comes to finding
> bizarre, off-worldy references via google.
>
> Reading the abstract makes it seem like the authors don't know the
> difference between the mutation and the selection parts of evolution.
> Mutation is random, selection is not. There's a reason the lion
> usually eats the zebra, rather than the other way around.
>
> However, some interesting commentary here [although there seems to be
> no "official" Dawkins on the "official" page]....
>
> http://richarddawkins.net/article,2141,New-Findings-Confirm-Darwins-Theory-Evolution-Not-Random,Science-Daily
A generally positive response.
Vend
> Vend wrote:
> > On 22 Apr, 19:56, Tim Tyler <seemy...@googlemail.com> wrote:
> >> Vend wrote:
> >>> On 22 Apr, 00:41, Tim Tyler <seemy...@googlemail.com> wrote:
> >>>> Vend wrote:
> >>>>> A system that can learn anything, that is, it performs well on any
> >>>>> arbitrary learning problem, is a mathematical impossibility.
> >>>> Hmm. The meaning of "arbitrary learning problem" would
> >>>> seem to depend on how you choose to enumerate learning
> >>>> problems - and so that statement is not well defined :-|
> >>> Why do you think so?
> >> Because its an infinite set - you cannot pick
> >> "arbitrary" problems (they would be enormous)
> >> and there are plenty of ways of enumerating
> >> the set so that if you pick a small index,
> >> you tend to get a problems which is soluble
> >> using a simple algorithm.
>
> > There are various ways of defining a size measure. The performance of
> > any algorithm averaged on the problem space of any given size (under
> > most definitions of size) is the same.
>
> Trivial counter-example - on the space of halting problems:
>
> Algorithm 1. Print "halts".
> Algorithm 2. Count to a million. Print "does not halt".
I fail to see any relevance to learning.
> >> The problem life solves is essentially the one all dissipative
> >> structures face: how to turn energy gradients into
> >> self-perpetuating order and exhaust rapidly.
>
> > What does it mean?
>
> Living systems rapidly transform resources into genomes and
> waste - such as heat.
Ok.
> That is what living systems do. They cannot just pick some
> other problem to solve - if they want to survive. God's
> Utility Function is quite specific.
But since this problem is extemely complex, it's worth considering it
as decomposable in sub-problems.
feedbackdroid
> feedbackdroid wrote:
> > On Apr 22, 3:50 pm, Tim Tyler <seemy...@googlemail.com> wrote:
> >> feedbackdroid wrote:
> >>> So, you think the originoflife.net guy is ever gonna get his
> >>> "inevitability of life" ideas published in refereed journals?
> >> That's my web site.
>
> > The question still stands. In triplicate.
>
> Probably not. I tend to put my work into the public domain -
>
Which of course qualifies as a "scientific" statement as regards the
validity of the "inevitability of human life" argument.
>
> and refereed journals often require material to be copyrighted,
> so they can use it to obtain money from readers to pay the
> wages of the referees.
>
>
>
> >> Plenty of material on this subject has been published - some
> >> of it in refereed journals. Some recent pubs on the subject
> >> include:
>
> >> New Findings Confirm Darwin's Theory: Evolution Not Random
>
> >> http://www.sciencedaily.com/releases/2008/01/080118134531.htm
>
> > And you still seem to be batting 100% when it comes to finding
> > bizarre, off-worldy references via google.
>
> > Reading the abstract makes it seem like the authors don't know the
> > difference between the mutation and the selection parts of evolution.
> > Mutation is random, selection is not. There's a reason the lion
> > usually eats the zebra, rather than the other way around.
>
> > However, some interesting commentary here [although there seems to be
> > no "official" Dawkins on the "official" page]....
>
> >http://richarddawkins.net/article,2141,New-Findings-Confirm-Darwins-T...
>
> A generally positive response.
> --
>
Right. I guess you didn't actually read the comments, nor the original
paper, but just the headline.
You also didn't address whether the people who supposedly criticized
Gould [we'll exclude your personal blog] actually did so because they
disagreed with his stance against the "inevitability of human life",
or disagreed on "other" matters [which I think you'd find to be the
case, were you to actually track it down].
It does make a difference.
Alpha
> feedbackdroid wrote:
> > On Apr 22, 3:50 pm, Tim Tyler <seemy...@googlemail.com> wrote:
> >> feedbackdroid wrote:
> >>> So, you think the originoflife.net guy is ever gonna get his
> >>> "inevitability of life" ideas published in refereed journals?
> >> That's my web site.
>
> > The question still stands. In triplicate.
>
> Probably not. I tend to put my work into the public domain -
> and refereed journals often require material to be copyrighted,
> so they can use it to obtain money from readers to pay the
> wages of the referees.
Your work would need to be coherent as well.
>
>
>
>
>
> >> Plenty of material on this subject has been published - some
> >> of it in refereed journals. Some recent pubs on the subject
> >> include:
>
> >> New Findings Confirm Darwin's Theory: Evolution Not Random
>
> >> http://www.sciencedaily.com/releases/2008/01/080118134531.htm
>
> > And you still seem to be batting 100% when it comes to finding
> > bizarre, off-worldy references via google.
>
> > Reading the abstract makes it seem like the authors don't know the
> > difference between the mutation and the selection parts of evolution.
> > Mutation is random, selection is not. There's a reason the lion
> > usually eats the zebra, rather than the other way around.
>
> > However, some interesting commentary here [although there seems to be
> > no "official" Dawkins on the "official" page]....
>
> >http://richarddawkins.net/article,2141,New-Findings-Confirm-Darwins-T...
>
> A generally positive response.
> --
> __________
Alpha
> On Apr 22, 3:50 pm, Tim Tyler <seemy...@googlemail.com> wrote:
>
> > feedbackdroid wrote:
> > > So, you think the originoflife.net guy is ever gonna get his
> > > "inevitability of life" ideas published in refereed journals?
>
> > That's my web site.
>
> The question still stands. In triplicate.
>
>
>
> > Plenty of material on this subject has been published - some
> > of it in refereed journals. Some recent pubs on the subject
> > include:
>
> > New Findings Confirm Darwin's Theory: Evolution Not Random
>
> > http://www.sciencedaily.com/releases/2008/01/080118134531.htm
>
> And you still seem to be batting 100% when it comes to finding
> bizarre, off-worldy references via google.
>
> Reading the abstract makes it seem like the authors don't know the
> difference between the mutation and the selection parts of evolution.
> Mutation is random, selection is not. There's a reason the lion
> usually eats the zebra, rather than the other way around.
Not to mention that mutation is a bona fide cause of evilution qua
evolution. Unlike natural selection, which is a process that results,
according to the Modern Synthesis, only in a (re)distribution of
allels in a population by acting upon the artifacts (phenotypes) that
already came into being *by other means* (e.g., mutation or de novo
emergence etc.).
See Reid's work.
>
> However, some interesting commentary here [although there seems to be
> no "official" Dawkins on the "official" page]....
>
> http://richarddawkins.net/article,2141,New-Findings-Confirm-Darwins-T...
>
>
>
>
>
> > Evolution Is Deterministic, Not Random, Biologists Conclude From
> > Multi-species Study
>
> > http://www.sciencedaily.com/releases/2007/11/071119123929.htm
> > --
> > __________
J.A. Legris
> feedbackdroid wrote:
> > On Apr 22, 3:50 pm, Tim Tyler <seemy...@googlemail.com> wrote:
> >> feedbackdroid wrote:
> >>> So, you think the originoflife.net guy is ever gonna get his
> >>> "inevitability of life" ideas published in refereed journals?
> >> That's my web site.
>
> > The question still stands. In triplicate.
>
> Probably not. I tend to put my work into the public domain -
> and refereed journals often require material to be copyrighted,
> so they can use it to obtain money from readers to pay the
> wages of the referees.
>
Got any citations yet?
feedbackdroid
I see it already was vetted by those guys. They were slightly
underwhelmed by the new breakthrough in darwinism ....
http://groups.google.com/group/sci.bio.evolution/browse_frm/thread/ad27424a6af3ae4d/1cbb44df87e8c5f2
Tim Tyler
> On Apr 23, 1:37 am, Tim Tyler <seemy...@googlemail.com> wrote:
>> feedbackdroid wrote:
>>> On Apr 22, 3:50 pm, Tim Tyler <seemy...@googlemail.com> wrote:
>>>> feedbackdroid wrote:
>>>>> So, you think the originoflife.net guy is ever gonna get his
>>>>> "inevitability of life" ideas published in refereed journals?
>>>> That's my web site.
>>> The question still stands. In triplicate.
>> Probably not. I tend to put my work into the public domain -
>
> Which of course qualifies as a "scientific" statement as regards the
> validity of the "inevitability of human life" argument.
It sounds as though you have lost track of your own questions :-|
> You also didn't address whether the people who supposedly criticized
> Gould [we'll exclude your personal blog] actually did so because they
> disagreed with his stance against the "inevitability of human life",
> or disagreed on "other" matters [which I think you'd find to be the
> case, were you to actually track it down].
>
> It does make a difference.
That raises a new issue. My original argument was of the
form: Gould thought a lot of nonsense, and so why should
what Gould thought on this issue make any difference?
Tim Tyler
Those are simple algorithms that happen to do no learning -
which are admissable since you said: "any algorithm". The
space of all Turing-machine programs naturally also includes
more complex algorithms - some of which will learn.
feedbackdroid
> feedbackdroid wrote:
> > On Apr 23, 1:37 am, Tim Tyler <seemy...@googlemail.com> wrote:
> >> feedbackdroid wrote:
> >>> On Apr 22, 3:50 pm, Tim Tyler <seemy...@googlemail.com> wrote:
> >>>> feedbackdroid wrote:
> >>>>> So, you think the originoflife.net guy is ever gonna get his
> >>>>> "inevitability of life" ideas published in refereed journals?
> >>>> That's my web site.
> >>> The question still stands. In triplicate.
> >> Probably not. I tend to put my work into the public domain -
>
> > Which of course qualifies as a "scientific" statement as regards the
> > validity of the "inevitability of human life" argument.
>
> It sounds as though you have lost track of your own questions :-|
>
> > You also didn't address whether the people who supposedly criticized
> > Gould [we'll exclude your personal blog] actually did so because they
> > disagreed with his stance against the "inevitability of human life",
> > or disagreed on "other" matters [which I think you'd find to be the
> > case, were you to actually track it down].
>
> > It does make a difference.
>
> That raises a new issue. My original argument was of the
> form: Gould thought a lot of nonsense, and so why should
> what Gould thought on this issue make any difference?
> --
Show that any of the Gould criticizers [other than yerself] have
evidence for, or even belief in, the inevitability of human life.
Start with your own links ...
=================
On Stephen Jay Gould's "Darwinian Fundamentalism" and "Evolution: The
Pleasures of Pluralism"
http://cogweb.ucla.edu/Debate/CEP_Gould.html
Wolf Kirchmeir
In that thread TT write:
...So: the issue arises of which /kind/ of explanation to invoke
when faced with the observed features of organisms:
Chance? Or necessity? ...
I don't see an either-or issue here. Why does TT think there is one?
--
wolf k.
Tim Tyler
> Show that any of the Gould criticizers [other than yerself] have
> evidence for, or even belief in, the inevitability of human life.
I think you have probably leaped to a dubious conclusion
simply from the title of Conway-Morris's book :-|
Anyhow, Conway-Morris is the cannonical antidote to
those who think chance rules in evolution:
``The reason for discussing convergence here is that its
recognition effectively undermines the main plank of
Gould's argument on the role of contingent processes in
shaping the tree of life and thereby determining the
outcome at any one time. Put simply, contingency is
inevitable, but unremarkable. It need not provoke
discussion, because it matters not. There are not an
unlimited number of ways of doing something. For all its
exuberance, the forms of life are restricted and channelled.
For the great majority of biologists such a conclusion will
hardly be surprising. The agenda, however, once again is
ideological, because the discussions on contingency versus
constraint seem to be more to provide the background and
focus of a very specific problem, that is the rise of human
intelligence. Gould's view is unequivocal. The likelihood
of Man evolving on any other planet is extraordinarily
unlikely. To paraphrase: if the history of evolution were
to be repeated, the world would teem with myriad forms of
life (note that the contingent likelihood of the origin of
life itself goes through on the nod), but certainly no
humans. As stated, this seems to be entirely unremarkable,
although again it presupposes that the constraints are
weak. It is not, however, the point. What we are interested
in is not the origin, destiny, or fate of a particular
lineage, but the likelihood of the emergence of a
particular property, say consciousness. Here the reality of
convergence suggests that the tape of life, to use Gould's
metaphor, can be run as many times as we like and in
principle intelligence will surely emerge.''
- http://www.stephenjaygould.org/library/conwaymorris_crucible.html
Tim Tyler
> In that thread TT write:
>
> ...So: the issue arises of which /kind/ of explanation to invoke
> when faced with the observed features of organisms:
> Chance? Or necessity? ...
>
> I don't see an either-or issue here. Why does TT think there is one?
I don't. It's rather like the nature/nurture debates: some features
are mostly down to nature, some are mostly nurture - and some are a
combination of the two. Similarly with chance and necessity.
Vend
I thought it was clear from the context that I meant any learning
algorithm.
In this context, performance is usually measured as utility measure
vs. number of training examples.
feedbackdroid
I notice you've had this discussion before, and the guys weren't very
impressed in that case, either. Cf, message #143/on.
http://groups.google.com/group/sci.bio.evolution/browse_frm/thread/1b91ea3a90e07eb9/2b6cf971b1543249
You think there's no criticism of Morris' perspective? Try googling
that.
Tim Tyler
> On 23 Apr, 21:18, Tim Tyler <seemy...@googlemail.com> wrote:
>> Vend wrote:
>>> On 23 Apr, 00:03, Tim Tyler <seemy...@googlemail.com> wrote:
>>>> Vend wrote:
>>>>> On 22 Apr, 19:56, Tim Tyler <seemy...@googlemail.com> wrote:
>>>>>> Vend wrote:
>>>>>>> On 22 Apr, 00:41, Tim Tyler <seemy...@googlemail.com> wrote:
[...]
>>>>>>>> Hmm. The meaning of "arbitrary learning problem" would
>>>>>>>> seem to depend on how you choose to enumerate learning
>>>>>>>> problems - and so that statement is not well defined :-|
>>>>>>> Why do you think so?
>>>>>> Because its an infinite set - you cannot pick
>>>>>> "arbitrary" problems (they would be enormous)
>>>>>> and there are plenty of ways of enumerating
>>>>>> the set so that if you pick a small index,
>>>>>> you tend to get a problems which is soluble
>>>>>> using a simple algorithm.
>>>>> There are various ways of defining a size measure. The performance of
>>>>> any algorithm averaged on the problem space of any given size (under
>>>>> most definitions of size) is the same.
>>>> Trivial counter-example - on the space of halting problems:
>>>> Algorithm 1. Print "halts".
>>>> Algorithm 2. Count to a million. Print "does not halt".
>>> I fail to see any relevance to learning.
>> Those are simple algorithms that happen to do no learning -
>> which are admissable since you said: "any algorithm".
>
> I thought it was clear from the context that I meant any learning
> algorithm.
> In this context, performance is usually measured as utility measure
> vs. number of training examples.
It /looks/ like you were /trying/ to express the "No Free Lunch" idea.
But what you actually said was incorrect nonsense.
The "No Free Lunch" idea is pretty worthless, IMO. It only works
at all if you quantify "size" over the /solution/ space, and constrain
algorithms so that they are unable to make repeated trials - both
highly unrealistic notions in most cases.
Tim Tyler
> On Apr 23, 2:03 pm, Tim Tyler <seemy...@googlemail.com> wrote:
>> feedbackdroid wrote:
>>> Show that any of the Gould criticizers [other than yerself] have
>>> evidence for, or even belief in, the inevitability of human life.
>> I think you have probably leaped to a dubious conclusion
>> simply from the title of Conway-Morris's book :-|
>>
>> Anyhow, Conway-Morris is the cannonical antidote to
>> those who think chance rules in evolution:
[...]
> I notice you've had this discussion before, and the guys weren't very
> impressed in that case, either. Cf, message #143/on.
>
> http://groups.google.com/group/sci.bio.evolution/browse_frm/thread/1b91ea3a90e07eb9/2b6cf971b1543249
Bob O'Hara agreed - and Jeffrey Turner disagreed, citing Gould.
So what?
> You think there's no criticism of Morris' perspective? Try googling
> that.
Everyone has their critics. I don't agree with Conway-Morris
about everything - but he's a good counter-cite for those who
are enamoured with Gould's views about the role of chance in
evolution - he has a whole book on the subject and it gives
many compelling examples.
feedbackdroid
> feedbackdroid wrote:
> > On Apr 23, 2:03 pm, Tim Tyler <seemy...@googlemail.com> wrote:
> >> feedbackdroid wrote:
> >>> Show that any of the Gould criticizers [other than yerself] have
> >>> evidence for, or even belief in, the inevitability of human life.
> >> I think you have probably leaped to a dubious conclusion
> >> simply from the title of Conway-Morris's book :-|
>
> >> Anyhow, Conway-Morris is the cannonical antidote to
> >> those who think chance rules in evolution:
>
> [...]
>
> > I notice you've had this discussion before, and the guys weren't very
> > impressed in that case, either. Cf, message #143/on.
>
>
> Bob O'Hara agreed - and Jeffrey Turner disagreed, citing Gould.
> So what?
>
> > You think there's no criticism of Morris' perspective? Try googling
> > that.
>
> Everyone has their critics. I don't agree with Conway-Morris
> about everything - but he's a good counter-cite for those who
> are enamoured with Gould's views about the role of chance in
> evolution - he has a whole book on the subject and it gives
> many compelling examples.
> --
Speaking of compelling examples, I've already wasted far too much time
on the subject of inevitability of human lifeforms. Boring. BTW, you
might find the following forum of some interest. Robert Stonjek
inhabits it, amongst others.