brain thoughts
rs...@nycap.rr.com
instructions written in the genome. It will drive the organism through
the universe that existed in the past. The future cannot be predicted,
but the genome contains information for a neural plasticity that
adapted to unforeseen aspects of the universe in the past. This
exercise of plasticity we term “learning”.
The nervous system includes circuitry that we call motor program
generators. This circuitry, when triggered, produces a motor program,
a sequenced set of axonal pulses that results in a motor act (such as
a step) when it arrives at the motor neurons.
The motor program generator is triggered by certain sensory impulses
that indicate the appropriateness of the motor act in the past.
Originally the sensory circuitry is established according to the
instructions in the genome, but it is altered by adjustment
(plasticity) as the organism travels through life. If this motor
program has produced a bad outcome in the past under the conditions
that presently exist, it can be halted by the thalamic reticular
nucleus.
The sum total of the motor acts constitutes the life of the organism.
The thalamic reticular nucleus can also halt sensory information as it
passes through the thalamus on its way to the neocortex. The neocrtex
relieved from the inward rush os sensory information is free to
associate. This results in thinking, reverie, and dreaming.
ray
Don Stockbauer
It's not what you say, it's what YOU DON'T SAY.
rs...@nycap.rr.com
>
> It's not what you say, it's what YOU DON'T SAY.- Hide quoted text -
>
It is almost impossible to speak seriously (scientifically) a out the
brain until we have made our peace with the concept of “mind”. This
early nineteenth century invention of the Anglo-Saxon philosophers was
held to be some junior-sized soul that was not a soul. Something that
could be carried about in a watch pocket. Something to be brought out
upon demand to be passed around. It was, and is, nothing but British
nonsense. We simply try to write always “soul (mind)”. When we speak
of the brain, we allow no soul (mind) to muck things up.
We would speak of the brain of the neuroscientist, a thing of neurons
that interact.
Cortical computation is the latest fad. The digital computer caught
the attention of academic newbies looking for a chance to establish
themselves. The Dartmouth Summer Conference was a coming together of
such people. The digital computer promised an ability to make
algorithms the subject of papers. The predicate calculus was ready at
hand. The soul (mind) as a fruitful subject was also at hand.
Beautiful.
There seemed to be no end. We will produce algorithms to manipulate
chess pieces (symbolically of course), and win at chess. Of course
that is too much to do in an afternoon, so we will use checkers. We
have watched half a century of this.
An alternative was to show our genius at clever programming. This
occupied the Artificial Intelligence Lab at MIT. Again, for half a
century.
From the beginning, there were those who looked at neurons. Neural
simulation of the brain was seen as the true future of digital
computation. But a hundred billion neurons were really too much. A net
of say six neurons was more to our liking. At once, we were faced by
those who would drop to the molecule. One to ten billion molecules in
a neuron was rally quite a bit to chew. So it goes.
Lately, we have the Blue Brain People who will deal with molecules at
astronomical numbers. They excite the envy of workers everywhere.
These workers must try to fish out papers with only the computational
facilities available to them. So we forego the molecule, and also the
one hundred billion neurons, and put our energy into little portions
of the brain.
So, what are we reduced to? We set out to simulate all the functions
of the Great Brain, and we are left only the Motor Program Generators
and the Thalamic Reticular Nucleus. What a come down.
The MPG’s execute and the organism moves through life. We have a
problem that the MPG’s may not be allowed to interfere with each
other. There must be a mechanism that allows only non-interference to
occur. Then the next step is to stop a motor act that will lead to a
bad outcome. This recognition of the possibility of a bad outcome, and
avoiding it, can be embroideried to encompass Plato’s reporting on
Socrates.
So it goes.
ray
Curt Welch
> On Nov 16, 5:38=A0am, Don Stockbauer <don.stockba...@gmail.com> wrote:
> >
> > It's not what you say, it's what YOU DON'T SAY.- Hide quoted text -
> >
>
> It is almost impossible to speak seriously (scientifically) a out the
> early nineteenth century invention of the Anglo-Saxon philosophers was
> held to be some junior-sized soul that was not a soul. Something that
> could be carried about in a watch pocket. Something to be brought out
> upon demand to be passed around. It was, and is, nothing but British
> of the brain, we allow no soul (mind) to muck things up.
Well, not many here are actually very confused about that so it's hardly
worth saying.
Some things John writes makes me believe that some of his ideas and
thoughts are illogical and incompatible with materialism, and I have
written many messages trying to get him to see that with little success.
But his far stronger and fundamental belief about all this brain/mind stuff
is strictly materialistic. So even he, how has some beliefs that tend to
be confused by issues of mind, has no doubt that our efforts to duplicate
the function of the brain is in fact just a mechanical problem to be
solved.
The people in the world who are truly still confused by the issues of
"mind" as being something non mechanical are simply not the people you
should waste your time talking to if you desire to understand how to solve
AI or how to build a working brain substitute (aka a machine that acts like
a human).
No, that's just not true. That's just your belief - and a belief that you
have completely failed to prove even in the least.
The truth is that all the research you talk about above is searching for
the correct way to describe the important features of a machine that can
produce human behavior. You just happen to believe that the important
features are described with the words "Motor Program Generators and the
Thalamic Reticular Nucleus".
I don't believe you are actually too far off, but those words simply fail
to describe the solution in enough detail to be of ANY use. Your words say
almost nothing more than "we just need a signal processing machine to
receive sensory inputs and produce the right effector outputs".
> The MPG's execute and the organism moves through life. We have a
> problem that the MPG's may not be allowed to interfere with each
> other. There must be a mechanism that allows only non-interference to
> occur. Then the next step is to stop a motor act that will lead to a
> bad outcome. This recognition of the possibility of a bad outcome, and
> avoiding it, can be embroideried to encompass Plato's reporting on
> Socrates.
So above, you go one tiny step further by bringing up the issues that motor
programs need to work without interference, but of course, it's far more
than that. You view is greatly oversimplified. Motor programs can and do
interfere with each other all the time. They interrupt each other when
conditions call for it. They merge together. We try to perform one act,
but our are hands start to move as if we are doing something else by
mistake. When we try to learn a new behavior, we end up producing odd
merged results of past beahviors.
Talking as you do, as if each MPG were a totally separate and distinct
module in the machine totally fails to explain the complexity of how they
interact, and merge. In short, real world behavior data shows quite
clearly that they are not separate entities as you would like to describe.
You talk as if each learned behavior had it's own little bit of hardware in
the brain called the MPG that produces that one behavior. As if we had one
MPG for eating ice cream, and another for riding a bike, and yet another
for typing the word "ice" and another for typing the word "cream".
But how then does the word "ice cream" get created? What controls that?
How do all the MPGs that allow me to type all these words get coordinated
so as to produce this entire message? I've never typed this message
before, so clearly, I don't have a single top level MPG hardware in me that
is in control of producing this behavior today. So how is it that I'm able
to produce this message? How does your "it's all just MPGs" theory explain
that I'm producing a behavior at the moment I've never produced before?
As someone else pointed out to you, it's what you don't say that's
important. What you have said, fails to explain human behavior. What you
have not said, is all the details messing to explain human behavior.
As you know, I too have my favorite high level (hand waving) words to
describe what features the machine needs. I say we need to build a
reinforcement trained reaction machine. Those words are not very different
(in level or detail) than your top level words of needing MPGs. But I do
at least attempt to build machines to implement my concepts. And they work
to some extent, so I have something more to point to.
If you think your idea of MPGs is the solution, then build one, and show us
how it works. I can tell you right now what problems you will run into
because I've seen people try to program robots by building lots of MPGs
into their design. It's very un-human (or un-animal) like in it's behavior
because it's simply not how our behavior is created. You might be able to
explain the behavior of an ant using hard wired MPGs, but not the behavior
of a rat. That's because the problem of learning new behaviors and of
regulating the use of the behaviors already learned in response to the
environment goes far beyond what the idea of MPGs can explain.
Thinking in terms of MPGs as the foundation of our behavior is a place to
start, but it's only a start. It might help someone who hasn't tried to
program robots to grasp where one might start, but it's only a start - and
not a very good one at that. Building a MPG is trial. What's hard, is
regulating the selection of which MPG(s) are currently in use, which is a
job of orchestrating the parallel production, and merging, and overlap of
many MPGs at the same time. Your over simplified ideas of "don't
interfere" and "stop when bad" is so naive I can't even grasp why you think
you have said something useful there. They must interfere at times, and
they must merge at times. It's not as simple as stop and go as you describe
it. It's more like 5% of MPG X combined with 12% of MPG Y, combined with
32% of MGP Z where those percentage numbers are changing dynamically in
response to a constant flow of new sensory information. The whole system
is so dynamic, that it really doesn't even make sense to think of it as a
MPG. The entire system must w3ork together to create one huge MPG that
produces human behavior. It's so complex and intertwined that it just
doesn't work to try and subdivide the system into separate MPGs.
So, if you want to do something useful here, build the machine based on
your ideas and see for yourself why the concept of MPGs doesn't work as an
architecture foundation. And if I'm wrong, then we would all love to see
the machine actually working and acting like a living animal.
--
Curt Welch http://CurtWelch.Com/
cu...@kcwc.com http://NewsReader.Com/
casey
>
> Some things John writes makes me believe that some of
> his ideas and thoughts are illogical and incompatible
> with materialism, and I have written many messages
> trying to get him to see that with little success.
Experiments and modern physics show a different foundation
to reality then the simple macro view generated by the
practical needs of an Ape.
> But his far stronger and fundamental belief about all
> this brain/mind stuff is strictly materialistic.
It is not a belief but a definition.
My understanding of ANY kind of machine, including what
is meant by machine, is as defined by Ross Ashby and that
is as an abstraction. It is so abstract it can even deal
with ghosts (see. 4/15 Materiality.)
It occurred to me that to the extent you talk any sense
at all you talk about a Universal Turing machine solution
which isn't practical. The biological brain is a working
solution to the problem you want to solve and it implements
solutions you like to deny as existing in real brains as
they don't fit your views on how it should be done.
Taking one of those issues, modularity, your view that
it is one module for every behavior is an exaggeration.
Your knowledge of programming should give you some
insight into the advantages and uses of modules to
produce stable hierarchical solutions to an evolving
or learning system but apparently not. It is something
I first understood before I taught myself how to program
when reading books on biological systems and how they
evolved in working stages.
JC
rs...@nycap.rr.com
> "rs...@nycap.rr.com" <rs...@nycap.rr.com> wrote:
> > On Nov 16, 5:38=A0am, Don Stockbauer <don.stockba...@gmail.com> wrote:
>
> > > It's not what you say, it's what YOU DON'T SAY.- Hide quoted text -
>
I have nothing to say to those who stand around with their fingers in
their mouth, marveling at what they see as he most complicated
structure in the universe.
> We try to perform one act,
> but our are hands start to move as if we are doing something else by
> mistake. When we try to learn a new behavior, we end up producing odd
> merged results of past beahviors.
Here I think you are invoking the soul (mind). What is this “we” you
speak of? The neuroscientist finds only neurons.
> > We simply try to write always soul (mind). When we speak
> > of the brain, we allow no soul (mind) to muck things up.
>
> Well, not many here are actually very confused about that so it's hardly
> worth saying.
I do think you might ask yourself just who this “we” is. Are you
confusing a soul (mind) with objective reality?
> Talking as you do, as if each MPG were a totally separate and distinct
> module in the machine totally fails to explain the complexity of how they
> interact, and merge. In short, real world behavior data shows quite
> clearly that they are not separate entities as you would like to describe.
> You talk as if each learned behavior had it's own little bit of hardware in
> the brain called the MPG that produces that one behavior. As if we had one
> MPG for eating ice cream, and another for riding a bike, and yet another
> for typing the word "ice" and another for typing the word "cream".
Module is a software concept. I think of the nervous system as
containing discrete circuitry set up by the genome (or the genome
contains a description that is read by the RNA). This circuitry we
call a motor program generator, an MPG. There is circuitry running
through the cortex and the basal ganglia (leading into the MPG) that
is effectively part of the MPG. There is circuitry running from the
MPG through the cerebellum, the thalamus, and the motor cortex that is
also part of the MPG.
The nervous system, as furnished by RNA, includes circuitry for
producing phonemes. When these circuits fire spontaneously at the age
of about one year, we say the infant is producing “baby talk”, or
babbling. The circuitry leading into the MPG’s that produced phonemes
can be altered to produce language. The circuitry leading out can be
altered to produce accent. This altering is not produced by an agent,
but by interaction with people.
Only if you insist on a soul (mind) with causal powers, do you find
difficulties.
The walking motor generator can be modified to produce dancing. It can
be modified to ride a bicycle. That riding a bicycle is only modified
walking should be obvious to all.
> If you think your idea of MPGs is the solution, then build one, and show us
> how it works.
Look at
http://crab-lab.zool.ohiou.edu/hooper/nerves.htm
for a nice picture and diagram of how neurons make a lobster’s stomach
work. Don’t object that this is a lobster, and you are interested in
human behavior. If you can rise above the soul (mind), you can
understand the neurons are neurons.
Or look at
http://crab-lab.zool.ohiou.edu/hooper/circuit.jpg
for a diagram of the essential neurons in the MPG. Called a Central
Pattern Generator in discussions of invertebrates. The circles are
individual neurons.
As for actually constructing an MPG at the vertebrate level, this is
beyond human capabilities. We do not have the time or capabilities by
many, many orders on magnitude. We can only talk.
ray
Curt Welch
> On Nov 17, 12:03 pm, c...@kcwc.com (Curt Welch) wrote:
> > "rs...@nycap.rr.com" <rs...@nycap.rr.com> wrote:
> > > wrote=
> :
> >
> > > > It's not what you say, it's what YOU DON'T SAY.- Hide quoted text -
> >
>
> I have nothing to say to those who stand around with their fingers in
> their mouth, marveling at what they see as he most complicated
> structure in the universe.
Hard to grasp what you are trying to say there.
> > We try to perform one act,
> > but our are hands start to move as if we are doing something else by
> > mistake. When we try to learn a new behavior, we end up producing odd
> > merged results of past beahviors.
>
> Here I think you are invoking the soul (mind). What is this =93we=94 you
> speak of? The neuroscientist finds only neurons.
You sir, are the one that thinks in terms of dualism if that's how you read
my words. When I talk about "myself", or "we do this", I'm making
reference not to the dualistic "we" (the soul in the machine), but to the
REAL "we", aka the human body that is performing the act.
> > > We simply try to write always soul (mind). When we speak
> > > of the brain, we allow no soul (mind) to muck things up.
> >
> > Well, not many here are actually very confused about that so it's
> > hardly worth saying.
>
> I do think you might ask yourself just who this =93we=94 is. Are you
> confusing a soul (mind) with objective reality?
Do you read any of my posts? Shit, I've written 1000's of posts here
arguing AGAINST all this talk of the soul, and then you accuse me of using
the word "we" as a reference to the soul in the machine instead of in
reference to the machine itself?
Again, I've long since driven the idea of the soul out of my own thinking
about myself. I am not a mind, or soul, in a body. I'm just a body with a
brain. When I say "I had a thought", I mean my body did something physical
just as when I write "I moved my hand".
> > Talking as you do, as if each MPG were a totally separate and distinct
> > module in the machine totally fails to explain the complexity of how
> > they interact, and merge. In short, real world behavior data shows
> > quite clearly that they are not separate entities as you would like to
> > describe=
> .
> > You talk as if each learned behavior had it's own little bit of
> > hardware =
> in
> > the brain called the MPG that produces that one behavior. As if we had
> > o=
> ne
> > MPG for eating ice cream, and another for riding a bike, and yet
> > another for typing the word "ice" and another for typing the word
> > "cream".
>
> Module is a software concept. I think of the nervous system as
> containing discrete circuitry set up by the genome (or the genome
> contains a description that is read by the RNA). This circuitry we
> call a motor program generator, an MPG. There is circuitry running
> through the cortex and the basal ganglia (leading into the MPG) that
> is effectively part of the MPG. There is circuitry running from the
> MPG through the cerebellum, the thalamus, and the motor cortex that is
> also part of the MPG.
I can't begin to argue what's actually there. I don't study
neurophysiology enough to debate brain structure. I study human behavior
enough to debate some issues of brain _function_ however and that was more
the point I was bringing up. Where in the brain, and how that function is
implemented I can only make wild speculations about. But to create human
behavior, that function MUST be in there somewhere.
> The nervous system, as furnished by RNA, includes circuitry for
> producing phonemes. When these circuits fire spontaneously at the age
> of about one year, we say the infant is producing =93baby talk=94, or
> babbling. The circuitry leading into the MPG=92s that produced phonemes
> can be altered to produce language. The circuitry leading out can be
> altered to produce accent. This altering is not produced by an agent,
> but by interaction with people.
>
> Only if you insist on a soul (mind) with causal powers, do you find
> difficulties.
>
> The walking motor generator can be modified to produce dancing. It can
> be modified to ride a bicycle. That riding a bicycle is only modified
> walking should be obvious to all.
>
> > If you think your idea of MPGs is the solution, then build one, and
> > show =
> us
> > how it works.
>
> Look at
>
> http://crab-lab.zool.ohiou.edu/hooper/nerves.htm
>
> for a nice picture and diagram of how neurons make a lobster=92s stomach
> work. Don=92t object that this is a lobster, and you are interested in
> human behavior. If you can rise above the soul (mind), you can
> understand the neurons are neurons.
In almost every post you make, you argue that we must rise above the soul
mind. I rose above it long ago and it's a complete waste of time to ask me
do so. It makes me wonder if every time you write about it like that, if
you are really just working on training yourself to rise above it?
"I" am not a neuron. I am a human body. Or, from the perspective I
sometimes write from, I am a brain with a human body attached. But what
I'm not, in any sense, is a soul (or mind) in a body.
If, BTW, I ever use the word "Mind" you can rewrite what I wrote and
replace "mind" with "brain" and the meaning to me will be identical. I
just use the word "Mind" in the places I've been taught to call the brain a
mind.
On the lobster bit, the fact that neurons are wired into circuits to
regulate the body is a DUH fact. Why have me look at a neuron circuit of a
lobster? What does that prove to someone like me that knows for a fact
that neurons regulate our behavior?
The only question at hand that I give a shit about (I'm an AI guy - not a
neurscientsits), is how do I build a circuit to make a robot act like a
human?
I know for a fact that human behavior is produced by a signal processing
device which regulates and produces what I talk about as "human behavior".
The only question if what type of signal processing architecture is
required to produce that behavior?
The architecture used in a lobster to control its stomach really doesn't
tell me shit about how a human learns to drive a car.
> Or look at
>
> http://crab-lab.zool.ohiou.edu/hooper/circuit.jpg
>
> for a diagram of the essential neurons in the MPG. Called a Central
> Pattern Generator in discussions of invertebrates. The circles are
> individual neurons.
>
> As for actually constructing an MPG at the vertebrate level, this is
> beyond human capabilities. We do not have the time or capabilities by
> many, many orders on magnitude. We can only talk.
That's just a cop out.
Yes, a human brain, once trained to the point of producing typical adult
human behavior, is a circuit so complex no human has a hope in
understanding it. You can't even make a team of a billion humans
understand it. It's simply beyond what any human will every ben able to
understand.
But, where does that complexity come from? Not our DAN, because our DNA is
relatively simple by comparison. I think it's less then a single CD worth
of data if I'm remembering correctly. You can't specify adult human
behavior in a "blue print" based on that small amount of data. The amount
of blue prints we create to define the structure of a complex machine like
a jet fighter, is far far greater than the amount of data in our DNA. So
the design of a human, (the data specified in our DNA), is well within the
grasp of human understanding. Your cop-out position of saying it's too
hard to understand doesn't apply to the blue-print data that creates us.
So where does the complexity come from? It comes from sensory data
collected over the life time. That sensory data is incorporated into the
very design of our brain circuits (though the process we call
conditioning). It's the nurture part of our machine complexity instead of
the nature part of our design.
If our DNA can specify a machine, which is able to change it's design from
interacting with a highly complex environment, and that DNA design is far
simpler than other machines human are already building, then it's highly
likely that we, as humans, can also design such a machine that change's
it's design though interaction with the environment, and in doing so, ends
up with a design that makes it at like a human. And that is exactly what I
think we must do to solve AI. We must build a learning machine. And that
machine, though we can fully understand how it learns, we shape itself into
a machine, that will be beyond our ability to understand.
And what I write here is not in any respect speculation. It's a fact, We
already have machines that do just that. Most learning algorithms work
exactly like that. We design, and build the learning algorithm, but what
the machine ends up learning, is too complex for any human to ever
understand. When a neural network gets trained, we understand fully why
the network converges on an optimal set of weights for a given problem.
But we have no clue, and never will, understand why one particular sets of
weights was the answer it was going to converge on - before it does it.
The solution to AI that humans will produce, is not the values of a
trillions of weights in a neural network. No human could every calculate
those. The solution, will be the architecture of the learning system that
can, on it's own, set those weights, to make it act like a human. And that
we will be able to understand.
casey
>
> "I" am not a neuron. I am a human body. Or, from the
> perspective I sometimes write from, I am a brain with a
> human body attached. But what I'm not, in any sense,
> is a soul (or mind) in a body.
The mind is of course not a physical or nonphysical entity,
it is something the brain does. The actual substance changes
and you assign physical parts to yourself according to a
set of rules which can in fact result in you denying an arm
attached to your body as being part of you and also
experiencing a fake arm (or a piece of wood) as being part
of you.
> The architecture used in a lobster to control its stomach
> really doesn't tell me shit about how a human learns to
> drive a car.
I would disagree. It will give you a hint as to the basic
computing units required to build one.
If you read the history of science you will find nothing
appeared out of a vacuum despite your belief to the contrary
with regards to mechanical flight. In practice I believe it
was the wings of birds and bats that pointed us in the right
direction.
Just as Darwin saw evolution as the result of natural
selection you see intelligence as the result of reinforcement
learning. But how do we find the mechanisms of natural
selection or reinforcement learning? How long would it
have taken to "invent" the idea of the dna protein complex?
The idea of natural selection is simple enough but knowing
how it can actually be implemented is another question.
You imagine it is nothing but conditioning. So how is
conditioning achieved in a neural circuit? In Aplysia we
know it is done by a modulatory circuit that increases
the weighted connections of the mediating circuit that
produces the stimulus response.
Of course we don't have to duplicate the physiology of
the neuron anymore than we have to duplicate the physiology
of a birds wing to duplicate the function of a wing but
it tells us what can work.
Researchers have gone beyond the simple stimulus response
models to revealing actual physical representations that
lead to behavior by merging cognitive psychology with
cellular neurobiology.
JC
Curt Welch
> On Nov 22, 12:07=A0pm, c...@kcwc.com (Curt Welch) wrote:
> >
> > "I" am not a neuron. I am a human body. Or, from the
> > perspective I sometimes write from, I am a brain with a
> > human body attached. But what I'm not, in any sense,
> > is a soul (or mind) in a body.
>
> The mind is of course not a physical or nonphysical entity,
> it is something the brain does.
It is not "of course". Only someone like you with a dualistic view of
reality sees action as non-physical.
> The actual substance changes
Which is why it's a physical thing.
> and you assign physical parts to yourself according to a
> set of rules which can in fact result in you denying an arm
> attached to your body as being part of you and also
> experiencing a fake arm (or a piece of wood) as being part
> of you.
>
> > The architecture used in a lobster to control its stomach
> > really doesn't tell me shit about how a human learns to
> > drive a car.
>
> I would disagree. It will give you a hint as to the basic
> computing units required to build one.
Well, sure, it's a biological neuron circuit so it must has some things in
common with how the brain is able to make the body drive a car. But so is
every other neural circuit in every sort of life form that has some sort of
nervous system.
> If you read the history of science you will find nothing
> appeared out of a vacuum despite your belief to the contrary
> with regards to mechanical flight.
Again, you trying to guess what my beliefs are and having no clue. I
thought you would have learned not to do that when I kept getting your
beliefs wrong! :)
> In practice I believe it
> was the wings of birds and bats that pointed us in the right
> direction.
No doubt, we wouldn't have even tried to build flying machines as soon as
we did if not for the fact that air was full of animals already showing us
it was possible. And yes, people studied animals to gain a basic
understanding of how it was possible.
But none of that has anything to do with a point I've made to you over and
over. That point being that the final design of a plsne like the Wright
Brothers had very little in common with any animal. It's power source was
nothing like that of any flying animal and required to be invented out of
thin air instead of copied from nature. The Wright Brothers control system
used a human brain of course, and the wing warping could easilly be
described as copied from nature. But most the details of the structure of
the air frame again had nothing in common with any animal. And most
important, 99.9% of the technology in a bird, was NOT used in the Wright
brothers plane. No heart to circulate blood, no tendons and muscles, no
digestive system, no complex self reproducing system in the plane, etc,
etc. Most of what exists in a bird, to make it fly, simply had no parallel
in our first planes.
Likewise, most of what exists in the human brain is simply not needed to
create machine intelligence. It's useful in us, to keep us alive, but not
to make us act intelligent.
To solve AI, we have to figu8re out what that .1% of the technology in the
human body is needed to make us act intelligent. If we can find it in the
brain, then sure, studying the brain is fine. But to be honest, I think
most people studying the brain have no clue what they are actually looking
for (or at).
> Just as Darwin saw evolution as the result of natural
> selection you see intelligence as the result of reinforcement
> learning. But how do we find the mechanisms of natural
> selection or reinforcement learning?
John, it doesn't need to be found. We already have it it working in 100's
of examples already. You are the only one here that seems to "need to find
it".
> How long would it
> have taken to "invent" the idea of the dna protein complex?
> The idea of natural selection is simple enough but knowing
> how it can actually be implemented is another question.
We know how it's implemented. What is it that you don't seem to know?
> You imagine it is nothing but conditioning. So how is
> conditioning achieved in a neural circuit?
Well, that's a good question for the neuroscientists. But frankly, I don't
give a shit because I already have it working with transistors and I see no
reason to switch to trying to build AI out of neurons.
> In Aplysia we
> know it is done by a modulatory circuit that increases
> the weighted connections of the mediating circuit that
> produces the stimulus response.
>
> Of course we don't have to duplicate the physiology of
> the neuron anymore than we have to duplicate the physiology
> of a birds wing to duplicate the function of a wing but
> it tells us what can work.
Yes, and the bigger more interesting question there is for someone to
figure out how the entire network of neurons is being conditioned so that
the entire network is able to work together to generate useful behaviors.
If no one knows how this works at any level of abstraction (a mathematical
model, or a computer model for example), then it's fairly unlikely someone
studying the brain is going to figure it out. That's because the network
behavior tends to be too complex to study. I'm fairly sure that we first
have to figure out the theory first as a branch of mathematics/computer
science before the neuroscientist will have a clue what to look for in the
hardware.
> Researchers have gone beyond the simple stimulus response
> models to revealing actual physical representations that
> lead to behavior by merging cognitive psychology with
> cellular neurobiology.
>
> JC
The only thing wrong with "stimulus response" is that when you use that
term, most people think the "stimulus" is a short term event (like a light
flash). In truth, the stimulus that drives human behavior is EVERYTHING
that has happened to the brain since birth. As long as you understand
"stimulus" in theory extends back to the machine's creation (all past
inputs), then stimulus response FULLY explains brain behavior and is not in
the least bit simple. It's actually "overly complex". In other words, if
you actually understand stimulus response, you would understand it's not
possible to go beyond it.
casey
> casey <jgkjca...@yahoo.com.au> wrote:
>
>> You imagine it is nothing but conditioning. So how is
>> conditioning achieved in a neural circuit?
>
>
> Well, that's a good question for the neuroscientists.
> But frankly, I don't give a shit because I already
> have it working with transistors and I see no reason
> to switch to trying to build AI out of neurons.
I wasn't suggesting you build AI out of neurons.
>> But how do we find the mechanisms of natural selection
>> or reinforcement learning?
>
>
> John, it doesn't need to be found. We already have it
> it working in 100's of examples already. You are the
> only one here that seems to "need to find it".
>
>
> We know how it's implemented.
We don't know how RL is implemented in a cat let alone
a human otherwise we would have copied it. The mechanisms
used in TD-Gammon or balancing a pole do not scale up.
You can see something like RL as being a solution and yet
be mistaken about what is required to actually implement
it at a higher level. Darwin for example thought that
inheritance was the result of analog blending which in
fact wouldn't have worked, it had to be digital.
> The only thing wrong with "stimulus response" is that
> when you use that term, most people think the "stimulus"
> is a short term event (like a light flash). In truth,
> the stimulus that drives human behavior is EVERYTHING
> that has happened to the brain since birth. As long as
> you understand "stimulus" in theory extends back to the
> machine's creation (all past inputs), then stimulus
> response FULLY explains brain behavior and is not in
> the least bit simple. It's actually "overly complex".
> In other words, if you actually understand stimulus
> response, you would understand it's not possible to go
> beyond it.
Although there is a lot we don't know about the brain it
seems churlish to ignore what we do know. None of it
reflects the views you have expressed about how nodes
should behave or how they should be wired.
The chicken wire network of nodes of the hydra brain did
not evolve into anything more complex. More behaviorally
complex brains for some reason formed clusters of neurons
called ganglia which are asymmetrical in their distribution.
And if it is simply "conditioning all the way up" why
didn't the old conditioning circuits just get bigger?
Why did new networks with different circuitry evolve?
JC
Wolf K
[...]
> But none of that has anything to do with a point I've made to you over and
> over. That point being that the final design of a plsne like the Wright
> Brothers had very little in common with any animal. It's power source was
> nothing like that of any flying animal and required to be invented out of
> thin air instead of copied from nature. The Wright Brothers control system
> used a human brain of course, and the wing warping could easilly be
> described as copied from nature. But most the details of the structure of
> the air frame again had nothing in common with any animal. And most
> important, 99.9% of the technology in a bird, was NOT used in the Wright
> brothers plane. No heart to circulate blood, no tendons and muscles, no
> digestive system, no complex self reproducing system in the plane, etc,
> etc. Most of what exists in a bird, to make it fly, simply had no parallel
> in our first planes.
The Wright plane was a conglomeration of existing technologies adapted
to the purpose. In no particular order:
a) Kites (which are very, very old), the box kite in particular;
b) Gasoline engine (which was by then developed enough to have a high
horsepower to weight ratio);
c) Bicycle wheel (an example of light weight framing with tension and
compression members);
d) Sailing ships (control lines, rudder);
e) Screw propeller (itself derived from the windmill).
NB that the dirigible balloon borrowed a good deal from the fixed wing
airplane.
I do not claim originality for the above list: SciAm had an article on
this topic a couple decades or so ago.
In AI terms, I suspect that a working autonomous robot (with limited
capabilities, bien sur) will turn out to be a conglomeration of existing
technologies. About the only thing I'm fairly sure of is that its
parts/subunits/modules/etc will be massively and multiply
interconnected. It will therefore not function as a digital computer,
but as a chaotic system. That will give it its free will (or whatever
you want to call that aspect of behaviour.)
cheers,
wolf k.
rs...@nycap.rr.com
> casey <jgkjca...@yahoo.com.au> wrote:
> > The mind is of course not a physical or nonphysical entity,
> > it is something the brain does.
>
> It is not "of course". Only someone like you with a dualistic view of
> reality sees action as non-physical.
I think you two are arguing about personal beliefs, although I must
say Casey has the better of it.
You, Welch, seem to be immersed in old-time Psychology, especially in
the view of the brain as coming into the world with the structure of a
bowl of porridge. A randomnized slew of neurons that is to trained to
meet up with the universe.
There is no randomness in the brain brought forth by the genome. It is
idle to hypothesize randomness where there is non.
Today, most would see the brain at birth as fully, precisely
interconnected with no randomness whatsoever. (At birth meaning up to
the age of six.)
I would say that it is connected up in circuitry that makes up the
motor program generators. The plasticity of the synapses is employed
to moderate, adjust, alter the MPG's to meet with the happenstances of
life.
ray
rs...@nycap.rr.com
> to the purpose. In no particular order:
> a) Kites (which are very, very old), the box kite in particular;
> b) Gasoline engine (which was by then developed enough to have a high
> horsepower to weight ratio);
> c) Bicycle wheel (an example of light weight framing with tension and
> compression members);
> d) Sailing ships (control lines, rudder);
> e) Screw propeller (itself derived from the windmill).
>
> NB that the dirigible balloon borrowed a good deal from the fixed wing
> airplane.
I think some would say that the Wright brothers should be credited
with inventing the banked turn. Certainly, when the Europeans first
saw Orville demonstrating flight, they leaped on the banked turn as
being the great innovation of the Wright brothers.
> In AI terms, I suspect that a working autonomous robot (with limited
> capabilities, bien sur) will turn out to be a conglomeration of existing
> technologies. About the only thing I'm fairly sure of is that its
> parts/subunits/modules/etc will be massively and multiply
> interconnected. It will therefore not function as a digital computer,
> but as a chaotic system. That will give it its free will (or whatever
> you want to call that aspect of behaviour.)
AI is ruled out as being far too costly. A brain of artificial neurons
is simple enough, just monstrously unfeasible.
ray
casey
> casey <jgkjca...@yahoo.com.au> wrote:
>>> The mind is of course not a physical or nonphysical
>>> entity, it is something the brain does.
>>
>> It is not "of course". Only someone like you with a
>> dualistic view of reality sees action as non-physical.
>
>
> I think you two are arguing about personal beliefs,
> although I must say Casey has the better of it.
My comment was not directed at Curt although he responded
to it for I already know how he thinks about these things.
The view I have tried to express isn't easy to grasp but
it is well understood by some heavy duty thinkers and
indeed it was from them that I came to understand it.
Curt confuses The View with The Reality. We assume there
is one Reality out there but there are many views.
> You, Welch, seem to be immersed in old-time Psychology,
I used the term "so 1960's" but it got deleted when I
reduced the size of the post.
> ... especially in the view of the brain as coming into
> the world with the structure of a bowl of porridge. A
> randomized slew of neurons that is to trained to
> meet up with the universe.
Essentially Curt wants to evolve the brain with the idea
that if you have the mechanisms to do this you have the
mechanisms for generating any kind of behavior. There are
practical reasons I don't see this as the best course of
action particularly given the way he wants to do it.
> There is no randomness in the brain brought forth by
> the genome. It is idle to hypothesize randomness where
> there is non.
>
>
> Today, most would see the brain at birth as fully,
> precisely interconnected with no randomness whatsoever.
> (At birth meaning up to the age of six.)
>
>
> I would say that it is connected up in circuitry that
> makes up the motor program generators. The plasticity
> of the synapses is employed to moderate, adjust, alter
> the MPG's to meet with the happenstances of life
It seems to me that the brain does start with default
connections that are modified by experience. The role
of the neocortex seems to add analytical power to the
older parts of the brain.
JC
casey
>
> In AI terms, I suspect that a working autonomous robot
> (with limited capabilities, bien sur) will turn out to
> be a conglomeration of existing technologies. About the
> only thing I'm fairly sure of is that its parts/subunits
> /modules/etc will be massively and multiply interconnected.
Whereas my view is they will only be potentially connected
to each other. The actual connections at any time will be
determined by higher level modules carrying out some goal.
> It will therefore not function as a digital computer, but
> as a chaotic system. That will give it its free will (or
> whatever you want to call that aspect of behaviour.)
The evidence is that the brain either uses chaotic behaviors
or it prevents them from happening or both. I also think you
are confusing the function of a digital computer with the
programs that might run on such a computer.
JC
Wolf K
> On Nov 24, 9:04 am, Wolf K <weki...@sympatico.ca> wrote:
>
>> to the purpose. In no particular order:
>> a) Kites (which are very, very old), the box kite in particular;
>> b) Gasoline engine (which was by then developed enough to have a high
>> horsepower to weight ratio);
>> c) Bicycle wheel (an example of light weight framing with tension and
>> compression members);
>> d) Sailing ships (control lines, rudder);
>> e) Screw propeller (itself derived from the windmill).
>>
>> NB that the dirigible balloon borrowed a good deal from the fixed wing
>> airplane.
>
> I think some would say that the Wright brothers should be credited
> with inventing the banked turn. Certainly, when the Europeans first
> saw Orville demonstrating flight, they leaped on the banked turn as
> being the great innovation of the Wright brothers.
Interesting tidbit. Thanks.
wolf k.
[...]
Curt Welch
Yes of course, but all those things were invented "out of thin air" so to
say by a long slow (hundreds or thousands of years) of technological
progress by humans. Not by just copying nature.
And likewise, when I build robots, I'm using a huge amount of technology
evolved by a long line of people before me - such as the technology to
control electricity and the transistor, and all the technology in a
computer etc. But to solve AI, we have to extend that technology a few
more steps still. In the end, when we invent, we add very little
originality of our own. We mostly just reuse old ideas with a few small
twists when we create something new. But we do, add originality of our own
in this process which is how, all the invention takes place over centuries.
Each of those little "twists" we add add up to be substantial change in
time.
> NB that the dirigible balloon borrowed a good deal from the fixed wing
> airplane.
>
> I do not claim originality for the above list: SciAm had an article on
> this topic a couple decades or so ago.
>
> In AI terms, I suspect that a working autonomous robot (with limited
> capabilities, bien sur) will turn out to be a conglomeration of existing
> technologies.
Of course. But which ones! :) I argue for neural network technologies
combined with RL technologies.
> About the only thing I'm fairly sure of is that its
> parts/subunits/modules/etc will be massively and multiply
> interconnected. It will therefore not function as a digital computer,
> but as a chaotic system. That will give it its free will (or whatever
> you want to call that aspect of behaviour.)
Well, I see nothing preventing it from both functioning as a digital
computer and as a chaotic system (shaped by reinforcement). It's easy
enough to write software which produces a chaotic system on a digital
computer.
> cheers,
> wolf k.
Curt Welch
> On Nov 23, 6:58=A0pm, c...@kcwc.com (Curt Welch) wrote:
> > casey <jgkjca...@yahoo.com.au> wrote:
>
> > > The mind is of course not a physical or nonphysical entity,
> > > it is something the brain does.
> >
> > of reality sees action as non-physical.
>
> I think you two are arguing about personal beliefs, although I must
> say Casey has the better of it.
Yeah, we are mostly arguing what we believe to the the philosophy of AI.
That is, how to approach finding a solution. Though we argue issues about
the brain, the real point I always argue is NEVER about the brain, how it
works, or what it does, but simply how to best approach the engineering
problem of developing man-made machines to duplicate what we think of a
human intelligent behavior.
> You, Welch, seem to be immersed in old-time Psychology, especially in
> the view of the brain as coming into the world with the structure of a
> bowl of porridge. A randomnized slew of neurons that is to trained to
> meet up with the universe.
Well, this is where many people who read my posts seem to be totally lost.
That is NOT IN THE LEAST MY VIEW OF THE BRAIN.
The brain is full of genetically specified modules and circuits which allow
it to do all the things it does.
However, 90% of what the brain does, I have zero interest in because I'm
not trying to build machines to keep humans alive. I'm interested only in
the problem of building control systems that make robots act intelligent.
Currently, our robots don't act very much like an intelligent human. We
can't even make them act like an "intelligent" rat yet. And there's no
real indication that what we are missing at this point is computing power,
or lines of code. Our computers are plenty powerful enough, and we know
how to get a lot of programmers working together to write billions of lines
of code.
What we are missing, is a fundamental understanding of how this type of
behavior is created by a control system. And all my arguments, are about
what that fundamental technology is - and how we might build it into our
machines.
I, for the most part, don't bring up the brain in my posts because I'm not
a brain researcher, I'm a computer programmer and engineer. I attempt to
design a solution to creating human behavior, not to duplicate the brain's
implementation of a control system.
What I do however, is try to point out how much of what people believe they
are looking at when they find features in the brain, is clearly
inconsistent with what the brain MUST do in order to create human behavior.
Though I often suggest how the brain might be working, I don't typically
take that to the position of saying it's how the brain DOES work, because
evolution could have created some very bizarre solutions to the problem of
producing adaptive behavior which could make it's implementation extremely
odd and highly different from what we might end up engineering.
> There is no randomness in the brain brought forth by the genome. It is
> idle to hypothesize randomness where there is non.
I NEVER use the word RANDOMNESS like that. You are using it because you
seem unable to understand how learning systems work - which is further
proof to me you haven't thought though the very basics of the engineering
problem the brain is actually solving.
> Today, most would see the brain at birth as fully, precisely
> interconnected with no randomness whatsoever. (At birth meaning up to
> the age of six.)
>
> I would say that it is connected up in circuitry that makes up the
> motor program generators. The plasticity of the synapses is employed
> to moderate, adjust, alter the MPG's to meet with the happenstances of
> life.
That might be true. That is, the idea that the wiring is fairly fixed,
with the learning mostly taking place in the synapses. I seriously doubt
that is true, but again, like I said above, I don't argue that level of
implementation issues for the brain.
I do, however, argue that your view of MPGs simply can't work (at least to
my best ability to understand what your view is based on what you have
written). You don't have a clue what you are talking about. You have
taken a highly over simplistic idea of behavior being control by a a set of
behavior generators and think that all learning can then be explained by
starting, and interrupting, these MPGs. Its' just absurd. Try to program
it and see what sort of behavior you can make a robot dog perform and you
will soon see how unlike human and animal behavior your approach is.
What I agree with of what you write, is that there must be hardware
behavior generators created by the genome which form the foundation of all
our behavior. Their use (aka when one is active and another is not) must
be triggered by the context of the environment as defined by the brain's
sensory system. So in this light, I would say you are conceptually close
to the right track. However, I suspect odds are that the "behavior
generator" at work in the human brain for creating most our learned
behaviors are not like the genetically wired neural circuits you call MPGs
to control actions in lobsters.
The type of learning humans do, can't be explained with the type of
hardware you seem to envision. You just can't get there from here. People
have tried to program robots the way you are suggesting and it doesn't make
them act at all like any of the living creatures which have strong adaptive
learning system. We know what you are suggesting doesn't work as an
engineering solution. If you build a robot using your ideas, and show it
me - them I would take your position with more seriousness. But from what
I can grasp, you study biology and don't have a clue how to do basic
engineering - which leaves you at a dead end when it comes to trying to
build the ideas you have about what you THINK the brain is doing, into real
hardware. That what's where your approach would get lost if you tried to
implement it.
One specific example of the problem with your approach is that human
perception and human behavior is hierarchical in nature. We combine small
sequences of learned behavior in different ways to create larger sequences
of learned behaviors - not just in two levels, but in a near infinite
number of hierarchies of behaviors using behaviors. Your MPG approach
doesn't offer a single explanation of how the hardware does that. In order
for that to work, one "behavior generator" must have the power to control
other "behavior generators". Nothing you have written indicates your
supposed brain architecture supports such a thing - yet, it's clearly and
obviousness required. You just don't seem to have a clue abiout even the
basics of the control problem the brain is solving. And people like you
that like to study brain research, but don't seem to have even an
introductory level understanding of the engineering control problem the
brain solves - or how our man made hardware solves some of the same
problems already - leaves with with a no faith vote on your opinion of how
the brain "works". I mostly ignore your posts because mostly I don't care
about how the brain solved the problems, and most important, I don't think
you even have a clue how far away from understanding the problem which the
brain is solving you are. The level of useful engineering information
contained in your posts is about equal to what I would expect a 9th grader
to produce - one who had no formal training, or experience with real
engineering problems and solutions.
John on the other hand, does do a good bit of both. That is, study brain
research AND produce working engineering solutions to some brain-like
problems. I argue with him because I think he has a chance of
understanding my points. He does at least, mostly understand my position
on using learning from the bottom up in a generic learning system. He just
doesn't believe it's possible (learning would be far far too slow that way
- maybe not even possible at all in a billion years), and as such, leans
more towards the idea that the low level is full of innate helper modules
created by evolution and that the learning is applied on top of that at a
higher level - which is more in line with your view that the MPGs are
innate, and learning is applied to regulate when each one starts and stops.
On the surface, what he suggest is very logical. And likewise, at the high
level, what you suggest can seem logical - and at the highest levels of
abstraction, I think both of you are right to a degree. But when you try
to actually make that high level learning work, you find the low level
"behavior generators" can't be "high level innate modules" and in fact,
have to be extremely primitive behaviors - like the firing of a neuron - or
in my AI networks, sorting a single pulse. The innate behavior modules
supplied by the genome can't be much higher than that and still allow us to
learn the high resolution behaviors we are able to learn which can't be
explained by innate modules like bike riding or sports playing, or high
speed typing of words on a computer keyboard. We do not have innate MPGs
modules for typing Usenet messages in English on a computer keyboard about
AI algorithms. There is no long evolutionary history of needing to use our
fingers in this way that would explain why we have innate MPGs that are
able to do this. It can only be explained by circuits that can
self-configure themselves into specialized MPG circuits that simply didn't
exist at birth as we "learn to type". What that "self configure" is done
only by adjusting synapse weights or whether it's dong by growing new
connections I have no clue. And I don't care. But I do know that if you
try to write a lot os MPG software modules for "finger motions" you can't
then use another module to regulate the stop and stop of those MPGs and end
up with the learned behavior of composing and typing Usenet messages about
AI and brain architecture.
Curt Welch
Well, that "practical reasons" you have is that you don't grasp how it
could work. That's what holding you back from understand it MUST work like
I say. That is, it must be a system that applies hierarchical learning to
an extremely low level innate behavior - like pulse sorting or the firing
of a single neuron. You can't apply the learning to high levels like to
innate MPGs or innate perception systems and get the high resolution (high
speed) learning humans have.
Just as a for example, we can hard-code a behavior sequence into a robot
dog to make it walk forward in a straight line. Or we can hard code a
sequence to make it sit. Or another one to make turn to the right. Or we
can hard code a behavior to make it balance itself on a moving platform
while otherwise standing still. That is typical of the tyupe of "scripts"
and "algorithms" people code into their real robots. But how do you blend
these together? If you have code that makes it turn right, and another
"module" that makes it walk straight, how do blend these two "MPGs"
together so that it can dynamically navigate from one side of the room to
another in order to get a bone? You can turn on one, and disable the
other. Then stop the "walk straight" and start up the "turn right for a
few milliseconds". Then stop that and start up the walk straight again.
But when you do that, your robot ends up acting NOTHING like real animals.
It looks like a stupid robot with nothing at all natural about it's
motions. So how do you blend many MPGs together to create a natural
looking motion so that it can move forward while turning in a slight arc to
the right?
The answer is that you can't do it. These sorts of "modules" just can't be
mixed when they were created separately. You can't create animal like
behavior by stopping and starting different behavior scripts - which is
Ray's idea of how it works.
To make the robot look at all natural, you have to do something like build
one complex algorithm from the ground up to do these multiple things
at the same time. So you do things like structuring it as a reverse
kinematic balancing algorithm that is good at moving legs and arms to keep
the robot balanced. And then you drive that algorithm with a higher level
algorithm that adjusts the balance point so the low level algorithm then
changes and keeps it off-balance in a particular direction which causes it
to start falling - but constantly catching itself in that direction That
makes it "walk" in a direction dynamically like humans do. The high level
code can then regulate how fast, and which direction the "balancing robot"
is moving, by regulating its balance point the low level balance code is
trying to achieve. These reverse kinematic algorithms only work if they
have pre-built into them, accurate physical descriptions of the robot body
however. It has to know how long and how heavy all the arms and leg
segments are.
But then you want to add in the ability to reach for and pick up an object
while all this is going on. So you have to add more code to the low level
balancing program to make it regulate hand position at the same time. So
instead of moving the arms and hands anywhere it needs to in order to
balance the robot, you code it so one hand can be constrained to be near a
certain spot in 3-D space. And you use yet another high level algorithm to
adjust that hand position constant code in the balancing code so as to make
it move the hand out to the location of the ball you want the robot to pick
up.
Now you have code that can balance, navigate around a room, and reach out
and touch stuff. But now we have a high level problem of regulating the
goal of this low level natural acting system. How does it decide where to
go, and where to put it's hand? How do you make it pick up a ball for
example? What happens if the visual/hand control system tells the low
level code to move the hand to the ball, but that it almost always ends up
kicking the ball with the feet in the act of trying to balance so it never
actually gets to the ball? Making it good comedian. :)
So we add some some MORE low level innate code in the balancing algorithm
to constrain where the feet go as it's trying to balance so the high level
motivation "module" which is trying to implement the "pick up the ball
behavior" is able to relay to the low level code both the hand and feet
constraints so that it can navigate over to the ball, and pick it up,
without kicking it out of the way before it can grab it.
My point being with this example, is that when you code it something like
the above, you end up with a robot that does in fact seem almost alive
because it's producing very dynamic-like actions with all these behaviors
somewhat merged together. But, EVERY TIME YOU WANT IT TO DO SOMETHING NEW
- you have to re-code, from the ground up, all the low level innate
systems.
Humans can learn to pick up a ball without kicking it out of the way FROM
EXPERIENCE. NOT because it's got a low level innate MODULE (or MPG) that
includes hand and foot constraint inputs.
The stuff humans can learn from experience (which could not possibly have
been needed for the past million years while the innate brain systems
evolved) invalidate the approach of hard-coding these complex interactive
behavior systems as you seem to envision it might work. Because of what
the human brain can learn from experience (like playing golf, or producing
langauge on a keyboard at the high speeds we can do it, or playing the
piano by sight reading sheet music), all the sorts of circuits I talked
about above have to be completely re-wired by the effects of conditioning.
So the question is, how do we implement an algorithm like the one I talked
about above, that has a hierarchy of learned control systems controlling
each other, and using each other in different combinations, all shaped by a
single reward signal?
The answer is that you need a hierarchical temporal reaction engine that
shapes it's reactions in response to the context defined by the stimulus
system as shaped by a reward signal. It's nothing more than a big "what do
I do know" reaction machine, where "what do I do" is a very low level
question such as sorting a single pulse, or making a single neuron fire.
No OTHER type of machine structure that I've seen sugegsted can explain how
this is achieved - and as it happens, the neocortex seems to be structured
just like that.
casey
>
> You can't apply the learning to high levels like to
> innate MPGs or innate perception systems and get the
> high resolution (high speed) learning humans have.
Of course you can.
> Just as a for example, we can hard-code a behavior
> sequence into a robot dog to make it walk forward
> in a straight line. Or we can hard code a sequence
> to make it sit. Or another one to make turn to the
> right. Or we can hard code a behavior to make it
> balance itself on a moving platform while otherwise
> standing still.
You hard code the behavior that causes the system to
balance itself. You then control it by unbalancing it.
The direction desired is not controlled at that level.
> That is typical of the type of "scripts" and
> "algorithms" people code into their real robots.
> But how do you blend these together?
You don't blend them together anymore than you blend
software modules. Think how you use modules in your
programs.
> If you have code that makes it turn right, and another
> "module" that makes it walk straight, how do blend
> these two "MPGs" together so that it can dynamically
> navigate from one side of the room to another in order
> to get a bone?
You don't have two modules you have one module and it
is controlled by another module that uses it to achieve
its own outcome.
< snip a lot of nonsense >
> But, EVERY TIME YOU WANT IT TO DO SOMETHING NEW - you
> have to re-code, from the ground up, all the low level
> innate systems.
Rubbish. Think about how you write high level programs
using modules.
> The stuff humans can learn from experience (which could
> not possibly have been needed for the past million years
> while the innate brain systems evolved) invalidate the
> approach of hard-coding these complex interactive
> behavior systems as you seem to envision it might work.
There is no suggestion that any high level "golf playing",
"bike riding", "English instead of French speaking" and
so on has be hard coded. We start with the same machinery
as any other animal minus their high level innate behaviors.
We also have extra connections etc.
> Because of what the human brain can learn from experience
> (like playing golf, or producing language on a keyboard
> at the high speeds we can do it, or playing the piano by
> sight reading sheet music), all the sorts of circuits I
> talked about above have to be completely re-wired by the
> effects of conditioning.
They don't have to be rewired any more than the modules in
your programs all have to be rewritten.
> So the question is, how do we implement an algorithm
> like the one I talked about above, that has a hierarchy
> of learned control systems controlling each other, and
> using each other in different combinations, all shaped
> by a single reward signal?
By a process of top down control.
> The answer is that you need a hierarchical temporal
> reaction engine that shapes it's reactions in response
> to the context defined by the stimulus system as shaped
> by a reward signal.
>
> It's nothing more than a big "what do I do know"
> reaction machine, where "what do I do" is a very low
> level question such as sorting a single pulse, or
> making a single neuron fire.
>
> No OTHER type of machine structure that I've seen
> suggested can explain how this is achieved - and as it
> happens, the cerebral cortex seems to be structured just
> like that.
The basic machinery exists without the cerebral cortex.
What the cerebral cortex seems to have added is increased
analytical power and increased memory capacity, in
particular episodic memories. Basic skills like riding a
bike do not require the cerebral cortex except as the
instigator to practice the skill in the first place.
Take a simple conditioning experiment. An audio tone
associated with a shock. Without the cerebral cortex the
animal will generalize over all sorts of audio tones.
The cerebral cortex however can analyze the tones into
many more complex categories. If you play one tune it
means a shock another tune no shock but the actual
association takes place in the amygdala. Before there
was a cerebral cortex the audio and pain signals were
all directed to the amygdala from the thalamus. Now
the signals are also sent to the cerebral cortex for
more complex analysis and then sent to the amygdala.
JC
Curt Welch
> On Nov 26, 1:34=A0pm, c...@kcwc.com (Curt Welch) wrote:
> > That is typical of the type of "scripts" and
> > "algorithms" people code into their real robots.
> > But how do you blend these together?
>
> You don't blend them together anymore than you blend
> software modules. Think how you use modules in your
> programs.
That's the point. We DON'T "blend" modules". IF the modules we have don't
do what we want, we write MORE modules.
What you are not thinking about, or grasping, is the LEARNING aspect of
this problem. How do you make these hard coded modules do something new
which they were never written to do, when you don't have the programmer
around to make modifications? That's the issue that shows up when you try
to make strong learning work.
It's the issue I'm trying to make you think about when you have two modules
for walking, but nether of the two modules walks in the way required to
solve some new problem that wasn't around when those two modules were
created.
You, and I, know damn well how to hard code lots of fun modules to make
robots do jsut about anything. BUT HOW DO YOU ADD LEARNING!!! How do you
take hard coded modules, and add learning on top of it?
Though you have constantly argued that it works by adding learning at the
high level and that the learning uses lots of hard coded low level modules
to reduce how much the system has to learn, you have NEVER once explained
how that's actually going to work by showing some type of prototype (or
working) hardware to demonstrate how your words can actually be
implemented. My entire argument against your position is that it can't be
implemented as you suggtest. They only defense against my argument, if for
you to show us a working example of learning applied to hard coded low
level modules which produces some aspect of intelligent like behavior.
> > If you have code that makes it turn right, and another
> > "module" that makes it walk straight, how do blend
> > these two "MPGs" together so that it can dynamically
> > navigate from one side of the room to another in order
> > to get a bone?
>
> You don't have two modules you have one module and it
> is controlled by another
You don't have two, you have "one conttrolled by another"??? In my math, 1
+ 1 was always 2.
> module that uses it to achieve
> its own outcome.
>
> < snip a lot of nonsense >
>
> > But, EVERY TIME YOU WANT IT TO DO SOMETHING NEW - you
> > have to re-code, from the ground up, all the low level
> > innate systems.
>
> Rubbish. Think about how you write high level programs
> using modules.
Have you actually ever worked on large (10 million line) software projects
or are these 1000 lines AI programs your only real programmer experience?
In REAL code programs with large systems, you ALWAYS have to go back and
re-factor the design of your low level modules because something always
comes up later that simply wasn't supported by the way you structured the
modules when you wrote them.
If you are lucky, you factored them correctly and gave the low level
modules all the parameters needed so they can be used by the high level
code without change. But in the real world, that never works, and everyone
ends up having to modify, or replace the low level modules they wrote.
In trying to explain how systems implement human behavior which is adaptive
- that is it manages to implement functions that were never needed in the
past (like the complex "code" in our brain that allows us to wiggle our
fingers to communicate by typing on a keyboard) that clearly is not part of
our evolutionary past.
So what EXACTLY are the low level modules you think exist in the brain to
allows us to move our fingers, but, with learning applied on top of those
level level module, they can be trained to type words on a keyboard?]
It's one thing to take the position that _some_ (unspecified) innate module
helps us use our fingers. It's another thing entirely when you stop
calling it "some module" and actually back up you view with something real
- like working code examples - which I've done since the beginning and you
have never done.
> > The stuff humans can learn from experience (which could
> > not possibly have been needed for the past million years
> > while the innate brain systems evolved) invalidate the
> > approach of hard-coding these complex interactive
> > behavior systems as you seem to envision it might work.
>
> There is no suggestion that any high level "golf playing",
> "bike riding", "English instead of French speaking" and
> so on has be hard coded. We start with the same machinery
> as any other animal minus their high level innate behaviors.
> We also have extra connections etc.
>
> > Because of what the human brain can learn from experience
> > (like playing golf, or producing language on a keyboard
> > at the high speeds we can do it, or playing the piano by
> > sight reading sheet music), all the sorts of circuits I
> > talked about above have to be completely re-wired by the
> > effects of conditioning.
>
> They don't have to be rewired any more than the modules in
> your programs all have to be rewritten.
It sure sounds to me like you have never worked on real programming
projects. They are CONSTANTLY being rewritten - many times from the ground
up. How many times do you think the Unix operating systems has been
rewritten by this point in time? Hundreds. Was all the new Unix systems
developed by just adding more high level code and re-using the low level
subroutines written in the 70's? Shit no. The old code was thrown out,
and totally re-written because it wasn't correctly structured to support
some of the new features someone wanted to added. The original Unix for
example has zero network support. The BSD guys added network support, and
to make it fit in, they had to re-write how signals worked and had to
rewrite how process scheduling worked because the old low level routines
simply did not support the features needed for the new behaviors.
In large complex systems, when you learn new beahviors, you have to rewrite
the code at the LOWEST levels in order to make it work. When humans learn
some new radically different behavior - like "talking with your fingers
using a keyboard" large amounts of the old "talking with your mouth" code,
and "moving your fingers to eat" code had to have major modifications at
the lowest levels to make this new behavior work.
So, again, stop taking abstractly and give some real examples of how the
hell learning can be applied to the "low level modules" you believe exist
in the brain.
> > So the question is, how do we implement an algorithm
> > like the one I talked about above, that has a hierarchy
> > of learned control systems controlling each other, and
> > using each other in different combinations, all shaped
> > by a single reward signal?
>
> By a process of top down control.
>
> > The answer is that you need a hierarchical temporal
> > reaction engine that shapes it's reactions in response
> > to the context defined by the stimulus system as shaped
> > by a reward signal.
> >
> > It's nothing more than a big "what do I do know"
> > reaction machine, where "what do I do" is a very low
> > level question such as sorting a single pulse, or
> > making a single neuron fire.
> >
> > No OTHER type of machine structure that I've seen
> > suggested can explain how this is achieved - and as it
> > happens, the cerebral cortex seems to be structured just
> > like that.
>
> The basic machinery exists without the cerebral cortex.
> What the cerebral cortex seems to have added is increased
> analytical power and increased memory capacity, in
> particular episodic memories. Basic skills like riding a
> bike do not require the cerebral cortex except as the
> instigator to practice the skill in the first place.
And you know that how? Can you point to a human or an animal that can read
a bike who doesn't have a cortex???? Really John, what's your evidence to
support such an odd statement?
> Take a simple conditioning experiment. An audio tone
> associated with a shock. Without the cerebral cortex the
> animal will generalize over all sorts of audio tones.
> The cerebral cortex however can analyze the tones into
> many more complex categories. If you play one tune it
> means a shock another tune no shock but the actual
> association takes place in the amygdala. Before there
> was a cerebral cortex the audio and pain signals were
> all directed to the amygdala from the thalamus. Now
> the signals are also sent to the cerebral cortex for
> more complex analysis and then sent to the amygdala.
>
> JC
Well, again, I don't give a shit how the brain works and whether
conditioning takes place in the cortrex or somewhere else.
As I've explained, and argued, for many years here, the learnign problem
has two major aspects to it. One is recognition - which is an unsupervised
problem of learning to recognize the temporal patterns that actually show
up in the sensory data, and the second half is the conditioning by a reward
signal, which adjusts the response produced by the pattern. If it turns
out the cortex is mostly connected with the first half of the problem, and
the second half happens elsewhere, that's still consistent with all my
arguments.
My argument is that the pattern recognition, and the conditioning halves of
the problems, must work at an extremely low level of behavior - such at or
very near to the level of a single neuron firing or not firing. It can't
work at the level of controlling high level behavior generating modules
like walking, or reaching, or balancing because there's now reasonable way
to apply learning to those types of modules unless you are a competent
engineer and are actually re-designing the modules to solve some new
problem - like typing on a keyboard.
But if your position is that these modules were designed by a million years
of evolution, then it would be impossible to learn a new behavior like
speaking words on a keyboard, without another million years of evolution
working to change the design - which clearly isn't the case here.
So again, give us an example of some low level module which is like the
things you think that exist in the brain, and show how it's actually made
to be conditioned by a reward signal to do something it was never designed
to do.
Wolf K
> grey...@ruddy.moss wrote:
[...]
>> About the only thing I'm fairly sure of is that its
>> parts/subunits/modules/etc will be massively and multiply
>> interconnected. It will therefore not function as a digital computer,
>> but as a chaotic system. That will give it its free will (or whatever
>> you want to call that aspect of behaviour.)
>
> Well, I see nothing preventing it from both functioning as a digital
> computer and as a chaotic system (shaped by reinforcement). It's easy
> enough to write software which produces a chaotic system on a digital
> computer.
>
>> cheers,
>> wolf k.
>
As I see it, that's just implementation of the formulas, which ain't the
same thing at all.
cheers,
wolf k.
casey
I haven't responded to most of your comments for I don't
want another long winded exchange of conflicting views.
We don't really agree on most of the fundamentals. I see
your pulse sorting network as trying to solve problems
that only exist in your imagination.
> How do you make these hard coded modules do something
> new which they were never written to do, when you don't
> have the programmer around to make modifications?
Biological systems have always had the programmer
(evolution) around to make modifications so why not
let man made systems have that privilege as well?
> How do you take hard coded modules, and add learning
> on top of it?
Easy. You connect the output of your hard coded module
to the input of the learning module.
Your approach was that each module should be the same
generic learning module. Now evolution did not select
this approach. Why? Time and resources. Instead it took
advantage of environmental constraints and balanced
the learning and innate components at each level
according to the constraints of time and resources.
The highest level is mostly learning and the lowest
level is mostly hard coded.
Essentially the hard coded modules do "learn", not
within individuals, but rather within the species in
an evolutionary process be it biological systems or
man made systems.
Thus instead of the resources and the time limit of
one individual you have all the resources and time
of many individuals and thus finding the right set of
weights becomes more likely. But note each individual
has its own unique history so the solution has to
be based on what they all have in common, that is,
unchanging constraints in their environment.
> So what EXACTLY are the low level modules you think
> exists in the brain to allows us to move our fingers,
> but, with learning applied on top of those low level
> modules, they can be trained to type words on a keyboard?
Learning to play the piano, touch type, tie a knot and so
on start out as plans which involve many parts of the brain
including the reward or motivating components but most of
all it requires the cerebral cortex. The cortical system
makes, carries out and monitors motor plans. It can plan
ahead and test the desirability of predicted future states.
This global action exists in a time frame which is broken
into smaller time frames of principle movements which are
in turn broken into smaller time frames of minor movements
down to real time motor automatisms controlled by position
and force feedback.
Learning to write may start out with trial motor actions
with pen on paper. Visual feedback provides an error signal
to punish or reward the lower system's response to the higher
level command. Perhaps this is an example of your notion
of the lower modules being rewritten.
Where are these motor commands held? If you try and write
with the opposite hand to the one you normally use the output
is not as desired, particularly if you try and do it quickly.
What you want as output is clear but now you are sending
this command to different lower motor sections and they also
need to be trained. The important thing to note however is
that the goal is top down.
>> The basic machinery exists without the cerebral cortex.
>> What the cerebral cortex seems to have added is increased
>> analytical power and increased memory capacity, in
>> particular episodic memories. Basic skills like riding a
>> bike do not require the cerebral cortex except as the
>> instigator to practice the skill in the first place.
>
>
> Can you point to a human or an animal that can ride
> a bike who doesn't have a cortex????
What I wrote was the basic machinery (conditioning) exists
without the cerebral cortex. I also wrote that the cerebral
cortex _is required_ as an instigator to learning skills
such as riding a bike.
The cerebral cortex however is not required to learn
that a tone will precede an electric shock. Aplysia can
learn this without any cerebral cortex.
JC
Curt Welch
Well then, what do you think a chaotic system is and why do you think a
computer made of transistors can't create one?
Or even more interesting, why is it that intelligent behavior can't be
created by a digital computer (which seems to be what you are saying)?
Curt Welch
> On Nov 27, 12:28=A0pm, c...@kcwc.com (Curt Welch) wrote:
>
> I haven't responded to most of your comments for I don't
> want another long winded exchange of conflicting views.
Likewise. :)
Wolf K
> grey...@ruddy.moss wrote:
[...]
> computer made of transistors can't create one?
>
> Or even more interesting, why is it that intelligent behavior can't be
> created by a digital computer (which seems to be what you are saying)?
>
I think that an electronics engineer can create a chaotic system quite
easily. Too easily, actually. But that's not the same as writing a
program that implements some formulas. IMO, a multi-processor system can
easily become chaotic, because AFAIK it's not well understood how to
program a massively parallel system and keep it reasonably sane (if
that's the right word ;-).) What's done is to allot chunks of processing
to different processors, and timing the data interchange very carefully
to prevent chaotic behaviour.
Since you asked, here's how I understand chaos:
A chaotic system is one whose state changes continuously, such that its
state at any time T(n) depends on its state at T(n-e), where e is as
close to zero as you care to make it (but in practical models is quite
large.) In this it differs from systems whose state at T(n) can be
calculated if you know n. If the system cycles through a sequence of
states, the cycles will resemble each other more or less closely, ie,
they cluster around an average pattern. But eventually the system flips
into a different pattern of cycles. Your heart is chaotic in this sense,
BTW, as is the climate, the economy, large institutions, etc.
"Eventually" can be a very long time after T(0), eg, it's been estimated
that the Sun will burn out before the solar system flips into a
different state. (Sorry, can't give you a reference.)
There are of course "linear" chaotic systems, which collapse into a
different state some time after start up without cycling through a
sequence of similar states.
There's more to be said, of course, eg, the fact that it's not possible
to build an exact model of a chaotic system that is simpler than the
system itself, IOW, the only exact model is the system itself. This
means that chaotic systems behave like random systems. And that fact
turns some people's minds towards deep questions about predictability,
cause and effect, randomness and order, and so on.
It's fun to think about, really. ;-)
wolf k.
rs...@nycap.rr.com
> "rs...@nycap.rr.com" <rs...@nycap.rr.com> wrote:
> > > casey <jgkjca...@yahoo.com.au> wrote:
You have had a great deal of fun beating up on a straw-man. A straw-
man that you portray as a young, naïve worker in the life sciences,
uninformed about computers and robotics. You should know that I
started programming on the first supercomputer during the late
‘fifties, and spent the remainder of my working life programming
computers and robotic mechanisms.
http://www.columbia.edu/acis/history/norc.html
> > I think you two are arguing about personal beliefs, although I must
> > say Casey has the better of it.
>
> Yeah, we are mostly arguing what we believe to the the philosophy of AI.
> That is, how to approach finding a solution. Though we argue issues about
> the brain, the real point I always argue is NEVER about the brain, how it
> works, or what it does, but simply how to best approach the engineering
> problem of developing man-made machines to duplicate what we think of a
> human intelligent behavior.
In my sad experience, programming robotic mechanisms has nothing
whatsoever to do with intelligence, artificial or otherwise. It has
more to do with picking cotton bolls, and setting ten pins, and stuff
like that.
I was introduced, early on, to grown men who had stumbled on the
notion that they could hook up an S-R box to a computer, and find
TRUTH. So go ahead and train your neural net. You shall do no harm. In
my younger days, I also simulated neurons. I did no harm.
We live today in the century of molecular biology. So, if you want to
study intelligence in an up-to-date manner, look to the meat brain.
Computers have never done anything but grind out numbers. Computers,
however, can find a use in simulating brain circuitry.
> That might be true. That is, the idea that the wiring is fairly fixed,
> with the learning mostly taking place in the synapses. I seriously doubt
> that is true, but again, like I said above, I don't argue that level of
> implementation issues for the brain.
>
> I do, however, argue that your view of MPGs simply can't work (at least to
> my best ability to understand what your view is based on what you have
> written). You don't have a clue what you are talking about. You have
> taken a highly over simplistic idea of behavior being control by a a set of
> behavior generators and think that all learning can then be explained by
> starting, and interrupting, these MPGs. Its' just absurd. Try to program
> it and see what sort of behavior you can make a robot dog perform and you
> will soon see how unlike human and animal behavior your approach is.
To understand the brain, you must first grasp that all motor acts
originate in a motor program generator. There are no other neurons
that can make motor acts. Without the MPG’s, a man is just a lump of
clay. All the neurons in your trained net cannot lift a little
finger.
Take a look at
http://www5.pbrc.hawaii.edu/STG/STGoverview.html
for all the information an electrical engineer needs to wire up a
lobster stomach. And don’t put your fingers in your mouth, and say
that you are not interested in lobsters. This is the wave of the
future, and has been since 1960. This is how your heart beats. This is
how you breathe.
Look at
http://crab-lab.zool.ohiou.edu/hooper/
to see a description of a university lab that is deep into the
present.
In man’s brainstem and spinal column, there is circuitry for
breathing, facial expression, vocalization, licking, chewing, and
swallowing. Circuitry for reaching, grasping, and manipulating. For
orienting movements, eyes, head, and neck. For posture, and
locomotion. When these circuits are activated we do all the things you
call human.
Go ahead and pound on your S-R box, You shall do no harm.
Ray
casey
> ...
> We live today in the century of molecular biology.
It is unclear to me what your interest here is?
John
Wolf K
Artificial intelligence.
Which is not congruent with programs running on digital machines. Not
even programs simulating neurons (though such programs may be useful for
learning about intelligent machines.)
Actually, I doubt that digital machines will play much of a role in AI.
For one thing, they're too inefficient, and generate too much heat for
the mount of work they do. A meat computer is much more efficient. Eg,
the human brain runs at less than 100 watts. The visual system of a frog
runs so cool that the poor thing must rely on external heat to bring
itself to operating temperature. Etc. I suppose a digital calculator
would be a useful subunit - would avoid having to learn calculus from
scratch, for example.
cheers,
wolf k.
Curt Welch
> Curt Welch wrote:
> > grey...@ruddy.moss wrote:
> [...]
> > Well then, what do you think a chaotic system is and why do you think a
> > computer made of transistors can't create one?
> >
> > Or even more interesting, why is it that intelligent behavior can't be
> > created by a digital computer (which seems to be what you are saying)?
> >
>
> I think that an electronics engineer can create a chaotic system quite
> easily. Too easily, actually. But that's not the same as writing a
> program that implements some formulas. IMO, a multi-processor system can
> easily become chaotic, because AFAIK it's not well understood how to
> program a massively parallel system and keep it reasonably sane (if
> that's the right word ;-).) What's done is to allot chunks of processing
> to different processors, and timing the data interchange very carefully
> to prevent chaotic behaviour.
>
> Since you asked, here's how I understand chaos:
>
> A chaotic system is one whose state changes continuously,
Well, ok, if you define chaotic as requiring continuous behavior, then
sure, discrete digital computers don't implement continuous (analog)
functions.
> such that its
> state at any time T(n) depends on its state at T(n-e), where e is as
> close to zero as you care to make it (but in practical models is quite
> large.) In this it differs from systems whose state at T(n) can be
> calculated if you know n. If the system cycles through a sequence of
> states, the cycles will resemble each other more or less closely, ie,
> they cluster around an average pattern. But eventually the system flips
> into a different pattern of cycles. Your heart is chaotic in this sense,
> BTW, as is the climate, the economy, large institutions, etc.
> "Eventually" can be a very long time after T(0), eg, it's been estimated
> that the Sun will burn out before the solar system flips into a
> different state. (Sorry, can't give you a reference.)
>
> There are of course "linear" chaotic systems, which collapse into a
> different state some time after start up without cycling through a
> sequence of similar states.
>
> There's more to be said, of course, eg, the fact that it's not possible
> to build an exact model of a chaotic system that is simpler than the
> system itself, IOW, the only exact model is the system itself.
Yes, that sounds valid.
But that doesn't mean a digital system or a model is not in itself chaotic
- just a different chaotic. Unless of course you just define the term to
_require_ analog state changes.
> This
> means that chaotic systems behave like random systems. And that fact
> turns some people's minds towards deep questions about predictability,
> cause and effect, randomness and order, and so on.
>
> It's fun to think about, really. ;-)
>
> wolf k.
Well, in the above, your definition used the words "can calculate" to
define what a chaotic vs non-chaotic system is. That of course is a fine
intuitive idea to get a basic grasp of what you are thinking, but not one
really helps us nail down precisely what we are talking about.
Does "can calculate" just mean, "too complex for a human brain to predict"?
Or too complex for any machine man has yet made to make predictions?
So is the limit not that it can't be calculated in theory, but that it's
just not _practical_ with current known technology to calculate it?
What if, the entire universe as we know it is just some sort of simulation
in a machine in some universe we have no awareness of yet, and in that
universe, there are many such machines, and they can each calculate and
predict with ease what the simulation on this machine will do? In which
case everything in this universe could be "calculated" before it happened.
I don't know if anyone else has done a better job of defining the line
between chaotic and none chaotic systems but I suspect there is no clear
line there. I believe the concept is just connected with large systems
that are beyond easy calculations and who's behavior tends not to average
out into some simple to describe and calculate behaviors - as happens when
a rock falls though air for example. Though the precise tumbling and speed
of the rock is very chaotic due to shifting air flow patterns the average
behavior of the falling rock can be described and predicted with very
simple Newtonian laws.
When you add multiple layers of feedback in to a system, it quickly becomes
complex in a way that makes it very difficult to "calculate". That's how
all encryption programs work. They are intentionally created "chaotic" in
that way because they are difficult to calculate the next step if you don't
know the full internal state of the system and you can't easily calculate
the last state (decrypt it). I think it's quite valid to call a computer
running an algorithm like that a chaotic system. But the state space is
highly discrete so it mostly fails your requirement of being continuous.
However, at the lower levels, it is continuous, but it's built to average
out the chaotic behaviors that exist at that level so at the abstraction
level of the running the program, we find highly simple and predictable
behavior - before we re-create chaotic behavior at even higher levels by
running a chaotic program. SO I think it's fairly hard with all this going
on in a computer to really argue it's "not chaotic" unless you can find a
more precise way to nail down the line between chaotic and non-chaotic
systems.
On the question of the brain, yes I would say it's basically a chaotic
system shaped by reinforcement in order to make it act in a useful way -
even though it's underlying engine is chaotic. And I think there's enough
analog implementations in the brain to allow it to fit your requirement of
continuous. But I don't agree that this all means we won't be able to
duplicate that type of behavior in a digital computer. We can create the
same sorts of chaotic engine in a computer, and we can I'm sure shape it at
the high level using a global reinforcement signal to make the chaotic
engine produce human-like intelligent behaviors.
Curt Welch
> On Nov 25, 8:52 pm, c...@kcwc.com (Curt Welch) wrote:
> > "rs...@nycap.rr.com" <rs...@nycap.rr.com> wrote:
> > > > casey <jgkjca...@yahoo.com.au> wrote:
>
> You have had a great deal of fun beating up on a straw-man. A straw-
> uninformed about computers and robotics. You should know that I
> started programming on the first supercomputer during the late
> computers and robotic mechanisms.
I was expecting you to say something like that. :)
Having programmed for 50 years doesn't mean you are good programmer
however. :)
> http://www.columbia.edu/acis/history/norc.html
>
> > > I think you two are arguing about personal beliefs, although I must
> > > say Casey has the better of it.
> >
> > Yeah, we are mostly arguing what we believe to the the philosophy of
> > AI. That is, how to approach finding a solution. Though we argue
> > issues abou=
> t
> > the brain, the real point I always argue is NEVER about the brain, how
> > it works, or what it does, but simply how to best approach the
> > engineering problem of developing man-made machines to duplicate what
> > we think of a human intelligent behavior.
>
> In my sad experience, programming robotic mechanisms has nothing
> whatsoever to do with intelligence, artificial or otherwise. It has
> more to do with picking cotton bolls, and setting ten pins, and stuff
> like that.
>
> I was introduced, early on, to grown men who had stumbled on the
> notion that they could hook up an S-R box to a computer, and find
> TRUTH. So go ahead and train your neural net. You shall do no harm. In
> my younger days, I also simulated neurons. I did no harm.
>
> We live today in the century of molecular biology. So, if you want to
> study intelligence in an up-to-date manner, look to the meat brain.
I don't have any problem with people studying the brain. They should of
course. And anything they can find to help us engineer smarter machines is
good. But from trying to build smart machines, I know the stuff you write
here is of no use to us (at least not in the form you present it here or
the web sites you point us to).
> Computers have never done anything but grind out numbers. Computers,
> however, can find a use in simulating brain circuitry.
>
> > That might be true. That is, the idea that the wiring is fairly fixed,
> > with the learning mostly taking place in the synapses. I seriously
> > doubt that is true, but again, like I said above, I don't argue that
> > level of implementation issues for the brain.
> >
> > I do, however, argue that your view of MPGs simply can't work (at least
> > t=
> o
> > my best ability to understand what your view is based on what you have
> > written). You don't have a clue what you are talking about. You have
> > taken a highly over simplistic idea of behavior being control by a a
> > set =
> of
> > behavior generators and think that all learning can then be explained
> > by starting, and interrupting, these MPGs. Its' just absurd. Try to
> > progra=
> m
> > it and see what sort of behavior you can make a robot dog perform and
> > you will soon see how unlike human and animal behavior your approach
> > is.
>
> To understand the brain, you must first grasp that all motor acts
> originate in a motor program generator. There are no other neurons
> that can make motor acts. Without the MPG=92s, a man is just a lump of
> clay. All the neurons in your trained net cannot lift a little
> finger.
>
> Take a look at
>
> http://www5.pbrc.hawaii.edu/STG/STGoverview.html
>
> for all the information an electrical engineer needs to wire up a
> lobster stomach. And don=92t put your fingers in your mouth, and say
> that you are not interested in lobsters. This is the wave of the
> future, and has been since 1960. This is how your heart beats. This is
> how you breathe.
>
> Look at
>
> http://crab-lab.zool.ohiou.edu/hooper/
>
> to see a description of a university lab that is deep into the
> present.
>
> In man=92s brainstem and spinal column, there is circuitry for
> breathing, facial expression, vocalization, licking, chewing, and
> swallowing. Circuitry for reaching, grasping, and manipulating. For
> orienting movements, eyes, head, and neck. For posture, and
> locomotion. When these circuits are activated we do all the things you
> call human.
Yes, it's not at all surprising that these important behaviors that have
been key to our survival long before we became anything near intelligent
(eating and breathing and heart pumping) are controlled by hard wired
neural circuits. They had to be created to keep a complex life form alive
long before evolution added on the strong generic learning systems that
made us intelligent.
But where is the bike riding, car driving, and keyboard typing MPGs? They
aren't there ray. They can't be there because those behaviors haven't been
a part of human life long enough for evolution to have built them.
So that means we have a high level generic learning system, that must use
the low level MPGs to learn all these new behaviors. How useful is the
hand-eye "grasping" MPGs for touch typing on a computer keyboard? Not
much.
There's much happening when we type other than the normal pushing with our
fingers - which could be a low level MPG. There must be hardware in our
brain that has been wired to produce all the complex finger and wrist
actions to type all the words we know know how to type - as well as
circuits that interconnect our language centers of the brain with whatever
circuits allow us to make these coordinated finger motion sequences.
NONE OF THAT IS A GENETIC MPG build by a million years of evolution. All
the wiring/configuration of that circuitry had to be created by
_conditioning_.
I can write lots of "modules" like John talks about to make a robot do lots
of important things like balance on two legs and move forward. And it's
easy to justify that I'm just performing the role of evolution as I design
these modules. But if I then want to make my robot type words on a
keyboard, I have to make some fairly drastic changes to all the code I've
already written to allow it to do something like stand up and balance
itself while at the same time typing on a keyboard.
The truth is, when you develop "modules" in software to make it "easier" to
write high level code - it's impossible to write modules to do _everything_
you need to do. No matter how large the library of "modules" is, when you
add a new function to your software (or robot), you still end up having to
use the lowest levels "modules" of the computer, like addition, and if
statements because there are no higher level modules to do what you need to
do.
How does this use of modules work when the changes to the system is not
done by evolution over millions of years (in which case we could just write
it off as - evolution redesigned it) - but instead, the changes have to
happen by conditioning?
I can tell you it's impossible to make much use of those medium to high
level MPGs for most of our learned behaviors. Most of our learned
behaviors have to be created from extremely low level behaviors.
> Go ahead and pound on your S-R box, You shall do no harm.
Go ahead and read about the brain - you shall do no good. :)
It makes no difference what the low level behaviors are - whether it's
pulse sorting as in my AI nets, or your MPGs for grasping. Either way, on
top of those low level behaviors, we have to figure out how to create
strong generic learning though conditioning. Nothing you have written here
gives us any hint how to implement that.
We know how to build MPGs into our robots. It's what everyone has been
doing for 1000 years. What no one has done yet, and what we have to solve
in order to make them act intelligent, is how to build strong learning on
top of whatever low level innate behaviors you think are best to learn
with. We can't solve AI by building more MPGs. We have to stop doing
that, and figure out how to do the hard stuff - the generic learning.
casey
>
> But where is the bike riding, car driving, and keyboard
> typing MPGs?
>
>
> typing on a computer keyboard? Not much.
What is useful is hand eye coordination. In order to type
you must direct your finger to the right location. To
learn a motor program it must first be imagined. It must
then be carried out by direction until it becomes automatic
with practice. Practice allows the lower systems to learn
via feedback how to carry out the motor program instigated
and planned by higher centers.
> There must be hardware in our brain that has been wired
> to produce all the complex finger and wrist actions to
> type all the words we know how to type -
The sequences of keys to tap are not in the motor program.
The sequence, held elsewhere, directs the motor program.
If you want to hit a key you first locate it. Move the finger
to the right position and then carry out the tap action.
The key, or sequence of keys, you want to hit are not in
the low level motor program.
> NONE OF THAT IS A GENETIC MPG build by a million years
> of evolution. All the wiring/configuration of that
> circuitry had to be created by _conditioning_.
And I began the discussion of what physically occurs when
novel behaviors are learned and how this learning is
directed and fixed by the reward system.
> But if I then want to make my robot type words on a keyboard,
> I have to make some fairly drastic changes to all the code
> I've already written to allow it to do something like stand
> up and balance itself while at the same time typing on a
> keyboard.
Or you simply add your typing program to the list of things
the robot can do. If it has to stand and type at the same time
then both programs can be running at the same time.
> We have to stop doing that, and figure out how to do the
> hard stuff - the generic learning.
And studying brain systems can show you possible hardware
or software solutions to doing just that.
JC
casey
> casey wrote:
> On Nov 28, 12:25 pm, "rs...@nycap.rr.com" <rs...@nycap.rr.com> wrote:
>> ...
>>> We live today in the century of molecular biology.
>>>
>>> It is unclear to me what your interest here is?
>
>
>
> Which is not congruent with programs running on digital
> machines. Not even programs simulating neurons (though
> such programs may be useful for learning about intelligent
> machines.)
A simulated cyclone cannot blow down a house but a robot
can fetch a can of beer. I don't see AI as a simulation.
A lot has to do with your notion of intelligence.
I don't see "intelligence" in the singular. There is no
one solution. It is a category word for certain kinds
of behavior.
> A meat computer is much more efficient.
Not relevant. The efficiency is either efficient enough
or it isn't for any given task.
JC
Curt Welch
> On Nov 29, 2:56=A0am, c...@kcwc.com (Curt Welch) wrote:
> >
> > But where is the bike riding, car driving, and keyboard
> > typing MPGs?
> >
> >
> > How useful is the hand-eye "grasping" MPGs for touch
> > typing on a computer keyboard? Not much.
>
> What is useful is hand eye coordination. In order to type
> you must direct your finger to the right location. To
> learn a motor program it must first be imagined.
I see no evidence at all to suggest that. I've learned many physical
actions in my life without ever once thinking about what it was I was
trying to learn. I don't for example remember thinking about how to stand
up, or sit down, or stick my fingers in my mouth. But somehow, these
skills were learned.
> It must
> then be carried out by direction until it becomes automatic
> with practice.
That's just not true at all. Yes, that's one way we do learn things, but
it's not the normal way in any sense. You don't learn to balance and
control your direction on snow skis by first thinking out the algorithm for
not falling over. You just put yourself on skis, and keep getting up every
time you fall. In the end, you tend to start falling less without no real
understanding, or prior thought about how you are doing it.
> Practice allows the lower systems to learn
> via feedback how to carry out the motor program instigated
> and planned by higher centers.
Yes, we do have the power to control motor actions using what you call the
higher center of our brain. And as we do that, other centers of our brain
tend to re-configure themselves so the behavior becomes more automatic. But
that is not, in any sense, the primary way we learn new physical actions.
> > There must be hardware in our brain that has been wired
> > to produce all the complex finger and wrist actions to
> > type all the words we know how to type -
>
> The sequences of keys to tap are not in the motor program.
> The sequence, held elsewhere, directs the motor program.
Why do you say some of the hardware controlling our actions is a "motor
program" and other parts of the hardware controlling our actions is not
"motor program"? How do you know the brain is actually divided like that
and what name do you give to the part of the hardware that you say is non
motor program?
> If you want to hit a key you first locate it. Move the finger
> to the right position and then carry out the tap action.
> The key, or sequence of keys, you want to hit are not in
> the low level motor program.
you my type by some "hunt and peck" method. I don't. I just talk to
myself and the fingers fly and the words come out on the screen. I never
for the most part think about what keys I'm hitting or where they are
located. It became highly automatic for me decades ago. I've now got some
hardware in my brain somewhere that translates words into finger actions
almost directly.
> > NONE OF THAT IS A GENETIC MPG build by a million years
> > of evolution. All the wiring/configuration of that
> > circuitry had to be created by _conditioning_.
>
> And I began the discussion of what physically occurs when
> novel behaviors are learned and how this learning is
> directed and fixed by the reward system.
Yes, you claim something to the effect that we can't learn anything unless
we direct it first by our "higher functions". First, no one I know
believes that's how it works. Second, in deferring the "learning" to
"desires of the higher level system", you have just pushed the problem of
how the low level system learns, into a question of how the high level
system learns without actually making any progress on explaining how
learning works. All you have done is said "it happens elsewhere". How
does the higher level system learn what to try to make the low level system
learn?????? You have the same problem explaining the high level system
left totally unanswered.
> > But if I then want to make my robot type words on a keyboard,
> > I have to make some fairly drastic changes to all the code
> > I've already written to allow it to do something like stand
> > up and balance itself while at the same time typing on a
> > keyboard.
>
> Or you simply add your typing program to the list of things
> the robot can do. If it has to stand and type at the same time
> then both programs can be running at the same time.
Sure, John, that's easy to say. Now actually try to code it. If you have
done any real coding on large systems, you would understand that what you
are suggesting can't work. You can't just write 100 different "behavior
scripts" and then make as many run at the same as you like and end up with
something that actually works. It's as absurd as trying to write to
printer drivers and trying to make then "run at the same time" controlling
a single printer.
How do you resolve the conflicts when the 10 behavior scripts you are
running at the same time all want to move the left arm in a different
direction?
This is the fundamental problem I've been trying to get you to think about,
and answer, for years now, and you just write it off as if there's no
problem there.
> > We have to stop doing that, and figure out how to do the
> > hard stuff - the generic learning.
>
> And studying brain systems can show you possible hardware
> or software solutions to doing just that.
Which is exactly why I have NEVER suggested someone should not study the
brain. I've only said I put no faith in people studying the brain solving
these fundamental questions because the standard way they all seem to work
is make up shit like "just run the modules at the same time" because it
sounds like a good answer and because they have never actually tried to
make such a system work on their own.
Yes, somehow, the brain does have to merge together multiple learned
behaviors so it can walk and chew gum at the same time - and so it can use
previously learned behaviors to boot-strap the learning of a new more
complex behavior. BUT HOW! That's the question that must be answered
other than doing what you and Ray do which is wave your hand and say "it's
easy" (just stop and start the MPGs at the right time) or the other defense
both of you have used at times "What evolution created is too complex for
us to understand".
I believe the only workable answer is closer to what Wolf was just talking
about. That is, I think it needs to have a chaotic behavior generator at
the core, and it needs to be configured so that generator can be shaped by
a reinforcement signal. What state (aka what behavior) the generator
produces next is always a very complex function of what sates it was just
in. And that mapping function which causes it to drift from one state to
the next is what gets shaped by years of reinforcement. At any point in
time, there aren't just 1 or 2 or 10 "MPGs" running in parallel. There are
literally millions of past learned experiences at work shaping the systems
current path though state space.
You can't implement the type of behaviors we produce by running behavior
scripts in parallel controlling the same hardware. But by shaping a huge
parallel state space transition function by reinforcement, you can produce
the type of smooth life-like behaviors we see in humans and animals.
The type of architecture you and Ray keep suggesting is just so far off
base it's not funny. People have tried long ago to implement such ideas in
robots and the robots either don't work at all when they try it, or it ends
up producing behavior that looks nothing like real animals.
Curt Welch
If efficiency was a problem, then yes, we might not be able to equal _full_
human behavior with computer parts because it might take more power than we
have available on the earth to drive it. But if that was the only thing
that was stopping us, we would still be able to build giant power hungry
machines that demonstrated simplistic animal-like behaviors - which no one
has yet done at any power level. It just seems totally invalid to argue
that efficiency has anything to do with the problem at this point. It's
just not a lack of power that's keeping our robots from acting like humans.
From most ways of measuring it, computers and electronics are far more
efficient than biological systems anyhow. Transistors can switch with far
less energy than what it takes to make a neuron fire.
The problem is that it's a type of machine we don't yet understand how to
build. As I just wrote in the other post, I think it's a machine that can
better be seen as a chaotic behavior generator that works in parallel but
which gets conditioned with a global reward signal that biases the path
though state space the machine likes to travel - where the "path though
state space" is the combined internal signals and outputs it produces in
response to the stimulus signals. It's got to be some type of constrained
chaotic system where the constraints get adjusted by conditioning. My
networks have all been attempts to create such systems.
zzbu...@netscape.net
> Curt Welch wrote:
> > greyw...@ruddy.moss wrote:
> [...]
> > Well then, what do you think a chaotic system is and why do you think a
> > computer made of transistors can't create one?
>
> > Or even more interesting, why is it that intelligent behavior can't be
> > created by a digital computer (which seems to be what you are saying)?
>
> I think that an electronics engineer can create a chaotic system quite
> easily.
Electronics engineers can create chaotic systems so easily, that's
why the real enginners invented Atomic Clock Wristwatches and HDTV,
rather than Physics.
And Digital Books, Holograms, and Home Broadband, rather than
AT&T.
And GPS, Digital Terrain Mapping, Cyber Batteries, Weather
Satellites, Data Fusion,
Desktop Publishing, Self-Assermbling Robots, Thermo-Electric
Cooling,
and Rapid-Prototyping, rather than GM.
And UAVs, Multiplexed Fiber Optics, mp3, mpeg, Blue Ray. USB,
Phalanx,
Self-Replicating Machines, and The 21st Century, rather than the
navvy.
casey
> ...
> I don't for example remember thinking about how to stand
Remembering "how to do something" is different to remembering
"learning how to do something".
We have more than one memory system. How do we know this?
One can be deactivated without having an effect on the others.
A subject called HM, after surgical removal of part of his
temporal lobe, which included damage to his hippocampus, was
unable to remember anything that he experienced just minutes
earlier. And yet his IQ was normal and he could be taught
new motor skills such as drawing while watching his hand in
a mirror which he mastered at almost the normal rate. However
he could never remember learning the skill even though it may
have just taken place an hour before!
> How does the higher level system learn what to try to
> make the low level system learn??????
It learns what to try, the same way all the systems learn
what to try, by feedback. The primal reward system of the
brain operates at all levels but on things that have a
more and more abstract and tokenized representation as you
move up levels.
Will I learn "to play the piano"? Depends how rewarding that
high level "learn to play the piano" is to me at the time.
JC
casey
> You can't implement the type of behaviors we produce
> by running behavior scripts in parallel controlling
> the same hardware. But by shaping a huge parallel
> state space transition function by reinforcement,
> you can produce the type of smooth life-like behaviors
> we see in humans and animals.
>
>
> that can better be seen as a chaotic behavior generator
> that works in parallel but which gets conditioned with
> a global reward signal that biases the path though
> state space the machine likes to travel - where the
> signals and outputs it produces in response to the
> stimulus signals. It's got to be some type of
> constrained chaotic system where the constraints get
> adjusted by conditioning. My networks have all been
> attempts to create such systems.
Whereas I see that as fundamentally the wrong approach
and contrary to what we see in the evolution of nature
which is highly modular with minimal interactions.
You can chew gum and walk at the same time providing
different hardware is being used. When two processes
are in conflict with each other it has to be resolved.
An example of such conflict is the Stroop test in
which there can only be one winner no merging.
If two processes both want to use the left arm they
will compete for it and a selection mechanism will
choose which one wins.
If one ear is being fed one conversation and the other
ear another conversation then when they both reach
a common piece of hardware at the top level only one
will selected. You can only pay attention to one
conversation at a time although up to some level they
are both being processed in parallel.
All brains start off with default connections which
have been determined by the evolutionary history of
the species and fine turned by sensory input during
the development of each brain.
Each brain uses the sensory input to construct an
internal reality which it then evaluates in order
to determine what action to take.
A simplified idea of this might be:
stimulus ---> world ---> evaluate ----> action
stimulus ---> bear ----> bad ----> retreat
stimulus ---> berries ---> good ----> approach
This can be modified by learning but the actual
architecture remains the same limiting what each
species can ever learn.
JC
rs...@nycap.rr.com
> On Nov 29, 9:49 am, c...@kcwc.com (Curt Welch) wrote:
On Nov 27 you wrote:
> It is unclear to me what your interest here is?
Your post here is part of my interest.
I listened to many eminent men (some of whom I would class as
brilliant) as they explained how a man exhibited intelligence, and how
they could reproduce it on a machine. Starting, let’s say, with
McCullough.
The machines they had at hand were pitiful. The supercomputer, I was
programming, was short three or four orders of magnitude of the
capabilities of the present day desktop, and their machines were
barely mechanized abacuses. But they were very, very smart…brilliant.
They knew what thought was; they had codified it in the predicate
calculus. All that was needed was to program a logic engine that would
mechanize the manipulative algebra of the calculus. Minsky disagreed.
He thought he had found intelligence in clever programming. He had
graduate students who were very clever, and they could do the work.
Half a century has gone by, and nothing has worked.
Today, everyone has a desktop, or a laptop, with unbelievable
capabilities, and still nothing works. So, my interest lies in
listening to what people say today, and regurgitating my personal
notions on where intelligence lies.
> > …. But by shaping a huge parallel
> > state space transition function by reinforcement,
> > you can produce the type of smooth life-like behaviors
> > we see in humans and animals.
Oh, yes. Parallel computation. Way to go.
> > My networks have all been
> > attempts to create such systems.
lol
> Whereas I see that as fundamentally the wrong approach
> and contrary to what we see in the evolution of nature
> which is highly modular with minimal interactions.
I agree completely. But don’t think of it as programming, but rather
as circuitry. Think of it as an electrical engineer haywiring up
discrete components.
> All brains start off with default connections which
> have been determined by the evolutionary history of
> the species and fine turned by sensory input during
> the development of each brain.
Yes! The genome contains the description of circuitry that managed to
navigate the environment in the past. These motor program generators
worked before, and maybe they will work again. Otherwise, the species
disappears.
The mechanisms of axonal growth and synaptic formation and
modification that are also written in the genome allow the nervous
system to adapt to a slightly different environment. These worked in
the past, and that’s all the genome has to go on.
So, lets look at the MPG’s to move the organism, a cerebellum to
handle any clashing, and a thalamic reticular nucleus to stop any
motor program that caused trouble with this particular environment in
the past. This is how we go bumbling through life.
You may call it intelligence. Why not?
Ray
casey
> On Nov 29, 10:29 pm, casey <jgkjca...@yahoo.com.au> wrote:
> I listened to many eminent men (some of whom I would
> class as brilliant) as they explained how a man exhibited
> intelligence, and how they could reproduce it on a machine.
> ...
> They knew what thought was; they had codified it in the
> predicate calculus. All that was needed was to program a
> logic engine that would mechanize the manipulative algebra
> of the calculus.
If you have a clear understanding of how you manipulate
symbols you can program it. As brilliant men they had a
clearer understanding of these things than the rest of
the population. However they had no understanding of how
the brain did things at the lower levels so not having
access to this information they couldn't program it.
> Minsky disagreed. He thought he had found intelligence
> in clever programming. He had graduate students who were
> very clever, and they could do the work.
I don't think intelligence is to be found. It is a category
word for a set of actions that achieve something be it
finding food or solving an equation. Each achievement uses
a different set of skills. You can be brilliant in one area
and dumb in others.
The confusion I think in that "being alive" is singular and
is equated with "being intelligent" or "sentient". However
you can be very much conscious or "alive" without showing
any intelligent behavior and when behaving "intelligently"
you are not any more alive than before. Of course the brain
does have degrees of alertness and for any given brain a
certain alertness is required to bring its full capacity
online.
> Half a century has gone by, and nothing has worked.
I think AI programs have shown sets of behaviors I would
categorize as intelligent but would not make them "alive".
Their ability to modify their behaviors (learn) is rather
limited and this is the important intelligent behavior
Curt wants to emulate.
>> Whereas I see that as fundamentally the wrong approach
>> and contrary to what we see in the evolution of nature
>> which is highly modular with minimal interactions.
>
>
> I agree completely. But don’t think of it as programming,
> but rather as circuitry. Think of it as an electrical
> engineer hardwiring up discrete components.
Well I see programming as wiring up the hardware of a
general purpose computer and thus equivalent to circuitry
except of course it runs much slower.
JC
Curt Welch
Yeah, and I think that since people have tried the wrong _type_ of
algorithms, the results haven't been close. Though the field of AI has
produced tons of examples of human-like behaviors in very limited domains
(like chess playing), nothing has come close to the general powers and
types of behaviors human have. I think Ray's classification as "nothing
has worked" is fairlly accurate in terms of how far away even the best AI
programs currently are from looking "alive" (as John said).
But I think we are going to see a breakthrough at some point where
suddenly, we get some of the right type of algorithms working, and then
everything these algorithms produce will "look alive".
On a scale of 0 to 10, where 10 is full human behavior, our best AI
programs are still lucky to get a score of 1. And that's for everything
even close to AI-like that's been done in my view. But I think when we get
the right type of algorithms working, suddenly, they will all look like a 7
or 8 on the scale of 1 to 10 because they will look, and act, very much as
if they were alive - even if their total intelligence is still highly
limited. At that time, I think it will become obvious to everyone what
"intelligence" is - that is, what the driving force behind animal and human
behavior is - it will be this new class of algorithms.
> >> Whereas I see that as fundamentally the wrong approach
> >> and contrary to what we see in the evolution of nature
> >> which is highly modular with minimal interactions.
> >
> >
> > I agree completely. But don=92t think of it as programming,
> > but rather as circuitry. Think of it as an electrical
> > engineer hardwiring up discrete components.
>
> Well I see programming as wiring up the hardware of a
> general purpose computer and thus equivalent to circuitry
> except of course it runs much slower.
>
> JC
Yeah, I've made this point may times. Programming is a physical act of
re-building physical hardware. It is JUST LIKE an electrical engineer
wiring up hardware - except that it's limited to discrete digital hardware.
and the re-wiring process is limited to moving elections from one place to
another in the hardware instead of moving wires and transistors.
And no, it's not always "a lot slower". Our modern computers are so fast
and do so much work in parallel that they can actually perform some digital
functions quite a bit faster than simple digital logical hardware wired by
hand.
I was once going to build some DES decryption hardware using TTL gates
once, but when I calculated how fast it could run, I found out the computer
chips at the time could do it faster in software. And that was back in the
90's. They are much faster now. The speed gain came from the fact that
you have to build circuits at the chip level with very short interconnect
wires to make it run as fast as our computers do. You can bread-board a
simple logic circuits anymore and make then run as fast. You have to build
custom LSI chips to equal the speed of what a modern processor can do with
just software modifications of the chip.
Some people think the analog aspect is important and will prevent us from
building the correct circuits using processors but I just don't buy that.
At least, I don't agree that we can't in theory specify the algorithm using
normal digital algorithms. We might, for practical reasons, end up
implementing it with some analog hardware for size and cost and power
reasons - but not because it's impossible to specify or build with digital
hardware. However, knowing how fast and cheap and low power (per
computation) our digital technology already is, I don't even believe that
will happen. It's easier and faster to use these cheap, but highly
optimized general processors to create parallel signal processing hardware
than to try and custom design new ICs.
Once the technology is proven with off the shelf CPU hardware, and there's
an obvious market for a million of these things, then it might become cost
justifiable to custom design some ICs.
casey
> So, lets look at the MPG’s to move the organism, a cerebellum to
> handle any clashing, and a thalamic reticular nucleus to stop any
> motor program that caused trouble with this particular environment in
> the past. This is how we go bumbling through life.
The problem is there is not much we can do with
your declarations about where things are happening
as we need to know how they are happening or how
we might get them to happen in enough detail to
actual test the idea.
For example if we have a set of modules that can
carry out various motor tasks how are they selected
and given the current state (data) so that they
can adjust themselves to the task.
For example if the task is to draw a circle you
might have a circle drawing module. However it
might also need the location and size of the circle
to be drawn. It in turn may call on other modules
to achieve its motor output.
I would say that first the sensory input is decoded
and the code acts to select both the module and the
data required by the module to allow it to carry out
its task. Although it may be implemented in hardware
I see it very much the way I write programs. We
select and call a module according to the current
gaol and input state and pass data to it. That
module in turn may or may not use shared modules
to achieve the desired outcome.
If you have a set of innate circle drawing motor
modules you can build another module or the system
can learn to build another module that makes use
of them to achieve a more complex motor output.
JC
casey
> casey <jgkjca...@yahoo.com.au> wrote:
>> Their ability to modify their behaviors (learn) is rather
>> limited and this is the important intelligent behavior
>> Curt wants to emulate.
>
>
> Yeah, and I think that since people have tried the wrong
> _type_ of algorithms, the results haven't been close.
> Though the field of AI has produced tons of examples of
> human-like behaviors in very limited domains (like chess
> playing), nothing has come close to the general powers
> and types of behaviors human have.
But this is not solved by going back to the notion of a
monolithic network of nodes as the sole computing agent
to implement reinforcement learning.
You see such a network in the neocortex, whereas I see
a collection of computing units that by themselves are
as useless as an array of flip flops in a memory circuit
without the address decoding circuitry, devoid of a program
and the instruction decoding circuitry or a cpu to carry
out any tasks.
Yes the neocortex IS required for human level learning.
We know it enables finer distinctions and generalizations
to be made for the analysis of the sensory input, for the
reward system and for the construction of motor programs.
Also you don't seem to appreciate the power of modules
even though you may practice object orientated programming.
When I tried to evolve a 2-bit adder using random connections
between NOR gates in a "monolithic" network I got sick of
waiting for it to happen. However when I broke the problem
down into evolving an XOR gate for the sum and an AND gate
for the carry it took place in a reasonable time.
This has been observed in other evolutionary networks.
In a monolithic network if one unit can be used in different
tasks then when it is changed for one task it messes up
its role in other tasks. This is what happens in the nets
you suggested. If you change weights for one outcome it
messes up any weight settings for other outcomes. It would
be like a radio in which adjusting the volume knob would
result in a change in the frequency setting and thus
requiring a change in the frequency setting which in turn
would change the volume setting, only many time worse.
In the biological world units show the autonomous properties
of wholes and the dependent properties of parts.
JC
Curt Welch
> On Dec 1, 8:24=A0am, c...@kcwc.com (Curt Welch) wrote:
> > casey <jgkjca...@yahoo.com.au> wrote:
> >> Their ability to modify their behaviors (learn) is rather
> >> limited and this is the important intelligent behavior
> >> Curt wants to emulate.
> >
> >
> > Yeah, and I think that since people have tried the wrong
> > _type_ of algorithms, the results haven't been close.
> > Though the field of AI has produced tons of examples of
> > human-like behaviors in very limited domains (like chess
> > playing), nothing has come close to the general powers
> > and types of behaviors human have.
>
> But this is not solved by going back to the notion of a
> monolithic network of nodes as the sole computing agent
> to implement reinforcement learning.
>
> You see such a network in the neocortex,
Well, actually, I don't see such a network by looking at what is known
about the neocortex. None of the current evidence about brain function
indicates that's what the neocortex is doing.
I see such a network because I attacked the problem from a different
direction. That is, from the problem of how to you build a machine, that
can learn to trigger any set of moves (aka behaviors) the body is able to
make, in response to any complex stimulus signal? The only way I've so far
figured out how to create such a machine is with a large fairly monolithic
network. I then look at the cortex and say, "gee, it's one large module
with repeated low level modules thoughout the entire cotex, maybe that's
what is is!".
So yes, I strongly _suspect_ that the underlying technology of the
neocortex is a large generic learning system - but I've not studied the
brain - or even studied the research of others enough to argue what it's
really doing.
> whereas I see
> a collection of computing units that by themselves are
> as useless as an array of flip flops in a memory circuit
> without the address decoding circuitry, devoid of a program
> and the instruction decoding circuitry or a cpu to carry
> out any tasks.
>
> Yes the neocortex IS required for human level learning.
> We know it enables finer distinctions and generalizations
> to be made for the analysis of the sensory input, for the
> reward system and for the construction of motor programs.
Well, do you grasp that that is exactly what I've been explaining to you I
think it is since I came to this group years ago?
> Also you don't seem to appreciate the power of modules
> even though you may practice object orientated programming.
I really don't grasp how after I'll I've written and all you know about me,
that you don't see how wrong that statement is?
I'm an engineer that has been building, and using, "modules" in every
machine and program I've designed for a good 50 years. Iknow the power of
modules.
Having said that, (again), why do you think I'm putting down your "module"
approach for _this_ application?
The answer is that there's no reasonable way to make a _learning_ system,
_modify_ or use the modules that I've every been able to think up.
Ok, so we take the Java SDK. Lots of good modules there for a HUMAN
PROGRAMMER to make use of when he wants to hard-code (using is own
intelligence) lots of new complex behaviors in a robot. Now, using the
Java SDK as your starting set of modules, how the hell do you write code to
replace the human programmer here? What Java code do you write, that takes
as input, the reward signal, and then re-connects the 1000's of java
modules together in different ways to make it perform a new task, like
driving a car?
You have answered a few questions like the above in a few of your recent
posts. But your answer is so vague and high level as to be useless.
There's nothing we can code from what you describe (the same error you
pointed out Ray is making). Your answer is that the modules get rewired or
reconfigured by the learning system. WEll, duh. But HOW??? What sort of
algorithm can take 100 different low level modules, and figure out how to
change their configuring in response to a reward signal??? What what you
describe, I hostilely have NO CLUE where to even begin to try and code an
exmaple. I have no clue how to write Java code, to make it re-write code
which uses the Java SDK libraries in response to a reward signal that has
even a slight chance of working for anything but the most trivial problem.
> When I tried to evolve a 2-bit adder using random connections
> between NOR gates in a "monolithic" network I got sick of
> waiting for it to happen. However when I broke the problem
> down into evolving an XOR gate for the sum and an AND gate
> for the carry it took place in a reasonable time.
Yes, because you SOLVED MOST THE PROBLEM FOR IT BY GIVING IT XOR GATES TO
WORK WITH. You made it run faster, by giving it an EASIER PROBLEM. Human
learning is not EASY no matter WHAT MODULES YOU GIVE IT. There is no "car
driving module" in us that makes learning to drive a car easy.
It's like trying to solve a hill climbing problem and you making the claim
the way it's solved, is by "adding a module" which moves the agent near the
top of the highest hill - without even attempting to describe how the magic
"make it all easy" module works.
> This has been observed in other evolutionary networks.
> In a monolithic network if one unit can be used in different
> tasks then when it is changed for one task it messes up
> its role in other tasks. This is what happens in the nets
> you suggested.
Do you grasp the design I've been attempting to implement in my networks?
I dount it based on what you just wrote.
My networks, which are very unlike most neural networks, have an important
design feature to address the very point you made here.
They are designed to function as _decoder_ networks which selects which
part of the network is being used to solve the problem based on the current
sensory context. Unlike many AI neural networks, most the nodes in the
network are unused most teh time. At any point in time, most the nodes had
nothing to do with what the network has recently done, and when learning is
applied, most the nodes don't get changed, because they weren't recently
active.
Yes, the issue you point out is important - but it's an issue I've been
designing networks to _solve_ for about 10 years now.
In the pulse sorting networks, only the nodes that a pulse passes though
play a role in what the network is doing. When you build _large_ pulse
sorting networks (as I have done at times), most the nodes are idle most
the time - only the nodes that are active based on the recent sensory data
patterns (the recent context) get used to create the networks recent
behaviors.
> If you change weights for one outcome it
> messes up any weight settings for other outcomes.
Which is why you must design the network in a way so that not all nodes get
adjusted the same for all conditions. This is how all neural networks in
general work (not just mine). They are solving a large linear equation
where every weight in the matrix effects every input and every output. But
yet, the correct setting of the weights, can make a matrix multiply produce
the right answers. The way you make a linear system like that converge on
the correct weights, is to adjust the weights relative to how much effect
they had on the current output. When the output is wrong, all the weights
that effected the output are adjusted - but the ones that had the most
effect at that moment, get adjusted more. This approach is what allows
neural networks in general to converge on a working answer even though each
learning tends to mess up other learning a little bit. But when done
correctly, those errors cancel out, and the network converges.
My network takes that even further by having most weights in the network
totally unused. A network which is 1000 wide by 10 deep only uses 10 out
of those 10,000 nodes for creating each pulse behavior.
> It would
> be like a radio in which adjusting the volume knob would
> result in a change in the frequency setting and thus
> requiring a change in the frequency setting which in turn
> would change the volume setting, only many time worse.
Yes, but if the frequency knob moved half as fast as the volume, when you
turned the volume, and the volume moved half as fast as the frequency, when
you turned the frequency, then you can always correctly set the two
parameters with only two adjustments. And if you couldn't pre-calculate
what those two correct adjustments were, you could just keep adjusting both
to their correct position and over time, the settings would get closer and
closer to correct - they would converge on what you were trying to set them
do - which is how learning in neural networks in general works.
> In the biological world units show the autonomous properties
> of wholes and the dependent properties of parts.
>
> JC
I don't think you gasp my argument, because you don't grasp how to build
learning networks in the first place. You don't understand how issues like
the one you just brought up, was already solved by the guys that created
the first neural networks many decades ago. And how that type of solution
was extend to multilayer networks, again, many decades ago.
There is nothing wrong with any of your general high level ideas of lots of
modules being trained by a high level learning system. At that level,
everything sounds logical and fits with what people believe is happening in
the cortex.
The problem shows up only when you try to implement your ideas.
The only learning you have ever talked about with "modules" is learning by
making random connections and then testing. Random connection changes and
testing is UNWORKABLE. It's slower than monkeys typing. The only way it
can work, as you talked about above, is if the solution is already there in
one of the modules, and all it has to do, is find the right module. But
most of what a human learns to do in their life time, is not implemented by
a module we were born with. We could well have a grasping, and a balancing
module, and some sort of basic hand-eye coordination modules just to name a
few of many that could reasonably be there. And it's also easy to
understand that some hand-eye task might be withing the range of what the
hand-eye module can learn with simple random connect and test approach.
But there are many high level human behaviors that are learned, and which
simply can not be explained as "having a pre-wired module to do it".
When we learn to drive for example, there are probably of millions of
different sensroy patterns we learn to recognize on the road - from road
conditions, to signs, to other cars, to the slope and curve of an
approaching turn, to the body languge of the driver in the other car that
is trying to signal us to do something. Each of these millions of
different sensory conditions generate in us, an appropriate response which
is created by moving our hands and legs to make the wheel and pedals move -
or to create body langauge to communicate with another human driver.
We DO NOT HAVE MODULES - to make us move our hands and feet move correct to
deal with an approaching bank in the road _before_ the car gets to the turn
- like what happens when we learn to drive.
It works, because we have g3eneric powers to recognize sensory patterns,
and we have generic patterns to generate _any_ type of motion needed to
respond to different sets of sensory patterns - whether the sensory pattern
is visual, or auditor, or tactual, or some combination of many sensory
conditions.
If we did have a "car response module" how would the learning system know
to try it out, instead of the "don't jump off a cliff" module, instead of
the "make a sandwich" module? It can't work by random connect and test
even if we had lots of good pre-wired modules to work with.
So, if none of the pre-wired modules can correctly produce the huge set of
stimulus response patterns we need to dive a car, how does the brain get
itself wired correctly in the process of "learning to drive"?
You have answered this by saying "we use our conscious brain to think about
what we want to do and then the low level brain learns to mimic it".
Taht's fine. But how did the high level "conscious brain" know what to do?
How did it get wired? How did it know what was resonable to try for
driving a car? If it learned it by reading about it, then how did it learn
how to respond to words with the the correct arm movements?
Every example of the complex learning problem the human brain can learn, is
an example of a large system recognition the correct patterns of sensory
context, and producing an appropriate response, for the appropriate set of
sensory patterns.
If the basic hardware is configured as doing that - and only that -
classifying sensory patterns - and producing a response - then you have
something that's easy to understand how to make it learn. When the system
receives a reward signal, the system is modified to increase the odds of
the same response happening again. And the easy way to do that, is widen
the response class to include more edge conditions in the same response.
That is, it's already working so that some _range_ of sensory conditions is
triggering the behavior. When it's rewarded, that range is simply expanded
a little bit - which must at the same time, shrink the set of other
responses near by.
There is no need in such an approach for "random search". Behaviors that
have worked out to be useful, become strengthened, which means the odds of
them being tried in a similar situation increases. When the network is
exposed to a pattern it's never seen before, it uses it's measure of
"closeness" to pick possible behaviors based on how well those behaviors
worked in similar situations in the past - greatly increasing the odds that
the behavior selected might actually be useful for the current condition.
If you instead have lots of "modules" then you are still playing the same
game. how does the learning system know where to start when a new sensory
condition shows up? Which module does it try first? What sort of system
does it use to rank the modules so the "right" answer is likely to be tried
soon, instead of waiting for it to be randomly selected?
All your arguments about how you seem to think it should work, and how you
tink my approach is going in the wrong directly, seem to me, to be based on
you not having a clue how to create learning systems. And how the learning
systems you have tired - failed - so your solution to making them work
better is to say "evolution does the work for us by building lots of high
level modules that pre-solve all the hard problems for us". But that's
impossible.
What evolution did, was not give up and punt like you are doing. It
actually figured out how to make strong learning work - and it build a
strong generic learning system that could map any sensory condition, to any
behavior needed, to maximize a reward signal. It didn't build modules that
had such mapping pre-coded. It didn't know what a stop sign would look
like and how we would have to respond to with just the right set of actions
to use the clutch to shift the car out of gear and then press the break.
It's nonsense to even suggest we have "modules" that do this stuff for us.
We have NO MODULE FOR making a car stop in response to a stop sign. We
have very low level modules that allow the brain to recognize patterns in
the sensory data across sensory modalities and very low level modules to
allow the brain to produce patterns of outputs controlling multiple muscles
in parallel, in response to those changing sensory patterns. We have
generic learning hardware that can wire any sensory pattern, to any learned
behavior - without the help of any other high level innate modules.
If you want to convince me that your "module" approach has merit - make a
system using modules that can learn some hard new behavior - any behavior -
just something that isn't trial. And of course, I need to do the same -
which I think I have done - but I know you have not seen my networks do
anything that you consider useful or interesting.
rs...@nycap.rr.com
> about the neocortex. None of the current evidence about brain function
> indicates that's what the neocortex is doing.
>
> I see such a network because I attacked the problem from a different
> direction. That is, from the problem of how to you build a machine, that
> can learn to trigger any set of moves (aka behaviors) the body is able to
> make, in response to any complex stimulus signal? The only way I've so far
> figured out how to create such a machine is with a large fairly monolithic
> network. I then look at the cortex and say, "gee, it's one large module
> with repeated low level modules thoughout the entire cotex, maybe that's
> what is is!".
>
> So yes, I strongly _suspect_ that the underlying technology of the
> neocortex is a large generic learning system - but I've not studied the
> brain - or even studied the research of others enough to argue what it's
> really doing.
There is a book that was written for a technologically oriented person
that describes area seventeen of the neocortex in exquisite detail.
“Eye, Brain, and Vision”, D. H. Hubel, 1988. Upon reading this text,
you will be left with the feeling that you know exactly what a neuron
in area seventeen (the part of the neocortex where optical signals
arrive from the thalamus) is doing. It is responding to a directed
segment in the visual field.
This neuron is readied by the genome (when it should come into the
world) to respond to a certain specific directed segment. What happens
when it does, or does not, is described.
This book is an easy read, and will leave you with a grasp as to how a
part of the brain is constructed by the genome, and how it adapts to
the world it encounters.
Ray
Curt Welch
> > Well, actually, I don't see such a network by looking at what is known
> On Nov 30, 9:44 pm, c...@kcwc.com (Curt Welch) wrote:
>
> > about the neocortex. None of the current evidence about brain function
> > indicates that's what the neocortex is doing.
> >
> > I see such a network because I attacked the problem from a different
> > direction. That is, from the problem of how to you build a machine,
> > that can learn to trigger any set of moves (aka behaviors) the body is
> > able to make, in response to any complex stimulus signal? The only way
> ar
> > figured out how to create such a machine is with a large fairly
> c
> > network. I then look at the cortex and say, "gee, it's one large
> > module with repeated low level modules thoughout the entire cotex,
> > maybe that's what is is!".
> >
> > So yes, I strongly _suspect_ that the underlying technology of the
> > neocortex is a large generic learning system - but I've not studied the
> > brain - or even studied the research of others enough to argue what
> > it's really doing.
>
> There is a book that was written for a technologically oriented person
> that describes area seventeen of the neocortex in exquisite detail.
> you will be left with the feeling that you know exactly what a neuron
> in area seventeen (the part of the neocortex where optical signals
> arrive from the thalamus) is doing. It is responding to a directed
> segment in the visual field.
>
> This neuron is readied by the genome (when it should come into the
> world) to respond to a certain specific directed segment. What happens
> when it does, or does not, is described.
>
> This book is an easy read, and will leave you with a grasp as to how a
> part of the brain is constructed by the genome, and how it adapts to
> the world it encounters.
>
> Ray
I've read many descriptions of the visual cortex. It's one of the best
documented ares of the brain. And it's been talked about lots of times
here over the years. What do you think I'll learn from that book that I've
not seen in all the other books and articles I've read about the visual
cortex???
What's important about human behavior, is how it changes over time. How
learning works. We are not born acting like adults. Nothing I've ever
seen about the visual cortex, explains even in the least, _how_ it adapts
and changes over time. They only document what it does, and what it
doesn't do, when it doesn't get a good data stream from the eyes to work
with.
I've not seen any documents on brain research that document how these
"modules" adapt to the world they are dealing with. To solve AI, we have
to figure out how to create a learning algorithm that equals the adaptive
abilities of the human brain. We don't have to build more pre-wired
modules that have the solutions hard-wired into them. We have to build a
module that can adapt to problems we never thought about.
When the mouse I posted about in the slashdot article were exposed to a
slot with food on the other side, their brain changed by rewiring the motor
cortex to make the behavior of the mouse change to fit the environment the
researcher placed the mouse into - an environment with food located on the
other side of a slot. This dynamic brain adaption allows the behavior of
the mouse to change so it could get more food in that environment. It
didn't work by stopping and staring innate MPGs. It rewired the MPGs that
were already there to make them act differently in response to this new
stimulus - the slot with food on the other side.
Show me some brain research that _explains_ how this sort of learning
works, and I will quickly grab a copy and read it. But so far I don't
think anyone knows how to create learning machines like that, or how the
brain does it. Yes, we do know lots of little facts about how neurons
adapt in response to stimulus. But no one has put the pieces together yet
into a coherent explanation of how the brain as a whole learns some new
simple behavior like grabbing food though a slot.
My interest is in making computer programs adapt like that. If someone
figures out, and can explain how the brain does it, I can translate that to
make the computer do it.
I suspect however, that engineers doing pure research on learning
technology will figure out the underlying technology long before anyone
looking at the brain will understand what they are looking at. I don't
think we have enough of the correct tools to really figure out the brain
yet. If they knew _what_ sort of technology to look for, I bet they could
find it fairly quickly. But not knowing ahead of time what they are
looking for, makes their job much much harder.
rs...@nycap.rr.com
> "rs...@nycap.rr.com" <rs...@nycap.rr.com> wrote:
> > > Well, actually, I don't see such a network by looking at what is known
> > On Nov 30, 9:44 pm, c...@kcwc.com (Curt Welch) wrote:
>
> > > about the neocortex. None of the current evidence about brain function
> > > indicates that's what the neocortex is doing.
Hubel and Wiesel did a fair job of puzzling out what area seventeen
is doing.
Take a look at
http://hubel.med.harvard.edu/index.html
This book speaks for itself, but I must say I find it beautifully
written.
> > > So yes, I strongly _suspect_ that the underlying technology of the
> > > neocortex is a large generic learning system - but I've not studied the
> > > brain - or even studied the research of others enough to argue what
> > > it's really doing.
I object to –learning-, possibly adapting is a better choice. The
genome furnishes a nervous system that can handle the job of life. The
system must adapt itself to what is served up by the environment.
> I've read many descriptions of the visual cortex. It's one of the best
> documented ares of the brain. And it's been talked about lots of times
> here over the years. What do you think I'll learn from that book that I've
> not seen in all the other books and articles I've read about the visual
> cortex???
You can find out how area seventeen works, and by much extrapolation
how the rest of the cortex might be doing. I doubt that you’ve read
anything like this book. But take a look, and let me know.
> What's important about human behavior, is how it changes over time. How
> learning works. We are not born acting like adults. Nothing I've ever
> seen about the visual cortex, explains even in the least, _how_ it adapts
> and changes over time. They only document what it does, and what it
> doesn't do, when it doesn't get a good data stream from the eyes to work
> with.
This is the place to find out how it does adapt, and how it changes
over time.
Ray
casey
> _learning_ system, _modify_ or use the modules that I've
> every been able to think up.
>
>
> programmer here?
I don't know how to replace the HUMAN programmer anymore
than you do. But we can use the analogy and ask how a higher
level controller (not as high yet as a human) selects low
level controllers to increase the value of the sum of its
rewards signals.
> What sort of algorithm can take 100 different low level
> modules, and figure out how to change their configuring
> in response to a reward signal???
First the selector of the modules does not figure out how
to change the configuration of the modules selected, it
simply provides them with a reward signal or not. It in
turn is provided with a reward signal if it has been good
at making the selections. If it makes a bad selection it
doesn't get a reward, it will change the criteria it uses
to make selections.
An example might be an employer who can take 100 different
employees and figure out which one to employ to maximize
the reward signal (business success). The employer by the
way doesn't have to figure out how to change the employees
he only needs to evaluate how well they do their job and
reward them with employment.
>> When I tried to evolve a 2-bit adder using random connections
>> between NOR gates in a "monolithic" network I got sick of
>> waiting for it to happen. However when I broke the problem
>> down into evolving an XOR gate for the sum and an AND gate
>> for the carry it took place in a reasonable time.
>
>
> Yes, because you SOLVED MOST THE PROBLEM FOR IT BY GIVING
> IT XOR GATES TO WORK WITH.
No, read it again I used NOR gates.
> You made it run faster, by giving it an EASIER PROBLEM.
The problem was the same but broken into two simple problems.
This is possible if each solution by itself can improve
the chances for a reward signal. When this is possible
we can reduce the size of the problem by breaking it
into sub problems.
> The only learning you have ever talked about with "modules"
> is learning by making random connections and then testing.
> Random connection changes and testing is UNWORKABLE. It's
> slower than monkeys typing.
The example with the NOR gates were random and shows that it
doesn't take much increase in size to make random connections
and testing unworkable. Clearly you need to learn non random
ways of connecting. I have played with non random connections
with interesting results which I did tell you about.
> It's like trying to solve a hill climbing problem and you
> making the claim the way it's solved, is by "adding a module"
> which moves the agent near the top of the highest hill -
> without even attempting to describe how the magic "make it
> all easy" module works.
If you want to write a program (the top of the hill) with a
window, task bars, buttons, sliders, images and so on don't
the modules make it easier? Of course I don't know how to
replace the Java programmer as that means building the
equivalent of a human brain. But the principle of making it
easier would be the same for a simpler system.
You select modules that select actions rather than select
the atomic actions themselves. Learning can be seen as an
act of selection.
> If you want to convince me that your "module" approach has
> merit - make a system using modules that can learn some hard
> new behavior - any behavior - just something that isn't trial.
It is a case of illustrating the plausibility of the modular
approach as having an advantage over the monolithic network you
suggest rather than me having a working human level solution.
Evolution seemed to have found it to be best to build bodies
and cells out of semiautonomous specialized modules. A colony of
ants or our evolving society is also made out of semiautonomous
modules (ants or humans)
> And of course, I need to do the same - which I think I have
> done - but I know you have not seen my networks do anything
> that you consider useful or interesting.
More importantly those with the expertise have also not found
anything useful in your approach.
The brain is of course made out of neural networks but not out
of one monolithic net just as the body is made out of organs.
The heart and lungs take on specialized functions rather than
trying to combine them into one organ.
With the evolution of brains you might ask why this modular
approach also won over just increasing the size of the first
primal net. Increasing the size of a net with a useful function
is of course an option and the neocortex I would say is an
example or 4Gb memory instead of 256K memory in a computer.
You like to point to the ANN as the way to go but there are
some things it is not good at (see Pinker) and there are some
things that are computations not associations. The ant for
example computes its position relative to the nest and a rat
can compute alternate paths through a blocked maze from some
kind of internal representation of the maze rather than an
from a SR sequence.
JC
Curt Welch
> On Dec 1, 10:26 am, c...@kcwc.com (Curt Welch) wrote:
> > "rs...@nycap.rr.com" <rs...@nycap.rr.com> wrote:
> > > > Well, actually, I don't see such a network by looking at what is
> n
> > > On Nov 30, 9:44 pm, c...@kcwc.com (Curt Welch) wrote:
> >
> > > > about the neocortex. None of the current evidence about brain
> on
> > > > indicates that's what the neocortex is doing.
>
> Hubel and Wiesel did a fair job of puzzling out what area seventeen
> is doing.
>
> Take a look at
>
> http://hubel.med.harvard.edu/index.html
>
> This book speaks for itself, but I must say I find it beautifully
> written.
Looks like a good book/web site.
> > > > So yes, I strongly _suspect_ that the underlying technology of the
> > > > neocortex is a large generic learning system - but I've not studied
> > > > t=
> he
> > > > brain - or even studied the research of others enough to argue what
> > > > it's really doing.
>
> I object to =96learning-, possibly adapting is a better choice.
It's just a word Ray. The concept is the same whether you call it learning
or adaptation. The end result is that over time, we can perform tasks that
we couldn't perform at birth.
> The
> genome furnishes a nervous system that can handle the job of life. The
> system must adapt itself to what is served up by the environment.
Right. But whether you call it a nervous what can learn, or a nervous
system that can adapt, the concept is the same. The genome had to build it
so it could perform that function (as well as any other functions it was
built to perform).
> > I've read many descriptions of the visual cortex. It's one of the best
> > documented ares of the brain. And it's been talked about lots of times
> > here over the years. What do you think I'll learn from that book that
> > I'=
> ve
> > not seen in all the other books and articles I've read about the visual
> > cortex???
>
> You can find out how area seventeen works, and by much extrapolation
> how the rest of the cortex might be doing. I doubt that you=92ve read
> anything like this book. But take a look, and let me know.
Well, I scanned it and didn't see in the index anything that seemed to be a
chapter explaining _HOW_ adaption worked in it. That is the only thing
that's important for AI and it's not even covered as far I can see from the
index. But I could easily be wrong - there may be some important and
interesting facts about the area's ability to adapt that would actually
help us solve AI.
> > What's important about human behavior, is how it changes over time.
> > How learning works. We are not born acting like adults. Nothing I've
> > ever seen about the visual cortex, explains even in the least, _how_ it
> > adapts and changes over time. They only document what it does, and
> > what it doesn't do, when it doesn't get a good data stream from the
> > eyes to work with.
>
> This is the place to find out how it does adapt, and how it changes
> over time.
Which place are you talking about? The book, or c.a.p., or the brain or
what?
If you mean the brain, sure, if we study it long enough, we will completely
uncover how it works and in doing that, we will solve AI at the same time.
The answer is in there, and research WILL uncover it. I just don't happen
to be a brain researcher so my interest is generally limited to researching
and creating new computer programs. If the brain researchers uncover
important facts about how to do the type of learning (adaption if you will)
in a large parallel real time signal processing system, that would be good
to know. But so far, nothing I've seen explains how these large parallel
systems manage to adapt in a way that's useful to the organism. So far,
all I've seen, is documentation on the fact that the brain _is_ adapting
(plasticity as they like to say), but not how it works. And we also have
the data from the behaviorists that show how external behavior changes as
the brain adapts (conditioning) - but again, no answers in how to implement
it - which to me, is the one and only important unanswered question of AI.
Curt Welch
> On Dec 1, 1:44=A0pm, c...@kcwc.com (Curt Welch) wrote:
> > ...
> > The answer is that there's no reasonable way to make a
> > _learning_ system, _modify_ or use the modules that I've
> > every been able to think up.
> >
> >
> > ... how the hell do you write code to replace the human
> > programmer here?
>
> I don't know how to replace the HUMAN programmer anymore
> than you do. But we can use the analogy and ask how a higher
> level controller (not as high yet as a human) selects low
> level controllers to increase the value of the sum of its
> rewards signals.
Yes, or a little more specifically, we know how to build reinforcement
learning algorithms, and that in my view IS the answer to how to replace
the human.
Which, if you accept that (which clearly I), the question becomes simpler.
The question becomes how to implement a reinforcement learning algorithm
that solves high dimension, real time, non-Markov, reinforcement learning
problems. In other words, we know exactly what has to be done, and the
only problem is finding a workable implementation.
There is no mystery at all in my view about what AI is and how it will be
solved. The only mystery is finding a workable _implementation_ to a very
well understood problem.
> > What sort of algorithm can take 100 different low level
> > modules, and figure out how to change their configuring
> > in response to a reward signal???
>
> First the selector of the modules does not figure out how
> to change the configuration of the modules selected, it
> simply provides them with a reward signal or not.
Are you saying it just passes through the rewards it is getting, or are you
saying it is somehow generating it's own reward signal? What do the
modules do with the reward signal? Are you saying they are all learning
modules but each built to learn a specific type of behavior?
> It in
> turn is provided with a reward signal if it has been good
> at making the selections. If it makes a bad selection it
> doesn't get a reward, it will change the criteria it uses
> to make selections.
Ok, but how does it select modules and how does it decide how to change
that selection algorithm when it gets a reward or doesn't get a reward?
> An example might be an employer who can take 100 different
> employees and figure out which one to employ to maximize
> the reward signal (business success). The employer by the
> way doesn't have to figure out how to change the employees
> he only needs to evaluate how well they do their job and
> reward them with employment.
Well, that is the basis for how every reinforcement learning algorithm
works. Do you get that? They measure the success of every "module" used in
terms of how much reward each module got and then use that measure of
accumulated reward to judge which module is the best.
You are being vague however about what a module is. Can two modules be
used at the same time? Or does the selection system have to disable one
module before it can enable the next module. How fast does it make
decisions about which module is currently in use? once very 10 seconds?
1000 times a second? What?
> >> When I tried to evolve a 2-bit adder using random connections
> >> between NOR gates in a "monolithic" network I got sick of
> >> waiting for it to happen. However when I broke the problem
> >> down into evolving an XOR gate for the sum and an AND gate
> >> for the carry it took place in a reasonable time.
> >
> >
> > Yes, because you SOLVED MOST THE PROBLEM FOR IT BY GIVING
> > IT XOR GATES TO WORK WITH.
>
> No, read it again I used NOR gates.
Ok, I did read it wrong, but the result is the same as I suspect.
Instead of crating a STRONGER learning algorithm, you do what you have done
many times - you made the problem easier so that your weak algorithm could
solve it.
Lowering the bar to make your bad algorithm work is not a possible solution
to AI. It's like switching from chess to TTT because your AI program
wasn't good enough to play chess. We can't solve AI that way. To solve
AI, we have to find a STRONG learning algorithm that doesn't need to be
hand fed simple problems. The stuff the brain can learn, IS NOT SIMPLE.
> > You made it run faster, by giving it an EASIER PROBLEM.
>
> The problem was the same but broken into two simple problems.
The learning problem WAS NOT THE SAME. Why on earth isn't that obvious to
you? Why do I have to spell out something so obvious?
> This is possible if each solution by itself can improve
> the chances for a reward signal. When this is possible
> we can reduce the size of the problem by breaking it
> into sub problems.
Yes, finding a solution that's only one step way is easier than finding a
solution that is 10 steps away - it's not just twice as easy either - it's
orders of magnitudes easier which is why your program was talking too long
to solve the first problem, but was able to quickly solve the easy problem.
Learning is a search problem. It's a search of some solution space to find
the best possible solution. If it takes 8 nor gates wired correctly to
create a full adder, then the learning system has to search all possible
combinations of 8 nor gates until it finds one of the possible soutions.
The space is very large, so it takes a long time to search - a long time to
learn.
If you change the problem, to create an XOR circuit first, then that might
only take 4 nor gates, and the search space of the number of ways you can
wire 4 gates is a few orders of magnitude smaller than the search space for
8 gates. So by giving it that problem as a steeping stone to the solution
to the harder problem, you have cut the search space down by many orders of
magnitude.
But how did you know that stepping stone was a good path to the solution?
You knew that because of your OWN INTELLIGENCE. You used your
intelligence, to do the work of the learning algorithm for it - you used
your intelligence to give it a short cut - a hint - which greatly
simplified it's job.
Now, if you can create a learning algorithm that finds that short cut on
its own, then that's great. But when you have to step in and use your own
intelligence to give it hints, then you have failed to make the software
more intelligent.
Evolution doesn't pre-wire into the brain, HINTS about how to dal with a
stop sign while driving a car John. Hints provided by eovlution which are
hard0-wired into "smart modules" can't explain the learning power of the
human brain.
We KNOW WHAT THE BRAIN IS ABLE TO LEARN. That is not in question. We have
thousand upon thousands of examples of things the brain can learn, which
can not be explained by "evolution giving the brain hints in the form of
hard-wired smart modules". It can learn these things because its got a damn
strong learning algorithm hat's beyond what any of our man-made learning
algorithm implementations can currently learn.
How you made the learning program easier, however is an important clue to
what we do actually have to do. Searching a large domain space takes
forever if the space is too large and if you don't have any hints how to
short cut the search.
So the question is, how large is the actual search space of various real
human learning problems, and how does the brain manage to reduce that
search space to something reasonable?
Such as for the rat that needs to learn how to get food though the slot in
the cage, how many different behaviors does it need to search before
finding one that gets the food? Should it run in circles for 10 minutes?
Roll over and over? Wiggle it's right toe on the left foot for 2 hours?
The number of possible behaviors it could try is effectively infinite - or
at least so large, it could try different behaviors randomly until the
universe died and still likely not have gotten around to trying one of the
sequences of behaviors that would have gotten food in it's mouth by
reaching thought he slot with a paw.
So how does the rat brain solve this learning (search) problem which is so
large it could never be expected to find the solution?
Your answer seems to be that evolution will build hints into the rat-brain
to make it know what behaviors to try first. And there might well be some
of that gone on - especially when it comes to locating food and getting it
in your mouth (something rats and their ancestors have needed to do for
billions of years). But we can't use the "evolution gave us hint" cop-out
for most of the intelligent behaviors humans are able to learn when dealing
with modern life in our modern society. The search space for new problems
we are given in our day to day live is way too large to explain how fast we
are able to solve new problems. The brain doesn't work by randomly trying
new behaviors.
The answer to all this, is simple. The brain needs a mechanism for making
educated guesses about what behaviors to try first when searching -
something your learning algorithm trying to build a full added didn't have
(or didn't have a very good "educated guess" system if it had one). It had
no ability to make an educated guess about what configuration of NOR gates
it should try next. It was forced to simply pick randomly from the list of
billions of possible configurations.
At no point, other than maybe at birth before the human baby has had any
experience with the world, does the brain have to "pick randomly" at the
billions of possible behaviors the brain is able to make the body perform.
It always has past experience to fall back on. It can use every past
experience it's had, to give it hints, about what to try first when
presented with the next new state of the environment.
Lets pretend the rat doesn't have any hardwired "hints" from evolution and
think about how it might still be able to deal with the problem of food on
the other side of a slot based on past experience.
To start with, it might recognize the food because it's the same stuff it's
been eating. But normally, the food is in a dish, and it uses a simple
"walk over and eat with the mouth" set of behaviors to get the reward from
eating the food. But lets say in this case, those behaviors fail, because
the rat's nose is too big to fit though the slot. So it tries the "walk
over and eat" behaviors (it's fist behavior choice in response to seeing
rat-food), but it fails to get the expected reward because the food is out
of reach.
What does it try next? Spin in circles? Nope because that's not the type
of behavior that's worked int eh past to get rewards when food is near by.
There are a lot of different behaviors the rat has used when food was near
by in the past, which led to reward - because the behavior got the food
into the mouth one way or another. One of those behaviors from the past
might have been to use the food to move the food closer to the rat - to
reach out and grab the food with the foot. It might have found that
behavior worked in the past while eating, even though it wasn't needed in
that case - it just discovered it "worked" (aka was rewarded) by accident.
But now, that past experience helps the rat-brain select a good alternative
behavior other than "walk and eat". So when walk and eat keeps failing,
other behaviors with high potential for the current context start to emerge
from the rat - such as the "move food with foot" behavior - which in this
case, ends up being a "winner" - because after that, the rat trys again the
"walk and eat" trick, and this time it works because the rat first used his
foot to move the food closer - which moved it through the slot so it can
now reach it. So the "walk and eat" gets rewarded, but so does the recent
"reach out with foot" behavior.
The whole trick to making this work, is that lessons learned from past
experience with _similar_ situations must be tried _first_ when faced with
a new situation.
And in order for that to work, the learning system must have the power rank
different situations as "similar" so the behaviors it tries, is based on
what it has learned works best, for similar situations from past learning.
And that's a big part of what we have to do to make our learning algorithms
stronger. We don't have to give them hints about what to try first. They
have to have internal systems that allow them to use their _own_ past
experience, to rank the order in which they search the behavior space for
better answers.
All reinforcement learning algorithms actually already do just that. But
they don't do it for high dimension sensory spaces. That's where most
simple RL algorithms fail to duplicate what the brain can do. They don't
create a good measure of "closeness" in a way that allows the system to
accurately apply old lessons learned, to new, but similar, situations.
This is really just another way to talk about the perception problem of AI.
If the visual system can't recognize that the same dog is in front of him
as was in front of him yesterday when it had that other learning
experience, then it can't possibly use the lessons learned form yesterday,
again today. The system in that case would be failing to recognize the two
sensory contexts were actually very similar (both had the same dog in
them).
This of course isn't just a visual problem. It's a measure of "similar"
that must span all sensory modalities.
So, to create STRONG learning, instead of just cheating and asking
evolution to give the learning system lots of hints about what behaviors to
try (many of which we know it can't possibly do for human learning
situations), we have to instead build a learning system that's very good at
applying lessons learned from past experience, to new situations, based on
an internal measure of "similar". This is just a perception system
implementation problem that must be solved before we can make our learning
algorithms stronger. Without a strong perception system to associate past
rewards with, there really isn't anything useful that it can learn.
> > The only learning you have ever talked about with "modules"
> > is learning by making random connections and then testing.
> > Random connection changes and testing is UNWORKABLE. It's
> > slower than monkeys typing.
>
> The example with the NOR gates were random and shows that it
> doesn't take much increase in size to make random connections
> and testing unworkable.
Yes, very true. To deal with high-dimension problem spaces (aka problem
spaces which are larger than large and totally unworkable if random search
is the best you can do), the agent must have its own way to use past
experience to bias the search (bias what behaviors it tries first) so as to
try the behaviors most likely to produce a good result first. If it can't
give itself hints based on it's own past experience, it won't have a rats
chance in hell of equaling the learning power of the human brain - which is
very good at applying old lessons learned to any new context it is faced
with, so that it makes very good "educated guesses" about what to try in a
new situations.
> Clearly you need to learn non random
> ways of connecting. I have played with non random connections
> with interesting results which I did tell you about.
>
> > It's like trying to solve a hill climbing problem and you
> > making the claim the way it's solved, is by "adding a module"
> > which moves the agent near the top of the highest hill -
> > without even attempting to describe how the magic "make it
> > all easy" module works.
>
> If you want to write a program (the top of the hill) with a
> window, task bars, buttons, sliders, images and so on don't
> the modules make it easier? Of course I don't know how to
> replace the Java programmer as that means building the
> equivalent of a human brain. But the principle of making it
> easier would be the same for a simpler system.
Yes, and there's nothing wrong with that point. Except of course, the
human brain is able to solve learning problems that evolution in now way
could possibly have "made it easier" by hard-wiring half the solution for
it like you did in the example above. And it's because the brain is able
to make very good use of the past experience. Stuff it just stumbles on by
pure chance, which turn out to help it get more rewards, gets stored away
for future potential use in similar future situations.
> You select modules that select actions rather than select
> the atomic actions themselves. Learning can be seen as an
> act of selection.
And that is EXACTLY how I described the operation of my pulse sorting nets
to you years ago. It almost sounds as if you are echoing back the things I
said to you in the past. When one of the nodes in my pulse sorting work
routes a pulse out one side to sub-net A, or out the other side to sub-net
B, it is doing exactly that. It's selecting which of the two sub-networks
it wants to use to create the current behavior. From the perspective of
each node in the network, the down-stream sub-network it sends to the pulse
to can be seen as a large "behavior module" that it has learned to select
to do the work. It activates each subset by sending it the "activation
pulse".
> > If you want to convince me that your "module" approach has
> > merit - make a system using modules that can learn some hard
> > new behavior - any behavior - just something that isn't trial.
>
> It is a case of illustrating the plausibility of the modular
> approach as having an advantage over the monolithic network you
> suggest rather than me having a working human level solution.
Yes, and as you describe your "module approach" you are also describing
exactly what I already have working in my "monolithic network" so I'm not
feeling all that impressed. Except in my approach, evolution doesn't have
to hard-code any of the modules. They all learn on their own.
> Evolution seemed to have found it to be best to build bodies
> and cells out of semiautonomous specialized modules. A colony of
> ants or our evolving society is also made out of semiautonomous
> modules (ants or humans)
Yes, but most the modules created by evolution were not meant to be
learning modules - they are the simple hard-coded modules we create in all
our technology that each perform their own specialized functions.
Intelligent behavior however is not hard-coded. It's adaptive. And the
more adaptive it is (the more it can change it's function in response to
the environment), the more intelligent it is.
The only important module we need to hard-code to solve AI - is a strong
generic behavior learning module.
> > And of course, I need to do the same - which I think I have
> > done - but I know you have not seen my networks do anything
> > that you consider useful or interesting.
>
> More importantly those with the expertise have also not found
> anything useful in your approach.
No on in this group who posts on a regular basis has real experience with
RL algorithms. There is SO LITTLE experience in this group that I have had
to explain, and re-explain, and re-explain, even the most fundamental and
basic concepts of RL to the the people like you who are willing to chat
about this stuff. And even after I've explained it many times, and pointed
to chapters in books about RL like Sutton's book, you still show that you
fail to truly understand important basic concepts of RL.
My approach is an advanced approach to solving a class of RL problems no
one knows how to solve yet. SO far, I haven't found ANYONE in this group,
that even understand what class of problems that is - yet alone,
understands the apparoch I'm trying to take to solve it.
If someone (anyone) in this group showed they understand what "high
dimension non-Markov" meant, I'd been impressed. So far, I don't think
anyone here really understands what the term means, and why that class of
RL learning problem is the one we have to find a workable implementation in
order to solve AI. It is a class of problems people in RL research are
working on - though I've also not seen any of them claim as I do that it's
key to solving AI. Mostly they just seem to think it's yet another
fun/hard learning problem to work on as far as I can grasp.
I suspect actually, that if I had more time to review and study all the RL
research out there, I would find that there are some approaches that are
conceptually very similar to my own - but that they are simply looking at
it differently and using different terminology (which I haven't learned) to
describe it - meaning I can't recognize the similarity by reading the
abstract.
If such similar works exists, as I suspect it might, then I could
re-describe my own work in their terms, and that would allow them at least,
to understand what I was doing. The fact that no one in this group
understands it, is no longer surprising to me.
> The brain is of course made out of neural networks but not out
> of one monolithic net just as the body is made out of organs.
> The heart and lungs take on specialized functions rather than
> trying to combine them into one organ.
THEY AREN'T LEARNING LUNGS AND LEARNING HEARTS JOHN! They were built to
provide a fixed function that's been a fixed need of our species for
millions of years.
Driving a car is NOT A FIXED FUNCTION THAT EVOLUTION COULD GIVE US MODULE
FOR. Ir had to build the "driving a car" module for us on the fly as an
on-demand adaptation.
Why on earth you have such a problem understanding the difference between
fixed-function modules and learning modules which have no fixed function
other than the fixed function of adaptation to the environment I don't
grasp.
It makes no difference how much of the brain are fixed function modules
created by evolution. What we can learn BEYOND what we were born with is a
learning problem no one has duplicated yet and until we can duplicate the
learning skill of the brain that happens AFTER/ABOVE the fixed function
modules, the debate over what the modules are is pointless. No amount of
fixed-function modules that could have been created by evolution will make
learning all the skills needed for a robot to drive a car as well as a
human possible unless you first solve the hard learning problem in front of
us.
> With the evolution of brains you might ask why this modular
> approach also won over just increasing the size of the first
> primal net. Increasing the size of a net with a useful function
> is of course an option and the neocortex I would say is an
> example or 4Gb memory instead of 256K memory in a computer.
>
> You like to point to the ANN as the way to go but there are
> some things it is not good at (see Pinker) and there are some
> things that are computations not associations. The ant for
> example computes its position relative to the nest and a rat
> can compute alternate paths through a blocked maze from some
> kind of internal representation of the maze rather than an
> from a SR sequence.
You are too easily fooled by nonsense John just because someone gives it a
different name like "compute".
There is no example of "compute" you can come up with that I can't
translate into a SR solution which uses temporal instead of only spatial
nodes - which means they HAVE MEMORY OF PAST EVENTS (in the terms you like
to use).
"Computing" addition for example can be easily replaced with the following
SR behaviors (as one of many examples I can create for how "compute" and SR
with feedback can be the same thing in all cases):
1+1 -> 2
1+2 -> 3
2+1 -> 3
2+2 -> 4
...
Now it might easily be true that an ant has special hardware for tracking
distance which has nothing to do with a generic behavior learning module.
But my point here is hat that a need to "compute" is not a valid
justification that there _must_ be innate modules doing the work.
casey
>
> No one in this group who posts on a regular basis has
> real experience with RL algorithms.
Then why not publish your ideas for those who you think
do understand RL algorithms. Send your ideas to Sutton.
Maybe start up your own RL yahoo group?
Here is some suggestions:
Clearly my posts are wasted on you.
JC
Curt Welch
> On Dec 2, 5:47=A0pm, c...@kcwc.com (Curt Welch) wrote:
> >
> > No one in this group who posts on a regular basis has
> > real experience with RL algorithms.
>
> Then why not publish your ideas for those who you think
> do understand RL algorithms. Send your ideas to Sutton.
> Maybe start up your own RL yahoo group?
Yeah, those are good ideas. I am subscribed to Rich Sutton's RL group and
have posted a few messages there. Mostly however, the group is not a place
where people seem to want to brainstorm/BS about RL like a typical Usenet
group.
The problem is that I'm not up to date on what the RL research community is
doing and I find it difficult to even understand some of the things they
like to talk about simply because I haven't put in the time to get up to
date on current RL research. In order to really communicate and engage
them, I have to get up to speed on current RL research and I just don't
have the time in my life right now to do that.
A couple of years back, I thought I was on track to go back to grad school
and put some serious time into AI work. But life doesn't always work out
as you expect. :)
At the moment, I have almost no time for AI because I've started a new
business as a blacksmith. Thanksgiving and the cold weather just game me
an excuse to come back and waste a little more time here in c.a.p.
> Here is some suggestions:
>
> http://groups.google.com.au/groups/dir?lnk=3Dnhpsfg&hl=3Den&q=3Dreinforce
> me= nt+learning&qt_s=3DSearch+for+a+group
>
> Clearly my posts are wasted on you.
>
> JC
The first group in that search list is the one I joined about a year ago.
casey
> ...
> At the moment, I have almost no time for AI because I've
> started a new business as a blacksmith. Thanksgiving and
> waste a little more time here in c.a.p.
I am not interested in wasting my time.
I didn't know blacksmiths still existed, I thought it was
your hobby. I have some images that you sent once, a metal
dice and a squirrel feeder.
JC
Curt Welch
> On Dec 3, 1:34=A0pm, c...@kcwc.com (Curt Welch) wrote:
> > ...
> > At the moment, I have almost no time for AI because I've
> > started a new business as a blacksmith. Thanksgiving and
> > the cold weather just gave me an excuse to come back and
> > waste a little more time here in c.a.p.
>
> I am not interested in wasting my time.
:)
By "wasting time" I mean, relaxing from work by entertaining myself with AI
debate.
> I didn't know blacksmiths still existed, I thought it was
> your hobby. I have some images that you sent once, a metal
> dice and a squirrel feeder.
Metalwork was just a hobby. Now I'm trying to make some sort of living out
of it. The Farrier (horse shoeing) side of blacksmithing is still a viable
business because horses still needs shoes about every 6 weeks even if most
the shoes are pre-fab now. But I don't do horseshoes. Pure blacksmithing
is mostly a hobby for people now but there are a few here and there that
manage to make a living at it. There's some demand still for hand made
iron work but it's half art and half functional production work and about
as easy to make money as any other art field (not easy). A few manage it,
but most who try don't make enough to really be able to call it a job.
Some pictures here if you care...
http://picasaweb.google.com/Curt.Welch/BlacksmithProjects#
Wolf K
Interesting. How did you line the forge?
And, no, I don't think an OT post is evil. ;-)
wolf k.
Curt Welch
With material purchased from Larry Zoeller ...
http://www.zoellerforge.com/flare.html
It's got two layers of 1" thick ceramic blanket, and then a thin coating of
his Plistix 900F which is a cement like covering on top of that. His web
site has lots of good information on building a gas forge.
I used his pre-made zburner as well instead of making my own though I did
add the piezoelectric starter to it.
> And, no, I don't think an OT post is evil. ;-)
Evil Wolf! There is the group alt.crafts.blacksmithing when this thread
gets too long and too off topic if you care but it's got very little
traffic.
Long live the global brain! (required AI content)
> wolf k.
Don Stockbauer
> greyw...@ruddy.moss wrote:
> > Curt Welch wrote:
I'll second that.
casey
> ...
> By "wasting time" I mean, relaxing from work by entertaining
> myself with AI debate.
My view on wasting time is the, yes it is, no it isn't, exchanges
and missing the point as I believe you did with the heart and
lung module example. Evolution is a learning process but at
the species, rather than the individual, level. There is always
a "programmer", a selector, even if that is in the form of
reproductive success.
I might still respond to your post indirectly with one on the
reason successful complex systems like bodies their brains are
made up of innate modules and how brain modules can be used
to learn high level stuff like tying a knot, driving a car, solving an
equation, making a cup of coffee.
> Metalwork was just a hobby. Now I'm trying to make some sort
> of living out of it. The Farrier (horse shoeing) side of
> blacksmithing is still a viable business because horses still
> need shoes about every 6 weeks even if most the shoes are
> pre-fab now. But I don't do horseshoes. Pure blacksmithing
> is mostly a hobby for people now but there are a few here and
> there that manage to make a living at it. There's some demand
> still for hand made iron work but it's half art and half
> functional production work and about as easy to make money as
> any other art field (not easy). A few manage it, but most who
> try don't make enough to really be able to call it a job.
I would say it would be a very therapeutic and satisfying job.
JC
Curt Welch
> On Dec 4, 1:14=A0am, c...@kcwc.com (Curt Welch) wrote:
> > ...
>
> > By "wasting time" I mean, relaxing from work by entertaining
> > myself with AI debate.
>
> My view on wasting time is the, yes it is, no it isn't, exchanges
> and missing the point as I believe you did with the heart and
> lung module example. Evolution is a learning process but at
> the species, rather than the individual, level. There is always
> a "programmer", a selector, even if that is in the form of
> reproductive success.
>
> I might still respond to your post indirectly with one on the
> reason successful complex systems like bodies their brains are
> made up of innate modules and how brain modules can be used
> to learn high level stuff like tying a knot, driving a car, solving an
> equation, making a cup of coffee.
Yes, I agree with all that. The "learning" system of evolution builds
modules which makes the complex system of the body as a whole function.
Learning is applied to the complex modules of the body over time as the
process of evolution so it's a clear example of how learning and complex
modules work together.
However, the lessons of evolution don't get applied to the modules
directly. The modules are not where the learning in evolution happens. It
happens instead, in a generic abstract monolithic module representation
system called DNA. Our DNA is effectively one large generic "module" that
implements a learning system that shapes the form and design of the modules
of the body.
casey
> directly. The modules are not where the learning in evolution
> happens. It happens instead, in a generic abstract monolithic
> module representation system called DNA. Our DNA is effectively
> one large generic "module" that implements a learning system
> that shapes the form and design of the modules of the body.
DNA is part of a bigger and very modular cellular system.
http://sysbio.harvard.edu/CSB/research/pdfs/Murray_ModularBiology.pdf
JC