AI approach couples biological computation with value
Alpha
contains many gems. One of which, and the focus of the book is:
"Nature has equipped biological computations with a measure of their value."
Read is professor of neuroscience at Baylor College of medicine, Director of
the Neuroimaging Lab, director of the Cente for Theoretical Neuroscience and
a fellow at the Institute for Advanced Study at Princeton.
Read visits the COTM (Computational Theory of Mind - if you do not know what
this refers to, in a nutshell, it is that mind is software running on the
brain; and that the mind is not equivalent to brain just as Microsoft Word
is not the logic gates that it runs on) and adheres to its tenets, along
with Turings ideas on computation. However...
Unlike computers,"... biological computations are not lifeless streams of
symbols, totally devoid of meaning [*]. Instead, biological computation
carry something extra - an extra measure of their overall worth. Instead of
just computation, there is computation plus something else, and that
something else is a measure of the value of the computation to the overall
success of the organism, its overall fitness. Biological computtions know
how to care"
He goes on to tell how brain implements goals, and how the energy efficiency
of a computation, selected via evolutionary mechanisms, puts meaning into
neural computations, assigning values and passing the values from one symbol
to another. Note that energy allocation schemes are missing from computers
today (which is one reason why your brain remains cool and your computer's
CPU is hot to the touch! All computations in today's computers get equal
energy billing.) Not so with biological computations, which are almost
"miraculously efficient", WRT energy Read says.
* Note that this observation and his subsequent approach, dovetails with the
philosophical position that a computer based on todays technology (without
some advance along the lines Read suggests) will continue to *NOT*
understand.
I just finished the first chapter and look forwrd to the rest of this
excellent (so far) book.
--
Posted via a free Usenet account from http://www.teranews.com
PeskyBee
news:45b0eb06$0$4765$8826...@free.teranews.com...
> Read Montague's new book: "Why Choose This Book: How We Make Decisions",
> contains many gems. One of which, and the focus of the book is:
>
> "Nature has equipped biological computations with a measure of their value."
>
> Read is professor of neuroscience at Baylor College of medicine, Director of the
> Neuroimaging Lab, director of the Cente for Theoretical Neuroscience and a
> fellow at the Institute for Advanced Study at Princeton.
>
> Read visits the COTM (Computational Theory of Mind - if you do not know what
> this refers to, in a nutshell, it is that mind is software running on the brain;
> and that the mind is not equivalent to brain just as Microsoft Word is not the
> logic gates that it runs on) and adheres to its tenets, along with Turings ideas
> on computation. However...
>
> Unlike computers,"... biological computations are not lifeless streams of
> symbols, totally devoid of meaning [*]. Instead, biological computation carry
> something extra - an extra measure of their overall worth. Instead of just
> computation, there is computation plus something else, and that something else
> is a measure of the value of the computation to the overall success of the
> organism, its overall fitness. Biological computtions know how to care"
Alpha, thanks for sharing this interesting book. I'll put it
in my reading list. However, I must confess that I'm somewhat
skeptical about claims that say symbolic systems can't have
"meaning". That reminds me of Searle's BS. If brains use
computation plus "something else", I'm willing to say that this
something else can be simulated computationally, provided we
understand what are the functional role of this thing.
*PB*
forbi...@msn.com
> Alpha, thanks for sharing this interesting book. I'll put it
> in my reading list. However, I must confess that I'm somewhat
> skeptical about claims that say symbolic systems can't have
> "meaning". That reminds me of Searle's BS. If brains use
> computation plus "something else", I'm willing to say that this
> something else can be simulated computationally, provided we
> understand what are the functional role of this thing.
I've gotten myself in hot water in another newgroup with some one
I respect so why not here too?
If you think Searle's writing is BS you probably don't understand
the point he was making. I'm going to try to make the point by
considering what it means to simulate computationally but before
I do I need to talk about what it means to compute.
Suppose we have observational only access to an otherwise
isolated system. While we observere it we find that we can
map particulars of the system at particular times to boolean
functions. The issues at hand is if we can meaningfully say
if boolean computation is going on in the system. For instance:
Suppose I can detect voltage at a point in the system and I can
map the times to when it is in excess to some value to "true"
otherwise to "false". Likewise I can do the same at another
point. We can label sequential and regular observations in
time, t, as A(t) for one and B(t) for the other and map the intervals
to intergers. Likewise we have determined that for sequential
observations, x (a time) B(x+1) = A(t). Are we entitled to
meaninfully label this a boolean computation? I think we can.
For many years we've been doing such mappings and the result
is today's computers. What else are computers doing if not
computations?
Now, for our needs it's not enough to have the system isolated
to observational access only, we want to have limited causal
impact on the system. OK, let's have two individuals, one as
before only has observational access where the other is allowed
to affect the system. For example:
There are three boxes on a table. Only the recorded observations
of person who cannot affect the system matters and the recording
ony goes on when the other person says one can be made. The
recorder observes that each box with either contain nothing or it
will contain a rock. By labeling the three boxes A, B, and C and
the instances when a rock is a box True, otherwise False, the
observer determines that the boolean function C = A or B holds
for all observations. Is the system of the boxes doing the compuation
or is the system of the boxes plus the person effecting the
changes to contents of the boxes doing the computation? From
what I know of physics, I'm pretty sure that the rock in box C doesn't
exist or not exists simply by putting or taking rocks out of either box
A or B so I'd say it was the system of the boxes and the person
that is doing the computation. The system of the boxes(and
their contents) is doing nothing other than existing as a point of
observation.
Now suppose there is only one box but the observer is irregular
and decides to choose a mapping between a rock being in the
box or not in the box as either true or false depending upon his
whim and actually make three recordings A B and C that the
person effecting the contents of the box can see map to the
boolean function C = A or B, that is it didn't really matter if
the box had a rock in it or not the observer just recorded a
member of the function. In what system can one meaningfully
say any computation is going on? I'd say it was the "observer"
or possibly the system of the box(and its contents) and the observer.
God, this could go on for ever and I'm still not sure I can get
to an uncontested point concerning what it means for a system
to be labelled "computational". Does it appear to you that I've
even headed in the right direction in identifying what we mean
when we call a system "computational" such that we would
be saying something of substance when we made the claim
that we could simulate something computationally?
Alpha
"PeskyBee" <pesk...@gmail.com> wrote in message
news:12r211d...@corp.supernews.com...
I have not read the whole book yet (only to page 50 of a 325pg book) but I
think that is the way he is leaning; that the something else can be
simulated. Give the IA system some goals, enable it to adapt to uncertainty
and unpredicted patterns, and allow it to value its decisions (determine
meaning within the symbol manipulation processes) in terms of energy
efficiency of computation and efficacy toward fullfillment of its goals.
Brains did this as a matter of how a selectional system ,under pressure from
the environment, produces mechanisms that, enable our value-laden mentation.
His four principles (that brain uses BTW - he provides copious examples) for
characterizing computationally efficient machines (as opposed, he
demonstraters, to our current crop of computational machines) are:
- Drain batteries slowly - consequence: compute as slowly and "softly" as
possible
- Save space - consequence: be as imprecise as possible and compress
everything (also, build as few wires as possble and build more short wires
than long wires)
- Save bandwidth - consequence: stay off the lines, don't repeat yourself
and be a s noisy as possible
- Have goals - consequence: physical signals representing goals, desires
(error signals) and values.
He elaborates and provides examples of each.
I'll post updates and his ideas as I read the book.
BTW, he strongly implies and explicity states that today'sompupters do not
have these features and do not have meaning/valuations tied to computations
now (especially not in terms of energy efficiency), bolstering the view that
computers manipulate meaningless symbols.
forbi...@msn.com
> Alpha, thanks for sharing this interesting book. I'll put it
> in my reading list. However, I must confess that I'm somewhat
> skeptical about claims that say symbolic systems can't have
> "meaning". That reminds me of Searle's BS. If brains use
> computation plus "something else", I'm willing to say that this
> something else can be simulated computationally, provided we
> understand what are the functional role of this thing.
OK, I wrote a long article that didn't get anywhere and didn't
get posted. Now I'm going to do the short version. I'll probably
draw the same flack anyway.
If you think Searle's writing is BS then you probably don't understand
it. Here's the issue at hand:
What do you mean by "can be simulated computationally" in
relationship to "meaning"?
I'm assuming that there must be a system that is doing the simuation
and the system can be observed in such a way that it is meaningful
to say it is doing computations. This is a bit of a regress since the
meaningfullness isn't an attribute of the system but of the system and
the attributer and the aquirer of the information the attributer is
trying
to convey.
As it turns out I've never gone very far in my many starts at trying to
do some significant AI work. I'm just a hobbyist. From time to time
I've tried investigating the automatic acquisition of regularities in
bit
streams. I've been wondering if an automation could, without prior
programming that preconditions it, identify 7, 8, 16, etc. bit
chuncking of
the bit stream as contianing meaningful regularities? What is it about
some particular bit patterns that makes them delimiters for the
contents
between them to be identified as English "words"? What is it about the
bit pattern we call the carriage return that makes its meaning
presentational
only?
By labelling a system as doing "symbolic processing" we've already
attributed to it something not in the system but in a mapping made by
an observer of the system and it's relationship to another system.
Neil W Rickert
>OK, I wrote a long article that didn't get anywhere and didn't
>get posted.
I have just finished reading the long article. How is that possible
if it didn't get posted?
>If you think Searle's writing is BS then you probably don't understand
>it. Here's the issue at hand:
Searle's arguments on AI are largely BS, and Searle doesn't fully
understand them either.
> What do you mean by "can be simulated computationally" in
> relationship to "meaning"?
You are correct that "can be simulated computationally" is a weak
claim, though one that AI proponents mistakenly take to be strong.
However, arguments over "meaning" tend to not get anywhere, mostly
because people don't agree on what the word means.
--
DO NOT REPLY BY EMAIL - The address above is a spamtrap.
Neil W. Rickert, Computer Science, Northern Illinois Univ., DeKalb, IL 60115
Don Geddis
> If you think Searle's writing is BS you probably don't understand
> the point he was making.
I understand Searle. His writing is still BS.
> Suppose I can detect voltage at a point in the system and I can map the
> times to when it is in excess to some value to "true" otherwise to "false".
> Likewise I can do the same at another point. We can label sequential and
> regular observations in time, t, as A(t) for one and B(t) for the other and
> map the intervals to intergers. Likewise we have determined that for
> sequential observations, x (a time) B(x+1) = A(t). Are we entitled to
> meaninfully label this a boolean computation? I think we can.
Sounds fine. Looks like computation.
> There are three boxes on a table. Only the recorded observations
> of person who cannot affect the system matters and the recording
> ony goes on when the other person says one can be made. The
> recorder observes that each box with either contain nothing or it
> will contain a rock. By labeling the three boxes A, B, and C and
> the instances when a rock is a box True, otherwise False, the
> observer determines that the boolean function C = A or B holds
> for all observations. Is the system of the boxes doing the compuation
> or is the system of the boxes plus the person effecting the
> changes to contents of the boxes doing the computation?
The system of (boxes + person) seems to be doing the computation.
Just like, in a computer, you need both the RAM and the CPU. Something
to store states, and something to change them.
> Now suppose there is only one box but the observer is irregular and decides
> to choose a mapping between a rock being in the box or not in the box as
> either true or false depending upon his whim and actually make three
> recordings A B and C that the person effecting the contents of the box can
> see map to the boolean function C = A or B, that is it didn't really matter
> if the box had a rock in it or not the observer just recorded a member of
> the function. In what system can one meaningfully say any computation is
> going on? I'd say it was the "observer" or possibly the system of the
> box(and its contents) and the observer.
I really don't understand this example, sorry.
To do a computation, the observer needs to be able to change the inputs,
and to view the output. If the black box keeps the logical relationship,
then it's doing computation.
> Does it appear to you that I've even headed in the right direction in
> identifying what we mean when we call a system "computational" such that we
> would be saying something of substance when we made the claim that we could
> simulate something computationally?
Usually, people arguing over this final phrase are more concerned about
"simulation" (is it "simulated intelligence", or "REAL intelligence"?) than
they are about "computation". I'm surprised that you think computation is
ill-defined.
-- Don
_______________________________________________________________________________
Don Geddis http://don.geddis.org/ d...@geddis.org
When I found the skull in the woods, the first thing I did was call the police.
But then I got curious about it. I picked it up, and started wondering who
this person was, and why he had deer horns.
-- Deep Thoughts, by Jack Handey [1999]
forbi...@msn.com
> forbi...@msn.com wrote on 20 Jan 2007 09:5:
> > Now suppose there is only one box but the observer is irregular and decides
> > to choose a mapping between a rock being in the box or not in the box as
> > either true or false depending upon his whim and actually make three
> > recordings A B and C that the person effecting the contents of the box can
> > see map to the boolean function C = A or B, that is it didn't really matter
> > if the box had a rock in it or not the observer just recorded a member of
> > the function. In what system can one meaningfully say any computation is
> > going on? I'd say it was the "observer" or possibly the system of the
> > box(and its contents) and the observer.
>
> I really don't understand this example, sorry.
OK, I am breathing in and out. I am thinking yes and no.
When the effector asks me to make an observation I assign
True to A if I am breathing in and there is a rock in the box.
I assing True to A if I am breathing out and there is no rock in
the box, otherwise I assign False to A. I use a similar algorithm
to assign B based upon my thinking yes or no at the time I
am asked to observe the box. I assign True or False to C
based upon C = A or B. Now it's clear that all three depend
upon there being or not being a rock in the box at the time
of observation and I do not effect the rock being or not being
in the box. There is compuation going on and the box and rock
are clearly involved in that compuation.
If you want to get into the issue as to what is a valid mapping
for symbolic processing to be going on the we can. I don't
think there can be a clear definition that disqualifies this example
except to the extent the definition is arbitary and not an empirical
observational requirement.
The thing is, what is going on in the universe isn't symbolic
processing
even when we describe it in symbolic terms. A bear's paw print in the
sand is a sign that a bear has passed by it is not a symbol that a bear
has passed by.
Landau Plotken
"Neil W Rickert" <phis...@cs.niu.edu> wrote in message news:k_ush.2781$O02....@newssvr11.news.prodigy.net...
> forbi...@msn.com writes:
>
>>OK, I wrote a long article that didn't get anywhere and didn't
>>get posted.
>
> I have just finished reading the long article. How is that possible
> if it didn't get posted?
>
>>If you think Searle's writing is BS then you probably don't understand
>>it. Here's the issue at hand:
>
> Searle's arguments on AI are largely BS, and Searle doesn't fully
> understand them either.
Searle questions good questions. Searle answer BS. Agree. Now you people
make own answer.
Forbisgaryg not mature. Forbisgaryg similar Harris (xgeorgiou). Lazy brain.
Mouth too big. Maybe grow. Maybe die. Read more and think.
>> What do you mean by "can be simulated computationally" in
>> relationship to "meaning"?
>
> You are correct that "can be simulated computationally" is a weak
> claim, though one that AI proponents mistakenly take to be strong.
Describe behaviour? Too hard. Approximate behaviour same simulate.
But who decide how much approximate? No matter. AI argument make
no difference for good approximate or bad. Too lazy. They not ever CS
Theory. Better answer put AI in Alife and see what survive.
> However, arguments over "meaning" tend to not get anywhere, mostly
> because people don't agree on what the word means.
Ontologist not agree. If meaning not agree, check attributes like Descartes
say 1637. AI people still argue Plato against Aristotle. They say about
Scruffy and Smooth I remember. Must move 1637 and after. Must ask "can do?"
and "how to know can do?".
LP.
feedbackdroid
Alpha wrote:
> >>
> >> "Nature has equipped biological computations with a measure of their
> >> value."
> >>
> >>
> >> Unlike computers,"... biological computations are not lifeless streams of
> >> symbols, totally devoid of meaning [*]. Instead, biological computation
> >> carry something extra - an extra measure of their overall worth. Instead
> >> of just computation, there is computation plus something else, and that
> >> something else is a measure of the value of the computation to the
> >> overall success of the organism, its overall fitness. Biological
> >> computtions know how to care"
...................
>
> BTW, he strongly implies and explicity states that today'sompupters do not
> have these features and do not have meaning/valuations tied to computations
> now (especially not in terms of energy efficiency), bolstering the view that
> computers manipulate meaningless symbols.
>
>
Cool, the term "ompupters" is kind of warm and fuzzy and cute. I like
it. OTOH, "... Biological computtions know how to care ..." is a little
TOO warm+fuzzy+cute for my tastes :).
Thanks for the link to the book. Might have some interesting
perspective. Clearly, the problem with our computer AI's is they're
simply computations, unconnected with the real-world. OTOH, ogranisms
are evolved entities which have **intimate** sensorimotor connection
with the outside world, and with biological goals built in from the
getgo.
High-level reasoning and consciousness and ability for symbol
manipulation didn't spring fully-formed from Zeus' rib, rather it
evolved over hypermillenia on top of much lower-level sensorimotor
consciousness. Compare the difference between Dennett and Edelman's
high-order versus primary sensory consciousness. We got to the second
by evolving through the first. And not the other way around.
If we ever really want to understand the second, we'll probably need to
crack the nut of the first beforehand, and if all we ever do is emulate
top-most logical processes [ie, high-level reasoning] via digital
computers, we're forever stuck in the philosopher's dilemma, voiced by
Searle and others. IOW, you can't get here from there using the means
available, ie ompupters and thought experiments.
Curt Welch
Well, the problem here is that computation is not what the machine is
doing, it's how we choose to describe what it's doing. Computation is a
language based description of behavior. It deals only with absolutes when
the world is mostly full of non absolutes. Either the rock is in the box
or it is not in the box (from you example above). Either something is true
or value. It's equal to 2 or it's equal to 3. It can't be both. Math is
a language of absolutes.
Even when we use the language of absolutes to talk about a continuum, (like
a line) we are forced to map all our concepts back to absolutes as we try
to talk about it, and as a result, problems arise (is 1.99999... the same
as 2.0000...)? The continuous value they try to represent (location on the
number line) is the same, but our representation in the language of
absolute symbols is different - so we are left with a problem in how to
express it using our language of absolute symbols.
When we try to describe the operation of a machine (any physical system) in
terms of these absolute symbols, we run into the same problems. How much
voltage exists in a wire a given point in space? We have problems
representing any location in space precisely with our symbol systems (you
need infinite precision numbers to do it) and we can't measure any value
precisely (again, we need infinite precision numbers).
Computers are symbol manipulators. They do what we do when we manipulate
absolute symbols to represent things in the physical world. But the
symbols we use to describe the action of physical things are not the
physical things himself. So when a computer "simulates" some physical
system (like a weather) it's clearly not being weather. It's just
manipulating symbols.
When it comes the the brain, there's no doubt that a computer simulation of
a brain using symbols is not a brain. But what we don't really know the
answer to, is whether the brain is nothing more than a symbol manipulation
system itself. I happen to believe it is.
Computers can simulate each other as well. I can write a transistor
simulator that models the complex current and voltage characteristics of a
transistor. And I can use a lot of these models to model an entire
computer. Or, since the purpose of the computer I'm trying to model is to
just manipulate symbols, I can skip all that, and just model it at the
level of the symbols it manipulates. And in doing so, I'm not as much
creating a simulation of the first computer, but instead, I've just
duplicated it's symbol manipulation functions in a different machine.
For the brain, we need to figure out if we can duplicate it's important
functions by simply duplicating it's symbol manipulation functions. Will
we be able to create a machine that acts human in all interesting ways by
simply duplicating the brain's basic symbols manipulation techniques? No
one really knows the answer to this yet and I don't see any way to prove it
one way or another. But what is clear, is that the brain is physical doing
complex things other than absolute symbol manipulation. The unanswered
question, is are those functions important for us to duplicate in any AI
machine we build?
I don't believe they are. I think the evidence from everything we have
been able to do with computers and digital signal processing systems makes
it clear that human behavior is nothing more than the operation of a large
symbol processing system. The brain uses symbols to abstractly represent
physical events. The pulse signals coming from the senors are systems that
abstractly represent the sensory events. The pulse signals sent to the
muscles are abstract symbols that represent muscle actions. The brain
doesn't "see" light, and "move" muscles, it just manipulates these abstract
symbols which represent those other things. As such, it's clear to me, the
brain is as much a symbol manipulation system as any computer is. Is there
some type of symbols manipulation the brain is able to do that a computer
can't do? Not likely. But again, I see no way to prove it one way or the
other.
Every time I see someone argue that a brain is doing "more" than
"computing" (aka symbol manipulation) my first impression is always that
the person is just hung up with the belief that they are special and that
they have some magic properties that computers don't have (even if they
have no clue what those properties are or what the actual evidence is that
these properties exist). So they pull all the shit like QM as evidence
that the brain could be something special.
I think we have no way (currently) to prove one way or another who is
right. The people that choose to believe humans are special, and are not
just symbol manipulation machines, are right in the sense that the brain is
doing stuff we don't yet understand - and as such, our computer and digital
signal processing devices might not be able to equal what the brain is
doing. So we can't really prove them wrong.
All we can do, is to continue working on symbol manipulation machines and
see just how much like humans we can make them act. And continue to study
the brain to see how much we can understand about it's total function.
In the end, I'm sure we will make machines that act so much like humans
that we will all agree that the function of the brain is the same as the
function of these machines. And when we get to that point, we will all be
able to describe exactly what that function is - whether we call it symbol
manipulation, or computation, or something else that hasn't been thought up
yet. But until we get to that point, we are all just making educated
guesses.
--
Curt Welch http://CurtWelch.Com/
cu...@kcwc.com http://NewsReader.Com/
Alpha
"feedbackdroid" <feedba...@yahoo.com> wrote in message
news:1169403824.8...@q2g2000cwa.googlegroups.com...
It does; the more I rad the more I appreciate his take on what brain is
doing. He is big on brain doing modeling as well, for predictive *and
learning* reasons! E.g., our imaginantion enables learning while "not on the
actual job". Hawkins is big on feedback as prediction_enabler.
He discusses RL at length too - something Curt should be interested in.
Critics and all that.
>Clearly, the problem with our computer AI's is they're
> simply computations, unconnected with the real-world. OTOH, ogranisms
Cool back to ya: the term "ogranisms" is a reference to granny's orgasims
right! ;^))
> are evolved entities which have **intimate** sensorimotor connection
> with the outside world, and with biological goals built in from the
> getgo.
Yes. They jumpstart/provide_criteria_for learning, attention (what should
the organism pay attention to), and valuation processes.
As goals mature they become more abstract but are drivers nonetheless.
E.g., I want to become President of the USA.
>
> High-level reasoning and consciousness and ability for symbol
> manipulation didn't spring fully-formed from Zeus' rib, rather it
> evolved over hypermillenia on top of much lower-level sensorimotor
> consciousness. Compare the difference between Dennett and Edelman's
> high-order versus primary sensory consciousness. We got to the second
> by evolving through the first. And not the other way around.
Probably; I think even ants have primary sensory consciousness (as I voiced
in my response to Don G. in another thread); that is, consciousness of the
sensory experience, but certainly not high-order/abstraction-formation
consciousness and probably not self-consciousness.
>
> If we ever really want to understand the second, we'll probably need to
> crack the nut of the first beforehand, and if all we ever do is emulate
> top-most logical processes [ie, high-level reasoning] via digital
> computers, we're forever stuck in the philosopher's dilemma, voiced by
> Searle and others. IOW, you can't get here from there using the means
> available, ie ompupters and thought experiments.
But if you ask granny about her orgasims, don't expect to get information
about organisms back.
Alpha
"Curt Welch" <cu...@kcwc.com> wrote in message
news:20070122152011.065$Z...@newsreader.com...
<snip>
> When it comes the the brain, there's no doubt that a computer simulation
> of
> a brain using symbols is not a brain. But what we don't really know the
> answer to, is whether the brain is nothing more than a symbol manipulation
> system itself. I happen to believe it is.
There you go again with the "nothing more" absolutism. I already posted
several things the brain is or is capable of or is or does, each of which is
true. So the brain is not "nothing more than a symbol manipulation system".
It is also all those other things/processes/capabilities I expressed, as
follows:
- pattern recognition
- categorization of recepts, percepts and concepts (and introcepts).
- memory machine (storage/recall/whatever)
- learning machine
- adaptable and configurable/reconfigurable (all those synapse groth
particulars that can be in turn named (there are several)
- genome as blueprint (its all in the interaction of proteins and other
chemicals, which is the interaction of molecules and atoms and thence quarks
and virtual particvles and so forth.) And BTW, the patterns which connect
one level of description to another so that we may *map* one level of
description onto another in terms of transforms adn constraints!
- Generator of a self
- Generator of consciousness processes
- experiencer of qualia and self and consciousness (and consciousness of
consciousness ad infinitum if you want to spend the time...)
- Has some degenerate processes (see Edelman)
- Auto-associative machine
- Feedback implementation machine
- Reaction machine
- Prediction machine
And I'll add more I thought of
- a modeling machine
- a symbol manipulation machine
- an energy_efficiency monitoring and control machine
Some questions one has to answer if one thinks brain is manipulating
sysmbols are :
What are the representations of those symbols?
How are they "manipulated" or "computed"?
When do symbols come into being in the brain? At birth or in fetal stage?
As brain starts sensing the environment? As brain learns to abstract
environment into models of same (i.e., *with* symbols for things/processes)?
And so forth.
<snip>
> But what is clear, is that the brain is physical doing
> complex things other than absolute symbol manipulation.
Well - above you generated a statement that contradicts your earlier
statement.
> The unanswered
> question, is are those functions important for us to duplicate in any AI
> machine we build?
>
> I don't believe they are. I think the evidence from everything we have
> been able to do with computers and digital signal processing systems makes
> it clear that human behavior is nothing more than the operation of a large
> symbol processing system.
That is ridiculous. Brain is doing lots of other things that are not symbol
processing that matter to its other operations to boot. E.g., energy
efficiency management is critical (see Montague's book Why Choose This Book"
for copious examples and reasoning) to most other operations of brain,
including symbol manipulation (if brain is even doing that the way computers
do so).
> The brain uses symbols to abstractly represent
> physical events. The pulse signals coming from the senors are systems
> that
> abstractly represent the sensory events. The pulse signals sent to the
> muscles are abstract symbols that represent muscle actions.
Exactly what is the abstract symbol then?
> The brain
> doesn't "see" light, and "move" muscles, it just manipulates these
> abstract
> symbols which represent those other things. As such, it's clear to me,
> the
> brain is as much a symbol manipulation system as any computer is. Is
> there
> some type of symbols manipulation the brain is able to do that a computer
> can't do? Not likely. But again, I see no way to prove it one way or the
> other.
>
> Every time I see someone argue that a brain is doing "more" than
> "computing" (aka symbol manipulation) my first impression is always that
> the person is just hung up with the belief that they are special and that
No, it is a fact that brain is doing much more than symbol manipulation. But
no magical properties involved (as far as we can tell).
> they have some magic properties that computers don't have (even if they
> have no clue what those properties are or what the actual evidence is that
> these properties exist). So they pull all the shit like QM as evidence
> that the brain could be something special.
It is a fact that QM underlies and subserves all higher-level operations in
Universe. Take it away and all the other things that happen in brain go away
as well. It is a necessary (but not suffcient) *part* of the *whole* of
processes Universe.
>
> I think we have no way (currently) to prove one way or another who is
> right. The people that choose to believe humans are special, and are not
Has zippo to do with being special, as all things in Unverse are based on QM
(and perhaps other even lower-level phenomena that key physicists like
Feynman thought physics we have not explored or found yet.)
> just symbol manipulation machines, are right in the sense that the brain
> is
> doing stuff we don't yet understand - and as such, our computer and
> digital
> signal processing devices might not be able to equal what the brain is
> doing. So we can't really prove them wrong.
>
> All we can do, is to continue working on symbol manipulation machines and
> see just how much like humans we can make them act.
And so far the results are absymal.
> And continue to study
> the brain to see how much we can understand about it's total function.
As opposed to its *partial* function, *perhaps* as a symbol manipulator.
>
> In the end, I'm sure we will make machines that act so much like humans
> that we will all agree that the function of the brain is the same as the
> function of these machines. And when we get to that point, we will all be
> able to describe exactly what that function is - whether we call it symbol
> manipulation, or computation, or something else that hasn't been thought
> up
> yet. But until we get to that point, we are all just making educated
> guesses.
>
> --
> Curt Welch
> http://CurtWelch.Com/
> cu...@kcwc.com
> http://NewsReader.Com/
--
feedbackdroid
Sorry, there's only room for one wiseguy on this forum. Me, not you.
BTW, I got over to Barnsesnoble to check out Montague's book. So far,
I'm very equivocal about it. First, no pictures :), and lots of
rambling. Very loose analogies. More importantly [from memory], about
page 65, his idea of the "mutual modeling principle", and the 2nd rule
of efficient computation = model everything, I feel are well off the
mark.
He gives an anecdote about a husband and wife driving to the same
airport from different directions, and how each must have a model of
the other's actions in their minds in order to eventually link up
successfully, given the limited amount of intercommunications available
to them [must be before the days of cell phones]. Then he tries to
extend this analogy to brain operation.
He states that brain areas which form reciprocal connections with other
areas "must" each both model themselves and also model the other areas
to which they're connectioned. This must be done to "reduce
intercommunications" in the brain, apparently in order to improve
energy efficiency, but this is plain impossible. He seems to think that
producing local models takes less energy than sending signals to other
areas.
In fact, a brain area will do well to create even "one" local model,
let alone multiple local models simultaneously. And this, especially
given that, on average, each cortical area is connected to close to
HALF of the other cortical areas, as Edelman is very fond of
illustrating in his various books. Even if a brain area does create
some kind of local internal model, it seems ridiculous to think that it
could also produce DOZENS of additional models of what other areas are
doing simultaneously. Naw.
Don Geddis
> Computers can simulate each other as well. I can write a transistor
> simulator that models the complex current and voltage characteristics of a
> transistor. And I can use a lot of these models to model an entire
> computer. Or, since the purpose of the computer I'm trying to model is to
> just manipulate symbols, I can skip all that, and just model it at the
> level of the symbols it manipulates. And in doing so, I'm not as much
> creating a simulation of the first computer, but instead, I've just
> duplicated it's symbol manipulation functions in a different machine.
I think this is actually a very insightful example.
It may be an interesting way to address people who worry "is the computer
REALLY intelligent, or just SIMULATING intelligence?" or "is the computer
REALLY conscious, or just SIMULATING consciousness?" The example people
always bring up is weather simulations, and argue that a simulation of a
thunderstorm is clearly not a real thunderstorm.
But your example is better. Consider a calculator. It's a physical device,
with voltages and transistors. It can also be modelled as a computational
device, with numbers and inputs and math operations as the function.
If you wanted to SIMULATE the calculator in another computer, you might
create a physics model, and code in the wiring diagram of the calculator,
and try to simulate as accurately as possible the rise and fall of voltages
across all the lines over time. And your simulation will always be a little
short of reality. And in any case, it will never really BE the calculator.
But if you realize that the calculator is really just acting as a symbol
processing device, you can abstract away the physics. Then you can construct
a new computation in the new computer, which achieves the same function as
the computation that the calculator has. This new algorithm differs from
the original calculator in many way: probably the different operations don't
take the same amount of time to complete as they did in the original device.
What you have now is no longer a SIMULATION of the calculator; instead what
you have is ANOTHER computational device, which performs the SAME computations
as the original calculator's functions.
That's what AI is doing. Attempting to build a computational device with
(about) the same I/O behavior as a human brain. AI is NOT trying to SIMULATE
any particular human brain.
-- Don
_______________________________________________________________________________
Don Geddis http://don.geddis.org/ d...@geddis.org
When I was in school, I cheated on my metaphysics exam: I looked into the soul
of the boy sitting next to me. -- Woody Allen
Curt Welch
> He discusses RL at length too - something Curt should be interested in.
> Critics and all that.
Ah, maybe he has hope afterall. :)
Vend
On 19 Gen, 17:54, "Alpha" <OmegaZero2...@yahoo.com> wrote:
> Read Montague's new book: "Why Choose This Book: How We Make Decisions",
> contains many gems. One of which, and the focus of the book is:
>
> "Nature has equipped biological computations with a measure of their value."
>
> Read is professor of neuroscience at Baylor College of medicine, Director of
> the Neuroimaging Lab, director of the Cente for Theoretical Neuroscience and
> a fellow at the Institute for Advanced Study at Princeton.
>
> Read visits the COTM (Computational Theory of Mind - if you do not know what
> this refers to, in a nutshell, it is that mind is software running on the
> brain; and that the mind is not equivalent to brain just as Microsoft Word
> is not the logic gates that it runs on) and adheres to its tenets, along
> with Turings ideas on computation. However...
But the machine (the logic gates) Word is running on can be emulated in
software, and Word can be in principle implemented in hardware,
therefore, formally, there is no difference.
> Unlike computers,"... biological computations are not lifeless streams of
> symbols, totally devoid of meaning [*]. Instead, biological computation
> carry something extra - an extra measure of their overall worth. Instead of
> just computation, there is computation plus something else, and that
> something else is a measure of the value of the computation to the overall
> success of the organism, its overall fitness. Biological computtions know
> how to care"
This looks like a loose definition of reinforcement learning, which
*is* symbolic processing.
> He goes on to tell how brain implements goals, and how the energy efficiency
> of a computation, selected via evolutionary mechanisms, puts meaning into
> neural computations, assigning values and passing the values from one symbol
> to another. Note that energy allocation schemes are missing from computers
> today (which is one reason why your brain remains cool and your computer's
> CPU is hot to the touch! All computations in today's computers get equal
> energy billing.) Not so with biological computations, which are almost
> "miraculously efficient", WRT energy Read says.
Actually energy efficiency is a major consideration in mobile devices.
zzbu...@netscape.net
On Jan 19, 11:54 am, "Alpha" <OmegaZero2...@yahoo.com> wrote:
> Read Montague's new book: "Why Choose This Book: How We Make Decisions",
> contains many gems. One of which, and the focus of the book is:
>
> "Nature has equipped biological computations with a measure of their value."
>
> Read is professor of neuroscience at Baylor College of medicine, Director of
> the Neuroimaging Lab, director of the Cente for Theoretical Neuroscience and
> a fellow at the Institute for Advanced Study at Princeton.
>
> Read visits the COTM (Computational Theory of Mind - if you do not know what
> this refers to, in a nutshell, it is that mind is software running on the
> brain; and that the mind is not equivalent to brain just as Microsoft Word
> is not the logic gates that it runs on) and adheres to its tenets, along
> with Turings ideas on computation. However...
>
> Unlike computers,"... biological computations are not lifeless streams of
> symbols, totally devoid of meaning [*]. Instead, biological computation
> carry something extra - an extra measure of their overall worth. Instead of
> just computation, there is computation plus something else, and that
> something else is a measure of the value of the computation to the overall
> success of the organism, its overall fitness. Biological computtions know
> how to care"
That's not really a new theory though.
It's a 21st century version of cartesian dualism,
Since Microsoft word doesn't actually
run anywhere, other than in library
reference manuals, Which are located
just across the aisle from the
Egyptology section.
feedbackdroid
zzbu...@netscape.net wrote:
> >
> > Unlike computers,"... biological computations are not lifeless streams of
> > symbols, totally devoid of meaning [*]. Instead, biological computation
> > carry something extra - an extra measure of their overall worth. Instead of
> > just computation, there is computation plus something else, and that
> > something else is a measure of the value of the computation to the overall
> > success of the organism, its overall fitness. Biological computtions know
> > how to care"
>
>
> That's not really a new theory though.
> It's a 21st century version of cartesian dualism,
> Since Microsoft word doesn't actually
> run anywhere, other than in library
> reference manuals, Which are located
> just across the aisle from the
> Egyptology section.
>
>
Yes, that particular passage is somewhat reflective of dualism. If
that's not what the author had in mind, he should probably have phrased
it differently. I already made comment to Alpha regards the overly
warm+fuzzy nature of the last sentence in the quote above.
Alpha
Hmmm; I don;t think so, the more a sender-receiver (of signals/messages)
know about ach other, the more they can craft such a message with less and
less ambiguity in the message; i.e., it can be crafted to fit the input
needs of the receiver (no translations necessary etc.) leading to more
efficient use of bandwidth and computing resources.
>
> He gives an anecdote about a husband and wife driving to the same
> airport from different directions, and how each must have a model of
> the other's actions in their minds in order to eventually link up
> successfully, given the limited amount of intercommunications available
> to them [must be before the days of cell phones]. Then he tries to
> extend this analogy to brain operation.
>
> He states that brain areas which form reciprocal connections with other
> areas "must" each both model themselves and also model the other areas
> to which they're connectioned. This must be done to "reduce
> intercommunications" in the brain, apparently in order to improve
> energy efficiency, but this is plain impossible. He seems to think that
> producing local models takes less energy than sending signals to other
> areas.
>
> In fact, a brain area will do well to create even "one" local model,
> let alone multiple local models simultaneously.
Why do you say this?
>And this, especially
> given that, on average, each cortical area is connected to close to
> HALF of the other cortical areas, as Edelman is very fond of
> illustrating in his various books. Even if a brain area does create
> some kind of local internal model, it seems ridiculous to think that it
> could also produce DOZENS of additional models of what other areas are
> doing simultaneously. Naw.
Ok; perhaps the "must" is too overbearing. Possible may be more like it.
For complex interactions between complex local neuronal groups I can see
modelling overcoming the communications bottleneck in terms of efficiency.
I don;t think each neuron models the activities of each and every other
neuron that it may communicate with.
Alpha
> "Alpha" <OmegaZ...@yahoo.com> wrote:
>
>> He discusses RL at length too - something Curt should be interested in.
>> Critics and all that.
>
> Ah, maybe he has hope afterall. :)
Indeed; I think you should pick up the book; he has lots to say (as I
continue reading) about such systems. I am beginning to see your points more
clearly now.
One dissonance with a conversation/thread that posed ideas about learning
from a "blank slate". Montague states quite explicity that there is no
learning system that starts with a blank slate in nature, to wit: " Without
a lot of hard-won assumptions about the learning problems, all learning
systems, including biological ones, would be rather helpless." and "The
idea that a mobile, adaptive creature starts as a blank slate is
contradicted by almost as much evidence as there is to support evulution by
natural selection." and " All animals start out this way [with some
assumptions about the problem space]. they all begin with rich models of
what they must learn and roughly how to do it."
I would think that artificial blank-slate learning systems would be equally
helpless; they need some historically- or otherwise-based information about
the nature of the problem (like categorization etc.) and an
architecture/basic_processes to support its learning a resolution.
>
> --
> Curt Welch
> http://CurtWelch.Com/
> cu...@kcwc.com
> http://NewsReader.Com/
--
Alpha
"Don Geddis" <d...@geddis.org> wrote in message
news:87ac09i...@geddis.org...
While that is true, the example does not give any ideas on how to approach
systems that have experience or consciousness or a self. One cannot assume
that one can abstract away *anything* in brain, because in so abstracting,
one can always miss properties and processes that contribute intimately to
the process. All that said, I am further saying that one must be careful
exactly what it is that is being abstracted/simulated. If one focuses on
(simulating) APs for example, then all the HPTA-axis functions are missed
and that such an abstraction would not come close to representing (or
simulating) the richness and architectural complexity of brain's functions;
i.e., it would NOT simulate the same computations as the original brain. It
is a question of completeness in description/abstraction.
But I agree with your take on what AI is doing and not doing.
>
> -- Don
> _______________________________________________________________________________
> Don Geddis http://don.geddis.org/
> d...@geddis.org
> When I was in school, I cheated on my metaphysics exam: I looked into the
> soul
> of the boy sitting next to me. -- Woody Allen
--
Michael Olea
<...>
> From time to time I've tried investigating the automatic acquisition of
>regularities in bit streams. ...
That is a problem that has been investigated in great depth, from many
angles. William Bialek, coauthor of "Spikes: Exploring the Neural Code", and
his colleagues have published some ground-breaking papers. The original
impetus for that work, I think, comes from the analysis of neural spike
trains, which can be treated as bit streams.
Another impetus comes from data compression, for example, the "entropy
encoder" that is the back-end of the JPEG standard (either an arithmetic
coder, or an adaptive Huffman coder).
Another context comes from solid-state physics - Ising systems and spin
glasses, for example (these are idealized models - 1D strings of components
each of which can be in one of two states: positive or negative spin).
Yet another context comes from the EAB - "choice dynamics" under two
concurrent VI (variable interval) schedules of reinforcement, for example.
Another is dynamical systems theory (e.g. the "symbolic dynamics" of the
logistic equation).
Another comes from basketball: do basketball players have "hot streaks" at
the free-throw line?
Should you believe that this coin is fair? W Bialek, q-bio.NC/0508044.
http://www.princeton.edu/~wbialek/our_papers/bialek_05.pdf
Suppose for starters that we are limited to observation of the bit-stream
itself - no experiments, and no other variables, just the stream of bits to
work with. The task can be formulated as predicting the future of the
stream from its past. Thinking of this as a time series, is there a
"compact" model for the dynamics of the stream? How quickly do various
abstract learning machines converge on a good model? That line of enquiry
leads to complexity classes (not the same as the P/NP notion, but rather
the complexity of the dynamics of a time-series) and a hierarchy of
hypothesis spaces (or learning machines) with trade-offs in speed of
learning, accuracy of prediction, and sensitivity to non-stationary
(changing) dynamics.
> ...I've been wondering if an automation could, without prior
> programming that preconditions it, identify 7, 8, 16, etc. bit
> chuncking of the bit stream as contianing meaningful regularities?
It depends on what you mean by "preconditions it". I'm guessing you mean
without being coded to chunk data into those specific boundaries. The
answer is a qualified yes. An example is LZW compression. I don't know if
you are familiar with the scheme or not - it works by building up a table
of substrings, and outputing the index rather than the substring. It starts
out building short strings, and progressively longer ones. So if
(sub)strings are repeated in the data stream this results in compression,
and the process will have detected regularities of various lengths.
But suppose the bit-stream is already compressed. Suppose the sequence of
bits has been decorrelated so that each bit is one or zero with equal
probability and independent of preceeding bits - a sequence of results of
flipping a "fair coin". Then all substrings occur with equal probability.
The stream cannot be compressed. Its "Kolmogorov complexity" is maximal -
there is no shorter description of the stream than the stream itself.
But this measure of complexity is unsatisfactory in that in this case there
is a simple, one parameter, model of the dynamics of the stream: P(bit=1) =
P(bit=0) = 1/2. In this case, given that you already know that p = 1/2, the
past provides no information about the future - all substrings are equally
likely, so you cannot predict the next bit given the previous n bits. You
can, however, predict the fraction of bits that will be on or off; you can
predict the probability that out of the next n bits k will be on.
But of course you don't know in advance that p = 1/2, or even that there is
a single number p that characterizes what there is to know about the
bit-stream.
Suppose you start out by making that assumption - that there is a single
number p that characterizes the dynamics, you just don't know what it is.
This is a hypothesis space - a bias over the sort of regularities to be
expected. It is a very simple hypothesis space - its "complexity" can be
quantified as the number of bits of resolution of p. This is, at least to
me, a somewhat subtle point, which is why I belabor it. To specify an
arbitrary real number in the interval [0, 1] exactly would require an
infinite number of bits. So if it were really true that a particular
bit-stream behaved like a sequence of flips of a coin with a bias p, then,
in a sense, the stream provides infinite information about p in that you
can continually refine your estimate of p. But you are learning less and
less about p with continued observation, narrowing it to an ever smaller
subinterval of [0, 1]. The rate at which you are improving your ability to
predict the behavior of the stream is diminishing rapidly. That you can
continue to refine the estimate of p without bound is not interesting in
terms of characterizing the complexity of the bit-stream or the complexity
of the hypothesis space, or the difficulty of the learning process. The
model space is low dimensional - one parameter, and if the bit-stream can
be characterized this way its complexity is low - one parameter. If you
further adopt the convention that for any given, particular, practical
purpose (like landing a rover on Mars) n bits of resolution for your
parameters are good enough, then the bit-stream provides at most n-bits of
predictive information. In that sense its complexity is finite, as is the
complexity of any time-series that can be fully characterized by a finite
number of parameters - the "finite parametric models" class of hypothesis
spaces.
It is a huge assumption, of course, that a bit-stream belongs to this class,
the one parameter family. But in domains where it happens to be true, a one
parameter learning machine is optimal: for any given number of observations
it will make more accurate predictions than, say, a two parameter learning
machine, or a non-parametric learning machine. Suppose, though, that the
assumption is wrong - how will a one parameter machine perform? It depends
on how the assumption is wrong. A one parameter machine might still be
"optimal" if quick, rough estimates are more important than more detailed
and more accurate predictions that take longer to learn.
Before leaving finite parametric models consider markov chains.
A two state markov chain is fully specified by 6 parameters: a 2x2
transition matrix and a 2-element initial state vector. In fact there are
only 3 degrees of freedom since these are probabilities (the probability of
a transition from state 1 to state 1, the probability of transition from
state 1 to state 2, state 2 to state 1, state 2 to state 2, the probability
of starting out in state 1, and the probability of starting out in state
2). The numbers are not independent since, for example, the probability of
a transition from state 1 to state 1 plus the probability of a transition
from state 1 to state 2 equals 1: P(1|1) + P(2|1) = 1. So it just takes 3
numbers to fully specify the model. And if we are not concerened with
predicting in advance the initial state of a bit-stream, just with
predicting its future from its past, then this is a 2 parameter model
space. It won't find the regularities you mentioned because it does not have
enough memory - it looks one bit into the past to predict the next bit. It
will find more regularities, if they exit, than the one-parameter model,
and make more accurate predictions, but it will take longer to do so - it
has a higher dimensional space of possibilities to explore. It is
estimating 4 frequencies (00, 01, 10, 11) instead of 2 (0, 1). How would a
one parameter machine perform on a stream generated by a 2-state Markov
process? Just to keep things simple suppose the transition matrix is:
[3/4 1/4]
[1/4 3/4]
On average the number of 1-bits equals the number of 0-bits, so p = 1/2, and
the 1-parameter model above will accurately predict the ratio of 1-bits to
0-bits, but it will underestimate the probability of runs of the same bit.
It will also overestimate the entropy of the stream: 1-bit per symbol rather
than about 0.8 bits per symbol. So it gets some things right and some
things wrong.
Notice that this bit-stream has the same complexity as a sequence of
independent coin flips - 1 parameter - due to the symmetry of the matrix:
[p 1-p]
[1-p p]
Its Kolmogorov complexity is lower (unless p = 1/2), since it is
compressible, but in terms of the dynamics of the stream, and in terms of
the difficulty of learning (prediction accuracy as a function of the number
of observations) the two streams have identical complexity. Notice also
that the entropy of the bit-stream does not quantify the difficulty of
learning a model of its dynamics - the two streams have different
entropies, but both are characterized by a single parameter.
A measure that does quantify the complexity of any model underlying the
dynamics of a time-series is "predictive information" (aka "excess
entropy") - the mutual information between the future and the past. If that
information is finite (for a given level of resolution) then the dynamics
are characterized by a finite parametric model - and the predictive
information tells you how many "effective" parameters that model must have
to capture the dynamics of the stream. In other words, I started with a
2-parameter parametric family:
[p 1-p]
[1-q q]
But the actual dynamics live in the subspace of models:
[p 1-p]
[1-p p]
The predictive information counts the parameters, but does not, on its own,
indicate the particular parametric family.
Suppose we add one more bit of memory to the Markov chain model - predict
the next bit on the basis of the previous 2 bits. The general transition
matrix is of the form:
|00 01 10 11
===+================+
00 [ p 1-p 0 0]
01 [ 0 0 q 1-q]
10 [ r 1-r 0 0]
11 [ 0 0 s 1-s]
So 4 degrees of freedom. The way to read this is that each state follows
from the previous by sliding a 2-bit window one bit to the right - you
cannot get from 00 to 10 or 11 no matter what the next bit is:
|00|???? -> 0|0?|???
You can only get to 00 or to 01.
Now further suppose that from a 1-D model we have learned that, on average,
over the long run, the number of 1-bits equals the number of 0-bits. This
adds some constraints on p, q, r, and s.
First, we cannot have any "absorbing states", meaning that once the state is
entered it can never be exited, which would be the case if p = 1, or s = 0.
But we could have, for example, p = 0 and s = 1:
|00 01 10 11
===+================+
00 [ 0 1 0 0]
01 [ 0 0 0 1]
10 [ 1 0 0 0]
11 [ 0 0 1 0]
This would be a fully deterministic system, an orbit of period 4:
00->01->11->10->00 = 001100110011...
In general, any matrix with r = 1-p, s = 1-q, p in [0, 1), q in [0, 1), will
result in the number of 0's and 1's equal on avereage, and the entropy of
the bit-stream will range from 0 to 1 bit per symbol, depending on the
values of p and q. This is a 2 parameter hypothesis space:
| 00 01 10 11
===+===================+
00 [ p 1-p 0 0]
01 [ 0 0 q 1-q]
10 [ 1-p p 0 0]
11 [ 0 0 1-q q]
We can, of course, continue to add states, looking more bits into the past,
at the cost of more memory, and longer learning times. The number of
states is exponential in the number of bits, but if the models can be
nested, lower dimensional models providing summary statistics constraining
the higher dimensional models, then the growth is subexponential.
Adding more memory, looking further back into the past, raises the absolute
complexity of the hypothesis space, but remains in the class of finite
parametric models, finite predictive information. In the limit the sequence
depends on its entire history - the future behavior depends on all of the
past. That marks a phase transition in the complexity of the dynamics
underlying the time series: divergent growth of predictive information. The
longer you observe the more there is to learn, even for finite resolution
of model parameters, though the amount you can learn per observation is
diminishing. It's a little like "escape velocity" - below escape velocity,
a rocket will reach a maximum altitude and then fall back to Earth, above
escape velocity it will slow down forever, but never reach zero, never
return (another phase-transition-like threshold).
Now consider a stochastic context free grammar - say strings of balanced
parantheses, with the encoding '(' = '0', ')' = '1'. A CFG for this language
is:
S -> '(' S ')' S | empty-string
This generates all finite-length strings of balanced parentheses. We can
make it stochastic by attaching probabilities to the productions:
S -> [p] '(' S ')' S | [1-p] empty-string
This is another case where, over the long run, the number of 0-bits equals
the number of 1-bits, but the long run can be arbitrarily long. There is no
finite number of bits into the past we can look at to predict the future of
the time-series. There are "long range correlations" in the data stream.
This is the case, for example, with RNA sequences, where 3-D folding leads
to long range correlations in nucleotides.
That's one way to make the grammar stochastic. It will generate perfectly
balanced strings only, but there is a distribution over nesting depth.
Another approach is to generate "nearly balanced" strings, with some
probability of violating perfect balance:
S -> [p] '(' S ')' S | [q] '(' S | [q] ')' S | [1-p-q] empty-string
This again has the property that the number of 1's equals the number of 0's,
and if q is much less than p, will be nearly balanced, with some
"impurities doping" the string.
So - regularities in the bit-stream can be "acquired", but not without
*some* kind of "preconditioning", some form of prior over hypothesis
spaces. But that is also true of people. Evolution equips us with priors.
> ...What is it about some particular bit patterns that makes them
>delimiters for the contents between them to be identified as English
>"words"?
It is possible to infer unit boundaries from co-ocurrence stats. That will
not identify certain bit patterns as "delimiters", but it will identify
recuring subunits. You can avoid combinatorial explosion by using
hierarchical models - assuming, of course, there is structure, regularity,
at different scales. So delimiters can be identified as high-frequency
patterns occuring at characteristic spacings (not many words are longer
than 10 letters) and with little or no mutual information between delimiter
and next or previous bit-patterns. You can build a machine that learns to
"tokenize" the bit-streams from the bit-streams alone, but not to attach
any semantics to the tokens other than as predictors of other tokens. For
that the machine would need other modalities - something with which to
correlate the dynamics of the stream.
> What is it about the bit pattern we call the carriage return that makes
>its meaning presentational only?
Nothing - that meaning emerges from the response of a device, like a tty, to
the bit-stream.
> By labelling a system as doing "symbolic processing" we've already
> attributed to it something not in the system but in a mapping made by
> an observer of the system and it's relationship to another system.
I am not sure why you think that is significant. Is there some form of
learning that is not discovery of relations? In any case, one way to
quantify information processing is by measuring the reduction in entropy of
system responses conditional on stimulus ensembles - coming under stimulus
control, or, if you prefer, synchronizing to the environment.
-- Michael