Responses to Wednesday: Continuous, real-valued brain?

[Shawn Taylor]

One of the postulates Wednesday seemed to be that the brain is a
continuous, real valued system (as opposed to a discrete binary system
ala most computer chips). It seems to me that the brain is a discrete
system. Nerve cells work (generically) by accumulating signal up to a
threshold and then firing an action potential down their axon. This
sounds like discretization. The transfer of this action potential to
other neurons is modulated by some discrete number of neurotransmitter
molecules present in the synaptic gap. So, at a basic level, the
brain is discrete. The question then, is at what level do "brain
algorithms" operate? Whatever level they operate at, because there is
an underlying discrete process to that operation, there will only be
continuous accuracy down to some level. This is the same boat we are
in with microelectronics. A 32-bit adder looks continuous until you
burrow down past the accuracy of 32-bits. Likewise, the brain looks
continuous until you borrow down past the accuracy of a single action
potential. Could it be, then, that the question we should be asking
is, "are we using enough accuracy to correctly model the brain?"

I think it's a great idea to develop an implemented general purpose
continuous valued algorithmic schema. We likely are losing a lot of
processing power potential by discretizing the way we do (we being
common engineering practice). I just don't see that the brain is
necessarily an existence proof of the power of continuous processing.
Maybe it is, we are still left with the fact that the brain is running
continuous algorithms on top of a discrete substrate (action
potentials).

Doing some back-of-the-envelope calculations; the brain has 10^11
neurons running at nominally 1kHz (rough frequency of action
potentials). That gives 10^14 binary operations per second. A flop
is a floating point operation, so very very roughly we might say the
brain has a peak performance of 10^13 flops. I.e. 10 teraflops.
Obviously we are doing something wrong when we can't simulate more
than a small portion of the brain using 10 teraflops. This
inefficiency was eluded to Wednesday with the statement that the
correct 100,000 lines of code could probably simulate the entire brain
on current hardware. Where does our main problem lie? Maybe the
brain is outperforming our best artificial efforts because is makes
better use of it's discrete substrate (I would label this better
discrete algorithms). Maybe the brain outperforms us because it runs
better continuous algorithms.

From an information theory standpoint, the complexity of these
algorithms (whatever kind they be) can only be a function of our
genetic code along with the inputs received during development of the
brain. Yes, there exists a fair amount of plasticity even after
development, but I think it's fair to says that the brain's
"processing algorithms" are fairly well fixed by adulthood. The data
observed during development is massive, but the genetic kernel by
which that data results in a functioning brain is relatively small (as
was mentioned in class). I point this out because another potential
method for replicating and/or bettering the brain is to come at the
problem from a developmental standpoint. If we can understand how the
algorithms form, we should be able to better understand where they end
up.

-shawn