We
are in the midst of a scientific and technological revolution. The
computers of today use artificial intelligence to learn from example and
to execute sophisticated functions that, until recently, were thought
impossible. These smart algorithms can recognize faces and even drive
autonomous vehicles. Deep learning networks, which are responsible for
many of these technological advances, are based on the same principles
that form the structure of our brain: they are composed of artificial
nerve cells that are connected to one another through artificial
synapses; these cells send signals to one another via these synapses.
Our
basic understanding of neural function dates back to the 1950's. Based
on this elementary understanding, present-day artificial neurons that
are used in deep learning operate by summing their synaptic inputs
linearly and generating in response one of two output states—"0" (OFF)
and "1" (ON). In recent decades, however, the field of neuroscience has
discovered that individual neurons are,
built from complex branching system that contains many functional
sub-regions. Indeed, the branching structure of neurons and the many
synapses that contact it over its distributed surface area implies that
single neurons might behave as an extensive network whereby each
sub-region its own local, that is, nonlinear input-output function.
New
research at the Hebrew University of Jerusalem (HU) seeks to understand
the computing power of a neuron in a systematic manner. If one maps the
input-output of a neuron for many synaptic inputs (many examples), then
one may be able to examine how "deep" an analogous network should be in
order to replicate the I/O characteristics of the neuron. Ph.D.
student, David Beniaguev, along with Professors Michael London and Idan
Segev, at HU's Edmond and Lily Safra Center for Brain Science (ELSC)
have undertaken this challenge and have published their findings in Neuron.
The objective of the study is to understand how individual nerve cells, the building blocks of the brain,
translate synaptic inputs to their electrical output. In doing so, the
researchers seek to create a new kind of deep learning artificial
infrastructure, that will act more like the human brain and
produce similarly impressive capabilities as the brain does. "The new
deep learning network that we propose is built from artificial neurons
whereby each of them is already 5-7 layers deep. These units are
connected, via artificial synapses, to the layers above and below it,"
Segev explained.
In
the present state of deep neuronal networks, every artificial neuron
responds to input data (synapses) with a "0" or a "1", based on the
synaptic strength it receives from the previous layer. Based on that
strength, the synapse either sends (excites) —or withholds (inhibits) —a
signal to neurons in the next layer. The neurons in the second layer
then process the data that they received and transfer the output to the
cells in the next level etc. For example, in a network that is supposed
to respond to cats (but not to other animals), this network should
respond for a cat with a "1" at the last (deepest) output neuron, and
with a "0" otherwise. Present-state deep neuronal networks demonstrated
that they can learn this task and perform it extremely well.
This
approach allows computers in driverless cars, for example, to learn
when they have arrived at a traffic light or at a pedestrian
crossing—even if the computer has never before seen that specific
crosswalk. "Despite the remarkable successes that are defined as a 'game
changer' for our world, we still don't completely appreciate how deep
learning is capable of doing what it does and many people across the
world are trying to figure it out," Segev shared .
The
ability of each deep-learning network is also limited to the specific
task that it's being asked to perform. A system that was taught to
identify cats isn't able to identify dogs. Furthermore, a dedicated
system needs to be in place to detect the connection between meow and
cats. While the success of deep learning is amazing for specific tasks,
these systems lag far behind the human brain in their ability to
multi-task. "We don't need more than one driverless car accident to
realize the inherent dangers in these limitations," Segev quipped.
Currently,
significant research is being focused on providing artificial deep
learning with more intelligent and all-encompassing abilities, such as
the ability to process and correlate between different stimuli and to
relate to different aspects of the cat (sight, hearing, touch, etc.) and
to learn how to translate those various aspects into meaning. These are
capabilities at which the human brain excels and those which deep
learning has not yet been able to achieve.
"Our
approach is to use deep learning capabilities to create a computerized
model that best replicates the I/O properties of individual neurons in
the brain," Beniaguev explained. To do so, the researchers relied on
mathematic modeling of single neurons, a set of differential equations
that was developed by Segev and London. This allows them to accurately
simulate the detailed electrical processes taking places in different
regions of the simulated neuron and to best map the complex
transformation for the barrage of synaptic inputs and the electrical
current that they produce through the tree-like structure (dendritic
tree) of the nerve cell. The researchers used this model to seek for a
deep neural network (DNN) that replicated the I/O of the simulated
neuron. They found that this task is achieved by a DNN of 5-7 layers
deep.
The team hopes that building deep-learning networks based
closely on real neurons which, as they have shown, are already quite
deep on their own, will enable them to perform more complex and more
efficient learning processes, which are more similar to the human brain.
"An illustration of this would be for the artificial network to
recognize a cat with fewer examples and to perform functions like
internalizing language meaning. However, these are processes that we
still have to prove possible by our suggested DNNs with continued
research," Segev stressed. Such a system wouldn't just mean changing the
representation of single neurons in the respective artificial neuronal
network but also combine in the artificial network the characteristics
of different neuron types, as is the case in the human brain. "The end
goal would be to create a computerized replica that mimics the
functionality, ability and diversity of the brain—to create, in every
way, true artificial intelligence," Segev said.
This
study also offered the first chance to map and compare the processing
power of the different types of neurons. "For example, in order to
simulate neuron A, we need to map seven different levels of deep learning from
specific neurons, while neuron B may need nine such layers," Segev
said. "In this way, we can quantitatively compare the processing power
of the nerve cell of a mouse with a comparable cell in a human brain, or
between two different types of neurons in the human brain."
On
an even more basic level, the development of a computer model based on a
machine learning approach that so accurately simulates brain function
is likely to provide new understanding of the brain itself. "Our brain
developed methods to build artificial networks that replicate its own
learning capabilities and this in return allows us to better understand
the brain and ourselves," Beniaguev said.