Porting a model with delta shaped input currents from NEST

[Christoph Hahn]

Hello,
We are working on simulating a spiking neural network converted with the algorithm from the paper "Conversion of Continuous-Valued Deep Networks to Efficient Event-Driven Networks for Image Classification" by Rueckauer et al. (https://www.frontiersin.org/articles/10.3389/fnins.2017.00682/full).
In NEST the neuron dynamics described in the paper can be relatively closely simulated using the iaf_psc_delta neuron (https://www.nest-simulator.org/helpindex/cc/iaf_psc_delta.html) with delta shaped input currents (that is after a spike the current is added to the potential of the post synaptic neuron in one leap). We are now trying to port the model to pyNN. Can you suggest a model with which the iaf_psc_delta neuron can be simulated?

Additionally, in the mentioned paper, membrane potentials are not reset to their resting potential, but instead the threshold is subtracted from their current potential. In NEST we are trying to simulate this using adaptive thresholds (e.g. adaptive thresholds with delta neurons: https://www.nest-simulator.org/helpindex/cc/pp_psc_delta.html). Is there something similar to this in pyNN?

We would really appreciate any tips on how to achieve this.

Thank you very much and Best Regards,

Christoph Hahn

[Andrew Davison]

Hi Christoph,

What is your motivation for porting to PyNN? More specifically, do you plan to simulate the network with a backend simulator other than NEST (e.g. NEURON, Brian, SpiNNaker,...)?

In any case, a good starting point would be to use the `native_cell_type()` function (http://neuralensemble.org/docs/PyNN/backends/NEST.html#using-native-cell-models) to use NEST models within PyNN.

If you then want to use a different simulator, your neuron model would have to be implemented for that simulator. One option that might be of interest would be to implement the neuron model in NineML (http://nineml.net), then use the pyNN.nineml module with code generation via Pype9, https://github.com/NeuralEnsemble/pype9

Best regards,

Andrew

[Christoph Hahn]

Hi Andrew,

Thanks for the quick answer! I'll have a look at the links you provided.

The goal of porting it to pyNN is to run it on SpiNNaker neuromorphic hardware.

As I understood it so far, the basic neuron models provided in pyNN work for all the different backends. That's why I thought I would need to use one of them.

Best,

Christoph

--
You received this message because you are subscribed to the Google Groups "Neural Ensemble" group.
To unsubscribe from this group and stop receiving emails from it, send an email to neuralensembl...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/neuralensemble/0b410b52-1ab8-4fce-83b5-2f3053bc6c1d%40googlegroups.com.

[Andrew Davison]

On Thursday, 7 November 2019 13:21:24 UTC+1, Christoph Hahn wrote:

Hi Andrew,

Thanks for the quick answer! I'll have a look at the links you provided.

The goal of porting it to pyNN is to run it on SpiNNaker neuromorphic hardware.

As I understood it so far, the basic neuron models provided in pyNN work for all the different backends. That's why I thought I would need to use one of them.

This is true, but if you really need the delta currents, none of the standard models provided by PyNN are suitable, so we would have to add a new model to PyNN's library.

I suggest you ask on the SpiNNaker user group (https://groups.google.com/forum/#!forum/spinnakerusers) whether they have a model that is similar to NEST's iaf_psc_delta model, and if not, how much effort it would be to add such a model.

An alternative approach would be to approximate the delta current using an exponential synapse with a very short time constant. The Izhikevich model is the only model with adaptation that is available on SpiNNaker. Do you think this could be used to approximate adaptive thresholds?

 

Best,

Christoph

[Christoph Hahn]

We are already trying to approximate the delta currents with the If_curr_exp neuron type in pyNN. I believe that this is based on the iaf_psc_exp from NEST (https://nest-simulator.readthedocs.io/en/latest/models/neurons.html#classnest_1_1iaf__psc__exp), right? From the source code of the neuron (https://github.com/nest/nest-simulator/blob/master/models/iaf_psc_exp.cpp), the membrane dynamics of the neuron are expressed as:

S_.V_m_ =

S_.V_m_ * V_.P22_ + S_.i_syn_ex_ * V_.P21ex_ + S_.i_syn_in_ * V_.P21in_ + ( P_.I_e_ + S_.i_0_ ) * V_.P20_

The synapse potentials are multiplied by V_.P21ex (or in) which itself depends both on the membrane time constant and the synapse time constant.

V_.P21ex = -tau / ( C * ( 1 - tau / tau_syn ) ) * std::exp( -h / tau_syn ) * numerics::expm1( h * ( 1 / tau_syn - 1 / tau ) ); from https://github.com/nest/nest-simulator/blob/master/libnestutil/propagator_stability.cpp, where tau is the membrane time constant and tau_syn is the membrane sysnapse constant

When setting the synapse time constant very low (which would approximate a delta current), P21ex also decreases such that the synapse potential does not get fully added to the membrane potential (I would like to have P21ex equals 1).

Also I would like the membrane potential to decay as little as possible as the original paper I'm using for the conversion actually uses Non-Leaky-Integrate-And-Fire neurons. So I set the membrane time constant very large.

I then tried to experimentally determine a good trade off value for the synapse time constant, but it is not too promising (I also tried to find it analytically, but didn't manage to).

I requested to join the SpiNNaker group and will ask there, if they have a neuron model with delta shaped currents. Until then, do you have any ideas, if the approximation can be improved or if my thinking is right in the first place?

Thanks again!

--
You received this message because you are subscribed to the Google Groups "Neural Ensemble" group.
To unsubscribe from this group and stop receiving emails from it, send an email to neuralensembl...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/neuralensemble/d36c3a28-7bf9-4455-96c7-1012999fd167%40googlegroups.com.

[Andrew Davison]

Hi Christoph,

Did you get a reply from the SpiNNaker group? In any case, please feel free to create a ticket on the PyNN issue tracker (https://github.com/NeuralEnsemble/PyNN/issues) so that this topic doesn't get forgotten about.

Regards,

Andrew

On Tuesday, 12 November 2019 10:37:57 UTC+1, Christoph Hahn wrote:

We are already trying to approximate the delta currents with the If_curr_exp neuron type in pyNN. I believe that this is based on the iaf_psc_exp from NEST (https://nest-simulator.readthedocs.io/en/latest/models/neurons.html#classnest_1_1iaf__psc__exp), right? From the source code of the neuron (https://github.com/nest/nest-simulator/blob/master/models/iaf_psc_exp.cpp), the membrane dynamics of the neuron are expressed as:

S_.V_m_ =

S_.V_m_ * V_.P22_ + S_.i_syn_ex_ * V_.P21ex_ + S_.i_syn_in_ * V_.P21in_ + ( P_.I_e_ + S_.i_0_ ) * V_.P20_

The synapse potentials are multiplied by V_.P21ex (or in) which itself depends both on the membrane time constant and the synapse time constant.

V_.P21ex = -tau / ( C * ( 1 - tau / tau_syn ) ) * std::exp( -h / tau_syn ) * numerics::expm1( h * ( 1 / tau_syn - 1 / tau ) ); from https://github.com/nest/nest-simulator/blob/master/libnestutil/propagator_stability.cpp, where tau is the membrane time constant and tau_syn is the membrane sysnapse constant

When setting the synapse time constant very low (which would approximate a delta current), P21ex also decreases such that the synapse potential does not get fully added to the membrane potential (I would like to have P21ex equals 1).

Also I would like the membrane potential to decay as little as possible as the original paper I'm using for the conversion actually uses Non-Leaky-Integrate-And-Fire neurons. So I set the membrane time constant very large.

I then tried to experimentally determine a good trade off value for the synapse time constant, but it is not too promising (I also tried to find it analytically, but didn't manage to).

I requested to join the SpiNNaker group and will ask there, if they have a neuron model with delta shaped currents. Until then, do you have any ideas, if the approximation can be improved or if my thinking is right in the first place?

Thanks again!

[Christoph Hahn]

Hi Andrew,

Sorry for the late response. Indeed, SpiNNaker has a neuron with delta shaped synaptic currents IfCurDelta which I used quite successfully for my problem. I think this issue can be closed. It might still be nice to have a similar neuron type in PyNN however, because it is a very simple model which is used in various papers to port algorithms from conventional neural networks to spiking neural networks.

Thank you very much for your help,

Best,

Christoph

--
You received this message because you are subscribed to the Google Groups "Neural Ensemble" group.
To unsubscribe from this group and stop receiving emails from it, send an email to neuralensembl...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/neuralensemble/2cb66b64-630f-4ff7-a555-06b498779ada%40googlegroups.com.

[Andrew Davison]

Hi Christoph,

I'm glad you resolved your problem. I've now opened a pull request to add a new standard model IF_curr_delta, which uses iaf_psc_delta for pyNN.nest, and IFCurDelta for pyNN.spiNNaker. This will hopefully be merged soon, once a few tests have been added.

- see https://github.com/NeuralEnsemble/PyNN/pull/664

Regards,

Andrew

On Tuesday, 7 January 2020 13:17:46 UTC+1, Christoph Hahn wrote:

Hi Andrew,

Sorry for the late response. Indeed, SpiNNaker has a neuron with delta shaped synaptic currents IfCurDelta which I used quite successfully for my problem. I think this issue can be closed. It might still be nice to have a similar neuron type in PyNN however, because it is a very simple model which is used in various papers to port algorithms from conventional neural networks to spiking neural networks.

Thank you very much for your help,

Best,

Christoph