Implementing reward-based STDP mechanism

61 views
Skip to first unread message

Daniel Casanueva Morato

unread,
Nov 20, 2024, 12:58:20 PM11/20/24
to SpiNNaker Users Group
Hi,

We are trying to explore new SNN-based memory networks implemented in SpiNNaker with python. For this, we need a very simple reward-based STDP learning mechanism. This STDP mechanism should apply the weight and time rules to modify the synaptic weight only when a third neuron (dopamine) has been activated in a time window, and not change the synaptic weight otherwise.

Checking the models present in the current sPyNNaker library, we have not found an implementation of STDP that achieves this operation, so we have tried to develop it ourselves. Our idea was as follows:

- A new LIF neuron model (Dopamine) that, when it generates an event, updates a global variable with the current timestamp.
- A new STDP model (Dopamine STDP) that simply checks that shared global variable and if the pairs of events to be evaluated are within the specified time window, it applies the learning/forgetting, or does not apply it otherwise.

After implementing and testing it on SpiNNaker, we have realised that the memory space of the new Dopamine neuron and that of the synapses with our new STDP model is not the same, therefore, this global variable is independent of each other, invalidating our implementation idea. When the dopamine neuron is activated, it updates the variable correctly in its memory space, but when we read it from the new STDP (debugged through the logs) this variable remains with its initial value.

Would there be a way, through C code or compilation directives, to make such a global variable shareable between the neuron model and the STDP model? If it is not possible, could you please give us an idea on how could this STDP model be implemented within SpiNNaker with the given tools?

Thank you very much in advance.
Best regards,
Daniel

Andrew Gait

unread,
Nov 20, 2024, 4:32:18 PM11/20/24
to SpiNNaker Users Group
Hi Daniel,

It's a bit hidden within the documentation site, but have you seen


which describes neuromodulated STDP, which I think you can adapt to what you are looking for?  We don't use a "dopamine neuron" as such but

The document is also somewhat out of date as you should be able to do this from the most recent sPyNNaker version (I think it is in from version 6.0 onwards?) without having to get the "master" branches from GitHub.

Andy

----------------------------------------------------
Dr. Andrew Gait (he/him),
Research Software Engineer,
Research IT,
Directorate of IT Services,
The University of Manchester,

I do not expect a reply to this email outside of normal working hours
From: spinnak...@googlegroups.com <spinnak...@googlegroups.com> on behalf of Daniel Casanueva Morato <danca...@gmail.com>
Sent: 20 November 2024 12:58
To: SpiNNaker Users Group <spinnak...@googlegroups.com>
Subject: [SpiNNaker Mailing List] Implementing reward-based STDP mechanism
 
Hi, We are trying to explore new SNN-based memory networks implemented in SpiNNaker with python. For this, we need a very simple reward-based STDP learning mechanism. This STDP mechanism should apply the weight and time rules to modify the synaptic
ZjQcmQRYFpfptBannerStart
This Message Is From a New External Sender
You have not previously corresponded with this sender. Please exercise caution when opening links or attachments included in this message.
 
ZjQcmQRYFpfptBannerEnd
--
You received this message because you are subscribed to the Google Groups "SpiNNaker Users Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to spinnakeruser...@googlegroups.com.
To view this discussion, visit https://groups.google.com/d/msgid/spinnakerusers/4d6f7115-0b00-4216-9eb5-a90a4848b726n%40googlegroups.com [groups.google.com].

Andrew Gait

unread,
Nov 20, 2024, 4:35:56 PM11/20/24
to SpiNNaker Users Group
Oops, my reply got sent early, that incomplete sentence was meant to say this:

We don't use a "dopamine neuron" as such but I think you can adapt what is in there already to work in the way that you want it to.

Andy

----------------------------------------------------
Dr. Andrew Gait (he/him),
Research Software Engineer,
Research IT,
Directorate of IT Services,
The University of Manchester,

I do not expect a reply to this email outside of normal working hours
From: Andrew Gait <Andre...@manchester.ac.uk>
Sent: 20 November 2024 16:32

To: SpiNNaker Users Group <spinnak...@googlegroups.com>
Subject: Re: [SpiNNaker Mailing List] Implementing reward-based STDP mechanism
 
Reply all
Reply to author
Forward
0 new messages