DA-STDP with CARLsim

[Mahadevuni Amarnath]

Dear All,

  Hope you are doing great!

  I am a Computer Engineering Graduate Student. I am trying to use CARLsim for Reinforcement Learning Applications.

  I created a DOPAMINERGIC_NEURON spike generator group and connected it to regular spiking (RS) neurons as mentioned in the User Guide. I also used the DA_MOD option to enable DA-STDP on RS neurons.

  Can anyone please let me know how to set the Reward / Dopamine for a Spiking Neural Network in CARLsim?

  Also if you know any Reinforcement Learning Projects that have been using CARLsim, kindly direct me to it.

Thanks,
Amarnath

[tingshuc]

Hello Amarnath:

    For reinforcement learning example, you can look into carlsim/test/stdp.cpp. There is a test case called "DASTDPWeightBoost". Briefly, you need to define what is good and then stimulate DA neurons to release dopamine on its target neural group. In the test case, the reward is only given when pre-ex fires before post-ex because we want to boost the weight. 

Best, Ting-Shuo