Weird problem with model we are doing
Stephen Read
Hi Everyone,
We are trying to train a model of depression in which activation of an individual motive node in a motivation layer will activate one of two possible behaviors in the behavior or output layer (so a socializing motive can activate either go out with friend or text friend). The network is set up so that there are two motivation layers, one with 5 Approach motives and the second with 3 Avoidance motives. The two motivation layers are directly connected to a Behavior layer that has 16 possible behaviors, organized into 8 pools of 2 behaviors each. This is to represent the idea that for each of the 8 motives there are two possible behaviors. We are trying to train the network, such that the first motive should activate one of the two behaviors in the first pool, the second motive should activate one of the two behaviors in the second pool and so forth. So the training data is set up so that the model should learn strong weights only between a motive and its two corresponding behaviors.
The problem we have is that when we train the network, it largely learns a reasonable pattern of weights, except that for ONE behavior (or 1 pair of behaviors) it learns weights of 1 to from ALL the motives, even though there is nothing in the training data to support that learning. (See image). Which behavior shows this behavior is not consistent, we see different ones for different random seeds. Different random seeds lead to a different behaviors learning weights of 1 from all the motive nodes.
We have done this with error correcting learning, Hebbian learning and a mix, as well as with and without an intervening hidden layer.
We have been working on this model for several years and had a version of this model working with the goki gui, but when we switched to the current core version for Leabra, we started getting this weird behavior. The parameters are identical for the two versions of the model and the network construction is identical.
I have attached several short videos of what this looks like and also attached a zip file of the program and its training data.
I have also attached a copy of the go.mod and go.sum for the older version that used goki.
Anyone have any idea why this is happening and whether there are parameters I can play with that will cure this problem?
Stephen J. Read
Mendel B. Silberberg Professor of Social Psychology
Department of Psychology
University of Southern California
Los Angeles, CA 90089-1061
Website: www.stephenjread.com
Randall O'Reilly
- Randy
Stephen Read
Thanks for the suggestions. I will let you know what I find out. Is this what you call a hog unit or is it something else?
Thanks, Steve
Stephen Read
So I figured out what seems to be happening, but haven’t figured out a fix yet. The key issue is that as part of the training, we first train the Behavior layer and then lesion the Behavior layer(the output layer) for the remainder of the training, but somehow the last activated behavior in the Behavior layer before lesioning stays active and that behavior is then strongly associated to all the motives during later training.
Further background is that we are training the network in two steps to try to capture the idea of Pavlovian to Instrumental Transfer (PIT), where an organism separately does Pavlovian learning, cues to reward, and then separately does Instrumental Learning where it learns the association between a reward and an instrumental response. So we have our network first learn the associations between reward and instrumental response, which are captured by the weights between the output behavior and the motive(reward representation) and then does Pavlovian learning, which is the association between the input cues and the motive(reward representation). To capture that, we first have the network learn the associations between the motives and the behaviors, and then lesion the behavior layer and have it learn the associations between input cues and behavior. What I have discovered is that even though the Behavior layer is lesioned, the activation of the last activated behavior during training remains active. So that when I do Instrumental learning between the input cues and the motives, that one behavior in the behavior layer stays chronically active. As a result the model learns a very strong relation between all of the motives and that one behavior. I also found that which of the behaviors stays active is random and is different with different random seeds.
Stephen J. Read
Mendel B. Silberberg Professor of Social Psychology
Department of Psychology
University of Southern California
Los Angeles, CA 90089-1061
Website: www.stephenjread.com
From: Randall O'Reilly <rcore...@gmail.com>
Date: Tuesday, September 23, 2025 at 3:10
PM
To: Stephen Read <re...@usc.edu>
Cc: emergent-users <emergen...@googlegroups.com>, O'Reilly Kai <koreil...@gmail.com>, Chaodan Luo <chao...@usc.edu>
Subject: Re: Weird problem with model we are doing
Randall O'Reilly
- Randy
> On Sep 26, 2025, at 6:40 PM, Stephen Read <re...@usc.edu> wrote:
>
> So I figured out what seems to be happening, but haven’t figured out a fix yet. The key issue is that as part of the training, we first train the Behavior layer and then lesion the Behavior layer(the output layer) for the remainder of the training, but somehow the last activated behavior in the Behavior layer before lesioning stays active and that behavior is then strongly associated to all the motives during later training.
> Further background is that we are training the network in two steps to try to capture the idea of Pavlovian to Instrumental Transfer (PIT), where an organism separately does Pavlovian learning, cues to reward, and then separately does Instrumental Learning where it learns the association between a reward and an instrumental response. So we have our network first learn the associations between reward and instrumental response, which are captured by the weights between the output behavior and the motive(reward representation) and then does Pavlovian learning, which is the association between the input cues and the motive(reward representation). To capture that, we first have the network learn the associations between the motives and the behaviors, and then lesion the behavior layer and have it learn the associations between input cues and behavior. What I have discovered is that even though the Behavior layer is lesioned, the activation of the last activated behavior during training remains active. So that when I do Instrumental learning between the input cues and the motives, that one behavior in the behavior layer stays chronically active. As a result the model learns a very strong relation between all of the motives and that one behavior. I also found that which of the behaviors stays active is random and is different with different random seeds.
> Stephen J. Read
> Mendel B. Silberberg Professor of Social Psychology
> Department of Psychology
> University of Southern California
> Los Angeles, CA 90089-1061
> Website: www.stephenjread.com
> From: Randall O'Reilly <rcore...@gmail.com>
> Date: Tuesday, September 23, 2025 at 3:10 PM
> To: Stephen Read <re...@usc.edu>
> Cc: emergent-users <emergen...@googlegroups.com>, O'Reilly Kai <koreil...@gmail.com>, Chaodan Luo <chao...@usc.edu>
> Subject: Re: Weird problem with model we are doing
Stephen Read
To: Stephen Read <re...@usc.edu>
Cc: emergent-users <emergen...@googlegroups.com>; O'Reilly Kai <koreil...@gmail.com>; Chaodan Luo <chao...@usc.edu>
Subject: Re: Weird problem with model we are doing
Steve -- nice detective work! Try calling InitActs() prior lesioning the Behavior layer, to clear the remaining activity. - Randy > On Sep 26, 2025, at 6:40 PM, Stephen Read <re...@usc.edu
> wrote: > > So I figured out what seems to be happening, but haven’t figured out a fix yet. The key issue is that as part of the training, we first train the Behavior layer and then lesion the Behavior layer(the output layer) for the remainder of the training, but somehow the last activated behavior in the Behavior layer before lesioning stays active and that behavior is then strongly associated to all the motives during later training. > Further background is that we are training the network in two steps to try to capture the idea of Pavlovian to Instrumental Transfer (PIT), where an organism separately does Pavlovian learning, cues to reward, and then separately does Instrumental Learning where it learns the association between a reward and an instrumental response. So we have our network first learn the associations between reward and instrumental response, which are captured by the weights between the output behavior and the motive(reward representation) and then does Pavlovian learning, which is the association between the input cues and the motive(reward representation). To capture that, we first have the network learn the associations between the motives and the behaviors, and then lesion the behavior layer and have it learn the associations between input cues and behavior. What I have discovered is that even though the Behavior layer is lesioned, the activation of the last activated behavior during training remains active. So that when I do Instrumental learning between the input cues and the motives, that one behavior in the behavior layer stays chronically active. As a result the model learns a very strong relation between all of the motives and that one behavior. I also found that which of the behaviors stays active is random and is different with different random seeds. > Stephen J. Read > Mendel B. Silberberg Professor of Social Psychology > Department of Psychology > University of Southern California > Los Angeles, CA 90089-1061 > Website: https://urldefense.com/v3/__http://www.stephenjread.com__;!!LIr3w8kk_Xxm!q36D6grPRYkc941xNS3nJs24i2-mv-d-po7-wGLslBZx5az7ma4-YXCH5ZxqbPl9rYcFEoFtGnmlyLU$
Stephen Read
Unfortunately, that doesn’t seem to make a difference.
Stephen J. Read
Mendel B. Silberberg Professor of Social Psychology
Department of Psychology
University of Southern California
Los Angeles, CA 90089-1061
From:
emergen...@googlegroups.com <emergen...@googlegroups.com> on behalf of Stephen Read <re...@usc.edu>
Date: Monday, September 29, 2025 at 6:56 AM
To: Randall O'Reilly <rcore...@gmail.com>
Cc: emergent-users <emergen...@googlegroups.com>, O'Reilly Kai <koreil...@gmail.com>, Chaodan Luo <chao...@usc.edu>
Subject: Re: Weird problem with model we are doing
Thanks! Will try that. I think I did try after lesioning, but will try this order. Get Outlook for iOS From: Randall O'Reilly <rcoreilly5@ gmail. com> Sent: Monday, September 29, 2025 6: 02: 03 AM To: Stephen Read <read@ usc. edu> Cc:
--
You received this message because you are subscribed to the Google Groups "emergent-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
emergent-user...@googlegroups.com.
To view this discussion visit
https://groups.google.com/d/msgid/emergent-users/BYAPR07MB4679C2C01752DED26B02E875D71BA%40BYAPR07MB4679.namprd07.prod.outlook.com.
Stephen Read
Unfortunately, that doesn’t seem to make a difference.
I tried InitActs() on both the target layer and on the entire network before lesioning, but it didn’t make a difference. The Target value from the last training trial still seems to be active during training of the other layers.