BehaviorNetwork race condition

[Matthew Lohbihler]

I placed a few logging statements into BehaviorNetwork to try and track down some suspected odd behaviour. Specifically, around lin 218:

public void receiveBehavior(Behavior b) {
    logger.log(Level.INFO, "Received behavior.", TaskManager.getCurrentTick()); // <== ADDED

Around line 251:

public void runThisFrameworkTask() {
    logger.log(Level.INFO, "BehaviorNetworkBackgroundTask.runThisFrameworkTask", TaskManager.getCurrentTick()); // <== ADDED

Around line 522:

public void decayModule(long t) {
    logger.log(Level.INFO, "decayModule", TaskManager.getCurrentTick()); // <== ADDED

I did this because it seemed like behaviours were not being executed as expected. In particular, there appeared to be long, seemingly random delays before actions took place, while activation levels in PAM and elsewhere looked ok. Here are some snips of the output.

In this snip, behaviours are received, but the BehaviourNetworkBackgroundTask has already run, and so they never get any excitation by the passActivationFromSchemes method. In the next tick, the decayModule method mops them up because their activations are 0.
0000010019 :0000004871 :INFO       :edu.memphis.ccrg.lida.actionselection.behaviornetwork.BehaviorNetwork     -> BehaviorNetworkBackgroundTask.runThisFrameworkTask
0000010020 :0000004872 :INFO       :edu.memphis.ccrg.lida.actionselection.behaviornetwork.BehaviorNetwork     -> decayModule
0000010021 :0000004872 :INFO       :edu.memphis.ccrg.lida.actionselection.behaviornetwork.BehaviorNetwork     -> BehaviorNetworkBackgroundTask.runThisFrameworkTask
0000010022 :0000004872 :INFO       :edu.memphis.ccrg.lida.actionselection.behaviornetwork.BehaviorNetwork     -> Received behavior.
0000010023 :0000004872 :INFO       :edu.memphis.ccrg.lida.actionselection.behaviornetwork.BehaviorNetwork     -> Received behavior.
0000010024 :0000004872 :INFO       :edu.memphis.ccrg.lida.actionselection.behaviornetwork.BehaviorNetwork     -> Received behavior.
0000010025 :0000004872 :INFO       :edu.memphis.ccrg.lida.actionselection.behaviornetwork.BehaviorNetwork     -> Received behavior.
0000010026 :0000004873 :INFO       :edu.memphis.ccrg.lida.actionselection.behaviornetwork.BehaviorNetwork     -> decayModule

In this snip, the behaviours are received and they receive activation because this time the BehaviorNetworkBackgroundTask runs after the thread that send them. Presumably, these resulted in actions being executes (although i did not add logging to confirm this... yet).
0000010227 :0000004971 :INFO       :edu.memphis.ccrg.lida.actionselection.behaviornetwork.BehaviorNetwork     -> BehaviorNetworkBackgroundTask.runThisFrameworkTask
0000010228 :0000004972 :INFO       :edu.memphis.ccrg.lida.actionselection.behaviornetwork.BehaviorNetwork     -> decayModule
0000010229 :0000004972 :INFO       :edu.memphis.ccrg.lida.actionselection.behaviornetwork.BehaviorNetwork     -> Received behavior.
0000010230 :0000004972 :INFO       :edu.memphis.ccrg.lida.actionselection.behaviornetwork.BehaviorNetwork     -> Received behavior.
0000010231 :0000004972 :INFO       :edu.memphis.ccrg.lida.actionselection.behaviornetwork.BehaviorNetwork     -> Received behavior.
0000010232 :0000004972 :INFO       :edu.memphis.ccrg.lida.actionselection.behaviornetwork.BehaviorNetwork     -> Received behavior.
0000010233 :0000004972 :INFO       :edu.memphis.ccrg.lida.actionselection.behaviornetwork.BehaviorNetwork     -> BehaviorNetworkBackgroundTask.runThisFrameworkTask
0000010234 :0000004973 :INFO       :edu.memphis.ccrg.lida.actionselection.behaviornetwork.BehaviorNetwork     -> decayModule

Here is one where the background task runs in the middle of the behaviour receipt:
0000011687 :0000005681 :INFO       :edu.memphis.ccrg.lida.actionselection.behaviornetwork.BehaviorNetwork     -> BehaviorNetworkBackgroundTask.runThisFrameworkTask
0000011688 :0000005682 :INFO       :edu.memphis.ccrg.lida.actionselection.behaviornetwork.BehaviorNetwork     -> decayModule
0000011689 :0000005682 :INFO       :edu.memphis.ccrg.lida.actionselection.behaviornetwork.BehaviorNetwork     -> Received behavior.
0000011690 :0000005682 :INFO       :edu.memphis.ccrg.lida.actionselection.behaviornetwork.BehaviorNetwork     -> BehaviorNetworkBackgroundTask.runThisFrameworkTask
0000011691 :0000005682 :INFO       :edu.memphis.ccrg.lida.actionselection.behaviornetwork.BehaviorNetwork     -> Received behavior.
0000011692 :0000005682 :INFO       :edu.memphis.ccrg.lida.actionselection.behaviornetwork.BehaviorNetwork     -> Received behavior.
0000011693 :0000005682 :INFO       :edu.memphis.ccrg.lida.actionselection.behaviornetwork.BehaviorNetwork     -> Received behavior.
0000011694 :0000005683 :INFO       :edu.memphis.ccrg.lida.actionselection.behaviornetwork.BehaviorNetwork     -> decayModule

By far the majority of these are like the first where the behaviours are removed before they receive any activation. I believe this also explains the odd behaviour of the ALife example, in which the agent would sit still for long periods while going hungry and being attacked by monkeys. I think the log snips above indicate a definite race condition in the threading.

Regards,
Matthew

[Ryan J. McCall]

Matthew,

Thanks again for your interest.

To simply this discussion I'll first note that module decay (via #decayModule) occurs "between" ticks (see TaskManager#goNextTick()), so we can rule out any race effects from the decay method.

Asynchrony is commitment of the LIDA model and this framework. Throughout the framework you'll find FrameworkModules that have both background tasks operating on the module data as well as other tasks bringing in new data into the module. (Here the background task does several things and another task in ProceduralMemory calls #receiveBehavior adding a new behavior.) If tasks are scheduled for the same tick then the framework cannot guarantee in which order they will run; this is a feature. So based on what you've said I would fault the decay rate parameter used to decay behaviors -- perhaps it's too high if the added behaviors are removed after only one decay.

Best,

Ryan

--
You received this message because you are subscribed to the Google Groups "Cognitive Computing Research Group - CCRG" group.
To view this discussion on the web visit https://groups.google.com/d/msg/ccrg-memphis/-/uDs4M-yQmJAJ.
To post to this group, send email to ccrg-m...@googlegroups.com.
To unsubscribe from this group, send email to ccrg-memphis...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/ccrg-memphis?hl=en.

Get the Information you need to Start Today. Search from a Complete List of
Local and Online Schools. Learn about Tuition, Classes, and More!
http://click.lavabit.com/utmudf88b1fg651moag3t7qexddyxnafift8jtk63uf3889r7oty/

--
Ryan J. McCall
Ph.D. Student, Dept. of Computer Science
Cognitive Computing Research Group
Institute for Intelligent Systems
The University of Memphis

[Matthew Lohbihler]

Thanks again Ryan. All good stuff to know.

However, the Behavior objects that are given in the receiveBehavior method start with an activation of 0. The objects are instantiated in the ProceduralMemoryImpl.createInstantiation method at line 368:

Behavior b = factory.getBehavior(s);

The Behavior gets its activation value in the ActivatibleImpl constructor, set to be the DEFAULT_ACTIVATION value, which is static. I don't immediately see a way to override this with configuration, and so - assuming there is no override - the order in which the receiveBehavior and BehaviorNetworkBackgroundTask.runThisFrameworkTask methods are called is critical to the behaviour being executed.

It would be better if the Behavior object got initialized more thoroughly before being added to the behaviors list in BehaviorNetwork, but i don't see a way to make this happen without code changes.

[Ryan J. McCall]

Matthew,

I see the problem now, ElementFactory#getBehavior does not set the created Behavior's initial activation based on the Scheme's total activation (or perhaps some fraction of the total.) I think we can call this a "bug." A workaround for now would be to override BehaviorNetwork#receiveBehavior in a subclass. Then after calling super the Behavior's activation can be set using the Scheme's total activation -- getTotalActivation(). This, by the way, points out another bug to me, which is that BehaviorNetwork#passActivationFromSchemes incorrectly uses Schemes' activation to excite Behaviors instead of total activation...

Many thanks,

Ryan

Have questions about Shop? Let Ask Find Your Answers Today!
http://click.lavabit.com/hp88b318sou9iufqqy4mkc7zesj5mhp9ckgb8x3rhqj51krw93ey/

[Matthew Lohbihler]

Thanks Ryan.

I created a subclass with this code, which seems to work better:

package edu.memphis.ccrg.lida.actionselection.behaviornetwork;

import edu.memphis.ccrg.lida.actionselection.Behavior;

public class BalanceBehaviorNetwork extends BehaviorNetwork {
    @Override
    public void receiveBehavior(Behavior b) {
        super.receiveBehavior(b);
        if (b != null)
            b.setActivation(b.getScheme().getTotalActivation());
    }

    @Override
    void passActivationFromSchemes() {
        // no op
    }
}

Still, i'm not sure if this is correct. The passActivationFromSchemes method multiplies the activation by the broadcastExcitationFactor field. Should my subclass do this as well?

Also, i override the passActivationFromSchemes method with a no-op to prevent the Behavior from receiving duplicate excitation. Because the method is package private i had to put my subclass into the same package as BehaviourNetwork.

Although this work better than before, it raises the problem of conflicting actions. The agent should not be executing both a left and right impulse at the same time, although from the action selection list it appears that this is the case fairly often. Is there a way to declare that the presence of one should suppress the other?

Regards,
Matthew

[Ryan J. McCall]

Matthew,

On 10/12/12 10:48 AM, Matthew Lohbihler wrote:

Thanks Ryan.

I created a subclass with this code, which seems to work better:

package edu.memphis.ccrg.lida.actionselection.behaviornetwork;

import edu.memphis.ccrg.lida.actionselection.Behavior;

public class BalanceBehaviorNetwork extends BehaviorNetwork {
    @Override
    public void receiveBehavior(Behavior b) {
        super.receiveBehavior(b);
        if (b != null)
            b.setActivation(b.getScheme().getTotalActivation());
    }

    @Override
    void passActivationFromSchemes() {
        // no op
    }
}

Still, i'm not sure if this is correct. The passActivationFromSchemes method multiplies the activation by the broadcastExcitationFactor field. Should my subclass do this as well?

The #receiveBehavior override is what I had in mind. I would not override #passActivationFromSchemes. The idea is that the Scheme's current activation, from which the Behavior's activation should derive, is changing back in the Procedural Memory module due to incoming broadcasts. So this method is incrementally updating (exciting) the Behaviors to keep them up-to-date. There is a double parameter that scales the amount of this excitation (see #init()).

Also, i override the passActivationFromSchemes method with a no-op to prevent the Behavior from receiving duplicate excitation. Because the method is package private i had to put my subclass into the same package as BehaviourNetwork.

Although this work better than before, it raises the problem of conflicting actions. The agent should not be executing both a left and right impulse at the same time, although from the action selection list it appears that this is the case fairly often. Is there a way to declare that the presence of one should suppress the other?

Conflicting actions, whose preconditions are negations of each other, along with "conflictor" links are not implemented in this version of the behavior network.

Best,

Ryan

Find, Compare and Save on Credit Cards by reviewing offers from top Credit Card
companies listed at Compare.US.
http://click.lavabit.com/ddg9nscadpnsu4u3qu7janojpxzb3jsnef6znnicz587igp4h98b/