Dorsal stream

[sandsmark]

Hi!

We're attempting to send location information and some identification
information from the sensory memory to the sensory-motor memory, and
as far as we understand this is what the dorsal stream is for. But we
are wondering if this stream is implemented in the current version of
the framework? There seems to be methods for adding listeners, but in
the default implementation of the sensory-motor memory the
receiveSensoryMemoryContent method contains only a comment "research
problem". Is this part not yet implemented?

Some background: We're attempting to use LIDA for implementing an
agent playing StarCraft: Brood War, and we have feature detectors for
detecting for example idle workers, but to be able to use this
information we need to send the location info or the unique unit ID to
the sensory-motor memory.

Regards,
Ken and Martin

[Ryan J. McCall]

Hi Ken and Martin,

Thank you for your interest in the LIDA software framework.

On 6/1/12 10:13 AM, sandsmark wrote:
> Some background: We're attempting to use LIDA for implementing an
> agent playing StarCraft: Brood War, and we have feature detectors for
> detecting for example idle workers, but to be able to use this
> information we need to send the location info or the unique unit ID to
> the sensory-motor memory.

I have some general questions for you. Is your primary goal for the
agent to play starcraft well? This is probably for a competition right?
To what degree do you want to model human-like (i.e. more complex)
processing? Maybe going for the latter improves performance, but not
always, depending on the task, "simple" approaches can yield great results.

> We're attempting to send location information and some identification
> information from the sensory memory to the sensory-motor memory, and
> as far as we understand this is what the dorsal stream is for.

Theoretically, in the LIDA model, I think of communication from Sensory
Memory to Sensory-Motor memory involving two kinds of signals:

1.) dorsal stream information which involves information about
location (and location only) of objects with respect to the agent and/or
its actuators in some internal coordinate system
2.) "emergency" signals that leads to a reflexive motor execution

So the dorsal stream is always sending the location information but it
seems that only in critical situations does the "what" information get
send to SMM. For example perhaps the agent should reflexively pull back
its army if it suddenly detects the opponent's army to avoid a bad
engagement.

Now to contrast suppose you immediately send information about every
unit and its position to the SMM all the time. This will probably not
give good results and does not really use the entirety of the LIDA model
-- but maybe it gives good results in some situations. For example one
pitfull with such an approach is that there is no prioritization
(attention) that selects the most "important" thing for the agent to
respond to.

> But we
> are wondering if this stream is implemented in the current version of
> the framework? There seems to be methods for adding listeners, but in
> the default implementation of the sensory-motor memory the
> receiveSensoryMemoryContent method contains only a comment "research
> problem". Is this part not yet implemented?

Right, at least initially, we've decided to leave SensoryMemory and
SensoryMotorMemory somewhat undefined since these modules can vary
depending on the agent's domain. There is no detailed implementation of
the dorsal stream currently in the framework . What is implemented is
what you mention; namely, that SensoryMotorMemory is a
SensoryMemoryListener and as such it must implement the following:

public abstract void receiveSensoryMemoryContent(Object arg0);

So you can have your SMM receiving any kind of Object from SM i.e. no
constraint on the information transmitted.

Hope this helps.

glhf,

Ryan

--
Ryan J. McCall
Ph.D. Student, Dept. of Computer Science
Cognitive Computing Research Group
Institute for Intelligent Systems
The University of Memphis

[sandsmark]

Hi!

On Mon, Jun 04, 2012 at 08:08:37PM -0500, Ryan J. McCall wrote:
> I have some general questions for you. Is your primary goal for the
> agent to play starcraft well? This is probably for a competition
> right?

No, our primary goal is to explore cognitive architectures, StarCraft is just
an interesting environment. :-)

I think using Java and JNIBWAPI precludes us from most of the competitions
anyways.

> Theoretically, in the LIDA model, I think of communication from
> Sensory Memory to Sensory-Motor memory involving two kinds of
> signals:
>
> 1.) dorsal stream information which involves information about
> location (and location only) of objects with respect to the agent
> and/or its actuators in some internal coordinate system
> 2.) "emergency" signals that leads to a reflexive motor execution
>
> So the dorsal stream is always sending the location information but
> it seems that only in critical situations does the "what"
> information get send to SMM. For example perhaps the agent should
> reflexively pull back its army if it suddenly detects the opponent's
> army to avoid a bad engagement.
>
> Now to contrast suppose you immediately send information about every
> unit and its position to the SMM all the time. This will probably
> not give good results and does not really use the entirety of the
> LIDA model -- but maybe it gives good results in some situations.
> For example one pitfull with such an approach is that there is no
> prioritization (attention) that selects the most "important" thing
> for the agent to respond to.

Yeah, doing that kind of loses the point of using LIDA in the first place, so
we'll want to avoid using reflexive actions like that as much as possible.

> Right, at least initially, we've decided to leave SensoryMemory and
> SensoryMotorMemory somewhat undefined since these modules can vary
> depending on the agent's domain. There is no detailed implementation
> of the dorsal stream currently in the framework . What is
> implemented is what you mention; namely, that SensoryMotorMemory is
> a SensoryMemoryListener and as such it must implement the following:
>
> public abstract void receiveSensoryMemoryContent(Object arg0);
>
> So you can have your SMM receiving any kind of Object from SM i.e.
> no constraint on the information transmitted.

Okay, makes sense, thanks for the clarification. :-)


> Hope this helps.

Indeed it does, thanks!


--
Martin & Ken

[sandsmark]

Hi again,

Just another question, if you don't mind. :-)

Do you know if there are any other larger projects (than the example projects)
using LIDA with source code available? We ask because it's not always easy to
see what options are available everywhere.

More specifically we want to be able to use the episodic memory modules more,
and it isn't well covered in the tutorials for the framework. If there's any
other resources (papers or anything) explaining this it would be nice as well.

--
Martin & Ken

[Matthew Lohbihler]

Hi Martin and Ken,

Are you guys still working on this? It would be good to hear how it turned out.

Thanks,
Matthew

[Ryan J. McCall]

Martin & Ken,

Sorry to have missed this message.

I'm not sure if you're referring to the AlifeAgent example, but I've recently updated the framework distribution (v1.2) to include an example project (AlifeAgent) that runs with version 1.2.

The other suggestion I have is to look at the javadoc of the default implementations of modules, e.g. EpisodicMemoryImpl. Also the init() methods of these classes have been updated with descriptions of the parameters that can be configuration for the module.

Regarding the EpisodicMemoryImpl module, it is based on sparse distributed memory by Kanera so you could study that to see the conceptual motivation. Recently, Javier Snaider, completed a dissertation on extensions to SDM which you can download from here: http://ccrg.cs.memphis.edu/theses-dissertations.html

Again, sorry for the late reply.

Ryan McCall