BECCA version 0.6.0 release

[Brandon Rohrer]

Happy New Year!

I'm excited to announce that there is a new major release of the BECCA code, https://github.com/brohrer/becca/releases/tag/0.6.0. Its core reinforcement learning algorithms have been heavily reworked. The hub is largely unchanged, but several other pieces have been added to it: spindle (attention), mainspring (short and long term declarative memory), and arborkey (a top level executive). Blocks are now named gearboxes and together they form a drivetrain.

It's a big step toward merging the higher-level cognitive processing of classic AI with more modern statistical and deep-learning approaches. More importantly, it's a big step closer to making BECCA a good fit for practical applications. 

Share and enjoy.

Brandon

[SeH]

thanks, this looks like a good set of updates.

in trying the examples, i looked into python performance options again.  'pypy' is a JIT compiler which is said to outperform standard c-python.  numba and theano libraries are other options. here's an article comparing numba to cython (python to C compiler): https://jakevdp.github.io/blog/2013/06/15/numba-vs-cython-take-2/  numba is shown faster and easier and can also apply GPU processing.  i'll see if i can run it with numba but i had some trouble importing numpy into pypy (though it should be possible once library issues are resolved).

OpenBECCA is currently written as mostly matrix and array calculations.  i had wondered if it would be possible to express all operations in an equivalent graph-based model which could allow for irregular kinds of system growth, even at the level of adding new sensor inputs and action outputs after the system has been running.  something like an asynchronous, decentralized, localized design that can adapt to any situation anywhere in a fractal heterarchy.  matrices and arrays can always be resized between cycles, however, operations on graph (whether adjacency is implemented as a sparse or dense matrix) may have certain architectural advantages.

BeccaJ was an earlier attempt to do a 1:1 translation of the python code to java, and currently it's not complete nor guaranteed to be correct.  individual matrix calculations, although they can be parallelized with BLAS for native CPU or OpenCL GPU calculation (like numba or theano promises), transferring large matrices back and forth to a python or java program in CPU RAM will not outperform a version which runs entirely in a GPU RAM compute kernel.  though it may be possible to "supercompile" such a kernel from python or java code.  ex: JuRLs is a function-approximation and q-learning library that is implemented as composeable expression trees. https://github.com/toki78/JuRLs#jurls

starting with the Ziptie component, i began imagining BECCA as a graph-ical design. some thoughts:

• Ziptie can be described as a bipartite graph mapping between cables and bundles.  Certain properties are calculated and stored for each cable vertex, and other properties are stored with each bundle vertex.
  
• It's possible to implement cable -> bundle partial memberships with weighted edges.  Other properties can be associated with each edge
• Agglomeration and nucleation processes could be entirely continuous (no thresholds) with partial membership, on a scale from 0=disconnected to 1=full connection.  the formulas can be modified (maybe simplified) to produce a curve with similar operational dynamics but no piecewise discontinuity from thresholds.
• with these changes, Ziptie might appear to have more in common with other unsupervised learning algorithms.  i'm thinking of the Autoencoder which could be used to implement an alternate sort of Ziptie. http://deeplearning.net/tutorial/dA.html Self-organizing map might also apply.
  

• the documentation compares the Daisychain to markov models. another comparison to consider is to a bayesian network, or dyanmic bayesian network.  https://en.wikipedia.org/wiki/Dynamic_Bayesian_network (HMM is a special case of dynamic bayesian network.)  this analogyt might suggest the possibility of an advanced Daisychain that can produce more accurate predictions with higher-order belief structures. (if it is even helpful or necessary at all to improve daisychain's power; such a self-contained component with a limited compartmentalized space/time scope may not require improvement)  continuously learning feedforward or recurrent neural-networks might be other alternate Daisychain implementations.
  

--
You received this message because you are subscribed to the Google Groups "BECCA_users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to becca_users...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Brandon Rohrer]

Thanks SeH! And thanks for the thoughtful analysis. I'm looking forward to digesting it.
Brandon

[Brandon Rohrer]

SeH,

You pointed out a lot of potentially fruitful lines of investigation. The algorithms are still rapidly evolving, so I'm not planning to direct my efforts toward speeding up the code yet. But when I do, I'll be glad of the thorough listing of tools to apply. If you decide to jump into it optimization yourself, timing the execution of the benchmark tasks would be a good way to compare alternate approaches.

Thanks!

Brandon

[SeH]

I agree it's more important to focus on correct design.  Except if the design involves multiple numeric parameters that affect results, correctness and optimization sometimes overlap.  As long as there remain values can not be determined well theoretically, automated parameter search may find something better than the original hardcoded "magic numbers" programmed in.

https://github.com/toki78/JuRLs/tree/master/JuRLs/src/jurls/core/reinforcementlearning/becca

Here's some new so-far incomplete BECCA experiments using the JuRLs project.  It's a start of an atlernate design of Ziptie and Daisychain components which would be used to form the entire feature learning "half" of BECCA.  no results yet except observable matrix views to see some activity.

AEZiptie is embedding a raw Autoencoder as a Ziptie; AEZiptie2 uses an Autoencoder only to determine the mapping connectivity (allowing partial membership when an autoencoder weight matrix cell is positive) and apply these to the original inputs according to the contribution and inhibition rules described by the manual. 

Is it ever useful for a ziptie to unbundle a cable?  I know this would confuse an above gearbox and probably the RL, but if the bundling and unbundling process is continuous and happens slowly enough, it might be able to successfully "switch gears" to new representations.  this is what i think something like the autoencoder ziptie can test.

the Daisychain in that directory is a very simple co-ocurrence matrix with a fading memory. as it exists now it may not be correct; it's a sketch for something more complex involving multi-level HMM / bayesian network which would model on/off to on/off transition maps between 2 or more timepoints.

[Brandon Rohrer]

Thanks for the JuRLs link, SeH. I've added some thoughts below.

 

On Sun, Jan 4, 2015 at 9:14 PM, SeH <seh...@gmail.com> wrote:

I agree it's more important to focus on correct design.  Except if the design involves multiple numeric parameters that affect results, correctness and optimization sometimes overlap.  As long as there remain values can not be determined well theoretically, automated parameter search may find something better than the original hardcoded "magic numbers" programmed in.

https://github.com/toki78/JuRLs/tree/master/JuRLs/src/jurls/core/reinforcementlearning/becca

Here's some new so-far incomplete BECCA experiments using the JuRLs project.  It's a start of an atlernate design of Ziptie and Daisychain components which would be used to form the entire feature learning "half" of BECCA.  no results yet except observable matrix views to see some activity.

[BR] Cool. I'd love to see what shakes out of it.

 

AEZiptie is embedding a raw Autoencoder as a Ziptie; AEZiptie2 uses an Autoencoder only to determine the mapping connectivity (allowing partial membership when an autoencoder weight matrix cell is positive) and apply these to the original inputs according to the contribution and inhibition rules described by the manual. 

[BR] Ditto for this. I haven't gone through the exercise of implementing AEs for this purpose and am curious about the results.

 

Is it ever useful for a ziptie to unbundle a cable?  I know this would confuse an above gearbox and probably the RL, but if the bundling and unbundling process is continuous and happens slowly enough, it might be able to successfully "switch gears" to new representations.  this is what i think something like the autoencoder ziptie can test.

[BR] I haven't found the need for unbundling yet, but it's still early days. BECCA adapts to large changes ('gear switching' as you say, for instance when an agent is exposed to an entirely new environment ad task) by developing new bundles. Inapplicable bundles are largely ignored in a process related to L-1 normalization. It's actually closer to L-0 normalization because it tends to drive sparsity aggressively. It's almost as aggressive as winner-take-all. With this approach, inapplicable bundles aren't much of a distraction. It also allows the agent to switch back to the old context quickly without having to do much re-learning.

 

the Daisychain in that directory is a very simple co-ocurrence matrix with a fading memory. as it exists now it may not be correct; it's a sketch for something more complex involving multi-level HMM / bayesian network which would model on/off to on/off transition maps between 2 or more timepoints.

[BR] Cool. I suspect there is a lot room for making the daisychain more sophisticated and increasing its performance. So far I've been trying to minimize complexity and get a baseline general agent in place. The investigations you're doing are exactly the type of things I hope will follow.