These are good discussions to have, for sure...
A few points
As Linas's comments on RelEx / link grammar indicate, having a separate repo and a mailing list and good documentation and solid, utterly modular code, and even a few publications, does not provide anywhere near a guarantee of garnering developer or user interest for a project...
I believe that OpenCog developer needs and OpenCog *user* needs (where by a user I mean a non-AI-researcher software developer who wants to use OpenCog in some product or serivce they are building) are going to be utterly different. I made a very rough sketch of my thoughts about a direction for an OpenCog interface for users, a little while ago
Building something like this would be a significant project on its own, of course leveraging existing OpenCog tools, probably including the RESTful API and the Scheme shell (or a similarly-functional python analogue), etc.
Some parts of OpenCog are messy and need massive refactoring. The Embodiment module contains a lot of mess, for example (though some of it is fine, e.g. Shujing's 3DSpaceMap and closely associated code).
On the other hand, some parts of OpenCog are quite nice -- take a look inside the PatternMatcher code that Linas wrote (as Misgana and I have been doing recently...)
Also, a number of infrastructure improvements have been suggested here
Many of these would help address the issues raised by Keyvan and Ramin (e.g. items 6, 8 and 9). This list also includes some cleaning up of Embodiment.
However, listing needed improvements takes less time than making them. Anyone who wants to pitch in with these, that would be great!
Modularizing code is a subtler matter in an OpenCog context than in the context of a typical software project.
As an analogy, the parts of a car are pretty modular; the parts of a human body much less so. There are lots of interdependencies between body organs because the organs have all evolved to work together. The same is true for the parts of the human brain.
Now, OpenCog is an engineered not evolved system so its parts don't have to be as interdependent of those as a human body or brain. On the other hand, it is a complex adaptive system (which a car is not, and neither are the current batch of self-driving cars), which probably is not compatible with the maximal level of clean modularity that can be achieved in a system where the parts can all operate fairly independently of each other, interacting only via simple and clean interfaces.
I realize this is a somewhat philosophical point, but I'll make it concrete below.
Now I want to talk about some stuff being currently developed in OpenCog. I think it's important to ground discussions like these in terms of specific AI modules or capabilities; otherwise the discussion can too easily turn into fancy-sounding articulations of generally good design practices...
I will enumerate a few cases, different in nature...
PLN. Our current plan is to replace the existing python PLN with a new PLN version that is much more tightly coupled with the Atomspace. This involves e.g.
-- rewriting the PLN inference rules as ImplicationLinks, living in the Atomspace
-- replacing the current custom PLN chainers with usage of a generic chainer that operates on the Atomspace, utilizing the PatternMatcher.
-- some PLN-customized callbacks for the PatternMatcher (so, custom C++ code)
On the face of it, this seems the opposite direction of what Ramin and Keyvan are advocating. Yet, I think it is the right direction for development of PLN as an AI system. Key advantages are:
-- it makes clear that the PLN inference rules are cognitive content, which can then be improved by PLN, studied by pattern mining, etc. (In the python version the rules were tangled up inside python code)
-- rather than proliferating another specialized chainer, it makes use of an elegant general purpose chainer (that Misgana is just starting to build)
So we are embodying two good principles here
-- whenever possible, make the rules of operation of an AI module into cognitive content that can be learned and adapted by the system
-- avoid making multiple modules in the code that do the same thing (in this case, many chainers)
But we are making a PLN that is interwoven w/ other OpenCog functions, in a careful and thought out way. This is not spaghetti code.
Currently it's broken into
-- a modular piece, which is a custom RelEx output generator (a separate Java class to be run w/in RelEx)
-- some Scheme code that postprocesses the output of said output generator
We intend to replace the former part with some ImplicationLinks to be executed via the general-purpose PatternMatcher based rule engine. Furthermore, it has become apparent that the most elegant/simple way to do RelEx2Logic will be to invoke PLN or a similar inference engine,
which does not really gel with the idea of having RelEx2Logic operate separately of other OpenCog modules.
Shujing has coded a frequent subgraph miner for the Atomspace. By its nature, this must be closely tied to the particulars of Atomspace representation (though not to the specific Atom types currently used). This makes heavy use of the PatternMatcher, and uses a separate Atomspace as an interim data-store to store the raw patterns in the process of being mined. (This makes the patterns themselves into cognitive content, which is important for many reasons, including that we can then do "meta-pattern mining" on this second Atomspace to learn patterns regarding which kinds of patterns tend to be worth searching for, and these meta-patterns can then be used to guide future pattern mining...)
The Pattern Miner is C++ code. It didn't have to be; Shujing could instead have extended the PatternMatcher Scheme API to encompass more of the functions of the PM's C++ API, or could have written a python API for the PM, etc. However, Shujing is a good C++ programmer, and the PM and Atomspace are in C++, so the choice of C++ seems fine enough to me here.
The Pattern Miner itself could of course be invoked via Scheme or Python or via the REST API or whatever, and its results could then be exported to non C++ code, etc. This will be done after the functionality has been more thoroughly tested and tuned.
The Pattern Miner can't really be a module separate from the Atomspace, Pattern Miner, etc. But it could have a specialized API allowing users who don't care about OpenCog to use it as a graph/hypergraph mining tool.
This has always been a separate module from the rest of OpenCog, as it originated separately. We are setting about reimplementing it, because the current C implementation is kind of a pain to work with, and not that flexible.
We considered implementing it very tightly coupled with the Atomspace, but the direction Tewodros (Teddy) and I are currently playing with is to reimplement it in python using the Theano GPU libraries developed by Yoshua Bengio's group at U. Montreal. This is convenient because
-- DeSTIN, unlike most of OpenCog, can make good use of GPUs for acceleration
-- As there is fairly good support for python in OpenCog, it will still be relatively straightforward to incorporate OpenCog functionality into DeSTIN dynamics if we do it this way
So in this case, we'd have a pretty separate module, and would continue to package and document it separately, as well as to use it together with OpenCog.
The purpose of ECAN is to spread STI and LTI among Atoms, and to build Atoms based on STI and LTI values. The goal here is to make these values accurately reflect the short and long term importance of Atoms, so as to effectively guide other cognitive processes.
As ECAN needs to loop through a lot of Atoms as quickly as possible, this seems a case where including delays between ECAN and the Atomspace is a bad idea. Also ECAN is specialized algorithmically for the Atomspace, and to use it for some other sort of AI system, woulc require lots of customization. Not straightforwardly made modular.
Eddie Monroe, Mike Duncan and I are looking at integration of MOSES w/ the Atomspace in gene expression data analysis problems.
For instance, within MOSES, if one has program trees representing patterns in genetic data, one may wish to mutate a program tree in a manner that makes use of OpenCog's assessment of similarity between two genes references in the program tree. This would require MOSES to make a call to PLN, or at least a lookup into the Atomspace, in the course of mutating a program tree.
This is a case where using a clean separation makes sense. E.g. the RESTful API in some variation could be used by MOSES to query the Atomspace or PLN and get an answer.
OK, there are more examples, but that's probably enough to get across the variety of scenarios.
To me, it is clear that a simplistic insistence on broad design principles like modularity doesn't really cut the mustard here. We're doing something really complicated here, and only a certain amount of that complicated can be banished via effective software design.