How to implement cell-automata/neural-net stuff ?
ne...@absamail.co.za
as described at:
http://www.rebelscience.org/Cosas/System.htm
I'm guessing that knowledge of how neural-networks are simulated
would help ?
On the one hand they seem to be screaming for an OOP
implementation, where each of the duplicated neuron is an object ?
OTOH using OOP hides the communication between neurons, which I'm
not comfortable with, and having arrays, which are scanned once per
cycle, [all changes occur in clocked cycles] seems natural.
The basic method[s] used by spreadsheets seem appropriate ?
Here's a description of some of the 'objects':
> A cell is an active synchronous objects that communicates with
> other cells via signals.
> There are two types of cells: sensors and effectors.
> -Sensors detect changes or patterns of changes
> - and effectors execute changes.
> Together with data (passive objects) cells comprise the basic
> building blocks of every COSA application.
>
> A synapse is part of a cell. It is a small data structure which is
> used to connect one cell to another.
Although lacking OOP experience, this sounds like:
- Sensors & effectors are descendants of cell-objects
- a class attribute of cell-objects is 'synapse'
with value=DestinationCell.
Personally I'd prefer to have tables/arrays where the
stepping is updated explicetly by accessing [running through]
the appropriate arrays.
Thanks for any ideas or advice,
== Chris Glur.
bob the builder
Hmm, i made a neural network in excel (long time ago), it didnt make
me happy. If you want to build a network from scratch then c++ or java
would be a better choice. And maybe you can program a more traditional
ANN first, to get the hang of it. If you decide to use c++ then you
could use Dev-c++ for example (http://www.bloodshed.net/dev/
devcpp.html) (its free).
SteveW
> We are trying to prototype some ideas to simulate the COSA system
> as described at:
> http://www.rebelscience.org/Cosas/System.htm
I read a few of the articles at the site and found them interesting
but beyond my ability to judge the merits of the arguments.
It is most interesting to see someone undertaking such an
ambitious and significantly different approach.
> I'm guessing that knowledge of how neural-networks are simulated
> would help ?
Seems like it would. That does seem to be the underlying model
they are basing their approach on. However, as you will
discover as you browse here and the web about NN's, there are
many different specialized types of NN's. Some (e.g. backprop) are
specialized
to perform a specific task accurately; these would seem the
least appropriate for the kind of thing they're doing.
Unsupervised recurrent NN's have feedback loops, are more along
the lines of a "free-running system", and there are
modifications that try to simulate parallel processing by
selecting randomly which cell(s) will do their calculations, rather
than running through an array in sequence.
See this group's FAQ document for a very good overview
of ANN's and introduction to the field, an
introduction to the terminology, and links. It's a good place to get
started.
> On the one hand they seem to be screaming for an OOP
> implementation
> (Although lacking OOP experience, )
For a large project (and even a small one), I believe OOP is
the only way to go. It helps keep you organized, keeps your
code clean, and helps makes your code understandable and maintainable.
There's now a variety of OOP languages, but most are offshoots of C++.
If you learn any of them, you'll know the fundamental concepts of all
of them. There are free compilers available for most of them.
What is not consistent and portable among the languages are
the libraries (Microsoft MFC or .NET, Borland VCL, Java whatever,
etc.),
so the operating system you want to use and the variety of functions
(e.g. graphics) you need will be major considerations in the choice.
There's a pretty steep learning curve for the libraries, much harder
than for the languages themselves, so choose with care.
After looking at various ones, I still prefer C++.
MS Visual Studio C++ 2005 Express is available as a free download.
I've been using it for a week or so, and like it. It will probably
become my new primary development language/IDE.
If it is of any interest, there is a free open source C++ neural
network program on my website at
http://25yearsofprogramming.com/ai/neuralnet/index.htm
The network classes are on the related page
http://25yearsofprogramming.com/ai/neuralnet/networkcpp.htm
I'm currently working on the (now functioning)
version for STL and MS Visual Studio C++ 2005 (.NET), and hope to have
it uploaded shortly.
What I thought might be of interest to you is the classes
(Node, Synapse, Network). The full program creates a backprop network,
definitely not what you'd want, but the classes should be easy to
press
into service for any type of network. The code is
heavily commented and should be easy to modify.
If those don't suit your purpose, you might browse around
http://www.sourceforge.net. However, most of the ANN programs I looked
at there are either quite specialized or have source code that is
pretty difficult to figure out by inspection.
> OTOH using OOP hides the communication between neurons,
Not sure what you mean by that.
> The basic method[s] used by spreadsheets seem appropriate ?
I doubt this. For such an ambitious venture, I suspect a spreadsheet
approach will quickly turn out to be more limiting than is tolerable.
Kent Paul Dolan
> as described at:
> http://www.rebelscience.org/Cosas/System.htm
If you'd _read_ the site, you'd see how counterproductive this idea
is. What make COSA "work" is precisely that it _is_ the operating
system, thoroughly debugged, and protected from user code twinking,
even at the applications level. _Simulating_ it opens it to twinking
and
to introduction of new bugs. If you want COSA, pick a machine to
dedicate to the enterprise, and run COSA on it. Otherwise, forego the
main benefit COSA offers. If you do that, why on earth bother with
COSA?
IMO
xanthian.
ne...@absamail.co.za
> > We are trying to prototype some ideas to simulate the COSA system
> > as described at:
> > http://www.rebelscience.org/Cosas/System.htm
The way 'objects communicate' [call methods] is hidden by the
language implementation. But I suspect I'm thinking at the wrong
level of granularity ?
> > The basic method[s] used by spreadsheets seem appropriate ?
>
> I doubt this. For such an ambitious venture, I suspect a spreadsheet
> approach will quickly turn out to be more limiting than is tolerable.
Well, it's so far removed from my experience that even initial
modelling with match-sticks could advance my understanding.
Thanks, I'm looking at your references.
><><><><><><>
xanthian wrote:
> is precisely that it _is_ the operating
> system, thoroughly debugged, and protected from user code twinking,
> even at the applications level. _Simulating_ it opens it to twinking
> and to introduction of new bugs. If you want COSA, pick a machine to
> dedicate to the enterprise, and run COSA on it. Otherwise, forego the
> main benefit COSA offers. If you do that, why on earth bother with
> COSA?
No ! COSA is only a concept. I'm in contact with the 'founder'
who's looking for collaborators to implement it. You can join too.
Thanks,
== Chris Glur.
John Ladasky
> implementation, where each of the duplicated neuron is an object ?
I wrote my own neural net implementation about a year ago, in Python.
It's slow, but it works, and I can see clearly into its inner
workings.
The reason that I chose to write my own is that I was studying neural
nets on paper, and discovered that many useful functions (for me)
needed four or even five layers of neurons to implement them. At the
same time, that I wanted to implement these long, tricky strings, I
didn't want to waste layers of neurons on functions which do not
require so many layers. Thus I implemented a "layerless" neural net.
Each node receives a number. The highest-numbered nodes are an output
layer of sorts -- but every node in the net, including the output
nodes, is capable of receiving imput from any lower-numbered node. In
the low-numbered nodes, it's essentially a free-for-all.
It would have been a huge waste of computing resources (and
programming time) for me to model each neuron as its own object. I
can think of a few reasons why you might want each neuron to be an
object, but none of them applied to me:
1) You have multiple types of neurons, with different topologies,
weighting, or summation functions. All of my neurons are homogeneous,
and nearly every neural net I've ever read about is the same way.
2) You need neurons to possess a persistent internal state -- for
memory, perhaps. In my system, if memory is required, it will arise
in feedback loops.
3) Your search/optimization algorithm involves a lot of neuron
transposition -- i.e., you're changing the topology of the network.
Although I can imagine that shuffing neurons around might be useful in
some circumstances, I've never seen it done.
+-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-+
| Ladasky Home Solar, Inc.: blowing sunshine up your |
| power grid since March 24, 2005. Fiat lux! |
+-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-+
| Uptime Downtime kWh generated kWh consumed |
| 772.5 days 13 hours 13485 14875 |
+-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-+
Tim Tyler
> On May 8, 5:48 am, n...@absamail.co.za wrote:
>> ...an OOP implementation, where each of the duplicated
>> neuron is an object ?
> It would have been a huge waste of computing resources (and
> programming time) for me to model each neuron as its own object. I
> can think of a few reasons why you might want each neuron to be an
> object, but none of them applied to me:
[snip]
Encapsulation and type safety would appear on many such lists.
--
__________
|im |yler http://timtyler.org/ t...@tt1lock.org Remove lock to reply.
Josip Almasi
> We are trying to prototype some ideas to simulate the COSA system
> as described at:
> http://www.rebelscience.org/Cosas/System.htm
>
> I'm guessing that knowledge of how neural-networks are simulated
> would help ?
>
> On the one hand they seem to be screaming for an OOP
> implementation, where each of the duplicated neuron is an object ?
Not really; you got detailed answer already.
OTOH I did implement 'neurons' with objects, in java, Observer, and
wrote a fuzzy essay to explain it to my OOP coleques,
http://vrspace.sourceforge.net/docs/zen_of_self-adaptive_code-jitsu.html
In my tests, I had 100000 'neurons' per node with response time ~100 ms.
With soft MRU cache (*) I got about 2M 'neurons' per node; 1000
'neurons' in memory, but memory requirements grew to 2G for references.
Response time remained about the same due to good cache hit rate.
So it works for me, but in general case, hardly.
It *could* also work for you, say your + 'neurons' don't have many
'synapses'...
But note these quotes I never forget to put around 'neurons'... it's not
ANN, it's *inspired* by NNs. Very basic difference is that my 'neurons'
are carefully crafted state machines with much more complex behaviour
and much less connections compared to (most of) ANNs, not to mention NNs.
Also note that due to that synchronous requirement it's not easy to use
either Observers or Listeners. It's possible, see that Checkpoint trick
in the essay.
HTH, questions, drop me a mail, I'll be glad to help.
WRT COSA, I have to say I'm ambivalent. I do tend to agree that
event-driven programming tends to produce less errors than say state
machines; OTOH there's lot of buggy electronics too. Especially in PC
world, like, I plug in woodoo and my scsi controler hangs. The way I see
it, it's due to open interfaces. And in both hardware and software,
protocols specify communication but not peer states, and you simply
can't avoid states.
OK to make long story short - proof of concept is required, either to
prove or disaprove.
Therefore - good luck guys.
Regards...
(*) like this:
http://vrspace.cvs.sourceforge.net/vrspace/vrspace/src/main/org/vrspace/neurogrid/SoftCache.java?view=log
Vend
> We are trying to prototype some ideas to simulate the COSA system
> as described at:
> http://www.rebelscience.org/Cosas/System.htm
The question is: why you would?
> I'm guessing that knowledge of how neural-networks are simulated
> would help ?
>
> On the one hand they seem to be screaming for an OOP
> implementation, where each of the duplicated neuron is an object ?
>
> OTOH using OOP hides the communication between neurons, which I'm
> not comfortable with, and having arrays, which are scanned once per
> cycle, [all changes occur in clocked cycles] seems natural.
Hint: you can put objects into arrays (or lists).
> The basic method[s] used by spreadsheets seem appropriate ?
>
> Here's a description of some of the 'objects':
>
> > A cell is an active synchronous objects that communicates with
> > other cells via signals.
> > There are two types of cells: sensors and effectors.
> > -Sensors detect changes or patterns of changes
> > - and effectors execute changes.
> > Together with data (passive objects) cells comprise the basic
> > building blocks of every COSA application.
>
> > A synapse is part of a cell. It is a small data structure which is
> > used to connect one cell to another.
>
> Although lacking OOP experience, this sounds like:
> - Sensors & effectors are descendants of cell-objects
> - a class attribute of cell-objects is 'synapse'
> with value=DestinationCell.
Don't use OOP if you aren't familiar with it.
A.L.
>We are trying to prototype some ideas to simulate the COSA system
>as described at:
> http://www.rebelscience.org/Cosas/System.htm
It looks like reinventing the wheel: it is very similar to Hoare's CSP
(Communicating Sequential Processes) where concurrently running
processes ("cells") comunicate through synchronous channels. Hoare's
book is available free on Internet. This concept was implemented in
the language Occam that was "high level assembly language" of platform
known as Transputer (now extinct). CSP has been implemented in Java
and implementation is known as JCSP. There are two implementations;
the best known is from University of Kent
http://www.cs.kent.ac.uk/projects/ofa/jcsp/
A.L.
Traveler
Well, the concept of synchronously communicating concurrent cells is
nothing new and I doubt that Hoare would lay claim to having inventing
the idea. Cells in a neural network do essentially the same thing.
However, I don't think that Hoare's primary goal was to build a
software environment for the construction of bug-free programs. It
seems that he was more concerned with creating a mathematical
formalism for parallel computation. Indeed, Occam was developed for
INMOS's Transputer, a computer that was ahead of its time.
The main problem with Occam (I am biased) is that it is a computer
language. In my opinion, the linguistic approach is ill-suited for
depicting communicating concurrent entities. The graphical approach is
much better if only because it presents the user with a easily
understood visual depiction of the communicating pathways between
objects. The other problem with Occam is that Hoare apparently did not
fully appreciate the advantage of using synchronous reactive entities
as a means to solve the software reliability crisis.
Although COSA is inherently concurrent and is thus well-suited to
parallel processing, parallelism is not its main strength. My primary
goal in designing COSA was to solve a vexing problem within the
software industry: unreliability. Unreliability imposes an upper limit
on the complexity of software systems. As an example, we could
conceivably be riding in self-driving vehicles right now (which would
prevent tens of thousands of accident-related deaths every year) but
concerns over safety and reliability will not allow it. In addition,
the cost of developing safety-critical software rises exponentially
with the level of complexity. The reason is that complex software is
much harder to test and debug.
It occurred to me that the only way to solve the problem was to devise
a software execution model and development environment that
facilitated and enforced deterministic control over the timing of all
events and operations: nothing is allowed to happen before or after
its determined time. The advantage of this approach is that it makes
it possible to build a software "sentinel" that automates the job of
discovering and enforcing the laws (constraints) that govern the
temporal behavior of a program. Additions and modifications to the
program are not allowed to break exisitng protocols thereby insuring
solid consistency throuhout the development cycle. The beauty of this
approach is that the number of constraints increases with complexity.
This enables the creation of bug-free software of arbitrary
complexity. Counterintuitively enough to a programmer, complexity
ceases to be the enemy of software reliability and becomes a welcome
and trusted friend.
There are other important advantages such as the automatic resolution
of data dependencies which go a long way toward creating bug-free
software. The idea that programs can be described exclusively with
actors (sensors and effectors) and their environment (data) simplifies
the development process and makes programs more comprehensible. It can
all be found at the URL below. It's nice to see Project COSA being
discussed on usenet.
Louis Savain
Why Software Is Bad and What We Can Do to Fix It:
http://www.rebelscience.org/Cosas/Reliability.htm
A.L.
wrote:
>
>It occurred to me that the only way to solve the problem was to devise
>a software execution model and development environment that
>facilitated and enforced deterministic control over the timing of all
>events and operations: nothing is allowed to happen before or after
>its determined time. The advantage of this approach is that it makes
>it possible to build a software "sentinel" that automates the job of
>discovering and enforcing the laws (constraints) that govern the
>temporal behavior of a program. Additions and modifications to the
>program are not allowed to break exisitng protocols thereby insuring
>solid consistency throuhout the development cycle. The beauty of this
>approach is that the number of constraints increases with complexity.
>This enables the creation of bug-free software of arbitrary
>complexity. Counterintuitively enough to a programmer, complexity
>ceases to be the enemy of software reliability and becomes a welcome
>and trusted friend.
Generally, I don't see anything in your project that would contribute
to the above claims, especially, quote: "environment that facilitated
and enforced deterministic control over the timing of all events and
operations: nothing is allowed to happen before or after its
determined time...". For me, this is still CPS-like environment where
you have all problems associated with concurrency, and that is not
free of such problems as, for example, deadlock. I don't see also any
"magic" that would allow, quote: "enables the creation of bug-free
software of arbitrary complexity..". You must be much more specific to
convince the public.
A.L.
P.S. Maybe you should check the following book: "Flow-Based
Programming: A New Approach to Application Development" by J. Paul
Morrison, Van Nostrand, 1994
Vend
> On Sun, 20 May 2007 21:21:06 -0400, Traveler <trave...@nospam.net>
I don't know much about CPS, but it seems to me that this COSA is a
kind of concurrent imperative language. Since everything is done
concurrently, it seems that the programmer has to explicitely take
care of timing constraints. This is a great complication, in
particular it severely impair the ability to make modular programs,
since the timing of a module, which is implementation dependent, must
be leaked to the module user.
Traveler
>Generally, I don't see anything in your project that would contribute
>to the above claims, especially, quote: "environment that facilitated
>and enforced deterministic control over the timing of all events and
>operations: nothing is allowed to happen before or after its
>determined time...". For me, this is still CPS-like environment where
>you have all problems associated with concurrency, and that is not
>free of such problems as, for example, deadlock. I don't see also any
>"magic" that would allow, quote: "enables the creation of bug-free
>software of arbitrary complexity..". You must be much more specific to
>convince the public.
>
>A.L.
I once had a brief exchange on usenet with either Hoare or someone
intimately familiar with Occam and he agreed with me that parallel
synchronous systems are inherently more reliable than algorithmic
systems. What Hoare and others fail to grasp is that the secret of
reliability can be summed up with one word, timing. I'll rephrase what
I wrote previously to make it even simpler to understand. Once you
gain control over the timing of events (this is automatic and
transparent in a synchronous system), then it is easy to create a
mechanism that automates the discovery and enforcement of the temporal
expectations. A temporal expectation is a simple constraint such as
event A always follows event B or event A and B are always
simultaneous.
It is not magic, as you call it, but it is definitely a silver bullet.
Best of all, it's extremely simple. Sorry you are having such a hard
time seeing how it works. I would not know how else to explain it. It
could be because I have been thinking about it for so long, I now take
certain things for granted.
>P.S. Maybe you should check the following book: "Flow-Based
>Programming: A New Approach to Application Development" by J. Paul
>Morrison, Van Nostrand, 1994
Well,I don't see why I should do that since COSA is not a flow-based
model, although it shares some similarities. COSA is a synchronous
reactive software model in the tradition of synchronous reactive
languages like Esterel, Signal, etc...
Traveler
[cut]
>
>I don't know much about CPS, but it seems to me that this COSA is a
>kind of concurrent imperative language. Since everything is done
>concurrently, it seems that the programmer has to explicitely take
>care of timing constraints. This is a great complication, in
>particular it severely impair the ability to make modular programs,
>since the timing of a module, which is implementation dependent, must
>be leaked to the module user.
Vend, why venture such an uninformed and negative criticism since you
admit that you don't know much?
Vend
> On 21 May 2007 03:26:08 -0700, Vend <ven...@virgilio.it> wrote:
>
> [cut]
>
>
>
> >I don't know much about CPS, but it seems to me that this COSA is a
> >kind of concurrent imperative language. Since everything is done
> >concurrently, it seems that the programmer has to explicitely take
> >care of timing constraints. This is a great complication, in
> >particular it severely impair the ability to make modular programs,
> >since the timing of a module, which is implementation dependent, must
> >be leaked to the module user.
>
> Vend, why venture such an uninformed and negative criticism since you
> admit that you don't know much?
Can you read? I said that I don't know much about CPS. I know your
project as much as is possible from the articles on your website.
Vend
>
> [cut]
>
>
>
> >I don't know much about CPS, but it seems to me that this COSA is a
> >kind of concurrent imperative language. Since everything is done
> >concurrently, it seems that the programmer has to explicitely take
> >care of timing constraints. This is a great complication, in
> >particular it severely impair the ability to make modular programs,
> >since the timing of a module, which is implementation dependent, must
> >be leaked to the module user.
>
> Vend, why venture such an uninformed and negative criticism since you
> admit that you don't know much?
Can you read? I said that I don't know much about CPS. I know your
George Maydwell
wrote:
>What Hoare and others fail to grasp
LOL
George Maydwell
--
Modern Cellular Automata: http://www.collidoscope.com/modernca
Collidoscope Hexagonal Screensaver: http://www.collidoscope.com
A.L.
>intimately familiar with Occam and he agreed with me that parallel
>synchronous systems are inherently more reliable than algorithmic
>systems.
The above statement has no sense. What is "algorithmic system"?
>What Hoare and others fail to grasp is that the secret of
>reliability can be summed up with one word, timing.
Sorry, nonsense.
> I'll rephrase what
>I wrote previously to make it even simpler to understand. Once you
>gain control over the timing of events (this is automatic and
>transparent in a synchronous system),
Sorry, not true
> then it is easy to create a
>mechanism that automates the discovery and enforcement of the temporal
>expectations. A temporal expectation is a simple constraint such as
>event A always follows event B or event A and B are always
>simultaneous.
>
Sorry, I will not comment the rest of your post...
A.L.
Traveler
Traveler
Maydwell) wrote:
>On Mon, 21 May 2007 09:15:58 -0400, Traveler <trav...@nospam.net>
>wrote:
>>What Hoare and others fail to grasp
>
>LOL
>
>George Maydwell
Are you easily amused or just kissing ass? ahahaha...
A.L.
wrote:
>On Mon, 21 May 2007 20:38:58 -0500, A.L. <fe...@2005.com> wrote:
>
>[cut]
>
>I get it. See ya.
Good. Get some textbooks. Basic. Study.
A.L.
Traveler
I'll do that when you stop kissing ass. How about that? ahahaha...
Tomasso
"Vend" <ven...@virgilio.it> wrote in message news:1179764775.3...@z24g2000prd.googlegroups.com...
Are you talking about CSP?
T.
Josip Almasi
>
> What Hoare and others fail to grasp is that the secret of
> reliability can be summed up with one word, timing. I'll rephrase what
> I wrote previously to make it even simpler to understand. Once you
> gain control over the timing of events (this is automatic and
> transparent in a synchronous system), then it is easy to create a
> mechanism that automates the discovery and enforcement of the temporal
> expectations. A temporal expectation is a simple constraint such as
> event A always follows event B or event A and B are always
> simultaneous.
Well this may be right but you can't have control of timing on the internet.
Did you address or ignore this issue, or am I just wrong?
Regards...
Vend
> Are you talking about CSP?
>
> T.
Yes, sorry ;)
Owen Rees
in <9ko453hvcou7lg61b...@4ax.com>:
>I'll do that when you stop kissing ass. How about that?
That is more or less the level of argument I would expect given what I
read on the web site. I don't have time to plough through long rants
about how everyone else is wrong to find out whether or not there is
anything of any substance there worth thinking about. If the ideas don't
stand up on their own without being wrapped in a fog of vituperation
then there is really not much point spending time on them.
--
Owen Rees
[one of] my preferred email address[es] and more stuff can be
found at <http://www.users.waitrose.com/~owenrees/index.html>
Traveler
wrote:
>On Mon, 21 May 2007 23:27:29 -0400, Traveler <trav...@nospam.net> wrote
>in <9ko453hvcou7lg61b...@4ax.com>:
>
>>I'll do that when you stop kissing ass. How about that?
>
>That is more or less the level of argument I would expect given what I
>read on the web site. I don't have time to plough through long rants
>about how everyone else is wrong to find out whether or not there is
>anything of any substance there worth thinking about. If the ideas don't
>stand up on their own without being wrapped in a fog of vituperation
>then there is really not much point spending time on them.
I am not running for office and Project COSA is not a popularity
contest. If you don't like it or if you don't like me personally, it's
your prerogative to ignore me or it. I don't have the time to convince
you personally nor do I really care to. I am too busy. There are those
who find COSA useful and interesting enough to want to participate in
the project. That's good enough for me. This is my last post in this
thread.
Kent Paul Dolan
> Are you easily amused or just kissing ass? ahahaha...
Well, you've just moved yourself firmly into the
column of Usenet kooks, crank "scientists" whose
fixed ideas cannot be overcome by any amount of
fact, logic, reason, or examples of error, and who
answer challenges to their claims with insults and
obscenities.
Please go be an annoying troll somewhere else. You
have nothing to offer the _discipline_ of computer
science but empty claims based on illogic.
xanthian.