looking for advice to help ONA learn rules of my alchemy world

[Gene]

I'm making an alchemy world so that ONA can learn its rules.

I'm having mixed results, hoping someone has suggestions.

I kind of suspect that the operations's output (which I feed back into ONA) is poorly constructed.

The rules of the world are 4 operations:

• ^mix increments the inventory of "mixed" things.
• ^boil decrements the mixed inventory, increments the "boiled" inventory.
• ^give produces a potion.  It decrements boiled & sends "g0" (which is the goal) back to ONA.
• ^blend does nothing.  It's wasted time.
• No inventory counter can go negative.  No inventory can go over 3.  (So in 100 consecutive ^mix calls, 97 are wasted.)

ONA quickly learns that ^blend is a waste; it still calls it occasionally, but not often.  That's cool.

ONA also quickly tends to call the ops in order: ^mix, ^boil, ^give.  Also cool.

It never quite figures out the max & min on the inventories, though.  For example, it might fall into a pattern of 5 more ^mixes (most of them wasted), a couple ^boils, a couple ^gives, couple more ^boils, couple more ^gives.  Due to the bounds on the inventory, a lot of those ops are wasted.

I presume that the ops can send some feedback into ONA.  I've experimented with that.  They seem to help it learn more quickly, but it still gets to about the same success rate (about 1/2 of the op calls are wasted).

Any advice?  Like, what format the feedback should be?  I've tried simple statements such as <mix --> N> (where N is the new inventory of mixed) and more complex sentences such as "<<((mix * 2) * (boil * 0)) --> ^boil> =/> ((mix * 1) * (boil * 1))>" (which is an attempt to show that calling ^boil when mix = 2 and boil = 0 results in mix = 1 and boil = 1).

Thanks for your thoughts.

Gene

[robe...@googlemail.com]

Hello,

One reason may be that it doesn't know the depth of the state ( <mix-->5> ) for example. You can easily fix this by using products with <(D*N)-->mix>, where D is the depth, this should simplify planning.

N could also be a combination of the counters of mixed and boiled, for example for mixed=5 and boiled=2 it can be 5A2, where A is the seperator. One can't yet use underscore _ as a seperator in ONA. This makes planing even easier, maybe you then don't need depth anymore.

NARS isn't that great to handle low level very mathematical ops in general. It was never built for these tasks.

Additionally the planning of any existing NARS could still be way better (as usual for NARS implementations).

That said.

One idea may be to "group" these low level ops into macros which are callable by ops, such as (^macro1, boil_and_give) so that it doesn't have to plan as deep.

Hope this helps.

--
You received this message because you are subscribed to the Google Groups "open-nars" group.
To unsubscribe from this group and stop receiving emails from it, send an email to open-nars+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/open-nars/34b7aac6-c8bd-4ec3-92c7-4119e93f866bn%40googlegroups.com.

[Patrick Hammer]

Hi Gene!

Cool example!

It has to take the size restrictions of the inventory into account in its behaviors, meaning you will need to let it perceive its inventory capacity, most effective would be simply to give it an event whenever its capacity is full.

@Robert: thank you for your valuable comments!

As an addition: _ in an atomic term name is fine actually, as long as it starts with a letter in one of the A-Z a-z 0-9 ranges.

Best regards,

Patrick

To view this discussion on the web visit https://groups.google.com/d/msgid/open-nars/CANYcpZw8wHzg7uUSfC3KjW5Va52YAx-Hj0pDQ4uBdc7aF5bFbQ%40mail.gmail.com.

[Gene]

Thank you both for the advice.  I'll play around with that & let you know how it goes.

[Gene]

Maybe I'm specifying the goal poorly.  I'm still new to Narsese, might be totally off on the syntax.

What if I want the goal to be "POTION good, EMPTY bad, and FULL bad"?  (Potion is the result of a successful ^give.  The others indicate that one of the inventory counters was already at a limit.)

Would that be:

<potion && (-- empty) && (-- full)>! :|:

?

That might be totally off.  Or maybe it's sort of right but has syntax problems somewhere?

Oh!  And maybe this matters:

• When ^give succeeds, it calls NAR_AddInputNarsese( "potion. :|:" );
• When an inventory is already at a limit, I call NAR_AddInputNarsese with "empty. :|:" or "full. :|:"
• Do those sound right?  I'm especially unsure about whether :|: is appropriate or necessary in these situations.

Thanks again.

Gene

[Patrick Hammer]

Hi Gene!

"When ^give succeeds, it calls NAR_AddInputNarsese( "potion. :|:" );
When an inventory is already at a limit, I call NAR_AddInputNarsese with "empty. :|:" or "full. :|:"
Do those sound right?  I'm especially unsure about whether :|: is appropriate or necessary in these situations."

Perfect! And yes these need :|: since they are events. Whenever you want it to be able to temporally relate the stimulus with other events, that's the way to go.

"What if I want the goal to be "POTION good, EMPTY bad, and FULL bad"?  (Potion is the result of a successful ^give.  The others indicate that one of the inventory counters was already at a limit.)

Would that be:

<potion && (-- empty) && (-- full)>! :|:"

Goal decomposition for && isn't implemented, for now simply let them periodically enter as individual goals:

potion! :|:

(-- empty)! :|:

(-- full)! :|:

though you can also force it to work on them on a specific order as goal decomposition for &/ is implemented:

((-- empty) &/ (-- full) &/ potion)! :|:

In case you don't want to provide any goal order or don't want to mention relevant subgoals, the system will be able to infer such by itself by learning the proper contingencies with the right sequences as preconditions. This happens naturally when the wrong hypotheses won't lead to the predicted outcome, making the hypotheses with "missing or incorrectly ordered preconditions" lose over the right ones in truth value.

Handling of multiple goals will be improved with the upcoming ONA versions btw., though under AIKR it's not a piece of cake.

Also: later you might also want to play with inheritance statements as inputs.

Simple inheritance statements such as <stimulus --> [property]>. :|: can mostly suffice and can easily be generated by sensors which report certain values etc.. From there, one can leave generalization and the building of more complex hypotheses (whatever it takes to predict and achieve the goals competently) to the system, though of course background knowledge can be helpful to the system.

Best regards,

Patrick

To view this discussion on the web visit https://groups.google.com/d/msgid/open-nars/151679cc-dcf7-4996-8372-476999cbad46n%40googlegroups.com.

[Gene]

I got it to work!  I plan to make my currently trivial "alchemy world" more complex to give NARS a bigger challenge.  I could post the results here.

I learned something important about ONA & have a question about it: After my first successes, the performance went back to about the performance of randomly calling the operations, sometimes even worse.  I assumed I was expressing things incorrectly in Narsese, so tried a lot of changes there.  I did a lot of debugging & figured out that when my operations send feedback to ONA before they return, ONA gets confused.  When I changed the ops to set flags, then I examine the flags in my main loop & send feedback from there, ONA learns so fast that I was surprised.

(I know that the examples work that way -- feedback from the main loop, but I had figured that sending it from the operations would give ONA more precise information.)

Q1. So is that an "anyone working with NARS needs to know" thing?

Q2. Is it caused by a race condition, where ONA is poking around & modifying its memory while NAR_AddInputNarsese then jumps in & modifies it, too?  Or is it fundamental to any implementation of NARS (ONA or otherwise)?

Once I figured that out, not only did it work well, but it learned way faster than I expected, even when the only thing I tell it about the world is the 4 operations I give it.

[Gene]

A useful technique I learned during the debugging was to dump ONA's memory to a file & read through those Narsese statements.

Interesting & useful.

At first, your eye sees it as a big pile of jibberish, but after reading a while, it makes sense just fine.

[Patrick Hammer]

Hi Gene!

Thank you for these very valuable observations!

Q1. So is that an "anyone working with NARS needs to know" thing?

Indeed, it's important to just set flags / args in the operations and provide feedback after the system's cycle returns from the "external loop". Else it's forced to do another cycle in its cycle, and what if this cycle triggers another operation execution, which is problematic, both from a timing obfuscation and real-time response time perspective.

Using flag ensures it's cycle is finished after the operation call as it's supposed to, so that it stays responsive to the environment / or the simulation can go on.

Q2. Is it caused by a race condition, where ONA is poking around & modifying its memory while NAR_AddInputNarsese then jumps in & modifies it, too?  Or is it fundamental to any implementation of NARS (ONA or otherwise)?

I agree, it's clearly something C/C++ API users should know, I can add this information to the usage page in the Wiki. Luckily in the Python interface this can't happen as it's not based on C function pointers called within the cycle, but interpretation of the operation execution print-out after the operation happened and the cycle finished.

Once I figured that out, not only did it work well, but it learned way faster than I expected, even when the only thing I tell it about the world is the 4 operations I give it.

Awesome to hear! 

Also, in case you do not use it for proprietary purposes and want to open-source the example: we could add it to the ONA procedure learning examples. In this case, it would be good to add your author information at the top of the file, and in case you have a Github user you could also tell me the name&email so that your contribution will also be listed there in the commit history. Alternatively you can also add the code by creating a pull request on Github in the ONA repository, or you could create your own "ONA examples" repository, whichever you prefer.

And: ONA v0.9.0 (to be released before this year is out!) is making good progress: https://github.com/opennars/OpenNARS-for-Applications/pull/132

Best regards,

Patrick

To view this discussion on the web visit https://groups.google.com/d/msgid/open-nars/0025b880-0d18-432a-b660-dbf717f8aca0n%40googlegroups.com.