memory violation error with Flora2.1 and XSB 4.0

[Marc Hadfield]

I get a non-deterministic error with some Flora-2 code:

++Error[XSB]: [Runtime/C] 

++Memory violation occurred during evaluation.

++Please report this problem using the XSB bug tracking system accessible from

++ http://sourceforge.net/projects/xsb

++Please supply the steps necessary to reproduce the bug.

++Error[XSB]: [Runtime/C] Exiting XSB abnormally...

with call stack:

Partial Forward Continuation...

... flora_db_reclaim_space/1  From /home/centos/FLORA21/Flora-2/flora2/syslib/flrstoragebase.xwam

... catch/3  From /home/centos/FLORA21/Flora-2/XSB/syslib/standard.xwam

... fllibprogramans/2  From /home/centos/FLORA21/Flora-2/flora2/syslib/flranswer.xwam

... call/1  From /home/centos/FLORA21/Flora-2/XSB/syslib/standard.xwam..........

This error occurs sometimes (not all the time) with the same code and same data.

If I go into flranswer.P, and simply comment out the reclaim call:

flora_end_of_answer_handler :-
        flora_clear_nested_timeouts.
        %% flora_reclaim_storage_space.

Then the error does not occur

I can (sometimes) trigger an error manually by calling the reclaim directly;  if I erasemodule the modules where data has changed and then call reclaim, then I don't seem to get an error.

As far as I can see the code is just using normal "insert", "delete", and "deleteall" directives to modify data and not anything unusual.

Any suggestions as to what may be going on? Or suggestions for diagnosing?  It seems that the module's data gets into a bad state via some race condition or something.

I imagine commenting out the "reclaim" would cause some memory leakage -- but would erasing the module reclaim that memory anyway?

Thanks for any help!

-- Marc

 

[Michael Kifer]

Marc, I need a small example to be able to investigate this.  Most likely it is an XSB bug, but it might be possible to work around it.

--

       --- michael


 

--
You received this message because you are subscribed to the Google Groups "ErgoAI, Flora-2, and XSB Users Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ErgoAI-Flora2-XSB...@coherentknowledge.com.
To view this discussion on the web visit https://groups.google.com/a/coherentknowledge.com/d/msgid/ErgoAI-Flora2-XSB-forum/CABgggufMaXKhhzQ1KnqL_3UXqSFo%3Dw%3DiS%3DeOiDrZw0TD%3Dzq8yg%40mail.gmail.com.

[Marc Hadfield]

I can test it out with the updated flora2 code synced with XSB 5.0 (once that is in SF) and see if the issue still persists.  Maybe whatever the issue is got resolved already.  But otherwise I'll try to produce an example.

--  Marc

To view this discussion on the web visit https://groups.google.com/a/coherentknowledge.com/d/msgid/ErgoAI-Flora2-XSB-forum/478c7294-ac68-40da-700c-4e0b1ec781f6%40coherentknowledge.com.

[Marc Hadfield]

Would whatever resources get reclaimed by "flora_reclaim_storage_space" get reclaimed by erasemodule ?

[Michael Kifer]

No. Those resources get reclaimed each time we get to the  top level so erase does not need to reclaim them.

Jun 27, 2022 11:17:31 Marc Hadfield <ma...@hadfield.org>:

[Marc Hadfield]

Right, I'm asking if erase *would* reclaim them.

To view this discussion on the web visit https://groups.google.com/a/coherentknowledge.com/d/msgid/ErgoAI-Flora2-XSB-forum/515d42b9-c2ab-4151-b4b6-ee179d706ffe%40coherentknowledge.com.

[Michael Kifer]

Don't understand. Do you mean to do reclamation in erase instead of at the top level? This pretty much makes this kind of garbage collection ineffective.

Jun 27, 2022 11:52:20 Marc Hadfield <ma...@hadfield.org>:

Right, I'm asking if erase *would* reclaim them.

On Mon, Jun 27, 2022 at 11:32 AM Michael Kifer <michae...@coherentknowledge.com> wrote:

No. Those resources get reclaimed each time we get to the  top level so erase does not need to reclaim them.

Jun 27, 2022 11:17:31 Marc Hadfield <ma...@hadfield.org>:

Would whatever resources get reclaimed by "flora_reclaim_storage_space" get reclaimed by erasemodule ?

On Sun, Jun 26, 2022 at 2:49 PM Marc Hadfield <ma...@hadfield.org> wrote:

I can test it out with the updated flora2 code synced with XSB 5.0 (once that is in SF) and see if the issue still persists.  Maybe whatever the issue is got resolved already.  But otherwise I'll try to produce an example.

--  Marc

On Sun, Jun 26, 2022 at 2:46 PM Michael Kifer <michae...@coherentknowledge.com> wrote:

Marc, I need a small example to be able to investigate this.  Most likely it is an XSB bug, but it might be possible to work around it.

--

       --- michael


 

On 6/25/22 5:48 PM, Marc Hadfield wrote:

…

--
You received this message because you are subscribed to the Google Groups "ErgoAI, Flora-2, and XSB Users Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ErgoAI-Flora2-XSB...@coherentknowledge.com.
To view this discussion on the web visit https://groups.google.com/a/coherentknowledge.com/d/msgid/ErgoAI-Flora2-XSB-forum/478c7294-ac68-40da-700c-4e0b1ec781f6%40coherentknowledge.com.

[Marc Hadfield]

I've found that 'flora_reclaim_storage_space" throws the Memory Violation error.

I've found that if I comment out the "flora_reclaim_storage_space" from the "flora_end_of_answer_handler" that this avoids the Memory Violation error.

But, this presumably causes a leak of resources.

I've found that calling erasemodule on modules where data changed and then calling flora_reclaim_storage_space does not cause the Memory Violation error.

So, it looks like erasemodule:

1) removes the resources that flora_reclaim_storage_space would but in a way that doesn't cause the Memory Violation error, 

or

2) updates the state in a way that leaks resources but updates whatever flora_reclaim_storage_space is looking at so it doesn't hit the Memory Violation error, 

or 

3) something else that I don't know.

So, knowing which of these is happening presumably will help find whatever the issue is with the Memory Violation error.

Also, this helps me understand the impact of my temporary workaround of modifying flora_end_of_answer_handler to not trigger the Memory Violation error.

-- Marc

To view this discussion on the web visit https://groups.google.com/a/coherentknowledge.com/d/msgid/ErgoAI-Flora2-XSB-forum/56eb7736-a1ed-4a33-8e75-56931e06ab50%40coherentknowledge.com.

[Michael Kifer]

Module erasure happens very rarely if at all.
The KE has to structure the whole thing so that module erasure would be meaningful and useful.
This memory violation shouldn't happen and I'd like to investigate it.

Michael

Jun 27, 2022 12:17:14 Marc Hadfield <ma...@hadfield.org>:

I've found that 'flora_reclaim_storage_space" throws the Memory Violation error.

I've found that if I comment out the "flora_reclaim_storage_space" from the "flora_end_of_answer_handler" that this avoids the Memory Violation error.

But, this presumably causes a leak of resources.

I've found that calling erasemodule on modules where data changed and then calling flora_reclaim_storage_space does not cause the Memory Violation error.

So, it looks like erasemodule:

1) removes the resources that flora_reclaim_storage_space would but in a way that doesn't cause the Memory Violation error, 

or

2) updates the state in a way that leaks resources but updates whatever flora_reclaim_storage_space is looking at so it doesn't hit the Memory Violation error, 

or 

3) something else that I don't know.

So, knowing which of these is happening presumably will help find whatever the issue is with the Memory Violation error.

Also, this helps me understand the impact of my temporary workaround of modifying flora_end_of_answer_handler to not trigger the Memory Violation error.

-- Marc

On Mon, Jun 27, 2022 at 12:03 PM Michael Kifer <michae...@coherentknowledge.com> wrote:

Don't understand. Do you mean to do reclamation in erase instead of at the top level? This pretty much makes this kind of garbage collection ineffective.

Jun 27, 2022 11:52:20 Marc Hadfield <ma...@hadfield.org>:

Right, I'm asking if erase *would* reclaim them.

On Mon, Jun 27, 2022 at 11:32 AM Michael Kifer <michae...@coherentknowledge.com> wrote:

No. Those resources get reclaimed each time we get to the  top level so erase does not need to reclaim them.

Jun 27, 2022 11:17:31 Marc Hadfield <ma...@hadfield.org>:

Would whatever resources get reclaimed by "flora_reclaim_storage_space" get reclaimed by erasemodule ?

On Sun, Jun 26, 2022 at 2:49 PM Marc Hadfield <ma...@hadfield.org> wrote:

…

--
You received this message because you are subscribed to the Google Groups "ErgoAI, Flora-2, and XSB Users Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ErgoAI-Flora2-XSB...@coherentknowledge.com.
To view this discussion on the web visit https://groups.google.com/a/coherentknowledge.com/d/msgid/ErgoAI-Flora2-XSB-forum/515d42b9-c2ab-4151-b4b6-ee179d706ffe%40coherentknowledge.com.

[Marc Hadfield]

Our code regularly uses erasemodule to clear out data, such as the "A Box" data relating to a particular business transaction.

But, my question is if "flora_reclaim_storage_space" is *not* used, does erasemodule free those same resources?  It looks like they use some of the same components like the XSB storage module, but I haven't deeply traced the code.

I'm asking to determine the impact of not using "flora_reclaim_storage_space" as a short term workaround as well as a means to help diagnose the underlying memory violation issue, although I'm hoping that issue just goes away once switched to XSB 5.

-- Marc

To view this discussion on the web visit https://groups.google.com/a/coherentknowledge.com/d/msgid/ErgoAI-Flora2-XSB-forum/57d15a81-3160-4c82-8b7c-98ea52f6c8f4%40coherentknowledge.com.

[Michael Kifer]

The resources freed up by flora_reclaim_storage_space  are the deleted facts. For efficiency, when a fact gets deleted, it is only marked as such as often these deleted facts get undeleted by backtracking. The deleted facts are disposed of by flora_reclaim_storage_space.

So, if there is a humongous number of deletions then memory leakage is significant.

My first reaction was to say that erasemodule does not  free up the space occupied by the marked deleted facts.

But thinking more about it, it occurred to me that

erasemodule deletes the entire data tries associated with the module, so the marked deleted facts are deleted as a part of that process.  Since you are saying that you do erasemodule frequently, it appears that commenting out the calls

to flora_reclaim_storage_space should be safe in your situation.

--

       --- michael


 

[Marc Hadfield]

Great! Thank you for the info Michael.  This helps with the short term work-around, and hopefully is a hint as to what the issue is if it still occurs once using XSB 5.

-- Marc