Direct SQL UPDATEs vs iDempiere Cache: Memory Leaks and Application Server Crashes

[Anna Fadeeva]

I want to discuss the topic of direct UPDATEs to the database bypassing the cache.

The first thing that becomes noticeable in the presence of direct UPDATEs via idempiereMonitor is that the cache size grows rapidly.This growth is visible on the charts.
When updating through the model, such growth does not occur, because the object is deleted back in young gen and does not have time to get into old gen.
During prolonged operation, the cache gets completely clogged and the server freezes if it is an isolated environment with a single user.
But under higher load, when there are many users and various integrations with other systems are running, the application server behaves differently: it crashes without any error in the logs. This looks very much like the operation of the Linux operating system's OOM Killer. The only salvation in this situation is a regular restart of the application server.

Let's consider two diagrams.

What diagram Scenario 1: Direct Database UPDATE (Bypassing Cache) shows:
1) The primary index (Table_ID + Record_ID) is overwritten and points to the NEW object
2) The secondary index (Table_ID + Value) still points to the OLD object (the 'OLD' key has not changed)
3) The eviction queue still holds the OLD object
4) GC cannot delete the old object — there are references to it
5) The heap gets clogged

What diagram Scenario 2: Standard Model-Driven Update (PO.save) shows:
1) The primary index is updated and points to the NEW object
2) The secondary index is updated — the old 'OLD' key is deleted, the new 'NEW' key is added
3) The eviction queue is updated — the old object is removed, it holds only the new one
4) GC CAN delete the old object — there are no references to it
5) The heap remains clean

I ask the community to write their explanation on this topic. Perhaps I did not describe the operation of the system accurately.