Main memory size grows more than 1 GB

[Vijay Veeraraghavan]

I know the java-chronicle is backed by the mapped file to write huge data kept outside the heap space, to the file.

While testing the IndexedChronicle and the VanillaChronicle in my application, I saw there is a raise in main memory by > 1 GB. I write around 10-30 million messages, runs for 2-3 minutes. While running the IndexedChronicle the main memory increases was very little. why? In case of VanillaChronicle, I see the increase in the memory by 1+GB. Now I am in a position to explain this to my QA, and also, my manager. 

Another problem I saw was, after writing 20 million records I put the writer thread to sleep for 30 seconds. The main memory did not decrease from the raised 1GB, till the application quits! Is it the expected behavior? When does chronicle free the main memory? Or do the chronicle always maintain the 1GB in the memory? Is there a way to change, increase/decrease, this amount? Anything to do with VanillaChronicleConfig?

regards

--V

[Vijay Veeraraghavan]

Peter

Your advise and explanation on this would be immensely valuable to me at this moment.

regards

--V

[Peter Lawrey]

The VanillaChronicle should be releasing the virtual memory in blocks.  There may be a bug where it is holding onto these blocks longer than it should be, however this makes *no difference to the amount of memory used*, only the amount of virtual memory used.

--
You received this message because you are subscribed to the Google Groups "Java Chronicle" group.
To unsubscribe from this group and stop receiving emails from it, send an email to java-chronicl...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Vijay Veeraraghavan]

Peter,

thanks for replying.

While integrating the chronicle with my application, my team member experienced a system crash (twise) (Windows 7, i5, 4 GB), with drastic increase in the memory. Now we suspect the cause of the crash is the VanillaChronicle. Will you be able to provide a patch, or shall a Cleaner outside the java-chronicle do that job? If so how it should be, else how can I release the memory?

regards

--V

[Peter Lawrey]

My computer I use as a TV has that spec, so I can try putting running it myself.  Can you send me a simple test which demonstrates the problem?

[Vijay Veeraraghavan]

Sorry for late reply, Peter.

I used the same example you provided for the VanillaChronicle. But after writing and reading the main thread sleeps for few seconds. I put the thread to sleep upto 30-45 seconds to check if the chronicle releases the increased memory. But I found it did not decrease (release??) till the application quits.

regards

--V

[Peter Lawrey]

Yes, it is not a bug as such, the default chunk size is 64 MB and the cache for these chunks is now 32.  This needs to be altered for smaller, windows machines as you note, this is too high.  It used to be a cache of 2 which is why you saw files around 128 MB at the start.

We are investigating whether resources are left open on Windows after you call close.

[Vijay Veeraraghavan]

Thanks Peter for investigating into this problem. It would be great if you could update this thread once you have made any progress on this bug.

regards

--V