How to limit memory usage?

[Jan.Eva...@seznam.cz]

Hello,

I would like to use hamsterdb in a memory-constrained environment. I created a simple test on Windows (create 2 million entries, and perform 50,000 random queries) with cache limited to 2 MB. The results show that database inserts do honor the cache limit while queries do not - the actual memory usage for queries shows peak of 60 MB private bytes without regard to the cache limit.

The memory space is actually used because the peak working set of the process is also 60 MB.

I tried to limit the memory use with HAM_CACHE_STRICT flag but hamsterdb cannot deal with files bigger than than the specified cache size when the flag is set (insert returns errors and queries fail to find some values).

Is there a way to limit actual memory usage in hamsterdb?

Thanks

Jan Evangelista

[Christoph Rupp]

Hi Jan,

which hamsterdb version are you using? There was a bug which caused the cache to grow too much, and not being purged properly. This was fixed in 2.1.5. (2.1.5 also removed the HAM_CACHE_STRICT flag because - as you noticed - it did not make too much sense).

If 2.1.5 does not work as expected then please tell me and i will fix the bug for the next release.

Best regards
Christoph

--
You received this message because you are subscribed to the Google Groups "hamsterdb User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hamsterdb-use...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Jan.Eva...@seznam.cz]

I am using 2.1.4. I will upgrade to 2.1.5 and retest.

Jan

[Jan.Eva...@seznam.cz]

I retested it with 2.1.5 and the memory limit was honored, but I was hit by another bug: the test created a 30 GB database when cache was limited to 1 KB (or 10 KB), while the database size is normally 192 M when cache is set to e.g. 50 M.

Jan

---------- Původní zpráva ----------
Od: Jan.Eva...@seznam.cz
Komu: hamster...@googlegroups.com
Datum: 21. 2. 2014 10:32:08
Předmět: Re: [hamsterdb-user] How to limit memory usage?

[Christoph Rupp]

Shame, I can reproduce this. I'll release a new version this weekend.

Best regards
Christoph

[Christoph Rupp]

Hi Jan,

here's the branch with the fix:

https://github.com/cruppstahl/hamsterdb/tree/v2

Windows builds and tarball should be available by sunday evening (CET).

Have a nice weekend,
Christoph