Low memory use on Linux

[Kjetil Kjernsmo]

All,

I'm struggling a bit with that 4store is trashing my disks (which are very
slow), when it really shouldn't because I have plenty of RAM available, enough
for it to keep the whole dataset in RAM:

http://pastebin.com/9bZvNhpM

As you can see, 4store is just taking up a few hundred MBs of resident memory,
while 12 GB is cached and up for grabs.

This is v1.1.5 on a Debian Squeeze system, but I've seen the same problem on
an more up to data Debian Wheezy system.

I suppose this is a kernel tuning problem, has anybody else seen the same and
solved it?

Best,

Kjetil

[Rinke Hoekstra]

Hi Kjetil,

I have the same problem. Did you manage to fix things?

Thanks,

Rinke

[Kjetil Kjernsmo]

On Monday 6. January 2014 12.50.13 Rinke Hoekstra wrote:
> I have the same problem. Did you manage to fix things?

Nope... In retrospect, I think the poor performance of my queries might have
been due to that I had an unbound predicate. I didn't suspect it at the time,
because the triple pattern that would eventually bind the predicate was highly
selective, so it shouldn't have to scan all that much, but I suppose the
structure of the index could have some strange effects...

Kjetil

[Manuel Salvadores]

Hi,

4store does not do much (except few small internal caches) to keep things in memory. Once you start accessing a KB  the data indexes  should be kept in RAM by the SO caches unless you have heavy Read & Write query workloads.

What I  see in my "mostly read" workloads is  that my first queries are slow after emptying SO caches with 'sync ; sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches'. Once the data is in cache I do not see much of a penalty of slow IO.

Try to use some of the tricks here if you want to pre-cache the KB in memory.

http://serverfault.com/questions/43383/caching-preloading-files-on-linux-into-ram

You should be able to get to the 4store files in '/var/lib/4store/NAME_KB/'. First you might want to check if your data fits in RAM.

Manuel

--
You received this message because you are subscribed to the Google Groups "4store-support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to 4store-suppor...@googlegroups.com.
To post to this group, send email to 4store-...@googlegroups.com.

Visit this group at http://groups.google.com/group/4store-support.

For more options, visit https://groups.google.com/groups/opt_out.