Mostly Disk Applicaiton Cache with extstore's ext_item_age=0

28 views
Skip to first unread message

Jacob Eisinger

unread,
Dec 4, 2020, 11:05:18 AM12/4/20
to memcached

Howdy,

We are investigating using Memcached's extstore as a single/non-sharded/non-clustered application cache of data larger than memory.  From my reading, it sounds like this is not the typical use case.  Due to that, I wanted to see if there were any unintended consequences of our configuration.

Specifically, we are looking to configure extstore with ext_item_age=0.  From our initial testing, it looks like this means that this effectively turns Memcached into a mostly disk-based cache of values.  In particular, Memcached will attempt to write out pages as soon as they fill to disk.  Then, when accessing values, the disk cache -> disk will be used.  Is that correct?

We noticed the extstore file grows and the memcache memory stays relatively low.  If we wanted to keep more of the recent values in memory, should we just increase the ext_item_age to a couple of hours?  Are there unintended consequences of this setup?  Any other configuration that we should be looking at?

Cheers,
Jacob

Jacob Eisinger

unread,
Dec 4, 2020, 11:45:36 AM12/4/20
to memcached
Oh, and, one more important aspect.  Our data is mostly write-once, read-many times.  There are no updates today.  And, in the future, there will be very few updates.

Tharanga Gamaethige

unread,
Dec 4, 2020, 2:30:18 PM12/4/20
to memcached
For the extstore, it doesn't matter whether your data set is sharded or not as the scope is a single Memcached process. Note that data is not durable, if the Memcached process crashes, data can not be recovered.

Yes, ext_item_age=0 will write data to the disk quickly, leaving only keys + meta data in memory. It's using FS cache as it does read/readv/pread/preadv.

If the probability of reading an item is high after it's written, you can increase the ext_item_age to keep it in memory. A possible downside of this is if your write rate is high, memory is full and the disk writes are slow, it will evict items before getting a chance to flush to the disk.

Extstore will work with both write-once or with updates.

Jacob Eisinger

unread,
Dec 4, 2020, 7:01:57 PM12/4/20
to memcached
Howdy,

Thank you for the quick response.  From what you said, I believe extstore is almost perfect for our usecase.

I had extensively read the wiki and I will read the blog posts now.  Thanks again!

Cheers,
Jacob
Reply all
Reply to author
Forward
0 new messages