Large slab class

[boaz shavit]

Hello,

I'm trying to understand how data is saved in memcached for items with size > 0.5M.

When I check the slabclass structure array, I see it only has values for classes up to .5 MB and another entry in place zero which looks like this:

(gdb) p slabclass[0]
$87 = {size = 0, perslab = 0, slots = 0x0, sl_curr = 0, slabs = 386, slab_list = 0x7f15b4015ef0, list_size = 2048}

when inserting a value which is very big (the key is 100 bytes, but the value is 800k) I see that the key goes to class 2 (which is 196 bytes) but I do not see where the value is stored. 

Can someone explain how this works for big values?

Thanks in advanced,

Bob.

[dormando]

Hey,

Items larger than the slab class max are "chunked" across multiple slab
chunks. See: https://github.com/memcached/memcached/wiki/ReleaseNotes1429

Since that release a "cap" chunk mode was added, so if chunk max is set to
16k and you store a 17k item, it will split into:

1) tiny chunk for key and header
2) 16k main chunk
3) attempt to "cap" with a 1k chunk

if it cannot allocate the 1k cap memory for some reason it will allocate
an extra 16k chunk instead.

The theory is that at larger item sizes we can spend a little CPU to
improve the memory efficiency. Making the "max slab class" smaller means
we can make better use of the slab classes. At some point I will be
reducing the default setting from 512k to 256k or lower, but I need to
revisit it and add some stats counters first.

-Dormando

On Tue, 6 Jun 2023, boaz shavit wrote:

> Hello,I'm trying to understand how data is saved in memcached for items with size > 0.5M.

> --
>
> ---
> You received this message because you are subscribed to the Google Groups "memcached" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to memcached+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/memcached/dedecb91-b3d2-4a83-8253-0ad965cecd68n%40googlegroups.com.
>
>

[boaz shavit]

Thanks a lot  Dormando.

We have made some customization to the code and we are using ITEM_data macro in order to retrieve the value of the key.

This work for all slabs except the last one. Is there any way to programmatically retrieve the data ?

Thanks,

Bob. 

[dormando]

Look for cases in the code for 'it_flags & ITEM_CHUNKED'. There are a few cases where the data is looped through. (append/prepend code).

On Jun 6, 2023, at 9:46 PM, boaz shavit <boa...@gmail.com> wrote:

Thanks a lot  Dormando.

To view this discussion on the web visit https://groups.google.com/d/msgid/memcached/e2254937-3369-4cc2-9de5-6982d1ef9066n%40googlegroups.com.