Proposal discussion: Invalidating multiple keys in a set or with a dependency relation

[Miguel D]

I wonder if memcached implements a mechanism for tagging keys with a piece of metadata such that in a single operation you can invalidate all keys with the same metadata tag, basically supporting sets of keys for the delete operation.

Alternatively, and more generally, I wonder if memcached can maintain a tree/DAG of relationships or dependencies between keys, like this:

k1 -> k2

     |-> k3 -> k4

such that when you delete k1,   k2, k3, and k4 are also deleted but if you delete k3, only k3 and k4 are deleted.

I'm not aware if these mechanisms exist already, since I'm not familiar with all of memcached. 

If they don't, I'd like to start a discussion on implementing one or both of these mechanisms or similar ones. I understand that memcached tries to be minimal and with predictable amortized constant time performance, so *maybe* these features are overkill. Nevertheless, I'd like to hear the devs opinion.

Currently, if you need such a feature you would implement it in the client, keeping the metadata there and performing multiple delete commands when needed.

Users who do not need such a feature should not suffer performance overhead if they don't use them.

Regards,

Mihai (gh:hMihaiDavid)

[Mihai D]

I think I should provide some common usage patterns related to the first idea in the previous mail (delete sets):

It is common to store a set of objects using setm in a single command and retrieve all of them together in a single getm command. Set aliases would spare the user having to care about creating and storing or recomputing the keys for each individual object. I think this would not add much complexity since I'm not proposing set operations like union or intersection.

On different note, it is also common to split a big object (>1MB) into small individual 1MB objects to store. There could be a command that would allow storing a big object and let memcached do the spliting as the data arrives. I wonder if this would prompt users to start using memcached in use cases it is not design for tho. I also wonder how common this use case is.

Regarding the DAG keys, maybe it is too general and adds too much complexity.

[dormando]

Hey,

Thought I had a specific FAQ page for this but maybe not?

The reason why memcached can't delete groups of keys atomically, is
because it's a distributed system by default and the servers don't
communicate. Keys are spread across many servers. You can use namespacing
instead:

https://github.com/memcached/memcached/wiki/ProgrammingTricks#namespacing

which uses a tertiary key as an index.

Large objects are stored internally by splitting them into small objects.
Set the item size limit (-I) as high as you need, reasonably.

RE: sets in general. I have ideas, but not sure when I'd get to them.

> --
>
> ---
> You received this message because you are subscribed to the Google Groups "memcached" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to memcached+...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/memcached/da094a37-f637-43d5-9306-4cc058c9d3c6n%40googlegroups.com.
>
>