Optimal way to delete large keys

[Tim K]

First of all, I'm new to Redis, so I'm not really sure what qualifies as "large keys". For example, if I have some keys whose values are ZSETs with thousands of elements, is that "big" or do they have to be in the millions for a simple DEL to be "slow"?

I know v4 will have async UNLINK, but I'm currently looking at using v3.2.x so there is no UNLINK.

The use pattern for my app will be something like this pseudo-code:

ZUNIONSTORE key2 2 key1 key2  (merge ZSET key1 into ZSET key2, keep key2, then delete key1)
DEL key1

1. Kind of a side question: when doing the ZUNIONSTORE above, are values actually copied from key1 into key2 or will these be just shared references and it won't require extra memory equal to (roughly) the size of key1?
2. If shared references, is "DEL key1" still potentially slow or just a simple ref count decrement?

I'm thinking of a few approaches for efficiently deleting large keys (these would happen in a MULTI/EXEC or Lua script, so transactional):
1. SET key1 ""  (change the type of key1 from ZSET to empty string -- maybe this doesn't help as changing the type may actually do a sync DEL key1 first anyway)
    DEL key1

2. PEXPIRE key1 1  (let Redis deal with the expiration whenever it can get around to it)

3. Variation of #2, that makes the key name available again
    RENAME key1 key1del
    PEXPIRE key1del 1  (let Redis deal with the expiration whenever it can get around to it)


Any suggestions/comments? Thanks.

Tim

[Salvatore Sanfilippo]

Hello, your only option is, if not to use Redis 4.0, to consume the
key incrementally.
So you start getting ranges of 100 elements from the sorted set, and
send a ZREM with all the elements, and so forth. This will not block
the server.

> --
> You received this message because you are subscribed to the Google Groups
> "Redis DB" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to redis-db+u...@googlegroups.com.
> To post to this group, send email to redi...@googlegroups.com.
> Visit this group at https://groups.google.com/group/redis-db.
> For more options, visit https://groups.google.com/d/optout.



--
Salvatore 'antirez' Sanfilippo
open source developer - Redis Labs https://redislabs.com

"If a system is to have conceptual integrity, someone must control the
concepts."
— Fred Brooks, "The Mythical Man-Month", 1975.

[Tim K]

Thanks. I played with it a bit and timed some operations. It looks like my #3 proposal (RENAME followed by PEXPIRE) is very fast. Is it a bad idea to let Redis do the garbage collection? I'm not that concerned about freeing memory immediately.

FWIW, SET key "" is as slow as DEL, I assume it does a sync DEL of the existing key?