New command: MHGET - command to get fields from multiple hashes

[Adam Renberg Tamm]

Hi!

I've made a short PR with a new command that I think fits well in Redis, https://github.com/redis-io/redis/pull/7347.

MHGET is a mix of MGET and HGET. It allows you to query for fields from multiple hashes in one command:

MHGET key field [key field ...]

I find this useful in a few cases. The main use-case I have currently is that we store user experiment assignments in Redis, as one hash per experiment, one field per user, value is the assignment group. By having the hash be per experiment, we can easily drop the hash once the experiment is over (we actually HSCAN through the hash to delete fields in batches before dropping the key, to not pay the O(N) cost in a single query). The drawback of this structure is that we need to execute several HGET commands to get all experiment assignments for a specific user. (We have usually <20 active experiments, but 10s of millions of active users).

What do you think, is this command useful for others? I think MHGET is slightly unfortunate as a command name. All other hash commands start with H. Perhaps HMHGET could work?

I'm not entirely sure who to request review from in the PR, given the recent maintainership news. Perhaps someone on this list can help me?

Cheers,

Adam

[Benjamin Sergeant]

One thing I would recommend is looking at the Redis Module API, and making a module out of your work.

Maybe your idea is great and generic enough, and could be merged, but with Redis Module you don’t even have to think about it, your command could get used right now.

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/redis-db/1369bc3f-3571-4783-a658-cb32cd0a78fdo%40googlegroups.com.

[Itamar Haber]

Hello Adam,

This is actually a long-time feature request and there's even an open PR for it (https://github.com/redis-io/redis/pull/7347).

However, until that PR is merged, as Benjamin said, modules are one way to go. Alternatively, you can use a Lua script, e.g.:

❯ redis-cli HSET foo bar 1
(integer) 1
❯ redis-cli HSET baz qux 2
(integer) 1
❯ redis-cli --eval /tmp/l foo baz , bar qux
1) "1"
2) "2"
❯ cat /tmp/l
if #KEYS ~= #ARGV then
  return redis.error_reply("KEYS and ARGV lengths must be equal")
end

local res = {}
for i, _ in ipairs(KEYS) do
  table.insert(res, redis.call('HGET', KEYS[i], ARGV[i]))
end
return res

To view this discussion on the web visit https://groups.google.com/d/msgid/redis-db/BFD17D35-67FF-4EE4-B2DB-BB7D29607364%40gmail.com.

--

Itamar Haber Technicalist Evangely Phone: +972.54.567.9692

Disclaimer

The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful.

[Adam Renberg Tamm]

Thanks Benjamin and Itamar for your replies. I hadn't looked into Redis Modules much before, and they seem like they could work. For this specific use-case, a pipelined call with multiple HGET commands could work too. (Which could be said for commands like MGET and MSET too).

I sent this email mostly to see if others have this use-case too, and if it makes sense to add the command as a core Redis command. I think people do, and it does :)

> This is actually a long-time feature request and there's even an open PR for it (https://github.com/redis-io/redis/pull/7347).

Yes, that's the PR I submitted and that I propose we merge :)

Cheers,

Adam

To view this discussion on the web visit https://groups.google.com/d/msgid/redis-db/CAFWLiejOpAzY8bHajhDgFrc%2BZiwJC%2BcUawKudhYKy6AjQ45Z-A%40mail.gmail.com.

[ky...@redislabs.com]

Hi Adam-

Love the idea, I think many would find it useful. Two things:

1. MHGET would need to be restricted in a cluster. Multiple key commands need all the keys to reside in the same hash slot. 
2. With your particular use case, you might actually want to consider Streams. Streams provide a similar (but not identical) field / value structure but with the added benefit of storing many together, each with their own unique identifier (a stream entry ID).  You can retrieve multiple entries and even multiple streams with the existing API. 

-Kyle

[Adam Renberg Tamm]

On Fri, Jul 3, 2020 at 5:45 PM ky...@redislabs.com <ky...@redislabs.com> wrote:

Hi Adam-

Love the idea, I think many would find it useful. Two things:

1. MHGET would need to be restricted in a cluster. Multiple key commands need all the keys to reside in the same hash slot. 

Do you mean I need to change the PR in some way to support this? I looked at MGET for inspiration, and it would have the same problem, right?

 

2. With your particular use case, you might actually want to consider Streams. Streams provide a similar (but not identical) field / value structure but with the added benefit of storing many together, each with their own unique identifier (a stream entry ID).  You can retrieve multiple entries and even multiple streams with the existing API. 

Streams seem interesting, but at a cursory glance, I don't see how they'd be useful here. Would you mind elaborating a bit more?

 

To view this discussion on the web visit https://groups.google.com/d/msgid/redis-db/fd24c6a4-c248-41ae-901f-7d6a78465b45n%40googlegroups.com.