Re: Is there an efficient way to retrieve integers from Redis when it stores all values as strings?

[the beest]

I want to use Redis to store a growing number of hashes that will each contain approximately 200 fields that will all be counters. I will have to retrieve such hashes on average 1,000/s, and more frequently in the future. How can I get the number value from Redis without having to cast 200,000 strings to integers on average, every second?

[the beest]

This would be the only thing I would use Redis for. Also, the keys for the hashes would be integer ids, so if there is an efficient solution for getting numbers from Redis, it would be used exclusively. I would never retrieve string values.

[Jan-Erik Rediger]

If all you need is to retrieve the value just store the raw bytes in
there (4 or 8 bytes, depending on if you need 32 or 64 bit integers).
On the client side you would need to cast this into an integer, but it
does not require parsing.
Of course you lose the ability to do any operations using Redis
directly besides setting and getting them.

On Sat, Jun 25, 2016 at 11:20:18AM -0700, the beest wrote:
> I want to use Redis to store a growing number of hashes that will each
> contain approximately 200 fields that will all be counters. I will have to

> retrieve such hashes on average 10,000/s, and more frequently in the

> future. How can I get the number value from Redis without having to cast

> 2,000,000 strings to integers on average, every second?
>
> --
> You received this message because you are subscribed to the Google Groups "Redis DB" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.
> To post to this group, send email to redi...@googlegroups.com.
> Visit this group at https://groups.google.com/group/redis-db.
> For more options, visit https://groups.google.com/d/optout.

[Itamar Haber]

Have you looked into the new BITFIELD command?

[the beest]

Unfortunately, the counters have to be functional -- i'm going to be incrementing them in Redis

[the beest]

I just checked it out. The reason I thought about using hashes is because they provide a logical grouping for a set of counters related to a specific item id. Which there'll be about 200 of for any given item. If I use the new BITFIELD command, I'm going to have to give up that logical structure and query for each of an item's counters individually when I want to retrieve those that belong to a given item; which is the only way they would be queried for -- all together.

Would the difference between retrieving all the fields of a hash and retrieving 200 separate key-values be significant?

[Itamar Haber]

With BITFIELD, each key can be treated as a bucket-of-counters. So, instead of using a Hash per item's counters, use the BITFIELD instead.

If your interested in getting all counters per item, you can either compose a BITFIELD GET or, possibly, just GET the entire String value and break it down at the client.

In either case, I suspect BITFIELD would perform better than HMGET/HGETALL and would definitely require less RAM.

[the beest]

That seems nearly perfect. Strings in Redis can be as large as 2^32 bits. If I have 200 BITFIELD's each u32 in size, and I get the entire String will the String be 200 x 32 bits in length?

[Itamar Haber]

Yep.

Itamar Haber | Chief Developer Advocate
Redis Watch Newsletter | &&(curat||edit||janit||)
Redis Labs ~/redis

Mobile: +972 (54) 567 9692
Email: ita...@redislabs.com
Twitter: @itamarhaber
Skype: itamar.haber

Blog  |  Twitter  |  LinkedIn