Re: Best way to query a list of IDs based on multiples booleans flags

[Felix Gallo]

With such small tables it's unlikely that you'll see a performance difference worth noting, unless you're running the lua script thousands of times per second.  And if you were, I would suggest caching the data directly in lua tables and bypassing redis altogether.

That said, I'd choose option 1, and create a set for each flag.  This maps cleanly and nicely to the way that the SQL table is organized if you're indexing on each flag, so moving data into and out of that data structure is fast and straightforward.

F.

On Tue, Nov 6, 2012 at 7:34 AM, Nicolas Bernard <nik...@nikkau.net> wrote:

Hi,

I have a SQL table with some records (between 500 and 10 000, no big dataset, it might help later).

Each object stored in this table have booleans properties based on linked data in multiples another tables.

Now I need a way to query all objects with a given list of boolean flag but I don't like SQL and don't want complex SQL query.

My idea is to calculate all these flags from my data in a background job, store them in Redis and for each query, ask Redis intersection of a given list of flags, then I get back a list of IDs and I can do a simple regular query on my primary table.

What's the best way to achieve that?

I have 2 ideas :

- For each flag, store a list of IDs in a set, so I have just to ask a SINTER of all sets of my query.

- For each flag, store IDs as offset in a string with SETBIT, so I have juste to ask a BITOP AND of all sets of my query.

The concept is pretty much the same but the first is simpler and the second is much memory-efficient.

But what is the fatest? (I will intersect a max of 30 criterias)

If it's matters, that will happen in a LUA script.

Or maybe I have miss a third and better way?

(Sorry for my terrible english ;))

Thanks in advance.

--Nicolas

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To view this discussion on the web visit https://groups.google.com/d/msg/redis-db/-/inljOOi2EAsJ.
To post to this group, send email to redi...@googlegroups.com.
To unsubscribe from this group, send email to redis-db+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.

[Josiah Carlson]

At 10k items, you won't see a lot of memory use regardless. Bits and
bitop will be more concise (obviously), and should be faster overall,
due to the better memory access patterns.

- Josiah