HSCAN folow ZADD write commands not allowed after no deterministic commands.

645 views
Skip to first unread message

Alan Perd

unread,
Jan 9, 2014, 11:37:15 AM1/9/14
to redi...@googlegroups.com
I have tried the new functionality "zscan" in redis version .2.8.3.
It works fine, however , when I try to save the result in a new sorted set I got the following error: write commands not allowed after no deterministic commands.

This is my example:

def testScan():
    import redis
    import time
    r = redis.StrictRedis(host='10.40.1.94')
    lua="""
   
      local function WriteZadd(value,score)
      redis.call("zadd","carephone_fiter2",value,score)
      return 'ok'
      end
     
      local function Filterby()
      local bSearch=1
      local aux={}
      local cursor=0
      local ok=nil
      while bSearch==1 do
      ok=redis.call("zscan","carephone_SerialNumber",cursor,"match","1*","count",10000)
      cursor=tonumber(ok[1])
      aux=ok[2]
      for i=0,table.getn(aux) do
      local res1=aux[i]
      local res2=aux[i+1]
      WriteZadd(res1,res2)
      end
      if (cursor==0) then
      bSearch=0
      end
      end
      return 'ok'
      end
     
      Filterby()
     
    """
   
    t1 = time.time()
    res=r.register_script(lua)
    response=res()
    t2 = time.time()
    print response
    print(t2 - t1)
if __name__ == "__main__":
    testScan()


Josiah Carlson

unread,
Jan 9, 2014, 2:46:28 PM1/9/14
to redi...@googlegroups.com
When performing ZSCAN, you are implicitly examining the internal state of Redis. The internal state of Redis during ZSCAN (and equivalently SCAN, HSCAN, and SSCAN) is a non-deterministic result based on the content, order, and sometimes timing of previously executed commands. So the order of members returned during ZSCAN (as well as SCAN, HSCAN, and SSCAN) is not consistent, and would not necessarily be the same if you performed the exact same operation on a fully up-to-date slave or just-restarted master (if the ZSET had 1 entry, it would, but that's not really interesting). But here's the thing: in order to allow for a write in a Lua script, it *must* be perfectly repeatable on the slave, or during re-execution of an AOF prior to rewrite - both of which execute the full script that was executed on the master initially. But because the order isn't consistent if you ran it on a master vs. a slave (or a restarted master), you would have inconsistent data on the master and slave, which would be an error. So Redis prevents it.

All of that said, you seem to be trying to copy the ZSET, but you have a few problems with your Lua script which you can read about below [1]. If you really want to copy a ZSET, I would recommend just using "ZINTERSTORE copy 1 source", or on the Python side of things, r.zinterstore('copy', ['source']) . If you really need an incremental copy operation (executed block by block, not just a single-shot copy operation), I would recommend looking at ZRANGE, ZRANK, ZRANGEBYSCORE, and ZSCORE. With the combination of those 4 commands, it would not be unreasonable to perform a fairly reliable copy operation even during ZSET updates (assuming that the number of elements copied per pass tended to be greater than the number of modifications occurring on the ZSET itself).

Those first two paragraphs are more dense than I intended, but hopefully explain the situation and perhaps why you shouldn't be using ZSCAN for this. But if you explain *why* you are trying to copy your ZSET in the first place, there might be other solutions that are better.

 - Josiah

[1] First, despite the Python Redis client's early (but continued) mistake, the actual order for ZADD is 'zadd key score member', not 'zadd key member score (I personally prefer the member,score order, as it is the same as the hash, but it's too late to change on either side). Second, tables in Lua start with index 1, not 0, so your loop will definitely fail. Third, your iteration over elements will actually alternate members and scores, passing your member as a score and scores as members, then scores as scores and members as members - you can use "for i=1,#aux,2 do" to fix that (when combined with the initial loop index issue).








--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.
To post to this group, send email to redi...@googlegroups.com.
Visit this group at http://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/groups/opt_out.

Alan Perd

unread,
Jan 13, 2014, 9:07:29 AM1/13/14
to redi...@googlegroups.com
Thank you very much, the idea was clear, however the objetive wasn't copy the zset, but doing a filter over this list for later doing other operations like filtering,ordening and paging.

Any other idea?

Josiah Carlson

unread,
Jan 13, 2014, 11:10:38 AM1/13/14
to redi...@googlegroups.com
Two questions:
1. How large is your zset?
2. Are you working in Python?

If #1 is something under say 100k items, and if your answer to #2 is 'yes', you may want to take a look at https://github.com/josiahcarlson/rom . It offers multi-attribute/column filtering, ordering, and pagination on data. In my opinion, it might be the best Python object-Redis mapper available (it certainly offers more sorting/filtering options than basically anything else in any other language).

Regards,
 - Josiah



Message has been deleted

Alan Perd

unread,
Jan 16, 2014, 5:29:52 AM1/16/14
to redi...@googlegroups.com

Hi Joshia,

 Thank you for your interest, yes, we’re using python. I’m implementing a grid showing data & status of a list of elements (probably ranging from 50K to 100k elements). I need to support filtering, pagination and ordering.

 So my idea is:

 ·         Have hashes with data for each element. Use id of element in the rest of sets, zsets.

·         For Filtering, depending on column type:

o   For integer fields (up to 15 digits): zset, score calculated using 4 bits per digit. This is similar to the “autocomplete” schema because I need to support “starts with”, “ends with” filtering.

o   For string fields: zset, score calculated reducing the number of valid characters, also for “starts with”, “ends with” filtering.

o   For enumerated value fields (for instance “status”:{ok,error,unknown}): sets with elements in each state.

o   After filtering each column some zunion/zinterstore magic to obtain a temporary zset with the result.

·         For ordering, paging:

o   A sort operation on the resulting zset using offset, count.

·         Reuse that temporary zset till the user changes filters or session timeouts.

 I’ve been testing your ROM and it looks quite good,  cleaner and (from my point of view) better than other similar libraries in python. It could be extended to support “autocomplete” filtering and using different score functions depending on data type (integers, strings, Unicode strings) and I think it would be a nice functionality to have.

 But I need to support also some wildcard search (something like al?n*) and that’s why I was trying to use zscan in lua. In the end I’ve made a test using a lua script to perform the zscan loop and call zadd for the results. The performance over 100k elements varies from 150ms to 2.5s depending on the quality of the filter (1* or *1* both return 90K elements, while 123* returns 72 and 12?3* even less). So, if the filter is “starts/ends with” I’ll use zset filtering and if it’s “*1*” or using ? wildcard I plan to use zscan.

 Your opinion about the filtering (and the whole scheme) is appreciated.

 Best regards,

Alan.

Josiah Carlson

unread,
Jan 16, 2014, 1:34:52 PM1/16/14
to redi...@googlegroups.com
On Thu, Jan 16, 2014 at 2:18 AM, Alan Perd <alanp...@gmail.com> wrote:

Hi Joshia,

 Thank you for your interest, yes, we’re using python. I’m implementing a grid showing data & status of a list of elements (probably ranging from 50K to 100k elements). I need to support filtering, pagination and ordering.

 So my idea is:

 ·         Have hashes with data for each element. Use id of element in the rest of sets, zsets.

·         For Filtering, depending on column type:

o   For integer fields (up to 15 digits): zset, score calculated using 4 bits per digit. This is similar to the “autocomplete” schema because I need to support “starts with”, “ends with” filtering.

Why not just use the integer itself? Do you need any integers greater than 2**52?

If so, I might suggest this bit of code to help: https://gist.github.com/josiahcarlson/8459874

o   For string fields: zset, score calculated reducing the number of valid characters, also for “starts with”, “ends with” filtering.

o   For enumerated value fields (for instance “status”:{ok,error,unknown}): sets with elements in each state.

o   After filtering each column some zunion/zinterstore magic to obtain a temporary zset with the result.

·         For ordering, paging:

o   A sort operation on the resulting zset using offset, count.

·         Reuse that temporary zset till the user changes filters or session timeouts.

I’ve been testing your ROM and it looks quite good,  cleaner and (from my point of view) better than other similar libraries in python. It could be extended to support “autocomplete” filtering and using different score functions depending on data type (integers, strings, Unicode strings) and I think it would be a nice functionality to have.


Autocomplete has been on my list of things to do for several months now. I've held off because I don't like ZSET score-based autocompletes due to the prefix limit (10 upper/lower + decimal, or 12 lowercase, or 6 full unicode characters), and I don't like requiring Lua scripting to use the library. But I suppose with Redis 2.6 having been out for 15 months now, and Redis 2.8 even 2 months old, I should get over it and just add the code from the book (modified to participate in a query chain, of course).
 

But I need to support also some wildcard search (something like al?n*) and that’s why I was trying to use zscan in lua. In the end I’ve made a test using a lua script to perform the zscan loop and call zadd for the results.


I'm going to guess you mean that you used ZRANGE and not ZSCAN.
 

The performance over 100k elements varies from 150ms to 2.5s depending on the quality of the filter (1* or *1* both return 90K elements, while 123* returns 72 and 12?3* even less). So, if the filter is “starts/ends with” I’ll use zset filtering and if it’s “*1*” or using ? wildcard I plan to use zscan.


That's not a bad plan. Though I will admit, those wildcard queries would be horribly nasty without using Lua's built-in regular-expression pattern matching with a little pattern pre-processing.

Your opinion about the filtering (and the whole scheme) is appreciated.


I think you've got a pretty good plan. If I can find time some time in the next week or so, I'll squeeze autocomplete on prefix/suffix and even the pattern matching into rom. I know that wasn't your intent, but those are useful tools to have available.

 - Josiah
 

Best regards,

Alan.



On Monday, January 13, 2014 5:10:38 PM UTC+1, Josiah Carlson wrote:

Alan Perd

unread,
Jan 17, 2014, 4:50:45 AM1/17/14
to redi...@googlegroups.com

Hi Josiah,


On Thursday, January 16, 2014 7:34:52 PM UTC+1, Josiah Carlson wrote:

On Thu, Jan 16, 2014 at 2:18 AM, Alan Perd <alanp...@gmail.com> wrote:


Hi Joshia,

 Thank you for your interest, yes, we’re using python. I’m implementing a grid showing data & status of a list of elements (probably ranging from 50K to 100k elements). I need to support filtering, pagination and ordering.

 So my idea is:

 ·         Have hashes with data for each element. Use id of element in the rest of sets, zsets.

·         For Filtering, depending on column type:

o   For integer fields (up to 15 digits): zset, score calculated using 4 bits per digit. This is similar to the “autocomplete” schema because I need to support “starts with”, “ends with” filtering.

Why not just use the integer itself? Do you need any integers greater than 2**52?
 

Because I need “starts/ends with” filtering, so I need lexicographical order. That’s why I manage integers like strings with a reduced character set. I don’t know if there is a better approach for this.


If so, I might suggest this bit of code to help: https://gist.github.com/josiahcarlson/8459874

Thank you! I was not aware of this conversion, I’m sure it’s going to be useful in other cases.


o   For string fields: zset, score calculated reducing the number of valid characters, also for “starts with”, “ends with” filtering.

o   For enumerated value fields (for instance “status”:{ok,error,unknown}): sets with elements in each state.

o   After filtering each column some zunion/zinterstore magic to obtain a temporary zset with the result.

·         For ordering, paging:

o   A sort operation on the resulting zset using offset, count.

·         Reuse that temporary zset till the user changes filters or session timeouts.

I’ve been testing your ROM and it looks quite good,  cleaner and (from my point of view) better than other similar libraries in python. It could be extended to support “autocomplete” filtering and using different score functions depending on data type (integers, strings, Unicode strings) and I think it would be a nice functionality to have.


Autocomplete has been on my list of things to do for several months now. I've held off because I don't like ZSET score-based autocompletes due to the prefix limit (10 upper/lower + decimal, or 12 lowercase, or 6 full unicode characters), and I don't like requiring Lua scripting to use the library. But I suppose with Redis 2.6 having been out for 15 months now, and Redis 2.8 even 2 months old, I should get over it and just add the code from the book (modified to participate in a query chain, of course).
 
 

Good news!

But I need to support also some wildcard search (something like al?n*) and that’s why I was trying to use zscan in lua. In the end I’ve made a test using a lua script to perform the zscan loop and call zadd for the results.


I'm going to guess you mean that you used ZRANGE and not ZSCAN.
 
 

No, I meant one lua script for zscan loop, returning data to client, sending data to a lua script issuing the zadd commands. I’ve just realized that performing a loop on zscan in a lua script defeats the whole purpose of the iterative/cursor approach of zscan as I’m going to block redis during that loop like ‘keys’ does.


The performance over 100k elements varies from 150ms to 2.5s depending on the quality of the filter (1* or *1* both return 90K elements, while 123* returns 72 and 12?3* even less). So, if the filter is “starts/ends with” I’ll use zset filtering and if it’s “*1*” or using ? wildcard I plan to use zscan.


That's not a bad plan. Though I will admit, those wildcard queries would be horribly nasty without using Lua's built-in regular-expression pattern matching with a little pattern pre-processing.

 

I don’t know exactly what you mean with your last statement regarding regular-expression and pattern pre-processing but it brings me another idea: for searching ‘al?n*’ I could use a lua script filtering with zrange or zrangebyscore on ‘al’ and parse the results using lua regular-expressions searching for ‘?n’ then returning data, worst case being patterns like ‘?lan*’ or ‘*lan*’ where it’s like a “full table scan” in SQL world.


 

Your opinion about the filtering (and the whole scheme) is appreciated.


I think you've got a pretty good plan. If I can find time some time in the next week or so, I'll squeeze autocomplete on prefix/suffix and even the pattern matching into rom. I know that wasn't your intent, but those are useful tools to have available.

 

It’ll be really useful not only to use it directly but also to learn from your code. Thank you for your help.

 

Josiah Carlson

unread,
Jan 18, 2014, 1:03:25 PM1/18/14
to redi...@googlegroups.com

On Fri, Jan 17, 2014 at 1:50 AM, Alan Perd <alanp...@gmail.com> wrote:

Hi Josiah,


Because I need “starts/ends with” filtering, so I need lexicographical order. That’s why I manage integers like strings with a reduced character set. I don’t know if there is a better approach for this.


That makes a lot of sense. Going the totally optimal bit-packing way can let you use 18 digits instead of 15 digits using ZSETs, but it's more code and not nearly as easily understood.
 

If so, I might suggest this bit of code to help: https://gist.github.com/josiahcarlson/8459874

Thank you! I was not aware of this conversion, I’m sure it’s going to be useful in other cases.


You are welcome, I just updated it to support negative numbers, because there is no reason not to :P
 



o   For string fields: zset, score calculated reducing the number of valid characters, also for “starts with”, “ends with” filtering.

o   For enumerated value fields (for instance “status”:{ok,error,unknown}): sets with elements in each state.

o   After filtering each column some zunion/zinterstore magic to obtain a temporary zset with the result.

·         For ordering, paging:

o   A sort operation on the resulting zset using offset, count.

·         Reuse that temporary zset till the user changes filters or session timeouts.

I’ve been testing your ROM and it looks quite good,  cleaner and (from my point of view) better than other similar libraries in python. It could be extended to support “autocomplete” filtering and using different score functions depending on data type (integers, strings, Unicode strings) and I think it would be a nice functionality to have.


Autocomplete has been on my list of things to do for several months now. I've held off because I don't like ZSET score-based autocompletes due to the prefix limit (10 upper/lower + decimal, or 12 lowercase, or 6 full unicode characters), and I don't like requiring Lua scripting to use the library. But I suppose with Redis 2.6 having been out for 15 months now, and Redis 2.8 even 2 months old, I should get over it and just add the code from the book (modified to participate in a query chain, of course).
 
 

Good news!

But I need to support also some wildcard search (something like al?n*) and that’s why I was trying to use zscan in lua. In the end I’ve made a test using a lua script to perform the zscan loop and call zadd for the results.


I'm going to guess you mean that you used ZRANGE and not ZSCAN.
 
 

No, I meant one lua script for zscan loop, returning data to client, sending data to a lua script issuing the zadd commands. I’ve just realized that performing a loop on zscan in a lua script defeats the whole purpose of the iterative/cursor approach of zscan as I’m going to block redis during that loop like ‘keys’ does.


Indeed!

But if you are very careful, you can use ZRANGE, ZRANGEBYSCORE, and ZRANK to do incremental scans of a ZSET using Lua (where you make multiple calls to Lua to complete a scan). That said, a non-incremental scan using ZRANGE is a lot easier:

local skey = KEYS[1]
local rkey = KEYS[2]
local pattern = ARGV[1]

for start = 0, redis.call('zcard', skey), 100) do
    local block = redis.call('zrange', skey, start, start+100, 'withscores')
    for i = 1,#block,2 do
        if string.find(block[i], pattern) then
            redis.call('zadd', rkey, block[i+1], block[i])
        end
    end
end

return results

... If you pass a proper 'pattern', the above function can perform prefix, suffix, and/or regular-expression pattern matching on the members of a ZSET. There are optimizations that can be done on prefix, suffix, and pattern matching in some cases, but this will work universally.


The performance over 100k elements varies from 150ms to 2.5s depending on the quality of the filter (1* or *1* both return 90K elements, while 123* returns 72 and 12?3* even less). So, if the filter is “starts/ends with” I’ll use zset filtering and if it’s “*1*” or using ? wildcard I plan to use zscan.


That's not a bad plan. Though I will admit, those wildcard queries would be horribly nasty without using Lua's built-in regular-expression pattern matching with a little pattern pre-processing.

 

I don’t know exactly what you mean with your last statement regarding regular-expression and pattern pre-processing but it brings me another idea: for searching ‘al?n*’ I could use a lua script filtering with zrange or zrangebyscore on ‘al’ and parse the results using lua regular-expressions searching for ‘?n’ then returning data, worst case being patterns like ‘?lan*’ or ‘*lan*’ where it’s like a “full table scan” in SQL world.


That's exactly what my implications were, and that's what I'm building for rom if I have the time. :)

Your opinion about the filtering (and the whole scheme) is appreciated.


I think you've got a pretty good plan. If I can find time some time in the next week or so, I'll squeeze autocomplete on prefix/suffix and even the pattern matching into rom. I know that wasn't your intent, but those are useful tools to have available.
 

It’ll be really useful not only to use it directly but also to learn from your code. Thank you for your help.


I'll do my best to squeeze it in. :)

 - Josiah

Josiah Carlson

unread,
Jan 19, 2014, 9:19:33 PM1/19/14
to redi...@googlegroups.com
Between yesterday and today, I managed to add support for prefix, suffix, and pattern matching in rom. I also added support for getting the key where the results of a query are stored for user-side pagination, etc.

I'm going to let it sit on my machine for the rest of the day to see if I'm able to remember anything that I might have forgotten. Assuming everything is good, I'll push it to the repository and the Python package index tomorrow morning Pacific time (UTC - 8).

 - Josiah

Alan Perd

unread,
Jan 23, 2014, 4:48:48 PM1/23/14
to redi...@googlegroups.com

Hello Josiah,

I have been testing your ROM and I am delighted with it, It is just what I was looking for.

In the time test results I have obtained the following information:

-- Populate time: Records: 100000  Time: 77.495000124

 

Filters

 

resutl: (1*)   --> records:9294 -- Time -> 0.0650000572205

resutl (*1)   --> records:9092 -- Time -> 0.066999912262

resutl (*1*)   --> records:60306 -- Time -> 0.538000106812

resutl (1?)   --> records:8163 -- Time -> 0.0599999427795

resutl (?1*)   --> records:16411 -- Time -> 0.227999925613

resutl (*1?2*)   --> records:10508 -- Time -> 0.202000141144

resutl (123*)   --> records:23 -- Time -> 0.0019998550415

resutl (*123)   --> records:24 -- Time -> 0.00300002098083


The results are awesome, they are even better than the ones I got with the scan commands. I have also tested the result cache and I found it very useful. 

However I think the ROM could be even better if the sorted operation could be performance using numbers in addition to the sorted string which is implemented right now.

 I mean, If I have to sort  numbers , I wold like to get the numeric sorted result, but I can not perform the operations (startswith,endwiths ... ) with integer types.

Besides, I have also missed the possibility of doing filters with "OR" logic, because right now I can only use "AND" logic on filters.

I am considering to use your ROM implementation and I think it would be very convenient to add these improvements.

I hope this comments are usefull for your future developments.

Thank you very much Josiah, you have helped me a lot. 

Regards.
Alan Perd.

By the way I gonna buy your book. :)

Josiah Carlson

unread,
Jan 23, 2014, 5:43:38 PM1/23/14
to redi...@googlegroups.com
Replies inline.

On Thu, Jan 23, 2014 at 1:48 PM, Alan Perd <alanp...@gmail.com> wrote:

Hello Josiah,

I have been testing your ROM and I am delighted with it, It is just what I was looking for.

You've got a feature request, so it's not "just" what you are looking for ;) But I'm glad you like it. :)

In the time test results I have obtained the following information:

-- Populate time: Records: 100000  Time: 77.495000124

 

Filters

 

resutl: (1*)   --> records:9294 -- Time -> 0.0650000572205

resutl (*1)   --> records:9092 -- Time -> 0.066999912262

resutl (*1*)   --> records:60306 -- Time -> 0.538000106812

resutl (1?)   --> records:8163 -- Time -> 0.0599999427795

resutl (?1*)   --> records:16411 -- Time -> 0.227999925613

resutl (*1?2*)   --> records:10508 -- Time -> 0.202000141144

resutl (123*)   --> records:23 -- Time -> 0.0019998550415

resutl (*123)   --> records:24 -- Time -> 0.00300002098083


The results are awesome, they are even better than the ones I got with the scan commands. I have also tested the result cache and I found it very useful. 

However I think the ROM could be even better if the sorted operation could be performance using numbers in addition to the sorted string which is implemented right now.

 I mean, If I have to sort  numbers , I wold like to get the numeric sorted result, but I can not perform the operations (startswith,endwiths ... ) with integer types.

Part of the reason was a matter of internal API. Thinking about it a bit more, I do have a method to allow for the option of numeric + prefix/suffix queries, but I may not have time before going on vacation. For the time being, you can duplicate your data into two columns to get the sorting and wildcard/prefix/suffix matching at the same time.

Besides, I have also missed the possibility of doing filters with "OR" logic, because right now I can only use "AND" logic on filters.

That's a good point, a glaring omission. When I get the chance, I'll add this along with the numeric + prefix/suffix stuff.

I am considering to use your ROM implementation and I think it would be very convenient to add these improvements.

I hope this comments are usefull for your future developments.

Thank you very much Josiah, you have helped me a lot.

You are quite welcome. I'd originally only planned on adding prefix matching for autocomplete, but your need for prefix, suffix, and pattern matching helped make me think harder about how to do it better. So thank you for having an interesting problem to solve :)
 
Regards.
Alan Perd.

By the way I gonna buy your book. :)

I'll take every sale :)
Message has been deleted
Message has been deleted

Alan Perd

unread,
Jan 24, 2014, 5:10:27 AM1/24/14
to redi...@googlegroups.com

Hello Josiah,
 
I’m sorry, maybe I didn’t explain myself properly. I meant that the "Startswidth" and "EndsWith" implementations satisfies the requirements my development needed and that’s great.
 
By now I’m going to follow your advice and I’m going to create another numeric  field to do the ordenation once the filter results are obtained.
 
On the other hand, what I meant by the logic expression “OR”, is that I can nest different types of queries, for example:
 
RomTestPSP.query.endswith(col='123').startswith(col='5').cached_result(30)


But let’s suppose that what I really want is to get the values of the columns which start with 123 or which end with 5. Is it possible to do this kind of filters with the ROM?
 
Thank you very much in advance.
I wish you have a nice vacations.

Kind regards.
 
Alan.
 
The book is comming .. :)

Josiah Carlson

unread,
Jan 24, 2014, 12:10:56 PM1/24/14
to redi...@googlegroups.com
Replies inline.

On Fri, Jan 24, 2014 at 2:08 AM, Alan Perd <alanp...@gmail.com> wrote:

Hello Josiah,
 
I’m sorry, maybe I didn’t explain myself properly. I meant that the "Startswidth" and "EndsWith" implementations satisfies the requirements my development needed and that’s great.
 
By now I’m going to follow your advice and I’m going to create another numeric  field to do the ordenation once the filter results are obtained.
 
On the other hand, what I meant by the logic expression “OR”, is that I can nest different types of queries, for example:
 
RomTestPSP.query.endswith(col='123').startswith(col='5').cached_result(30)

I get what you want, and I'm going to build it for you. Many of the other types of queries have similar functionality: RomTestPSP.query.filter(scol=['foo', 'bar']) performs a query where the 'scol' has values 'foo' or 'bar'. I will be adding a SQLAlchemy-style query mechanism so that your desired query becomes:

RomTestPSP.query.filter(rom.or_(RomTestPSP.col.endswith('123'), RomTestPSP.col.startswith('5'))).cached_result(30)

How does that sound? All you've got to do is to wait for me to build it :P

 - Josiah


But let’s suppose that what I really want is to get the values of the columns which start with 123 or which end with 5. Is it possible to do this kind of filters with the ROM?
 
Thank you very much in advance.
I wish you have a nice vacations.

Kind regards.
 
Alan.
 
The book is comming .. :)
On Thursday, January 9, 2014 5:37:15 PM UTC+1, Alan Perd wrote:
Message has been deleted

Alan Perd

unread,
Feb 3, 2014, 9:08:19 AM2/3/14
to redi...@googlegroups.com
Thank you Josiah,

It sounds very well.

Regards,
Alan.

Alan Perd

unread,
May 22, 2014, 5:02:56 AM5/22/14
to redi...@googlegroups.com
Hi Josiah,
I have a case of practice making use ROM that I don't know very well as solve,I need to perform filters "different from"  like this ".filter(field!=value)",  but I don't know how to make these filters.
Could you give me a hint?


Best Regards,
Alan.

Josiah Carlson

unread,
May 23, 2014, 11:11:55 AM5/23/14
to redi...@googlegroups.com
Alan,

Bad news, good news. Bad news first: I did not build that filtering mechanism into rom, but I will add it to my list of feature requests. The good news is that if the value you are searching for is indexed numerically, you can either calculate the query with two queries, or you can perform a single query, get the cached result key, pull those non-matching entries out via ZREMRANGEBYSCORE, then using ZRANGE + Model.get(items) get the results.

I can describe one of the two numeric-based queries further if you need something in the short-term.

 - Josiah



For more options, visit https://groups.google.com/d/optout.

Reply all
Reply to author
Forward
0 new messages