lua: conversion of KEYS to args in redis.call()
Raghava Mutharaju
Javier Guerra Giraldez
<m.vijay...@gmail.com> wrote:
> redis.call('ZINTERSTORE', 'dummyKey123', numKeys, KEYS[1], KEYS[2], KEYS[3])
redis.call('ZINTERSTORE', 'dummyKey123', numKeys, unpack(KEYS))
--
Javier
Raghava Mutharaju
--
Javier
--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To post to this group, send email to redi...@googlegroups.com.
To unsubscribe from this group, send email to redis-db+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.
catwell
> redis.call('ZINTERSTORE', 'dummyKey123', numKeys, unpack(KEYS))
This is a good solution but be careful not to overflow the stack in
unpack. Typically this will work with 1000 keys but not with 10000.
Kristoffer Berggren
Since I am new on Redis:
How do I execute lua scripts? Can it be sent to server over client connection?
Regards,
Kristoffer
Alexander Gladysh
Not in unpack, but in redis.call().
Not stack, but the limit on number of function arguments.
This is a problem in Redis API, BTW.
Maybe allow redis.call to optionally accept table with arguments
(keeping the current API as well, of course)?
I.e.
table.insert(KEYS, 1, "ZINTERSTORE")
table.insert(KEYS, 1, "dummyKey123")
table.insert(KEYS, 1, numKeys)
redis.call(KEYS)
Inserting at the beginning of the table is not very efficient, though,
so, maybe allow to mix styles?
redis.call("ZINTERSTORE", "dummyKey123", numKeys, KEYS)
Alexander.
catwell
> > On Feb 29, 3:50 am, Javier Guerra Giraldez <jav...@guerrag.com> wrote:
>
> >> redis.call('ZINTERSTORE', 'dummyKey123', numKeys, unpack(KEYS))
>
> > This is a good solution but be careful not to overflow the stack in
> > unpack. Typically this will work with 1000 keys but not with 10000.
>
> Not in unpack, but in redis.call().
>
> Not stack, but the limit on number of function arguments.
> a= {}; for i=1,10000 do a[i]=1 end; unpack(a)
stdin:1: too many results to unpack
stack traceback:
[C]: in function 'unpack'
stdin:1: in main chunk
[C]: ?
And afaik the limit is LUAI_MAXCSTACK.
> This is a problem in Redis API, BTW.
>
> Maybe allow redis.call to optionally accept table with arguments
> (keeping the current API as well, of course)?
>
> I.e.
>
> table.insert(KEYS, 1, "ZINTERSTORE")
> table.insert(KEYS, 1, "dummyKey123")
> table.insert(KEYS, 1, numKeys)
> redis.call(KEYS)
>
> Inserting at the beginning of the table is not very efficient, though,
> so, maybe allow to mix styles?
>
> redis.call("ZINTERSTORE", "dummyKey123", numKeys, KEYS)
the Ruby Redis client by the way (although the limit was higher):
https://github.com/ezmobius/redis-rb/issues/103
Alexander Gladysh
> On Feb 29, 9:58 am, Alexander Gladysh <aglad...@gmail.com> wrote:
>> On Wed, Feb 29, 2012 at 12:36, catwell <catwell-goo...@catwell.info> wrote:
>> > On Feb 29, 3:50 am, Javier Guerra Giraldez <jav...@guerrag.com> wrote:
>>
>> >> redis.call('ZINTERSTORE', 'dummyKey123', numKeys, unpack(KEYS))
>>
>> > This is a good solution but be careful not to overflow the stack in
>> > unpack. Typically this will work with 1000 keys but not with 10000.
>>
>> Not in unpack, but in redis.call().
>>
>> Not stack, but the limit on number of function arguments.
>
> Unpack will break first:
>
> > a= {}; for i=1,10000 do a[i]=1 end; unpack(a)
> stdin:1: too many results to unpack
> stack traceback:
> [C]: in function 'unpack'
> stdin:1: in main chunk
> [C]: ?
>
> And afaik the limit is LUAI_MAXCSTACK.
Number of return values, right.
Sorry, not fully awake yet :-)
Alexander.
Raghava Mutharaju
Salvatore Sanfilippo
> And afaik the limit is LUAI_MAXCSTACK.
Hey, we are not running into Raspberry Pi (for now), so we can alter
this default value to an higher value. Makes sense?
Salvatore
--
Salvatore 'antirez' Sanfilippo
open source developer - VMware
http://invece.org
"We are what we repeatedly do. Excellence, therefore, is not an act,
but a habit." -- Aristotele
Alexander Gladysh
> On Wed, Feb 29, 2012 at 10:24 AM, catwell <catwell...@catwell.info> wrote:
>
>> And afaik the limit is LUAI_MAXCSTACK.
>
> Hey, we are not running into Raspberry Pi (for now), so we can alter
> this default value to an higher value. Makes sense?
No. You will hit the limit anyway eventually. (And the default should
be large enough, I think.)
The point is to remove the limit (or rather provide a limitless, if
less convenient option :-) ) — and this can only be done by offering a
different API.
Alexander.
catwell
> Hey, we are not running into Raspberry Pi (for now)
> so we can alter this default value to an higher value. Makes sense?
Otherwise I would just use several calls to redis.call() in the script.
Salvatore Sanfilippo
> No. You will hit the limit anyway eventually. (And the default should
> be large enough, I think.)
>
> The point is to remove the limit (or rather provide a limitless, if
> less convenient option :-) ) — and this can only be done by offering a
> different API.
Well that's entirely possible because:
redis 127.0.0.1:6379> EVAL 'return redis.call({1,2,3})' 0
(error) Lua redis() command arguments must be strings or integers
So we can introduce a new calling convention so that tables are
automatically expanded into arguments.
This way calling: (1,2,3) or (1,{2,3}) is he same. The most
straightforward use would be to just a single call, but also this
allows to just write: redis.call("MGET",mykeys).
Sounds good?
Pieter Noordhuis
from taking nested arguments in the future. Why not go the route of
EITHER taking in all strings/integers, OR taking in a single table?
Cheers,
Pieter
Salvatore Sanfilippo
> While it solves this particular issue, it does prevent the Redis API
> from taking nested arguments in the future. Why not go the route of
> EITHER taking in all strings/integers, OR taking in a single table?
Good point, thanks you Pieter... maybe it's time to use our secret
weapon of waiting a few months before proceeding ;)
Cheers,
Salvatore
Jak Sprats
parsing nested tables opens up a whole new realm of possibilities, and
is pretty simple w/ Lua's limited number of data-types.
it may have a ripple effect out to clients that can't handle the
nested data ... dont know.
still having nested table support opens up so much, I personally think
its worth doing
- jak
On Feb 29, 6:22 pm, Salvatore Sanfilippo <anti...@gmail.com> wrote:
> > While it solves this particular issue, it does prevent the Redis API
> > from taking nested arguments in the future. Why not go the route of
> > EITHER taking in all strings/integers, OR taking in a single table?
>
> Good point, thanks you Pieter... maybe it's time to use our secret
> weapon of waiting a few months before proceeding ;)
>
> Cheers,
> Salvatore
>
>
>
>
>
>
>
>
>
>
>
> > Cheers,
> > Pieter
>
> > --
> > You received this message because you are subscribed to the Google Groups "Redis DB" group.
> > To post to this group, send email to redi...@googlegroups.com.
> > To unsubscribe from this group, send email to redis-db+u...@googlegroups.com.
Alexander Gladysh
> On Wed, Feb 29, 2012 at 7:23 PM, Pieter Noordhuis <pcnoo...@gmail.com> wrote:
>> While it solves this particular issue, it does prevent the Redis API
>> from taking nested arguments in the future. Why not go the route of
>> EITHER taking in all strings/integers, OR taking in a single table?
Probably this is the best way. BTW, the problem with list head
insertion overhead may be alleviated if there would be a way to append
result of command to the existing table.
Something like (pseudocode):
local command = { 'ZINTERSTORE', 'dummyKey123', 0 }
local num_keys = redis.call_append_result(command)
command[3] = num_keys
redis.call(command)
Not so pretty, of course...
> Good point, thanks you Pieter... maybe it's time to use our secret
> weapon of waiting a few months before proceeding ;)
May be a good idea.
Alexander.
Alexander Gladysh
> On Thu, Mar 1, 2012 at 01:22, Salvatore Sanfilippo <ant...@gmail.com> wrote:
>> On Wed, Feb 29, 2012 at 7:23 PM, Pieter Noordhuis <pcnoo...@gmail.com> wrote:
>>> While it solves this particular issue, it does prevent the Redis API
>>> from taking nested arguments in the future. Why not go the route of
>>> EITHER taking in all strings/integers, OR taking in a single table?
>
> Probably this is the best way. BTW, the problem with list head
> insertion overhead may be alleviated if there would be a way to append
> result of command to the existing table.
>
> Something like (pseudocode):
>
> local command = { 'ZINTERSTORE', 'dummyKey123', 0 }
> local num_keys = redis.call_append_result(command)
Above should be:
local num_keys = redis.call_append_result(command, 'KEYS', '*')
Alexander Schepanovski
Hi, Salvatore.
Looks like you've been waiting enough :). I ran into this issue for real here - https://github.com/Suor/django-cacheops/blob/fc13ce96d8a198090238d75f4800042e8c72f703/cacheops/lua/invalidate.lua#L29 - trying to delete to many keys at a time.
I am going to use this workaround now:
local call_in_chunks = function (command, args)
local step = 100
for i = 1, #args, step do
redis.call(command, unpack(args, i, math.min(i + step - 1, #args)))
end
end
call_in_chunks('del', cache_keys)
However, it would be much nicer just write
redis.call('del', cache_keys)
Regards, Alexander.
четверг, 1 марта 2012 г., 5:22:54 UTC+8 пользователь Salvatore Sanfilippo написал: