Finally we have some doc on the current implementation of the
scripting feature in Redis:
Please help me improving this documentation.
Also I would like to use this thread to discuss the Redis scripting
feature semantics and implementation details in client libraries.
Yesterday I saw a few comments on twitter and tried to reply, but now
with the actual document published and more than 140 characters I bet
it's a better environment for discussions ;)
Ciao,
Salvatore
--
Salvatore 'antirez' Sanfilippo
open source developer - VMware
http://invece.org
"We are what we repeatedly do. Excellence, therefore, is not an act,
but a habit." -- Aristotele
--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To post to this group, send email to redi...@googlegroups.com.
To unsubscribe from this group, send email to redis-db+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.
Good idea, sorry, fixing.
> I still think that defining commands with a given name instead of
> using SHAs can be interesting for some use cases, and that SCRIPT LOAD
> should work that way. Why?
>
> * You can still compute the SHA on the client side and obtain the same
> result if you want.
>
> * You can still version the command manually if needed (eg. MYCMD ->
> MYCMD2...). OTOH you can also *not* do it and hot-patch a script if
> you find a bug / need to fix its behavior.
>
> * This makes it possible to separate applications that load scripts
> from applications that use scripts. For instance there could be a
> script in redis-extend that loads a command into a Redis server.
>
> * I don't think there's a semantic problem: if you use scripts this
> way you will document them exactly in the same way that you would
> document Redis commands.
>
> * I don't think this makes deployment harder. I think it makes it
> simpler because it separates it from the application. If you can
> deploy a Redis cluster then you can probably deploys scripts too. OTOH
> if you're like me you probably have more applicative servers than
> Redis servers, and you would currently have to modify them all if you
> wanted to update a script.
>
> The difference between both approaches lies IMO in whether you
> consider scripts as one-shot tools or real new commands, whether
> they're the job of dev or ops.
Hi, thanks for the feedback!
I think there is some fundamental error in your reasoning, as already
stated on twitter ;)
And this error is in mixing the way applications will end scripts to
the server, and the semantics and API from the point of view of the
client.
From the point of view of a client lib there are many ways to expose scripting.
One client lib will just expose EVAL and EVALSHA directly.
Another client lib instead will use an API like the following, perhaps:
redis.register_script("shuffte-list", SHUFFLE_LIST_LUA_CODE,1,0); //
Note: final 1,0 means for instance: expect 1 key, 0 arguments.
redis.shuffle_list("mylist");
This is a way, others will just prefer a raw approach were EVAL is a
wrapper for sending EVALSHA, then retrying with EVAL if NOSCRIPT is
returned. And so forth.
So clients can pick different designs about it, completely legit.
Then there is the other side. Sending scripts to the server. Whatever
you do on the client, why on the earth should you fight with the idea
that a Redis server needs to be instructed with commands after it is
restarted or upgraded? Anyway you don't want the idea of an external
program managing your script collection, or registering scripts server
side in a way that you can define/delete them.
This will just make your operations more complex, without a single gain.
SCRIPT LOAD is only a way to ensure a given script exists, and this is
useful for pipelining. You can also use it if your API does something
like redis.register_script() since it makes sense.
But when you call your command you should ALWAYS have a body that is
associated with a command, compute the SHA1 (you can of course cache
it in your Redis object) and send EVALSHA, and if it fails, redefine
it.
Stopping and Restarting Redis. Or even using a different instance
while the server is running (imagine Redis Cluster) should never pose
problems about what scripts are defined and what not.
So, every client that will not:
1) For normal requests: use EVALSHA and later EVAL if needed.
2) For pipelined requests: use SCRIPT EXISTS ..., and pipeline SCRIPT
LOAD commands before the actual pipeline, and send just EVALSHA in the
pipeline.
is from my point of view broken.
Note that the Redis script cache has a very interesting semantic of
not forgetting scripts, and SCRIPT FLUSH should only be called when a
Redis instance is emptied in a cluster environment to provide it to
another user, or in general when You Know What You Are Doing, so... if
you are taking a permanent connection to Redis in your client library
like redis-rb does, you can even remember that you did SCRIPT LOAD for
a given script already, so you can just go for EVALSHA in a pipeline
if all your scripts are already defined. Just: make sure to flush your
scripts client-side cache if you reconnect to Redis after a network
error.
Cheers,
This IMHO not only does not make sense as a script that does no longer
exists from the point of view of an application should never get its
SHA called, but is remarkably dangerous as it violates our script
cache contracts of not deleting scripts.
Salvatore
> --
> You received this message because you are subscribed to the Google Groups "Redis DB" group.
> To post to this group, send email to redi...@googlegroups.com.
> To unsubscribe from this group, send email to redis-db+u...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.
>
>
--
> --
> You received this message because you are subscribed to the Google Groups "Redis DB" group.
> To post to this group, send email to redi...@googlegroups.com.
> To unsubscribe from this group, send email to redis-db+u...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.
>
>
--
> I do want to be able to call scripts like normal Redis
> commands on the client side so I will probably write
> code that allows me to do that (using EVALSHA with a
> fallback to EVAL). This will make all the client apps
> require the source code of the scripts and ability to
> compute SHAs though.
As I said this is very legit, as it is an API stuff so arguments are
about aesthetic not functionality.
Even separating Lua and Ruby/Python/Perl/Whatever code this way what
is good is that Lua scripts are *still* part of the applications,
inside some scripts.rb file or alike, so that the full semantics of
the application is evincible reading the full source code.
Isn't inconsistent to use "ok" as field for all status?
Lua number -> Redis integer reply
Clarification on non integers numbers. By my tests math.floor (or
truncation) is being applied:
redis 127.0.0.1:6379> EVAL "return 1.9" 0
(integer) 1
Should add a note about multi level recursive multi-bulk being added?
Previous version of Redis clients may not support it
I don't know if it's a bug or not, but lua's tables not starting on 1
are not handled as I would expect it. E.g.:
redis 127.0.0.1:6379> EVAL "t = {}; t[1] = 3; return t;" 0
1) (integer) 3
redis 127.0.0.1:6379> EVAL "t = {}; t[2] = 3; return t;" 0
(empty list or set)
> Scripts also are subject to a maxium execution time of five seconds.
This default timeout is huge since a script should run usually in a sub
millisecond amount of time. The limit is mostly needed in order to avoid
problems when developing scripts that may loop forever for a programming
error.
What happens if the master executes a script and a slave hits timeout?
Cheers,
Seppo
hset redis-internal-reserved:: scriptSHA1 script
Regarding the "ok" field: there is no Redis protocol equivalent for a
table with such an entry. Only tables with numeric indexes are
converted to Redis multi bulks. The float maps to an integer reply; if
you want to return the string (just as double scores for sorted sets),
you need to call "tostring" on it to make Redis convert it to a bulk.
You are correct about deeper nested multi bulks: not all clients
support them. The Ruby client (redis-rb) and the C client (hiredis),
do.
Converting a Lua table to the Redis protocol is done by iterating
starting at field 1. Anything that is not defined in between is a mark
for Redis to stop. Lua tables are key/value maps, so iterating over
numeric keys in order can only be done by starting at 1.
Preventing timeout on slaves is necessary, thanks for mentioning.
Cheers,
Pieter
Indeed... good point, simply if an instance is a slave it will not
care about enforcing timeouts.
Thanks,
What do you think?
Cheers,
Pieter
This is definitely an issue... actually the timeout should only happen
when things are really... serious, that is, you have a bug in your
script that is going to loop forever (or a lot more than expected),
since even the 5 seconds default are an huge value.
For replication it is not a big issue, as we can just close the
replication link when the master script timeouts. This will force a
resync.
For AOF it is more complex than that... thinking about it... Thanks
for the good feedback.
Salvatore
Typos fixed.
I slept on this, and I think we are going to be something rather extreme.
Let alone that we end with problems in the AOF/replication link, but
the biggest problem is that the scripting contract is broken if we
abort the script half-way, since the application has no way to know
what was done and what was not: in short this breaks scripts
consistency.
So this is my idea: if a script that has done no writes at all so far
goes over the setup max time we abort it.
If a script that already sent even a single write command goes after
the max time, we need to call SHUTDOWN and terminate the server.
The above solution can appear bold at first, but it is the *only* way
to preserve consistency.
Also:
1) Scripts should never run usually for a long time. This timeout
should be set to two order of magnitudes bigger value compared to what
the max script execution time should be.
2) Scripts usually run a lot of time because of bugs, for debugging it
is not going to be a big problem that the server shuts down.
I really think we don't have other options but would love to get some
feedback on that.
Salvatore
Instead of replicating the script, how about just replicating the resulting actions of the script? That would simplify more than just this scenario.
-Fritzy
Hi Nathan,
unfortunately things are different. Even if Redis had no replication,
nor consistency at all, a script running, doing half the work it
should, and then terminating for timeout, will leave an inconsistent
data set.
So actually the problems with AOF and replication were trying to tell
us that our semantic of interrupting scripts was broken in a much more
fundamental way.
Salvatore
Hi Louis-Philippe, I think not shutting down is not an option if the
script already did even a single write, otherwise the data set is left
into an inconsistency state.
Salvatore
Every client to a Redis server has root-powers in Redis land.
Executing a FLUSHALL or a long-running script is all the same, and we
trust clients to "do the right thing". So, instead of terminating the
script, why not let it run forever when it needs to and thereby avoid
introducing consistency problems. Even with a default timeout that is
huge, CPUs will eventually get pegged, and perfectly fine scripts will
eventually hit that timeout. Having Redis shutdown in that case,
doesn't sound right to me. I think having a latency burst in such
cases is much better than shutting down the server altogether.
Again, we trust users to do the right thing, and users should know
that with scripting comes great responsibility ;-)
In short: I'm in favor of not timing out scripts.
Cheers,
Pieter
> In short: I'm in favor of not timing out scripts.
+1...
I want to just add this: the trap that used to terminate the script
instead of being removed at all can be turned into:
redisLog(REDIS_WARNING, "Hey... but we wanted to trust our users :(");
Makes sense?
On Wed, Oct 26, 2011 at 5:31 PM, Pieter Noordhuis <pcnoo...@gmail.com> wrote:
> Again, we trust users to do the right thing, and users should know
> that with scripting comes great responsibility ;-)
>
> In short: I'm in favor of not timing out scripts.
+1. I've been reading this thread and so far this one seems to be the
saner approach. SHUTDOWN is too aggressive, recover from something
like this without MVCC is next to impossible...
If a programmer is using Redis scripting in a way that it can block
the entire server for a long time, Redis should do what the programmer
asked him (her?) to do.
Bye,
--
Pedro Melo
@pedromelo
http://www.simplicidade.org/
http://about.me/melo
xmpp:me...@simplicidade.org
mailto:me...@simplicidade.org
"You just shoot yourself in the foot... with style!"
> Makes sense?
IMHO, best of the current scenarios/solutions.
>> In short: I'm in favor of not timing out scripts.
kind regards
---
Thomas FRITZ
web http://fritzthomas.com
twitter http://twitter.com/thomasf
2011/10/26 catwell <catwell...@catwell.info>:
On Thu, Oct 27, 2011 at 12:09 PM, Thomas Fritz <frit...@gmail.com> wrote:
> Maybe i am overseeing something, but why not encapsulate every script
> in a transaction, and if the script is timing out throws any error,
> redis performs a rollback?
There is no "rollback" in Redis. MVCC or something like that is not
part of the implementation.
---
Thomas FRITZ
web http://fritzthomas.com
twitter http://twitter.com/thomasf
2011/10/27 Pedro Melo <me...@simplicidade.org>:
If you lose the ability to read values from scripts most of the value
of scripting is lost.
When you are inside MULTI/EXEC you can't read stuff, you always get queued.
Salvatore
--
I think I finally found a solution that makes me completely happy. It
is already implemented by the last commit in the unstable branch.
This is how it works:
* You can set the max execution limit as always.
* When that limit is reached Redis logs a warning, about a slow script detected.
* Redis starts re-entering the event loop once that condition is
detected, to accept new clients queries.
* Those queries will always be handled returning a -SLOWSCRIPT error,
that notifies the clients why the server is busy and can't reply.
* However the SHUTDOWN command is accepted in this condition to allow
the sysadmin to shutdown the server in a safe way without consistency
concerns.
I'm very happy with this solution, I hope you like it as well.
Salvatore
What about it? It doesn't do rollbacks. You can't even find that word
on the page.
Its called a transaction because all commands are executed or none at
all, and with WATCH you can make sure that some precondition you based
your MULTI-EXEC'ed operations didn't change.
But it doesn't use speculative execution like a more traditional DBMS
that executes each command as it receives it, while keeping track of
undo information in a rollback segment somewhere. No. Redis accepts
all commands but it doesn't execute none of them until the final EXEC,
and then, atomically (as in without other clients operations in the
middle) it does all the ops that were sent after the MULTI.
Cheers,
Pieter
On Thursday, October 27, 2011 at 5:52 AM, Salvatore Sanfilippo wrote:
Hi again,I think I finally found a solution that makes me completely happy. Itis already implemented by the last commit in the unstable branch.This is how it works:* You can set the max execution limit as always.* When that limit is reached Redis logs a warning, about a slow script detected.* Redis starts re-entering the event loop once that condition isdetected, to accept new clients queries.* Those queries will always be handled returning a -SLOWSCRIPT error,that notifies the clients why the server is busy and can't reply.
* However the SHUTDOWN command is accepted in this condition to allowthe sysadmin to shutdown the server in a safe way without consistencyconcerns.
I'm very happy with this solution, I hope you like it as well.SalvatoreOn Wed, Oct 26, 2011 at 6:32 PM, Salvatore Sanfilippo <ant...@gmail.com> wrote:On Wed, Oct 26, 2011 at 6:31 PM, Pieter Noordhuis <pcnoo...@gmail.com> wrote:In short: I'm in favor of not timing out scripts.+1...I want to just add this: the trap that used to terminate the scriptinstead of being removed at all can be turned into:redisLog(REDIS_WARNING, "Hey... but we wanted to trust our users :(");Makes sense?--Salvatore 'antirez' Sanfilippoopen source developer - VMware"We are what we repeatedly do. Excellence, therefore, is not an act,but a habit." -- Aristotele--Salvatore 'antirez' Sanfilippoopen source developer - VMware"We are what we repeatedly do. Excellence, therefore, is not an act,but a habit." -- Aristotele
On Thu, Oct 27, 2011 at 6:05 PM, Pieter Noordhuis <pcnoo...@gmail.com> wrote:
> Exactly why I dislike the term "transaction" for MULTI/EXEC. The name
> "atomic execution group" would be a better fit..
+1.
On Thursday, October 27, 2011 at 5:52 AM, Salvatore Sanfilippo wrote:
Hi again,I think I finally found a solution that makes me completely happy. Itis already implemented by the last commit in the unstable branch.This is how it works:* You can set the max execution limit as always.* When that limit is reached Redis logs a warning, about a slow script detected.* Redis starts re-entering the event loop once that condition isdetected, to accept new clients queries.* Those queries will always be handled returning a -SLOWSCRIPT error,that notifies the clients why the server is busy and can't reply.Perhaps we should have a more generic -BUSY. Might be able to reuse this in other situations where the server gets slowed down.
* However the SHUTDOWN command is accepted in this condition to allowthe sysadmin to shutdown the server in a safe way without consistencyconcerns.I know it's a bit more work, but a SCRIPT KILL <some id> command would be nicer. Larger DBs can take minutes to save/load. Seems overkill to shutdown the entire system to get rid of one connection. I certainly don't have to shutdown PostgreSQL or MySQL to kill a poorly written stored proc or long running query.SCRIPT KILL would also make it easy for an external automated process to be able to respond to -SLOWSCRIPT errors so I don't have to wake up my sysadmin at 3:00AM :)
Hey Jak, I would simply return the error to the caller, as the -BUSY
(was SLOWSCRIPT) stuff is really a critical condition that should
never happen in production if the code is sane. The default is now
five seconds but I can make it even bigger, so that this will happen
only in pathological conditions.
Cheers,
Salvatore
> Perhaps we should have a more generic -BUSY. Might be able to reuse this in
> other situations where the server gets slowed down.
Good idea Andy, done.
> I know it's a bit more work, but a SCRIPT KILL <some id> command would be
I would be more than willing to do the work, the problem is that this
breaks script semantics, you can't know how much of its work the
script was able to do, so this will leave the database in a possibly
inconsistent state.
That's why we can't stop it, otherwise implementing script kill would
be trivial. Actually *it is* trivial since no ID at all is needed,
there can be only a single script at time.
Cheers,
Salvatore
After all there will be people using Redis "computationally" that will
run log scripts for a reason, they have just one client.
Or to do big schema migrations.
They'll be very well served with the current behavior since the script
will not terminate (if they don't send SHUTDOWN), but at the same time
we are serving all the other users with a buggy script well, since
they'll be both the error reported by clients to understand what is
happening, and can select if let the script run or if kill the server,
restart it, and fix the program.
Cheers,
Salvatore
On Thursday, October 27, 2011 at 12:00 PM, Salvatore Sanfilippo wrote:
On Thu, Oct 27, 2011 at 7:17 PM, Andy McCurdy <sed...@gmail.com> wrote:Perhaps we should have a more generic -BUSY. Might be able to reuse this inother situations where the server gets slowed down.Good idea Andy, done.I know it's a bit more work, but a SCRIPT KILL <some id> command would beI would be more than willing to do the work, the problem is that thisbreaks script semantics, you can't know how much of its work thescript was able to do, so this will leave the database in a possiblyinconsistent state.
That's why we can't stop it, otherwise implementing script kill wouldbe trivial. Actually *it is* trivial since no ID at all is needed,there can be only a single script at time.Cheers,Salvatore--Salvatore 'antirez' Sanfilippoopen source developer - VMware"We are what we repeatedly do. Excellence, therefore, is not an act,but a habit." -- Aristotele
> But SHUTDOWN does an implicit SAVE. Any changes the script has already made
> to would get saved to disk, right? It seems like SHUTDOWN would have the
> same problem that killing the script would, leaving things in an
> inconsistent state.
That's true, with RDB we are basically out of luck, but at least with
AOF with SHUTDOWN we are safe and the DB is left in a consistent way.
With RDB there is this problem, maybe when SHUTDOWN is sent in this
condition better avoiding to SAVE at all?
Everything like now, but:
1) SCRIPT KILL will work if a script did no write at all so far. This
is sensible as it saves the user from a restart of Redis if the issue
is just a read-only script trapped into an infinite loop.
2) SHUTDOWN when called in that context will not save the RDB at all.
Alternatively we can add an ABORT command just for that if it is more
sensible to make the user understand that this is not going to save
the RDB file.
3) Ability to call SAVE when the script is busy? The user may still
want a non consistent DB instead of loosing the altest queries.
Suggestions? Thanks.
Salvatore
> Ok that's my revisited proposal.
>
> Everything like now, but:
>
> 1) SCRIPT KILL will work if a script did no write at all so far. This
> is sensible as it saves the user from a restart of Redis if the issue
> is just a read-only script trapped into an infinite loop.
How about an optional parameter to kill the script even if writes have
been made? SCRIPT KILL FORCE maybe? That would leave the DB
inconsistent, but not shutdown the server.
> 2) SHUTDOWN when called in that context will not save the RDB at all.
> Alternatively we can add an ABORT command just for that if it is more
> sensible to make the user understand that this is not going to save
> the RDB file.
Maybe SHUTDOWN NOSAVE?
> 3) Ability to call SAVE when the script is busy? The user may still
> want a non consistent DB instead of loosing the altest queries.
>
Don't need this with script kill force.
So you are proposing something like that:
* We have a busy script that have never been doing writes yet: SCRIPT
KILL -> OK.
* We have a busy script that already performed at least one write
operaton -> SCRIPT KILL -> "ERR script already wrote so terminating it
may result in inconsistent database, use SCRIPT KILL FORCE if you know
what you are doing."
It makes sense but the problem I see here is about the "user
interface" we want to have with the programmer.
In this mailing list our context is biased, we know about what we are
talking about. Users are not all Redis experts, so they'll pick SCRIPT
KILL FORCE just since the previous command did not worked. They don't
exactly know what it will do, and why it may be a bad idea IMHO.
So I'm currently more biased towards not allowing the termination of
scripts that wrote on the DB.
>> 2) SHUTDOWN when called in that context will not save the RDB at all.
>> Alternatively we can add an ABORT command just for that if it is more
>> sensible to make the user understand that this is not going to save
>> the RDB file.
>
> Maybe SHUTDOWN NOSAVE?
We already saw a similar, reversed, feature request: SHUTDOWN
FORCESAVE or alike (there is an issue open about it), since SHUTDOWN
may not save if no save point is configured.
Still ABORT seems more clear to me, since currently the only
difference is that it does not save, but the general semantics is:
close the server without trying to be an hero saving or doing any
other cleanup. Just delete the pid and exit.
>> 3) Ability to call SAVE when the script is busy? The user may still
>> want a non consistent DB instead of loosing the altest queries.
>>
>
> Don't need this with script kill force.
Yep, I actually don't think it is a good idea at all to let the user
to recover from that condition performing a SAVE.
Either it cared about durability and has AOF enabled, or he was ok
with a N-minutes snapshot so there is the old file.
I don't want to allow in any case an inconsistent DB to survive.
Thanks for your help, very appreciated and useful in investigating
better this important aspect.
Salvatore
On Friday, October 28, 2011 at 8:56 AM, Salvatore Sanfilippo wrote:
On Fri, Oct 28, 2011 at 4:17 PM, Andy McCurdy <sed...@gmail.com> wrote:On Oct 28, 2011, at 5:38 AM, Salvatore Sanfilippo <ant...@gmail.com> wrote:Ok that's my revisited proposal.Everything like now, but:1) SCRIPT KILL will work if a script did no write at all so far. Thisis sensible as it saves the user from a restart of Redis if the issueis just a read-only script trapped into an infinite loop.How about an optional parameter to kill the script even if writes havebeen made? SCRIPT KILL FORCE maybe? That would leave the DBinconsistent, but not shutdown the server.So you are proposing something like that:* We have a busy script that have never been doing writes yet: SCRIPTKILL -> OK.* We have a busy script that already performed at least one writeoperaton -> SCRIPT KILL -> "ERR script already wrote so terminating itmay result in inconsistent database, use SCRIPT KILL FORCE if you knowwhat you are doing."It makes sense but the problem I see here is about the "userinterface" we want to have with the programmer.In this mailing list our context is biased, we know about what we aretalking about. Users are not all Redis experts, so they'll pick SCRIPTKILL FORCE just since the previous command did not worked. They don'texactly know what it will do, and why it may be a bad idea IMHO.
2) SHUTDOWN when called in that context will not save the RDB at all.Alternatively we can add an ABORT command just for that if it is moresensible to make the user understand that this is not going to savethe RDB file.Maybe SHUTDOWN NOSAVE?We already saw a similar, reversed, feature request: SHUTDOWNFORCESAVE or alike (there is an issue open about it), since SHUTDOWNmay not save if no save point is configured.Still ABORT seems more clear to me, since currently the onlydifference is that it does not save, but the general semantics is:close the server without trying to be an hero saving or doing anyother cleanup. Just delete the pid and exit.
Blocking from LUA will not work at all, the blocking commands will
actually return an error in the final version...
Cheers,
Salvatore
> -Fritzy