calling WATCH inside a MULTI/EXEC

[Aaron Boxer]

Hello!
I have two MULTI/EXEC blocks, and I would like to start WATCHing a few keys
immediately after the first MULTI/EXEC runs. Does this make sense:

MULTI
COMMAND1
COMMAND2
.
.
.
WATCH key1, key2, key3
EXEC

MULTI
COMMAND3
COMMAND4
.
.
.
EXEC

I would like the second transaction to fail on the WATCH, but not the first.

[Salvatore Sanfilippo]

On Sun, Dec 19, 2010 at 2:10 PM, Aaron Boxer <box...@gmail.com> wrote:

> MULTI
> COMMAND1
> COMMAND2
> .
> .
> .
> WATCH key1, key2, key3
> EXEC
>
> MULTI
> COMMAND3
> COMMAND4

Hello, please can you tell us what is your goal instead of *how* you
want to accomplish it?
In general I think that what you want to do can be modeled with a
single MUTLI/EXEC/WATCH block but it's better to have a few more
details.

Thank you,
Salvatore

--
Salvatore 'antirez' Sanfilippo
http://invece.org

"We are what we repeatedly do. Excellence, therefore, is not an act,
but a habit." -- Aristotele

[Aaron Boxer]

Thanks, Salvatore. My use case is check and set:

1) call transaction A, which calls GET K on a key K, returning value V1
2) compare V1 with some in-memory value, and decide whether to call
transaction B
3) transaction B calls SET K V2, with new value V2
4) fail transaction B if K has changed, because the decision in step 2
may no longer
be valid if K has changed

If I issue

WATCH K

after the first MULTI/EXEC, then it is possible that another
client will have changed K before the WATCH was issued,
so I will miss the change. So, I would like a guarantee that no other
commands get processed
before the WATCH gets issued. If I issued the WATCH at the very
beginning, then of course the first
transaction would reset the WATCH after it executed.

Note: I need to use transactions, because there are a few other
commands I need to call
along with the GET and SET.

> --
> You received this message because you are subscribed to the Google Groups "Redis DB" group.
> To post to this group, send email to redi...@googlegroups.com.
> To unsubscribe from this group, send email to redis-db+u...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.
>
>

[Derek Williams]

Can you GET K before your first MULTI? If so, can you combine the two transactions? After getting K you can decide which commands to use inside the MULTI. Either that or perform the WATCH inbetween the transactions, you can always call DISCARD if the value isn't what you want.

[Aaron Boxer]

Thanks, Derek.A WATCH between transactions could miss a change
directly after the first
transaction but before the WATCH gets called. And DISCARD wouldn't
work because I wouldn't
know that I needed to call it.

I've re-arranged my code so that I can get away with only a single MULTI/EXEC.
Problem solved :) I still think it could be useful to have EXEC pass in a flag
that disabled the post-transaction UNWATCH. Then it would be possible
to choose which transaction would fail on the WATCH. Or, alternatively
, if EXEC detected that a WATCH
was queued, then it could disable post-transaction UNWATCH.

[Derek Williams]

On 2010-12-19 6:58 PM, "Aaron Boxer" <box...@gmail.com> wrote:
>
> Thanks, Derek.A WATCH between transactions could miss a change
> directly after the first
> transaction but before the WATCH gets called. And DISCARD wouldn't
> work because I wouldn't
> know that I needed to call it.

I don't know enough about your code, but I meant something like this after the first transaction:

- WATCH K
- GET K
- decide if 2nd transaction should run based on previous value from first transaction, or else DISCARD.

Doing it in one transaction is probably best though.

>
> I've re-arranged my code so that I can get away with only a single MULTI/EXEC.
> Problem solved :)  I still think it could be useful to have EXEC pass in a flag
> that disabled the post-transaction UNWATCH. Then it would be possible
> to choose which transaction would fail on the WATCH. Or, alternatively
> , if EXEC detected that a WATCH
> was queued, then it could disable post-transaction UNWATCH.
>

Isn't checking manually as above just about as useful?

[Aaron Boxer]

Thanks again for your help, Derek. Sorry for not providing more details earlier.

Here is my original check and set:

MULTI
GET K
other stuff
EXEC

Decide if I am going to SET K with a new value, based on the value I
received with GET.
If so, then:

MULTI
SET K NEW_VALUE
other stuff
EXEC

otherwise

UNWATCH

I need to be assured that once I GET K, that K will not change until after SET.
So, where can I put the WATCH?

If I put it before the first transaction, then the
first transaction will UNWATCH when it executes.

If I put it after the first transaction, the key K may have changed,
but I wouldn't
know about it.

That's why it would be nice be able to call

EXEC NO_UNWATCH

for the first transaction.

[Derek Williams]

MULTI
GET K
other stuff
EXEC

WATCH K
GET K

compare to K that was received above, if it is different or if you
don't want to SET K, UNWATCH, or else carry on:

MULTI
SET K NEW_VALUE
other stuff
EXEC

I would think this would be okay since the first transaction is
already done and can no longer fail. if you were able to use one WATCH
for both then the first transaction would still be completed even if
the second failed due to a changing of K. If the first transaction
should fail if the second one does, then they probably should be done
as one. It isn't too important though as this is probably slower then
doing it with a single transaction, which I assume you did something
like this:

WATCH K
GET K
decide what commands to use for multi depending on K
MULTI
do stuff
possibly SET K
EXEC

--
Derek

[Aaron Boxer]

ahhhhh, ok, now I understand. Yes, that could work as well.

Thanks! Will add that to my bag of tricks.

[Salvatore Sanfilippo]

On Mon, Dec 20, 2010 at 4:54 AM, Aaron Boxer <box...@gmail.com> wrote:
> ahhhhh, ok, now I understand. Yes, that could work as well.
>
> Thanks! Will add that to my bag of tricks.

In general you know there is something strange in your code if you
have a GET or other read operation inside a transaction.
The whole point of WATCH is the following pattern:

WATCH key
... some read operation against key ...
... possibly a conditional ...
MULTI
... write stuff ...
EXEC

Cheers,
Salvatore

--

[Aaron Boxer]

Yes, good point. I need to study my code a bit more....

[Ingvar Bogdahn]

Hi,

I would be very interested in the answer to the general question of the thread's title. imho, it was not answered. 

In my usecase, I'm almost sure, it's inevitable to call watch after multi.   

The redis client and jedis give an error "WATCH inside MULTI is not allowed"  but I wonder, which it shouldn't be possible?

this would save my a lot of pain..

thanks,

Ingvar

[Salvatore Sanfilippo]

On Mon, May 30, 2011 at 3:22 PM, Ingvar Bogdahn
<ingvar....@googlemail.com> wrote:
> would be very interested in the answer to the general question of the
> thread's title. imho, it was not answered.
> In my usecase, I'm almost sure, it's inevitable to call watch after multi.
>
> The redis client and jedis give an error "WATCH inside MULTI is not allowed"
>  but I wonder, which it shouldn't be possible?

Hello, I can assure you it is never useful to call WATCH inside MULTI.
Actually it does not make any sense at all because of the semantics of
Redis transactions.

Salvatore

--
Salvatore 'antirez' Sanfilippo
open source developer - VMware

[Ingvar Bogdahn]

yes, I think it would be useful, but admitted, my use case is rather exotic: I'm adapting jedis as a "primitive store" for HypergraphDB (a graph NoSQL). So jedis is the underlying database for another database. A hypergraphDB operation causes several jedis operations, with different keys at different places in the code. Some later operations of the same transaction, depend on earlier results of that same transaction. The creation of the Transaction Object ( multi) is discoupled of the actual code using it, since several different methods have to share the same Transaction Object and HypergraphDB has a means to communicate the current & correct transactionObject to the primitive store. However, it is impossible to specify all the keys involved in the transaction, in advance and before call of multi. If the Transaction Object itself would have a method watch, which could be called at some later point during the transaction, everything would be perfect. But now, I have the thing more than half done, and realized that this is dealbreaker, i.e. the jedis-HypergraphDB would be non-transactional (isolation is important for hypergraphDB) :-(  Is there some hard reason, why Redis cannot call watch after multi?  ( sorry for bad english)

regards,

Ingvar

[Santiago Perez]

Ingvar, everything executed inside a MULTI/EXEC block is executed atomically, so any key you watch after the MULTI cannot change before the EXEC.

--

[Ingvar Bogdahn]

Ingvar, everything executed inside a MULTI/EXEC block is executed atomically, so any key you watch after the MULTI cannot change before the EXEC.

huu?!? i'm not sure I follow your reasoning here. We are only talking about the time before exec, but the first part of your sentence refers to the time after exec.

Maybe I was badly describing my problem. Let me reformulate: is there a way to "update watchs", somewhere in the timespan between multi and exec ?

Wouldn't it be good to have the possibility that commands in a transaction block automatically infer watch for the keys they have been called with? 

 

regards,

ingvar

[Santiago Perez]

I may be totally off since a have never really used transactions and watches on redis, but as I understand it, while on a transaction every command you send (including watches) are buffered on the server, so you can't really "read" anything in that time. Once you send the EXEC the buffered commands are executed serially with no concurrency with other client's commands. So the only changes made to redis within the transaction are the ones done in that transaction.

The use case for watch is always to read something before a transaction (otherwise you will only know the actual value after exec), but only execute the transaction if the values you have read (and watched) haven't changed. This gives you the illusion of having the reads and the writes executed atomically by assuming nothing will change in the middle and failing otherwise.

Regards,

Santiago