Command order guarantee?

[Vali Dr]

Hello

I'm trying to debug something:

If I run these commands on host A, is it possible that host B could get the baz published event before the foo key is created?

(Redis server is on a 3rd host)

Producer

// Pipe to socket:
MULTI
SET foo 1
EXPIRE foo 30
EXEC

// Some php code that can take a few MS

// Pipe to socket:
MULTI
RPUSH bar 1
EXPIRE bar 30
EXPIRE foo 25
PUBLISH baz 1
EXEC

And since I pipe the commands, do I need the MULTI/EXEC? or is the order guaranteed?

[Itamar Haber]

Hello Vali

Order is always guaranteed from the perspective of host A. Pipelining is a networking optimization. MULTI makes the block atomic so no other requests (from other servers) are served during the block's execution.

> If I run these commands on host A, is it possible that host B could get the baz published event before the foo key is created?

No, it isn't possible.

> And since I pipe the commands, do I need the MULTI/EXEC? or is the order guaranteed?

No, you don't need pipelining or MULTI/EXEC here to guarantee the order.

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/redis-db/a1c7a2b6-1b1d-4f6d-9b9e-6c2cc52ecda8%40googlegroups.com.

--

Itamar Haber Technicalist Evangely Phone: +972.54.567.9692

Disclaimer

The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful.

This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast Ltd, an innovator in Software as a Service (SaaS) for business. Providing a safer and more useful place for your human generated data. Specializing in; Security, archiving and compliance. To find out more Click Here.

[Vali Dr]

Thank you, that's what I thought, but very rarely, foo is not there after I get the baz event on Host B.

One thing was not clear in your explanation tho:

My code does:

1) Pipe to socket some commands.

2) Process some other stuff

3) Pipe to the same socket other commands.

So between 1 and 3, something else can run commands to redis no problem.

Now to clarify 100% my previous question:

- Can redis somehow run the commands 3) before 1) ?

Theoretically, on a non blocking connection, there could be a network glitch, to delay the 1) commands, and then 3) commands could reach the redis server before 1).

But I'm waiting for the response every time, so I know redis accepted 1) and then accepted 3).

So I want to make 100% sure, that:

- If redis accepted and replied to a command, it actually blocked the entire process, ran those commands, waited for them to finish, sent a reply to the user, and continues with it's queue.

- And NOT, that it accepted the commands, confirmed the receipt with a reply, and then blocked the thread to run the commands. (like it does with expire)

[Itamar Haber]

> very rarely, foo is not there after I get the baz event on Host B.

It could have expired... Or maybe some other client had deleted it...

> If redis accepted and replied to a command, it actually blocked the entire process, ran those commands, waited for them to finish, sent a reply to the user, and continues with it's queue.

Correct. Almost all commands are blocking until they are completed, and a small subset is asynchronous (e.g. BGSAVE and UNLINK). EXPIRE is also executed immediately in the sense that it sets the key's TTL. Actual expiration of the key is another matter and is handled by the server when needed.

--

You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/redis-db/b0c81f8e-2307-40e7-a91c-4a9c3d69c306%40googlegroups.com.

[Vali Dr]

Great, thank you. :)

On Friday, May 1, 2020 at 9:49:42 AM UTC-4, Itamar Haber wrote:

> very rarely, foo is not there after I get the baz event on Host B.

It could have expired... Or maybe some other client had deleted it...

Actually, it's within a few 1-12ms of each-other, with a 30sec TTL , so has to be something else.

> If redis accepted and replied to a command, it actually blocked the entire process, ran those commands, waited for them to finish, sent a reply to the user, and continues with it's queue.

Correct. Almost all commands are blocking until they are completed, and a small subset is asynchronous (e.g. BGSAVE and UNLINK). EXPIRE is also executed immediately in the sense that it sets the key's TTL. Actual expiration of the key is another matter and is handled by the server when needed.

On Fri, May 1, 2020 at 4:12 PM Vali Dr <val...@gmail.com> wrote:

Thank you, that's what I thought, but very rarely, foo is not there after I get the baz event on Host B.

One thing was not clear in your explanation tho:

My code does:

1) Pipe to socket some commands.

2) Process some other stuff

3) Pipe to the same socket other commands.

So between 1 and 3, something else can run commands to redis no problem.

Now to clarify 100% my previous question:

- Can redis somehow run the commands 3) before 1) ?

Theoretically, on a non blocking connection, there could be a network glitch, to delay the 1) commands, and then 3) commands could reach the redis server before 1).

But I'm waiting for the response every time, so I know redis accepted 1) and then accepted 3).

So I want to make 100% sure, that:

- If redis accepted and replied to a command, it actually blocked the entire process, ran those commands, waited for them to finish, sent a reply to the user, and continues with it's queue.

- And NOT, that it accepted the commands, confirmed the receipt with a reply, and then blocked the thread to run the commands. (like it does with expire)

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.

To unsubscribe from this group and stop receiving emails from it, send an email to redi...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/redis-db/b0c81f8e-2307-40e7-a91c-4a9c3d69c306%40googlegroups.com.

[Greg Andrews]

Are you checking the results that are returned from your MULTI...EXEC block commands?

The documentation page https://redis.io/topics/transactions discusses situations in which a MULTI...EXEC block might not execute all the commands.

In your description of the symptoms, I'm also seeing an assumption that the client subscribed to the 'baz' channel receives the '1' message before the key has expired.  This is worth testing.  Perhaps by publishing the current timestamp in the Unix/Linux seconds-since-1970 format rather than just '1'.  Your client that receives the published message but gets an error on the 'foo' key can log the time it tried to read 'foo' along with the timestamp it received the message from 'baz' and the message contents of the message.  This can show you if there are delays between the message being published, being received, and the key being read.

It's also worth testing that the expiration applied to the key is indeed the value as it's supposed to be.  Another client saving the output of the MONITOR command can be useful to see what your PHP client library is actually sending to Redis.