Redis Twitter Clone has a N + 1 Get?

[Subskii]

Hi all

I'm looking at the Twitter Clone @ http://redis.io/topics/twitter-clone
and noticed a N + 1 -like "get" in the code:

function showPost($id) {
$r = redisLink(); $postdata = $r->get("post:$id"); ...

foreach($posts as $p) {
if (showPost($p)) $c++; if ($c == $count) break; }

The above code seems rather suboptimal, if my understanding is
correct.

Are there any ways to improve upon this?

Thanks in advance!

[Dvir Volk]

you can do 2 things to accelerate this:

1. use pipelining to get all the posts in one server roundtrip (won't change the code much and be much faster)

2. use SORT ... GET semantics to get all the post data at once from the list of ids (should be somewhat faster than 1)

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To post to this group, send email to redi...@googlegroups.com.
To unsubscribe from this group, send email to redis-db+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.

--

Dvir Volk

System Architect, Do@, http://doat.com

[Pierre Chapuis]

On Mon, 21 Feb 2011 09:57:12 +0200, Dvir Volk wrote:
> you can do 2 things to accelerate this:

> 1. use pipelining to get all the posts in one server roundtrip (wont

> change the code much and be much faster)
> 2. use SORT ... GET semantics to get all the post data at once from
> the list of ids (should be somewhat faster than 1)

3. Use MGET to get all the post data at once. Should be somewhat
faster than 2.

--
Pierre 'catwell' Chapuis

[Sebastian]

Why do you say so? I think
redis> SORT uid:123:posts BY nosort GET post:*

may be faster, since it's only one roundtrip (to use MGET you need to
fetch the list first), and not receiving+sending the whole list of ids.
I'm not sure if nosort will guarantee to keep the list order, thought.

[Dvir Volk]

right, I'm automatically thinking of this stuff as being inside HASHEs, but the example refers to single field values.

--

You received this message because you are subscribed to the Google Groups "Redis DB" group.
To post to this group, send email to redi...@googlegroups.com.
To unsubscribe from this group, send email to redis-db+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.

[Demis Bellot]

Hi,

Just want to add pipelining really helps.

My RedisStackOverflow clone at: 

http://www.servicestack.net/RedisStackOverflow/

Does an N+1 here to batch together 30 operations and still maintains good speed since pipelining lets you treat it like a single request/response IO call:

//Batch multiple operations in a single pipelined transaction (i.e. for a single network request/response)

RedisManager.ExecTrans(trans => {

foreach (var question in questions) {

  var q = question;

  trans.QueueCommand(r => r.GetSetCount(QuestionUserIndex.UpVotes(q.Id)),

  voteUpCount => resultsMap[q.Id].VotesUpCount = voteUpCount);

  trans.QueueCommand(r => r.GetSetCount(QuestionUserIndex.DownVotes(q.Id)),

  voteDownCount => resultsMap[q.Id].VotesDownCount = voteDownCount);

  trans.QueueCommand(r => r.As<Question>().GetRelatedEntitiesCount<Answer>(q.Id),

  answersCount => resultsMap[q.Id].AnswersCount = answersCount);

}

});

Having said that, if you have the opportunity to take advantage of batchful operations like MGET take it, as it makes your client code a lot more readable.

Cheers,

Demis

- Demis

http://twitter.com/demisbellot

http://www.servicestack.net/mythz_blog

[Pierre Chapuis]

On Mon, 21 Feb 2011 10:48:08 -0300, Sebastian wrote:
> Why do you say so? I think
> redis> SORT uid:123:posts BY nosort GET post:*
>
> may be faster, since it's only one roundtrip (to use MGET you need to
> fetch the list first), and not receiving+sending the whole list of
> ids.

You're right, MGET will only be faster than pipelining, not
than SORT ... GET. I should learn how to use this one :)

--
Pierre 'catwell' Chapuis

[George Bashi]

I used approach #2 in my sinatra-backed twitter clone here:

https://github.com/georgebashi/tweedis

You'll find all the redis commands in model.rb.

George

[Salvatore Sanfilippo]

The first problem is that this example was modeled after Redis 0.9 or
something like that.
Today we could write it 100 times better, finding the time.... :)

Cheers,
Salvatore

> --
> You received this message because you are subscribed to the Google Groups "Redis DB" group.
> To post to this group, send email to redi...@googlegroups.com.
> To unsubscribe from this group, send email to redis-db+u...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.
>
>

--
Salvatore 'antirez' Sanfilippo
open source developer - VMware

http://invece.org
"We are what we repeatedly do. Excellence, therefore, is not an act,
but a habit." -- Aristotele