Redis module: please speed up RedisModule_Call(ctx, "PUBLISH", ...)

[Zongheng Yang]

I benchmarked two things:

1) vanilla redis, SET

2) my custom module's PUT, which basically does (i) RedisModule_StringSet, then (ii) RedisModule_Call(ctx, "PUBLISH", ...) to notify subscribers that the write has been finalized.

I used Google's pprof to profile these two variants.  Visualizations show that step (2)'s (ii), the publish, has 4x aggregated time than (2)'s (i), which is similar to (1).

Here's the svg for (2) (download & open with Chrome to zoom in/out):

https://drive.google.com/file/d/14bSJ0M9eqNetjJ-9cMAk2BpS4JQXBZhM/view?usp=sharing

Specific questions:

1) I see RM_Call(.., "PUBLISH", ..) spends a lot of time on createClient(), freeClient().  

How does each publish work?  Does it create and destroy a client every time it needs to publish a message?

If so, anyway to avoid this?  I can ensure that each channel has exactly 1 subscriber.

2) In general, can you optimize the publish codepath for Redis Module?  Preferably something like native RedisModule_Publish, to avoid looking up the command by string "PUBLISH".

Zongheng

[Dvir Volk]

Hi Zongheng.

Yes, this is a painful subject :)

I'm not sure whether CALL can be optimized by a lot. Adding a low level call to Publish is a good idea IMO, but whether it should be part of the API is up to Salvatore. 

I'm willing to have a go at both items :)

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.
To post to this group, send email to redi...@googlegroups.com.
Visit this group at https://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.

[Zongheng Yang]

Dvir: Good to know I'm not completely crazy.  Let me know if you do get something better working!

You received this message because you are subscribed to a topic in the Google Groups "Redis DB" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/redis-db/kSv5xbUpxsw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to redis-db+u...@googlegroups.com.

[Dvir Volk]

May I ask what module are you writing?

[Dvir Volk]

I've made a PR that optimizes repeated calls to RedisModule_Call. It speeds up 1M calls to PING from 3.5 seconds to 1.3 seconds on my laptop.

https://github.com/antirez/redis/pull/4623

It won't help much if your module does just one call per command it implements, but maybe we can optimize that too.