Attaching hiredis' redisAsyncContext from within a redis module to server.c's event loop

[Zongheng Yang]

Hi all,

(Apologies first for the long email.  This might be one of the more contrived use cases of redis+ae+hiredis+redis module :D.)

I am implementing a lightweight redis module, MyModule, within which I create a redisAsyncContext (from hiredis) to connect to another instance of the same module:

MyModule:
     redisAsyncContext *child = redisAsyncConnect(addr, port);

Let's say I have 2 such servers running, with architecture "HEAD -> TAIL".  The tail server does not have a child.

Then, I hacked redis' server.c to expose server.el, the event loop of the server, and did something like this:

MyModule:
     redisAsyncContext *child = redisAsyncConnect(addr, port);
     // Check child is connected.
     aeEventLoop *loop = HACK_getEventLoop();
     redisAeAttach(loop, child);

     // HEAD_PROPAGATE n: call CHILD_PROPAGATE n times
     int HeadPropagate_RedisCommand(...):
           long long N = /* process argv[1] */
           for (int i = 0; i < N; ++i): 
               redisAsyncCommand(child, NULL, NULL, "CHILD_PROPAGATE %b", /* process i into string */)
           RedisModule_ReplyWithNull(ctx);
           return REDISMODULE_OK;

     // CHILD_PROPAGATE i: just print out i
     int ChildPropagate_RedisCommand(...):
           // Just print out i.

Now, this works only sporadically, with the following symptoms.

From a client that connects to the HEAD, running

    HeadPropagate 1

always returns immediately with nil.  However, inspecting the HEAD server's log, sometimes HeadPropagate_RedisCommand() is not invoked immediately, and arbitrarily delated.  (I judge this by printing out a line at the top of this function body.)  In such a case, subsequent invocations of the same command, with a same or different N, sporadically (read: may or may not) causes the HEAD server to process currently queued HeadPropagate.  There seems to be no pattern between # of such queued requests, and the next time HEAD server becomes reactive to these queued commands.

However, if I hack server.c further, to print out a message at the top of serverCron(), beforeSleep(), and afterSleep(), I can see that the loop driving the HEAD server is being continuously run, it's just that some HeadPropagate_RedisCommand() get queued somewhere (instead of dropped permanently).

Another complication: if I don't actually issue redisAsyncCommand and instead just change HeadPropagate_RedisCommand to be a trivial echo command -- but still keep the redisAeAttach() bit -- no delaying/queueing can be observed anymore.

What's up with all this?  Can someone provide help on why my setup doesn't work?

redis: commit cf9a3f70 (Nov 8, 2017)

hiredis: 07d414731b (Nov 7, 2017)

Tested on Mac OSX:

Apple LLVM version 9.0.0 (clang-900.0.38)

Target: x86_64-apple-darwin17.2.0

Thread model: posix

Thanks,

Zongheng

[Zongheng Yang]

Update: I tested on an Ubuntu machine and it had the same issue.  

$ gcc --version

gcc (Ubuntu 5.4.0-6ubuntu1~16.04.5) 5.4.0 20160609

$ uname -a

Linux ip-172-31-40-47 4.4.0-1039-aws #48-Ubuntu SMP Wed Oct 11 15:15:01 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

[Dvir Volk]

Don't do this, you're just asking for trouble. 

I'm doing the same, but with two main differences:

1. I'm using an external hiredis library, and I'm making sure the symbols don't get mixed with the internal hiredis implementation. 

In Linux you should add to your compile flags:

-shared -Bsymbolic -Bsymbolic-functions

And on Mac you should add:

-exported_symbol _RedisModule_OnLoad 

2. I'm using another event loop, not even libae, I'm linking hiredis against libuv. This works great and has no issues with redis. 

In general, as a rule of thumb, when developing modules - don't try to hack redis to expose anything it doesn't expose. It's almost always a bad idea.

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.
To post to this group, send email to redi...@googlegroups.com.
Visit this group at https://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.

[Zongheng Yang]

Adding redis-dev@

[Zongheng Yang]

Adding redis-dev@.