Pointers on developing zeromq nginx module

[Jai]

[resending the mail to group as recommend by @agentzh]

We have a zeromq backend module speaking protobuf protocol and was looking for giving an http interface to it. One solution I was thinking about was to create an nginx module to connect http with our backend zeromq server. This module needs custom protocol conversion handler, hooks for event handling , send-recv handlers and keep alive. [zeromq has custom protocol over tcp, needs custom send-recv handlers and needs the fd to be injected to nginx event loop]

I was looking at https://github.com/FRiCKLE/ngx_zeromq as a starting point and but realized it did not have keep alive, a proxy handler (for custom protocol conversion) and was not touched for 2 years. If I were to start on this now, what would you recommend as a good starting point ?  Is there a simple nginx module template that I can use for developing such a module. Is this stretching nginx module beyond what it is meant to be , has anybody attempted doing this and running in production.

Any pointers and suggestions would be invaluable.

regards,

Jai

[Fasih]

Hi

I have developed an nginx module that talks to a custom backend server in protobuf on a persistent connection. I had evaluated using a custom proxy handler but when I did, it didnt have keepalive support, so I ended up writing my own handler that adds an fd into the nginx event loop. Given that there is very minimal documentation on this, I would say that this falls in "stretching" nginx but it is doable and after the initial quirks, it has been working fine. I couldnt find any template to start off with, but I think you could use nginx lua's tcp module to get some clues on how this would work. Be cautioned that you would have to handle backpressure, downstream errors, upstream errors and more by yourself. If you do use the proxy handler to do this (which I think is the right way to do it), please update the thread/mail me.

Regards

[Jai]

Thanks Fasih for the reply.  Are you referring to this module - https://github.com/bigplum/nginx-tcp-lua-module ?

Also have you looked at the drizzle module  (https://github.com/chaoslawful/drizzle-nginx-module) , on a quick pass it appears to be doing all that I might need, it has custom send/recv handlers, persistent connection to the backend (hopefully the connection doesn't  get closed after every request) and also seems to be using a proxy handler encompassing a load balancing handler implemented internally (pardon if I am wrong, spend just a day looking at it).  I was trying to use that as a reference to see whether I can get zeromq connections/ handlers plugged in , but if one of the experienced folks here can see a dead end on this path , please warn me early.

The trouble here is a simple proxy module alone wouldn't suffice, since I also need to handle the send/recv part as zeromq has custom sockets and protocol. All it provides is a fd which can be waited on using a epoll loop, but not a regular tcp socket.

Suggestions welcome

Jai