[Lua Module]: Variable persistent across HTTP requests with lifetime of TCP connection

[Suraj Jaiswal]

Hello,

I am trying to implement variable support in nginx lua module which persists across HTTP requests.

[Disclaimer: Its an attempted hack to verify the implementation possibility]

The idea is exactly similar to the ngx.ctx.XYZ construct available in the module.

I have done the following:

------------------------------------

1. Used the similar construct as ngx.ctx.XYZ such that the context which stores the reference to the table returned by luaL_ref() is stored off ngx_connection_t (r->connection->lua_ref, where lua_ref is the ref returned by luaL_ref())

2. The cleanup handler is attached to connection pool thereby giving the persistence across HTTP requests

As of now I am getting:

---------------------------------

I call this construct ngx.conn_ctx.XYZ

So I correctly get the persistence of variables across HTTP requests as long as my script does not use ngx.conn_ctx.XYZ in the HTTP header filter.

More precisely:

location / {

    rewrite_by_lua '

                  if not ngx.conn_ctx.foo then ngx.conn_ctx.foo = 17 end ngx.conn_ctx.foo = ngx.conn_ctx.foo + 1 if ngx.var.uri == "/foo/" then return ngx.redirect("https://www.foo.com/".. ngx.conn_ctx.foo) end'

    proxy_pass ....

    header_filter_by_lua ' if ngx.var.uri == "/bar/" then ngx.conn_ctx.foo = ngx.conn_ctx.foo + 3 ngx.header.FOO = ngx.conn_ctx.foo end'

}

For 3 requests per connection:

1. GET /foo/, I get as expected

https://www.foo.com/18,

https://www.foo.com/19

https://www.foo.com/20

2. GET /bar/, I dont get expected persistence value in the FOO header in HTTP response

FOO: 21

FOO: 21

FOO: 21

Seems like the ngx.conn_ctx.foo table is accessed correctly across the 3 coroutines created as part of rewrite directive, but the table seems to be cleared after the header_filter is executed.

The magic seems to be in ngx_http_lua_header_filter_by_lua_env().

Yichun, any comment from top of your head?

I will try to send the diff as well. I dont have access to my dev source tree now.

Best

Suraj 

[Yichun Zhang (agentzh)]

Hello!

On Tue, Mar 24, 2015 at 6:30 PM, Suraj Jaiswal wrote:
> 1. Used the similar construct as ngx.ctx.XYZ such that the context which
> stores the reference to the table returned by luaL_ref() is stored off
> ngx_connection_t (r->connection->lua_ref, where lua_ref is the ref returned
> by luaL_ref())
> 2. The cleanup handler is attached to connection pool thereby giving the
> persistence across HTTP requests
>

Are you trying to share data across pipelined or keep-alive requests
on each TCP connection? Not sure why you need that because those
requests are still independent of each other. Registering the cleanup
hook on c->pool won't work for you because c->pool MAY get freed after
processing each request (when the connection has no pending data in
the receive buffers). To quote nginx's related comment:

/*
* To keep a memory footprint as small as possible for an idle keepalive
* connection we try to free c->buffer's memory if it was allocated outside
* the c->pool. The large header buffers are always allocated outside the
* c->pool and are freed too.
*/

See the ngx_http_set_keepalive function in the nginx core for more details.

Not sure why header_filter_by_lua makes a difference for you. I'm not
really interested in your own hack, sorry :)

Regards,
-agentzh

[Suraj Jaiswal]

Thanks Yichun for pointing out the behavior of c->pool.

We have a scenario where we want to persist data across ALL requests in a TCP connection.

I read about the ngx.ctx.XYZ implementation and just tried a quick hack which does not works as you pointed out :-)

From the Lua module's execution model (coroutine creation per request processing and using the original parent lua_State VM on the response in the header filter), would you be able to suggest if implementing a construct such as ngx.conn_ctx.XYZ to persist data across all HTTP requests is good idea or not? 

Best

Suraj

[Yichun Zhang (agentzh)]

Hello!

On Tue, Mar 24, 2015 at 7:23 PM, Suraj Jaiswal wrote:
> We have a scenario where we want to persist data across ALL requests in a
> TCP connection.
> I read about the ngx.ctx.XYZ implementation and just tried a quick hack
> which does not works as you pointed out :-)
>

It's possible with a simple approach (without patching ngx_lua nor the
nginx core). You can just write a simple nginx C module [1] that
exposes a simple Lua API function that returns the fd for the current
"c" as an approximated connection identifier. To detect fd recycling
among different connections, you can also check c->requests in Lua to
ensure it's increasing monotonically (otherwise it means the current
fd is a new connection).

Regards,
-agentzh

[1] Similar to my ngx_lua_upstream module
(https://github.com/openresty/lua-upstream-nginx-module ) or Brian
Akins's ngx_example_lua module
(https://github.com/bakins/nginx-example-lua-module ).