Setting up client_max_body_size dynamically

[tzi...@mozilla.com]

Hello

I have a lua module that route requests dynamically by setting proxy_pass on the fly - reading an external server to get the configuration. That works well.

What I want to do now is return a 413 when the request body is too large with a limit value that's also provided on the fly.

At first I was thinking about reading the body in a stream and raise a 413 if the limit is reached, but that seems to be a lot of extra work.

So I tried to set up the client_max_body_size dynamically using :

  set $max_body_size 1M;
  ...  some access by lua code runs here...
  client_max_body_size $max_body_size;

Which leads to : "client_max_body_size" directive invalid value..


How can I do this ? Can I avoid reading the body content in the lua code ? or is this mandatory ? is this compatible with using proxy_pass ?


Thanks!
Tarek

[Yichun Zhang (agentzh)]

Hello!

On Sat, Jun 28, 2014 at 3:18 AM, tziade wrote:
> So I tried to set up the client_max_body_size dynamically using :
>
> set $max_body_size 1M;
> ... some access by lua code runs here...
> client_max_body_size $max_body_size;
>
> Which leads to : "client_max_body_size" directive invalid value..
>

This requires patching the nginx core to add nginx variable support to
the client_max_body_size directive. I could add an patch to the nginx
core bundled by OpenResty for this (at CloudFlare we use such a patch
internally, which has not yet been incorporated into OpenResty yet) if
you really want it. You can also consider requesting this feature in
the nginx mailing list and it's better to be included in the
mainstream nginx.

One (partial) workaround is to test the Content-Length request header
in Lua before reading the request body (either explicitly by
ngx.req.read_body() in Lua or implicitly by ngx_proxy and etc). But
requests using chunked encoding for their request bodies won't work
with this workaround.

>
> How can I do this ? Can I avoid reading the body content in the lua code ?
> or is this mandatory ? is this compatible with using proxy_pass ?
>

If you use the ngx.req.socket() API to read the request body yourself,
then you should use the ngx.req.init_body(), ngx.req.append_body(),
and ngx.req.finish_body() to add the body data you're reading into the
internal nginx data structures for request bodies such that other
nginx modules like ngx_proxy can see it.

Best regards,
-agentzh

[tzi...@mozilla.com]

On Sunday, June 29, 2014 3:02:30 AM UTC+2, agentzh wrote:

..

This requires patching the nginx core to add nginx variable support to
the client_max_body_size directive. I could add an patch to the nginx
core bundled by OpenResty for this (at CloudFlare we use such a patch
internally, which has not yet been incorporated into OpenResty yet) if
you really want it. You can also consider requesting this feature in
the nginx mailing list and it's better to be included in the
mainstream nginx.

  I suppose it depends on the release cycle of both. I will send an e-mail to the nginx mailing list,
  but in the interim I'd be delighted if OpenResty gets the feature - this is quite useful imho :)

[..]

If you use the ngx.req.socket() API to read the request body yourself,
then you should use the ngx.req.init_body(), ngx.req.append_body(),
and ngx.req.finish_body() to add the body data you're reading into the
internal nginx data structures for request bodies such that other
nginx modules like ngx_proxy can see it.

ok thanks for all the info, it seems like it could introduce some overhead but I guess I can give it a shot.


Thanks!

Tarek

[tzi...@mozilla.com]

Added on Nginx side at http://trac.nginx.org/nginx/ticket/586