HttpLuaModule + subrequest (ngx.location.capture) body problem

[Jędrzej Nowak]

Hello,

Consider following files:

nginx.conf: http://wklej.to/1e0qX/text

lua_tester.lua: http://wklej.to/Tdk9O/text

(you need some HTTP app or something)

now when sending GET or any other request without body everything works fine.

BUT

when I will send any POST request (or PUT in fact), the backend receives no body. It receives correct headers with correct sizes, but it looks for me like openresty / lua is not sending it.

When I will specify the body by hand like body = "xyz", everything works perfect.

So my question is, how to do POST / PUT subrequest with body. 

[Luis Gasca]

You need to set 

lua_need_request_body

Regards,

Luis

--
You received this message because you are subscribed to the Google Groups "openresty-en" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openresty-en...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Jędrzej Nowak]

Wouldn't it cause of reading whole body into memory, and then passing it from memory ?

I thought that:

• always_forward_body

when set to true, the current (parent) request's request body will always be forwarded to the subrequest being created if the body option is not specified. By default, this option is false and when the body option is not specified, the request body of the current (parent) request is only forwarded when the subrequest takes the PUT or POST request method.

Will forward it without copying...

There is no other possibility ? I really need something like zero copy of body.

[Yichun Zhang (agentzh)]

Hello!

On Sun, Oct 6, 2013 at 9:08 AM, Jędrzej Nowak wrote:
> Wouldn't it cause of reading whole body into memory, and then passing it
> from memory ?
>

Both "lua_need_request_body on" and ngx.req.read_body() use the
built-in request body reader in the Nginx core, which always buffers
the whole requesty body in either the memory or a temporary file
(depending on whether your request body size is bigger than the
client_body_buffer_size setting in nginx.conf). As long as you use
Nginx core's builtin request body reader, you have to buffer the whole
body, no matter it is the ngx_lua module or nginx standard modules
like ngx_proxy, ngx_fastcgi, or ngx_dav.

The ngx_lua module provides, however, an alternative request body
reader itself, that supports streaming processing:

http://wiki.nginx.org/HttpLuaModule#ngx.req.socket

With this ngx.req.socket API, you can read the request body data in
chunks and pass them to the backend services via the cosocket API at
the same time. Note that, for streaming request body processing, you
cannot use subrequests and traditional nginx modules as the subrequest
targets, because traditional nginx modules like ngx_proxy and
ngx_fastcgi always expect the whole requesty body to be ready (because
they assume that Nginx core's builtin requesty body reader is used).

One real-world example using this ngx.req.socket API to read big
request bodies in chunks is the lua-resty-upload library that reads
and parses multi-part request body data in a streaming fashion:

https://github.com/agentzh/lua-resty-upload

> I thought that:
>
> always_forward_body
>
[...]

>
> Will forward it without copying...
>

When this option is set and in effect, the request body read by the
builtin Nginx request body reader will be directly forwarded to the
subrequest without copying the whole request body data when creating
the subrequest (matter the request body data is buffered in memory
buffers or temporary files). Because you're using subrequests, you
have to buffer the whole request body anyway.

Regards,
-agentzh

[Jędrzej Nowak]

Ok, got it. I think it would be nice to mention it in docs though (near to the always_forward_body)

On Saturday, October 5, 2013 10:15:21 PM UTC+2, Jędrzej Nowak wrote:

[Yichun Zhang (agentzh)]

Hello!

On Sun, Oct 6, 2013 at 12:33 PM, Jędrzej Nowak wrote:
> Ok, got it. I think it would be nice to mention it in docs though (near to
> the always_forward_body)
>

Updated the documentation a bit:

http://wiki.nginx.org/HttpLuaModule#ngx.location.capture

Thank you for the suggestion :)

Best regards,
-agentzh