Request Hijack and Chunked Transfer Support

[vsen]

Hi,

We have followed the implementation of StreamingWriteHandler from

 https://github.com/nginx-clojure/nginx-clojure/blob/master/test/java/nginx/clojure/java/StreamingWriteHandler.java

We are observing that data is written to the downstream in smaller byte array but we also observed that gzip and chunked transfer is not applied

i.e. we are are not seeing Transfer-Encoding : chunked in the response header

with request.hijack(false) we were expecting that nginx will intercept the response and apply the filters like gzip, chunked transfer.

Can anyone help us to achieve Chunked transfer , Gzip with Request hijack

Thanks in advance

vsen

[Yuexiang Zhang]

Hi,

NginxHttpServerChannel.write is of the low level APIs and only send the raw data to the client, e.g. websocket data.

Please try the high level API such as NginxHttpServerChannel.sendHeader/send/sendBody, e.g.

NginxHttpServerChannel channel = r.hijack(false);

channel.sendHeader(200, null, true, false);

channel.send("first part.\r\n", true, false);

channel.send("second part.\r\n", true, false);

channel.send("third part.\r\n", true, false);

channel.send("last part.\r\n", true, true);

--
You received this message because you are subscribed to the Google Groups "Nginx-Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nginx-clojure+unsubscribe@googlegroups.com.
To post to this group, send email to nginx-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nginx-clojure/ddb38755-013a-4909-a840-6c44ffaf38e1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[vsen]

Hi,

Thanks for clarification, with the suggested changes we were able to see NGINX intercepting the hijacked request and we are seeing the below response header that is related to gzip and chunked transfer. But the issue we are observing is that there is no chunk size before the content. Instead the size is getting added to the end. Because of which the GET request fails with 0 bytes transfer after the header is received.

Can you please let us know how to specify the chunk-size at nginx level?

Or how do we ensure the chunksize is written before the content?

HTTP/1.1 200 OK

Date: Fri, 07 Jul 2017 12:22:53 GMT

Content-Type: application/vnd.hp-pcl

Transfer-Encoding: chunked

Connection: close

Vary: Accept-Encoding

Server: nginx-clojure/0.4.3

X-my-header-L: my-header-content-L

..............................

.........................................................................................2bdc6

Curl gives the below output

{ [data not shown]
* Problem (2) in the Chunked-Encoded data
  0     0    0     0    0     0      0      0 --:--:--  0:00:04 --:--:--     0
* Closing connection 0
curl: (56) Problem (2) in the Chunked-Encoded data

Please let me know what is going wrong here...

Thanks

vsen

On Thursday, 6 July 2017 23:30:43 UTC+5:30, xfeep wrote:

Hi,

NginxHttpServerChannel.write is of the low level APIs and only send the raw data to the client, e.g. websocket data.

Please try the high level API such as NginxHttpServerChannel.sendHeader/send/sendBody, e.g.

NginxHttpServerChannel channel = r.hijack(false);

channel.sendHeader(200, null, true, false);

channel.send("first part.\r\n", true, false);

channel.send("second part.\r\n", true, false);

channel.send("third part.\r\n", true, false);

channel.send("last part.\r\n", true, true);

On Thu, Jul 6, 2017 at 9:01 PM, vsen <senth...@gmail.com> wrote:

Hi,

We have followed the implementation of StreamingWriteHandler from

 https://github.com/nginx-clojure/nginx-clojure/blob/master/test/java/nginx/clojure/java/StreamingWriteHandler.java

We are observing that data is written to the downstream in smaller byte array but we also observed that gzip and chunked transfer is not applied

i.e. we are are not seeing Transfer-Encoding : chunked in the response header

with request.hijack(false) we were expecting that nginx will intercept the response and apply the filters like gzip, chunked transfer.

Can anyone help us to achieve Chunked transfer , Gzip with Request hijack

Thanks in advance

vsen

--
You received this message because you are subscribed to the Google Groups "Nginx-Clojure" group.

To unsubscribe from this group and stop receiving emails from it, send an email to nginx-clojur...@googlegroups.com.

[vsen]

For the below sample code, the tcpdump is captured where we could see the first chunk getting repeated once without the chunk size and once with the chunk size.

How to avoid the duplicate first chunk?

Note: same behavior is observed irrespective of we adding \r\n to the end of each chunk.

Sample code :

-----------------------------------------------------

private void doWriteChunked(NginxHttpServerChannel serverChannel) throws IOException {

    
    serverChannel.sendHeader(NGX_HTTP_OK,null,true, false);
    serverChannel.send("FirstChunk.",true, false);
    serverChannel.send("SecondChunk.",true, false);
    serverChannel.send("ThirdChunk.",true, false);
    serverChannel.send("LastChunk.",true, true);
}

TCP Packet :

------------------------------------------------------

HTTP/1.1 200 OK

Date: Tue, 11 Jul 2017 06:53:55 GMT

Content-Type: application/octet-stream

Transfer-Encoding: chunked

Connection: close

Server: nginx-clojure/0.4.3

X-my-header-L: my-header-content-L

FirstChunk.b

FirstChunk.

c

SecondChunk.

b

ThirdChunk.

a

LastChunk.

0

-------------------------------------------------------

[Yuexiang Zhang]

Hi,

Did you try nginx-clojure v0.4.5 ?

I can not reproduce it. The similar example is just one of our unit tests.

To unsubscribe from this group and stop receiving emails from it, send an email to nginx-clojure+unsubscribe@googlegroups.com.

To post to this group, send email to nginx-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/nginx-clojure/af520761-e500-428e-a2c7-ae943c1bf134%40googlegroups.com.