Exception at NginxJavaBodyFilter when "Accept-Encoding:gzip" on last part

[Georgios Papageorgiou]

Hello,

I have the above groovy NginxJavaBodyFilter and i want to read the response body from the remote server and maybe to change it in condition. The response body is encoded as gzip.

public class ResponseFilterBody implements NginxJavaBodyFilter {

public ResponseFilterBody() {

}

@Override

public Object[] doFilter(Map<String, Object> request, InputStream bodyChunk, boolean isLast) throws IOException {

System.err.println("============= Inside invoke ResponseFilterBody =============");

System.err.println(isLast);

System.err.println(ungzip(bodyChunk));

((NginxChainWrappedInputStream)bodyChunk).rewind();

if (isLast) {

return [200, null, bodyChunk];

}else {

return [null, null, bodyChunk];

}

}

private String ungzip(InputStream is) throws Exception{

InputStreamReader isr = new InputStreamReader(new GZIPInputStream(is), "UTF-8");

StringWriter sw = new StringWriter();

char[] chars = new char[1024];

for (int len; (len = isr.read(chars)) > 0; ) {

sw.write(chars, 0, len);

}

return sw.toString();

}

}

Here is the logged response in error.log file.

============= Inside invoke ResponseFilterBody =============

false

[{"name":"George","streetAddress":"St Georges 18"}]

============= Inside invoke ResponseFilterBody =============

true

2017-12-18 16:29:51[error][6552][main]server unhandled exception!

java.io.EOFException

at java.util.zip.GZIPInputStream.readUByte(GZIPInputStream.java:268)

at java.util.zip.GZIPInputStream.readUShort(GZIPInputStream.java:258)

at java.util.zip.GZIPInputStream.readHeader(GZIPInputStream.java:164)

at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:79)

at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:91)

at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)

at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

at java.lang.reflect.Constructor.newInstance(Constructor.java:423)

at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:83)

at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrapNoCoerce.callConstructor(ConstructorSite.java:105)

at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:247)

at gr.test.body.ResponseFilterBody.ungzip(ResponseFilterBody.groovy:38)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:210)

at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:59)

at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:166)

at gr.test.body.ResponseFilterBody.doFilter(ResponseFilterBody.groovy:26)

at nginx.clojure.java.NginxJavaHandler.process(NginxJavaHandler.java:114)

at nginx.clojure.NginxSimpleHandler.handleRequest(NginxSimpleHandler.java:187)

at nginx.clojure.NginxSimpleHandler.execute(NginxSimpleHandler.java:105)

at nginx.clojure.NginxClojureRT.eval(NginxClojureRT.java:1133)

2017/12/18 16:29:51 [alert] 6552#11256: *4 zero size buf in writer t:1 r:0 f:0 0000000001CC4DA0 0000000001CC516B-0000000001CC516B 0000000000000000 0-0 while sending to client, client: 127.0.0.1, server: localhost, request: "POST /test/api/rest/customer HTTP/1.1", upstream: "https://my.remote.server/test/api/rest/customer", host: "localhost:8080"

As you can see, if its the last part i can not ungzip the body.

When I comment this line System.err.println(ungzip(bodyChunk));

I do not see any error in the log, but i know that something went wrong, as i get the exception in the client side.

Caused by: java.net.SocketException: Unexpected end of file from server

at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:86)

at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:58)

at java.lang.reflect.Constructor.newInstance(Constructor.java:542)

at sun.net.www.protocol.http.HttpURLConnection$6.run(HttpURLConnection.java:1689)

at sun.net.www.protocol.http.HttpURLConnection$6.run(HttpURLConnection.java:1687)

at java.security.AccessController.doPrivileged(AccessController.java:452)

at sun.net.www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java:1685)

at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1258)

at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:95)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:56)

at java.lang.reflect.Method.invoke(Method.java:620)

at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)

at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)

at org.codehaus.groovy.runtime.metaclass.MethodMetaProperty$GetBeanMethodMetaProperty.getProperty(MethodMetaProperty.java:76)

at org.codehaus.groovy.runtime.callsite.GetEffectivePojoPropertySite.getProperty(GetEffectivePojoPropertySite.java:64)

at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callGetProperty(AbstractCallSite.java:296)

at wslite.http.HTTPClient.buildResponse(HTTPClient.groovy:152)

at wslite.http.HTTPClient.this$2$buildResponse(HTTPClient.groovy)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:95)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:56)

at java.lang.reflect.Method.invoke(Method.java:620)

at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:210)

at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:59)

at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:52)

at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:154)

at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:174)

at wslite.http.HTTPClient.execute(HTTPClient.groovy:60)

at wslite.http.HTTPClient$execute.call(Unknown Source)

at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)

at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)

at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)

at wslite.rest.RESTClient.executeMethod(RESTClient.groovy:105)

... 17 more

Caused by: java.net.SocketException: Unexpected end of file from server

at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:808)

at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:663)

at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:805)

at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:663)

at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1337)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:95)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:56)

at java.lang.reflect.Method.invoke(Method.java:620)

at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)

at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)

at org.codehaus.groovy.runtime.metaclass.MethodMetaProperty$GetBeanMethodMetaProperty.getProperty(MethodMetaProperty.java:76)

at org.codehaus.groovy.runtime.callsite.GetEffectivePojoPropertySite.getProperty(GetEffectivePojoPropertySite.java:64)

at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callGetProperty(AbstractCallSite.java:296)

at wslite.http.HTTPClient.execute(HTTPClient.groovy:55)

... 22 more

If I remove the body filter from the nginx conf I get the correct body at the rest client side.

Thanks for the help.

[Yuexiang Zhang]

Hi, there're two problems.

1. "System.err.println(ungzip(bodyChunk));" won't work every time because  a bodyChunk is not always a complete valid gzip data but part of gzip data sometimes.

2. "zero size buf in writer" is a bug of nginx-clojure since v0.4.5. It should be fixed in the master branch by the latest commit.

Please try again.

--
You received this message because you are subscribed to the Google Groups "Nginx-Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nginx-clojure+unsubscribe@googlegroups.com.
To post to this group, send email to nginx-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nginx-clojure/eef49cc3-912e-42bd-a26a-31b1324c152a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Georgios Papageorgiou]

Hi,

I have already tried the following with no success.

1 ) Download the new src and build nginx-clojure.jar

2) Replace the old jar with the new one in the ‘jars’ and ‘libs’ folders where I start nginx-win64.exe

Your commit is a change in a .c file so maybe I have to rebuild my nginx instance for windows like here for the change to take effect?

Thanks

To unsubscribe from this group and stop receiving emails from it, send an email to nginx-clojur...@googlegroups.com.