Differences in response body encoding from two different services?

[Roderik van der Veer]

I have 2 services on kubernetes (software not written by me), one in java, one in go. Both use JSONRPC with the same methods. 

I capture request and response bodies for logging purposes with the following loa script mounted in nginx-ingress

local ngx = ngx

local _M = {}

function _M.body_filter()

  if ngx.header['content-type'] == 'application/json' then

    ngx.var.req_body = ngx.req.get_body_data()

    ngx.var.resp_body = string.sub(ngx.arg[1], 1, 1000)

    local resp_body = string.sub(ngx.arg[1], 1, 1000)

    ngx.ctx.buffered = (ngx.ctx.buffered or "") .. resp_body

    if ngx.arg[2] then

      ngx.var.resp_body = ngx.ctx.buffered

    end

  end

end

return _M

The go one has {"jsonrpc":"2.0","id":0,"result":"0xc42df5"} in the resp_body in the first line where we set it.

The java one has \u001F\b\u0000\u0000\u0000\u0000\u0000\u0000\u0000RPP*+*HVRRP23P\u0001\tf\u0006`vQjqiN\tXޠ$Q\u0016\u0000\u0000\u0000\u0003\u0000Ǿ8\u0000\u0000\u0000 or when using ngx log a buch of question marks in my terminal. 

Using curl -i -vvv returns both times a json object, with no difference in headers, content types etc.

It appears the java response is binary encoded in some way, but it escapes me how i can 'decode' it so it becomes readable in my logs.

[Rainer Canavan]

On Mon, Jul 19, 2021 at 2:15 PM Roderik van der Veer
<rod...@settlemint.com> wrote:
>
> I have 2 services on kubernetes (software not written by me), one in java, one in go. Both use JSONRPC with the same methods.

[...]

> The go one has {"jsonrpc":"2.0","id":0,"result":"0xc42df5"} in the resp_body in the first line where we set it.
>
> The java one has \u001F\b\u0000\u0000\u0000\u0000\u0000\u0000\u0000RPP*+*HVRRP23P\u0001\tf\u0006`vQjqiN\tXޠ$Q\u0016\u0000\u0000\u0000\u0003\u0000Ǿ8\u0000\u0000\u0000 or when using ngx log a buch of question marks in my terminal.
>
> Using curl -i -vvv returns both times a json object, with no difference in headers, content types etc.
>
> It appears the java response is binary encoded in some way, but it escapes me how i can 'decode' it so it becomes readable in my logs.

Looks like the java response uses 'content-encoding: gzip'. Check the
accept-encoding request header and the vary response header as well.
It is unclear how your "logged" response differs from what you are
doing with curl, since you haven't posted any request or response
headers at all. If you're going to reproduce a problem with curl,
always include all the request headers of the original request, e.g.
via appropriate -H ... statements, even if you believe they aren't
relevant.

rainer