Disable URL encode when writing to access log

[Gil Luz]

Hi.

I have an openresty that creates  a json value in lua code and write it to the access log by transfer it as nginx variable.

This access log is then transfered to logstash and loaded to elastic search.

My problem is that the quotes are getting url encoded so it's a problem for me to load it as json file.

Currently I logrotate it and then perform an awk command to fix it but it's harming my near real time ability.

Is there an option to disable the url encode or to write in other way to a file while keeping good performance?

Thanks in advance.

Gil

[Yichun Zhang (agentzh)]

Hello!

On Tue, Jun 16, 2015 at 5:41 PM, Gil Luz wrote:
> I have an openresty that creates a json value in lua code and write it to
> the access log by transfer it as nginx variable.
> This access log is then transfered to logstash and loaded to elastic search.
> My problem is that the quotes are getting url encoded so it's a problem for
> me to load it as json file.
> Currently I logrotate it and then perform an awk command to fix it but it's
> harming my near real time ability.
>

Well, I guess you're getting things like \x22\xE4 in your access logs.
Well, it's not URL encoding but NGINX's own access log encoding for
special chars in logs.

There's no easy way to disable that because it's hard-coded in the
nginx core (though OpenResty's bundled version of nginx does provide a
log_escape_non_ascii configuration directive to tune the behaviour a
bit).

One recommended way is to use custom logger like below:

https://github.com/cloudflare/lua-resty-logger-socket

This can bypass the nginx's access logging completely. It is also 100%
nonblocking on I/O and can also hook up with syslog-ng, for example.

Regards,
-agentzh

[Gil Luz]

Thanks. I will like into it.

--
You received this message because you are subscribed to the Google Groups "openresty-en" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openresty-en...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.