String replacement in body_filter_by_lua
3,832 views
Skip to first unread message
Guilherme
Nov 9, 2012, 5:05:58 PM11/9/12
to openre...@googlegroups.com
Hello!
I'm trying to make string replacement in body_filter_by_lua, but the replaced content is being truncated. The mainly purpose of this is to append, dynamically (per website), some javascripts in response body.
body_filter_by_lua
'
replacestr = "<script>sdsdsdsdsd</script></head>"
ngx.arg[1] = ngx.re.sub(ngx.arg[1],"</head>", replacestr)
';
The original html version:
<html>
<head>
</head>
<body>
test
</body>
</html>
After body_filter_by_lua:
<html>
<head>
<script>sdsdsdsd
It looks like body_filter_by_lua is truncating in 16bytes.
Any idea?
I made this kind of replacement in response body using nginx_substitution_filter (3rd party), and it worked, but it's less dynamic than lua (I cannot use ngx.ctx to use variables from other phases, for example).
What is the recommended approach in this case?
Best regards,
Guilherme
I'm trying to make string replacement in body_filter_by_lua, but the replaced content is being truncated. The mainly purpose of this is to append, dynamically (per website), some javascripts in response body.
body_filter_by_lua
'
replacestr = "<script>sdsdsdsdsd</script></head>"
ngx.arg[1] = ngx.re.sub(ngx.arg[1],"</head>", replacestr)
';
The original html version:
<html>
<head>
</head>
<body>
test
</body>
</html>
After body_filter_by_lua:
<html>
<head>
<script>sdsdsdsd
It looks like body_filter_by_lua is truncating in 16bytes.
Any idea?
I made this kind of replacement in response body using nginx_substitution_filter (3rd party), and it worked, but it's less dynamic than lua (I cannot use ngx.ctx to use variables from other phases, for example).
What is the recommended approach in this case?
Best regards,
Guilherme
agentzh
Nov 9, 2012, 6:14:25 PM11/9/12
to openre...@googlegroups.com
Hello!
On Fri, Nov 9, 2012 at 2:05 PM, Guilherme wrote:
> I'm trying to make string replacement in body_filter_by_lua, but the
> replaced content is being truncated. The mainly purpose of this is to
> append, dynamically (per website), some javascripts in response body.
>
>
> body_filter_by_lua
> '
> replacestr = "<script>sdsdsdsdsd</script></head>"
> ngx.arg[1] = ngx.re.sub(ngx.arg[1],"</head>", replacestr)
> ';
>
> The original html version:
>
> <html>
> <head>
> </head>
> <body>
> test
> </body>
> </html>
>
> After body_filter_by_lua:
>
> <html>
> <head>
> <script>sdsdsdsd
>
> It looks like body_filter_by_lua is truncating in 16bytes.
>
Does your original response contain a Content-Length response header?
If yes, then you should clear it first in header_filter_by_lua,
because you change the response body length.
You can try something like this:
location = /test.html {
header_filter_by_lua '
ngx.header.content_length = nil
';
body_filter_by_lua
'
replacestr = "<script>sdsdsdsdsd</script></head>"
ngx.arg[1] = ngx.re.sub(ngx.arg[1],"</head>", replacestr)
';
}
Best regards,
-agentzh
On Fri, Nov 9, 2012 at 2:05 PM, Guilherme wrote:
> I'm trying to make string replacement in body_filter_by_lua, but the
> replaced content is being truncated. The mainly purpose of this is to
> append, dynamically (per website), some javascripts in response body.
>
>
> body_filter_by_lua
> '
> replacestr = "<script>sdsdsdsdsd</script></head>"
> ngx.arg[1] = ngx.re.sub(ngx.arg[1],"</head>", replacestr)
> ';
>
> The original html version:
>
> <html>
> <head>
> </head>
> <body>
> test
> </body>
> </html>
>
> After body_filter_by_lua:
>
> <html>
> <head>
> <script>sdsdsdsd
>
> It looks like body_filter_by_lua is truncating in 16bytes.
>
If yes, then you should clear it first in header_filter_by_lua,
because you change the response body length.
You can try something like this:
location = /test.html {
header_filter_by_lua '
ngx.header.content_length = nil
';
body_filter_by_lua
'
replacestr = "<script>sdsdsdsdsd</script></head>"
ngx.arg[1] = ngx.re.sub(ngx.arg[1],"</head>", replacestr)
';
}
-agentzh
Guilherme
Nov 9, 2012, 9:17:05 PM11/9/12
to openre...@googlegroups.com
Great! Now it worked!
Tks,
Guilherme
Tks,
Guilherme
--
You received this message because you are subscribed to the Google Groups "openresty-en" group.
To unsubscribe from this group, send email to openresty-en...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Viktor malezink
Jul 8, 2013, 11:05:11 PM7/8/13
to openre...@googlegroups.com
Hi
I have very strange bug with this.
When i replace like that
local str = ngx.arg[1]
local script = "<script>alert('Hello');</script></body>"
ngx.arg[1] = ngx.re.gsub(str,'</body>',script)
So when i proxy site like nginx.com(here config)
location = / {
proxy_pass http://nginx.com/;
proxy_set_header Host "nginx.com";
proxy_ignore_headers Expires Cache-Control;
header_filter_by_lua 'ngx.header.content_length = nil';
body_filter_by_lua_file conf/filter.lua;
}
location ~ \.(png|jpg|jpeg|gif|js|css|ico|bmp)$ {
rewrite ^ http://nginx.com/$request_uri? permanent;
}
Than i see result in browser, but when i try to proxy something like google.co.uk or some other sites.
location = / {
proxy_pass http://www.google.co.uk/;
proxy_set_header Host "www.google.co.uk";
proxy_ignore_headers Expires Cache-Control;
header_filter_by_lua 'ngx.header.content_length = nil';
body_filter_by_lua_file conf/filter.lua;
}
location ~ \.(png|jpg|jpeg|gif|js|css|ico|bmp)$ {
rewrite ^ http://www.google.co.uk/$request_uri? permanent;
}
Ain't see changes in html from browsers(Chrome,SrWare,Firefox,IE,Opera) but when i load page from curl or wget i see that html has changed.
So i decided(maybe mistake) that webbrowser ignore(how?) html from openresty and load from origin source. But how to deal with that?
Thanks.
Yichun Zhang (agentzh)
Jul 8, 2013, 11:21:46 PM7/8/13
to openresty-en
Hello!
On Mon, Jul 8, 2013 at 8:05 PM, Viktor malezink wrote:
> Ain't see changes in html from browsers(Chrome,SrWare,Firefox,IE,Opera) but
> when i load page from curl or wget i see that html has changed.
> So i decided(maybe mistake) that webbrowser ignore(how?) html from openresty
> and load from origin source. But how to deal with that?
One major difference between curl/wget and your web browsers is that
the former does not send the "Accept-Encoding: gzip" request header by
default. If your backend server compresses the HTTP response, nginx
will not automatically uncompress the data before feeding it into its
output filter chain. You can also test this case with curl by
specifying the --compressed command-line option, as in
curl --compressed http://localhost:8080/
One work-around is to disable gzip compression on your backend servers
by adding the following line to your nginx.conf:
proxy_set_header Accept-Encoding "";
And you can still enable gzip compression on your nginx side because
the standard ngx_gzip module runs *after* body_filter_by_lua.
Regards,
-agentzh
On Mon, Jul 8, 2013 at 8:05 PM, Viktor malezink wrote:
> Ain't see changes in html from browsers(Chrome,SrWare,Firefox,IE,Opera) but
> when i load page from curl or wget i see that html has changed.
> So i decided(maybe mistake) that webbrowser ignore(how?) html from openresty
> and load from origin source. But how to deal with that?
the former does not send the "Accept-Encoding: gzip" request header by
default. If your backend server compresses the HTTP response, nginx
will not automatically uncompress the data before feeding it into its
output filter chain. You can also test this case with curl by
specifying the --compressed command-line option, as in
curl --compressed http://localhost:8080/
One work-around is to disable gzip compression on your backend servers
by adding the following line to your nginx.conf:
proxy_set_header Accept-Encoding "";
And you can still enable gzip compression on your nginx side because
the standard ngx_gzip module runs *after* body_filter_by_lua.
Regards,
-agentzh
Viktor malezink
Jul 9, 2013, 12:14:11 AM7/9/13
to openre...@googlegroups.com
You are amazingly clever and experienced man, and yeah, you absolutely right.
It's unfortunate that my ignorance is so great.
Thanks for the enlightenment.
Reply all
Reply to author
Forward
0 new messages