Build as dynamic module, against nginx mainline

[Frederic Trudeau]

Greetings

I'm having a hard time building ngx_pagespeed against official nginx Mainline version

( aka ppa:nginx/development ), using build_ngx_pagespeed.sh located here https://github.com/pagespeed/ngx_pagespeed/blob/master/scripts/build_ngx_pagespeed.sh

For instance, by passing the additional arguments ( -a flag ) and trying to change the builddir ( -b flag ), the script cannot find the existing modules ( ex : /build/nginx-y2T5Oj/nginx-1.11.9/debian/modules/nginx-auth-pam   ) 

The idea is to use *exactly* the official pre-build nginx version provided byt the PPA, and simply add ngx_pagespeed as a dynamic module module, without having to rebuild nginx at all.

Is it at all possible ? 

= = =

# nginx -V

nginx version: nginx/1.11.9

built with OpenSSL 1.0.2g  1 Mar 2016

TLS SNI support enabled

configure arguments: --with-cc-opt='-g -O2 -fPIE -fstack-protector-strong -Wformat -Werror=format-security -fPIC -Wdate-time -D_FORTIFY_SOURCE=2' --with-ld-opt='-Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now -fPIC' --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --modules-path=/usr/lib/nginx/modules --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --with-debug --with-pcre-jit --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-http_v2_module --with-http_dav_module --with-http_slice_module --with-threads --with-http_addition_module --with-http_geoip_module=dynamic --with-http_gunzip_module --with-http_gzip_static_module --with-http_image_filter_module=dynamic --with-http_sub_module --with-http_xslt_module=dynamic --with-stream=dynamic --with-stream_ssl_module --with-mail=dynamic --with-mail_ssl_module --add-dynamic-module=/build/nginx-y2T5Oj/nginx-1.11.9/debian/modules/nginx-auth-pam --add-dynamic-module=/build/nginx-y2T5Oj/nginx-1.11.9/debian/modules/nginx-dav-ext-module --add-dynamic-module=/build/nginx-y2T5Oj/nginx-1.11.9/debian/modules/nginx-echo --add-dynamic-module=/build/nginx-y2T5Oj/nginx-1.11.9/debian/modules/nginx-upstream-fair --add-dynamic-module=/build/nginx-y2T5Oj/nginx-1.11.9/debian/modules/ngx_http_substitutions_filter_module

[Otto van der Schaaf]

I think it is possible, but it may make life easier to only worry about configure options that actually matter when it comes to compatibility between the dynamic module and the nginx binary. 

You can leave out the --add-dynamic-module arguments when building ngx_pagespeed.so

Unfortunately, I can't find an exact list of which configure arguments are relevant here.

But the following may be worth trying, what happens when you only pass in just these?:

 --with-debug --with-pcre-jit --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-http_v2_module --with-http_dav_module --with-http_slice_module --with-threads --with-http_addition_module --with-http_geoip_module=dynamic --with-http_gunzip_module --with-http_gzip_static_module --with-http_image_filter_module=dynamic --with-http_sub_module --with-http_xslt_module=dynamic --with-stream=dynamic --with-stream_ssl_module --with-mail=dynamic --with-mail_ssl_module 

To figure out the exact arguments we should worry about here, one approach may be to search the nginx code base looking for the string NGX_COMPAT. Because I think that define affects all the possible structures that are involved. And then the next step would be to figure out which configure arguments the fields that are conditionally included in the structs relate to, because these are the ones that affect compatibility. 

If we have an exact list of arguments we need to consider for passing through for compatibility reasons, then I suspect it's not very hard to add an option to build_ngx_pagespeed.sh to automatically do that for you when building the dynamic module version.

Otto

[Frederic Trudeau]

Thanks for the reply,

I first tried this :

# ./build_ngx_pagespeed.sh -y -v 1.11.33.4 -m -n 1.11.9 -a "--with-debug --with-pcre-jit --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-http_v2_module --with-http_dav_module --with-http_slice_module --with-threads --with-http_addition_module --with-http_geoip_module=dynamic --with-http_gunzip_module --with-http_gzip_static_module --with-http_image_filter_module=dynamic --with-http_sub_module --with-http_xslt_module=dynamic --with-stream=dynamic --with-stream_ssl_module --with-mail=dynamic --with-mail_ssl_module"

After moving the compiled ngx_pagespeed.so module to /usr/lib/nginx/modules, I tried restarting the original untouched nginx binary, and got this error :

Mar 10 16:09:24 img01 systemd[1]: Starting A high performance web server and a reverse proxy server...

Mar 10 16:09:24 img01 nginx[31803]: nginx: [emerg] module "/usr/share/nginx/modules/ngx_pagespeed.so" is not binary compatible in /etc/nginx/nginx.conf:1

Mar 10 16:09:24 img01 nginx[31803]: nginx: configuration file /etc/nginx/nginx.conf test failed

Mar 10 16:09:24 img01 systemd[1]: nginx.service: Control process exited, code=exited status=1

Mar 10 16:09:24 img01 systemd[1]: Failed to start A high performance web server and a reverse proxy server.

Mar 10 16:09:24 img01 systemd[1]: nginx.service: Unit entered failed state.

Mar 10 16:09:24 img01 systemd[1]: nginx.service: Failed with result 'exit-code'.

I also tried this :

# ./build_ngx_pagespeed.sh -y -v 1.11.33.4 -m -n 1.11.9 -a "--with-cc-opt='-g -O2 -fPIE -fstack-protector-strong -Wformat -Werror=format-security -fPIC -Wdate-time -D_FORTIFY_SOURCE=2' --with-ld-opt='-Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now -fPIC' --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --modules-path=/usr/lib/nginx/modules --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --with-debug --with-pcre-jit --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-http_v2_module --with-http_dav_module --with-http_slice_module --with-threads --with-http_addition_module --with-http_geoip_module=dynamic --with-http_gunzip_module --with-http_gzip_static_module --with-http_image_filter_module=dynamic --with-http_sub_module --with-http_xslt_module=dynamic --with-stream=dynamic --with-stream_ssl_module --with-mail=dynamic --with-mail_ssl_module"

Again, same error message.

Of course, if I replace the current nginx binary with the compiled one, everything works fine, but that is not what I'm trying to achieve here.

FT

--
You received this message because you are subscribed to the Google Groups "ngx-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ngx-pagespeed-di...@googlegroups.com.
Visit this group at https://groups.google.com/group/ngx-pagespeed-discuss.
For more options, visit https://groups.google.com/d/optout.

[Otto van der Schaaf]

One more thing to try: it is possible the nginx server binary was actually configured with "--with-compat":

If that is the case, then adding it when building ngx_pagespeed as well should fix the problem:

 ./build_ngx_pagespeed.sh -y -v 1.11.33.4 -m -n 1.11.9 -a "--with-compat"

Otto

[Frederic Trudeau]

If it was configured with --with-compat, would'nt it be shown with nginx -V ?

Does not seam to be the case with nginx version: nginx/1.11.9 

[Otto van der Schaaf]

I tested that, --with-compat does show up in the output of nginx -V for version 1.11.9 when specified. 

Did you get a chance to diff the outputs of nginx -V for both the installed and newly build versions?

If that does not show any differences, all I can think of is that one of the compatibility flags is set based upon some environment condition of the build system.

It is hard to figure out which flag it is, because the signatures nginx uses to compare from the server binary and the dynamic module are not logged.

Otto

[Otto van der Schaaf]

Cross linking a potential cause for binary incompatibility of ngx_pagespeed build as a dynamic module with official nginx ppa versions (and how to address):

https://github.com/pagespeed/ngx_pagespeed/issues/1440#issuecomment-315520779

Otto

To unsubscribe from this group and stop receiving emails from it, send an email to ngx-pagespeed-discuss+unsub...@googlegroups.com.

Visit this group at https://groups.google.com/group/ngx-pagespeed-discuss.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "ngx-pagespeed-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ngx-pagespeed-discuss+unsub...@googlegroups.com.

Visit this group at https://groups.google.com/group/ngx-pagespeed-discuss.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "ngx-pagespeed-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ngx-pagespeed-discuss+unsub...@googlegroups.com.

Visit this group at https://groups.google.com/group/ngx-pagespeed-discuss.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "ngx-pagespeed-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ngx-pagespeed-discuss+unsub...@googlegroups.com.