Nginx Crashes As Ngx_pagespeed Can Not Start Worker Thread

Yalçın Özveren

unread,

Nov 1, 2014, 1:45:38 PM11/1/14

to ngx-pagesp...@googlegroups.com

Hi,

I had a problem with ngx_pagespeed described below:

Error: As I realised that my Nginx server was down, I restarted it and then read the log file (/var/log/nginx/nginx.log). In the logs I saw that ngx_pagespeed threw some warnings about gzip ("pagespeed: cannot set gzip, command not found in /etc/nginx/nginx.conf:10") and some alerts about open sockets ("*3887621 open socket #59 left in connection 17") for the last few days. And today after it threw an alert about worker thread initialization ("2014/11/01 17:10:24 [error] 14745#0: [ngx_pagespeed 1.9.32.1-4238] [1101/171024:ERROR:worker.cc(120)] Unable to start worker thread"), the server stopped working. I pasted a more detailed output below.

OS: CentOS release 6.6 (Final)

LSB_VERSION=base-4.0-amd64:base-4.0-noarch:core-4.0-amd64:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-noarch

CentOS release 6.6 (Final)

Nginx Version: 1.6.1

Ngx_pagespeed Version: 1.9.32

Some Output Examples:

...............................................................................

2014/10/26 09:55:59 [warn] 3263#0: [ngx_pagespeed 1.9.32.1-4238] [1026/095559:WARNING:resource_fetch.cc(197)] Fetch failed for resource url http://www.oyunuoyna.com.tr/uploads/images/cay-evi.jpg.pagespeed.ce.jhenHtGTbi.jpg

2014/10/26 10:38:26 [warn] 3257#0: [ngx_pagespeed 1.9.32.1-4238] Fetch timed out: http://savas.oyunuoyna.com.tr/assets/images/sosyal.png (1) waiting for 50 ms

2014/10/26 10:40:22 [warn] 3260#0: [ngx_pagespeed 1.9.32.1-4238] Fetch timed out: http://macera.oyunuoyna.com.tr/assets/images/sosyal.png (1) waiting for 50 ms

2014/10/26 10:44:15 [warn] 3258#0: [ngx_pagespeed 1.9.32.1-4238] Fetch timed out: http://2kisilik.oyunuoyna.com.tr/assets/images/sosyal.png (1) waiting for 50 ms

2014/10/26 10:45:38 [warn] 3267#0: [ngx_pagespeed 1.9.32.1-4238] Fetch timed out: http://macera.oyunuoyna.com.tr/assets/images/sosyal.png (1) waiting for 50 ms

2014/10/26 10:55:48 [warn] 3256#0: [ngx_pagespeed 1.9.32.1-4238] Fetch timed out: http://aslan.oyunuoyna.com.tr/assets/images/sosyal.png (1) waiting for 50 ms

2014/10/26 11:02:56 [warn] 2339#0: pagespeed: cannot set gzip, command not found in /etc/nginx/nginx.conf:10

2014/10/26 11:02:56 [alert] 3263#0: *3838137 open socket #68 left in connection 29

2014/10/26 11:02:56 [alert] 3263#0: *3838140 open socket #70 left in connection 52

2014/10/26 11:02:56 [alert] 3263#0: aborting

2014/10/26 11:03:10 [alert] 3265#0: *3731861 open socket #59 left in connection 13

2014/10/26 11:03:10 [alert] 3265#0: *3839866 open socket #91 left in connection 57

2014/10/26 11:03:10 [alert] 3265#0: *3839882 open socket #104 left in connection 98

2014/10/26 11:03:10 [alert] 3265#0: *3839884 open socket #110 left in connection 105

2014/10/26 11:03:10 [alert] 3265#0: *3839881 open socket #101 left in connection 128

.................................................................................

2014/10/29 15:43:24 [alert] 10340#0: *2842118 open socket #205 left in connection 123

2014/10/29 15:43:24 [alert] 10340#0: aborting

NgxBaseFetch::RequestCollection: Broken pipe

2014/10/29 17:05:02 [warn] 28101#0: pagespeed: cannot set gzip, command not found in /etc/nginx/nginx.conf:10

2014/10/29 17:06:40 [alert] 2835#0: *3170881 open socket #60 left in connection 28

2014/10/29 17:06:40 [alert] 2835#0: aborting

NgxBaseFetch::RequestCollection: Broken pipe

2014/10/29 19:01:47 [warn] 28101#0: pagespeed: cannot set gzip, command not found in /etc/nginx/nginx.conf:10

...................................................................................

2014/11/01 17:10:20 [warn] 407#0: [ngx_pagespeed 1.9.32.1-4238] [1101/171020:WARNING:serf_url_async_fetcher.cc(1196)] Aborting fetch of http://3d-dovus.oyunuoyna.com.tr/assets/images/star_full.png

2014/11/01 17:10:20 [warn] 407#0: [ngx_pagespeed 1.9.32.1-4238] [1101/171020:WARNING:serf_url_async_fetcher.cc(1196)] Aborting fetch of http://3d-dovus.oyunuoyna.com.tr/assets/images/add_icon.png

terminate called after throwing an instance of 'St9bad_alloc'

what(): std::bad_alloc

2014/11/01 17:10:23 [alert] 14498#0: worker process 14499 exited on signal 6 (core dumped)

terminate called after throwing an instance of 'St9bad_alloc'

what(): std::bad_alloc

2014/11/01 17:10:23 [alert] 14498#0: worker process 14501 exited on signal 6 (core dumped)

terminate called after throwing an instance of 'St9bad_alloc'

what(): std::bad_alloc

.................................................................................................

2014/11/01 17:10:24 [error] 14745#0: [ngx_pagespeed 1.9.32.1-4238] [1101/171024:ERROR:worker.cc(120)] Unable to start worker thread

2014/11/01 17:10:24 [alert] 14745#0: [ngx_pagespeed 1.9.32.1-4238] [1101/171024:FATAL:serf_url_async_fetcher.cc(819)] Check failed: 0 == apr_thread_create(&thread_id_, __null, SerfThreadFn, this, pool_) (0 vs. 11)

nginx: worker process() [0x4ae7ea]

nginx: worker process() [0x47c2d6]

nginx: worker process() [0x4ab702]

nginx: worker process() [0x776d00]

nginx: worker process() [0x5b1bb1]

nginx: worker process() [0x7732f1]

nginx: worker process() [0x7bf5c9]

..............................................................................................

2014/11/01 17:10:27 [alert] 14498#0: worker process 15038 exited on signal 6 (core dumped)

2014/11/01 17:10:28 [error] 15057#0: [ngx_pagespeed 1.9.32.1-4238] [1101/171028:ERROR:worker.cc(120)] Unable to start worker thread

2014/11/01 17:10:29 [error] 15057#0: [ngx_pagespeed 1.9.32.1-4238] [1101/171029:ERROR:worker.cc(120)] Unable to start worker thread

2014/11/01 17:10:32 [error] 15057#0: [ngx_pagespeed 1.9.32.1-4238] [1101/171032:ERROR:worker.cc(120)] Unable to start worker thread

2014/11/01 17:10:36 [warn] 14498#0: pagespeed: cannot set gzip, command not found in /etc/nginx/nginx.conf:10

terminate called after throwing an instance of 'St9bad_alloc'

what(): std::bad_alloc

2014/11/01 17:10:37 [alert] 14498#0: worker process 15145 exited on signal 6 (core dumped)

terminate called after throwing an instance of 'St9bad_alloc'

what(): std::bad_alloc

2014/11/01 17:10:37 [alert] 14498#0: worker process 15140 exited on signal 6 (core dumped)

terminate called after throwing an instance of 'St9bad_alloc'

what(): std::bad_alloc

2014/11/01 17:10:37 [alert] 14498#0: worker process 15147 exited on signal 6 (core dumped)

terminate called after throwing an instance of 'St9bad_alloc'

what(): std::bad_alloc

2014/11/01 17:10:37 [alert] 14498#0: worker process 15222 exited on signal 6 (core dumped)

terminate called after throwing an instance of 'St9bad_alloc'

what(): std::bad_alloc

and the "terminate called after throwing..." errors goes all the way till the end of file for many lines.

Otto van der Schaaf

unread,

Nov 4, 2014, 7:00:49 AM11/4/14

to ngx-pagesp...@googlegroups.com

Are you by chance running SELinux?

That might explain the messages about threads failing to start in the logs you posted.

Otto

--
You received this message because you are subscribed to the Google Groups "ngx-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ngx-pagespeed-di...@googlegroups.com.
Visit this group at http://groups.google.com/group/ngx-pagespeed-discuss.
For more options, visit https://groups.google.com/d/optout.

Yalçın Özveren

unread,

Nov 6, 2014, 9:07:47 AM11/6/14

to ngx-pagesp...@googlegroups.com

Our server is running CentOS 6.6.

To unsubscribe from this group and stop receiving emails from it, send an email to ngx-pagespeed-discuss+unsub...@googlegroups.com.

Otto van der Schaaf

unread,

Nov 7, 2014, 5:17:57 PM11/7/14

to ngx-pagesp...@googlegroups.com

Sorry, let me rephrase, is SELinux enabled on your CentOS server?

You can check SELinux's status with this command:

sestatus | grep -i mode

If the output of that says that the current mode is enforcing, that means SELinux

could potentially interfere with ngx_pagespeed from operating as intended.

If SELinux turns out to be enforcing its policies, I would expect traces in the log for

operations being disallowed in the logs (probably /var/log/audit/audit.log), so inspecting

what is logged there when nginx fails might be helpful.

Otto

To unsubscribe from this group and stop receiving emails from it, send an email to ngx-pagespeed-di...@googlegroups.com.

Reply all

Reply to author

Forward