Seeing 'Failed to read cache clean timestamp' multiple times on single request

Lauri Juusela

unread,

Mar 18, 2017, 9:35:48 PM3/18/17

to ngx-pagespeed-discuss

I'm having followed error given in pagespeed message history '[Warning] [1] Failed to read cache clean timestamp /var/cache/ngx_pagespeed/!clean!time!. Doing an extra cache clean to be safe.' multiple times on single request and many times withing minute. I'm having low traffic site. Also i think that cache misses are quite high about 75%. I think this might be reason for poor performance too. Serving pages without pagespeed (300 req/s) and with pagespeed about 30 req/s. I'm not very good with these things so its very probably my mistake somewhere.

Joshua Marantz

unread,

Mar 19, 2017, 5:39:09 PM3/19/17

to ngx-pagesp...@googlegroups.com

My first guess is that the process running Apache doesn't have write-access to /var/cache/ngx_pagespeed. Can you check that before we go any further with analysis?

I would expect, though, that you would see other messages in your logfile to that effect.

On Sat, Mar 18, 2017 at 9:35 PM, Lauri Juusela <lauri....@gmail.com> wrote:

I'm having followed error given in pagespeed message history '[Warning] [1] Failed to read cache clean timestamp /var/cache/ngx_pagespeed/!clean!time!. Doing an extra cache clean to be safe.' multiple times on single request and many times withing minute. I'm having low traffic site. Also i think that cache misses are quite high about 75%. I think this might be reason for poor performance too. Serving pages without pagespeed (300 req/s) and with pagespeed about 30 req/s. I'm not very good with these things so its very probably my mistake somewhere.

--
You received this message because you are subscribed to the Google Groups "ngx-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ngx-pagespeed-discuss+unsub...@googlegroups.com.
Visit this group at https://groups.google.com/group/ngx-pagespeed-discuss.
For more options, visit https://groups.google.com/d/optout.

Lauri Juusela

unread,

Mar 20, 2017, 1:48:32 AM3/20/17

to ngx-pagespeed-discuss

Its running on nginx and nginx process is running as root for now. Nginx process has write-access to /var/cache/ngx_pagespeed. Nginx pagespeed writes and shm_metadata_cache folder and files but nothing else to folder /var/cache/ngx_pagespeed. I'm using memcached as external cache so maybe this is reason? There is no other related messages in logfile. I'm mostly worried about this part of the warning -> "Doing an extra cache clean to be safe" because that might be the cause for performance issues?

Joshua Marantz

unread,

Mar 20, 2017, 7:58:15 AM3/20/17

to ngx-pagesp...@googlegroups.com

Can you paste the /pagespeed_admin/statistics values related to your cache? E.g.

wget -q -O - YOUR_SITE/pagespeed_admin/statistics|grep cache

and in particular your file cache:

wget -q -O - YOUR_SITE/pagespeed_admin/statistics|grep file_cache

To unsubscribe from this group and stop receiving emails from it, send an email to ngx-pagespeed-discuss+unsubscri...@googlegroups.com.

Visit this group at https://groups.google.com/group/ngx-pagespeed-discuss.
For more options, visit https://groups.google.com/d/optout.

Lauri Juusela

unread,

Mar 20, 2017, 10:39:11 AM3/20/17

to ngx-pagespeed-discuss

Here are the results from all cache related lines from statistics. ( using wget -q -O - YOUR_SITE/pagespeed_admin/statistics|grep cache )

cache_time_us: 8726720

cache_hits: 6680

cache_misses: 31694

cache_backend_hits: 7548

cache_backend_misses: 30826

cache_fallbacks: 0

cache_expirations: 850

cache_inserts: 2758

cache_deletes: 0

cache_extensions: 463845

not_cacheable: 0

font_service_input_resource_recent_uncacheable_miss: 0

font_service_input_resource_recent_uncacheable_failure: 0

in_place_uncacheable_rewrites: 0

num_local_storage_cache_candidates_found: 0

num_local_storage_cache_stored_total: 0

num_local_storage_cache_stored_images: 0

num_local_storage_cache_stored_css: 0

num_local_storage_cache_candidates_added: 0

num_local_storage_cache_candidates_removed: 0

url_input_resource_recent_uncacheable_miss: 0

url_input_resource_recent_uncacheable_failure: 14

rewrite_cached_output_missed_deadline: 10346

rewrite_cached_output_hits: 1632492

rewrite_cached_output_misses: 75298

resource_fetches_cached: 6071

num_cache_control_rewritable_resources: 9964

num_cache_control_not_rewritable_resources: 1307

ipro_not_in_cache: 28668

downstream_cache_purge_attempts: 0

successful_downstream_cache_purges: 0

cache_batcher_dropped_gets: 0

cache_batcher_coalesced_gets: 0

cache_batcher_queued_gets: 0

memcache_timeouts: 0

file_cache_bytes_freed_in_cleanup: 0

file_cache_cleanups: 0

file_cache_disk_checks: 0

file_cache_evictions: 0

file_cache_skipped_cleanups: 0

file_cache_started_cleanups: 0

file_cache_write_errors: 0

file_cache_deletes: 0

file_cache_hits: 0

file_cache_inserts: 0

file_cache_misses: 0

lru_cache_deletes: 0

lru_cache_hits: 0

lru_cache_inserts: 0

lru_cache_misses: 0

shm_cache_deletes: 0

shm_cache_hits: 0

shm_cache_inserts: 0

shm_cache_misses: 0

memcached_async_deletes: 0

memcached_async_hits: 0

memcached_async_inserts: 0

memcached_async_misses: 0

memcached_blocking_deletes: 0

memcached_blocking_hits: 0

memcached_blocking_inserts: 0

memcached_blocking_misses: 0

compressed_cache_corrupt_payloads: 0

compressed_cache_original_size: 52047430

compressed_cache_compressed_size: 11281423

pcache-cohorts-beacon_cohort_deletes: 0

pcache-cohorts-beacon_cohort_hits: 11473

pcache-cohorts-beacon_cohort_inserts: 6742

pcache-cohorts-beacon_cohort_misses: 653

pcache-cohorts-dom_deletes: 0

pcache-cohorts-dom_hits: 11012

pcache-cohorts-dom_inserts: 11383

pcache-cohorts-dom_misses: 620

pcache-cohorts-dependencies_cohort_deletes: 0

pcache-cohorts-dependencies_cohort_hits: 9386

pcache-cohorts-dependencies_cohort_inserts: 9682

pcache-cohorts-dependencies_cohort_misses: 2246

ipro_recorder_inserted_into_cache: 0

ipro_recorder_not_cacheable: 2

cache_flush_count: 0

memcache_last_error_checkpoint_ms: 0

memcache_error_burst_size: 0

cache_flush_timestamp_ms: 0

maanantai 20. maaliskuuta 2017 13.58.15 UTC+2 Joshua Marantz kirjoitti:

Can you paste the /pagespeed_admin/statistics values related to your cache? E.g.

wget -q -O - YOUR_SITE/pagespeed_admin/statistics|grep cache

and in particular your file cache:

wget -q -O - YOUR_SITE/pagespeed_admin/statistics|grep file_cache

On Mon, Mar 20, 2017 at 1:48 AM, Lauri Juusela <lauri....@gmail.com> wrote:

Its running on nginx and nginx process is running as root for now. Nginx process has write-access to /var/cache/ngx_pagespeed. Nginx pagespeed writes and shm_metadata_cache folder and files but nothing else to folder /var/cache/ngx_pagespeed. I'm using memcached as external cache so maybe this is reason? There is no other related messages in logfile. I'm mostly worried about this part of the warning -> "Doing an extra cache clean to be safe" because that might be the cause for performance issues?

sunnuntai 19. maaliskuuta 2017 23.39.09 UTC+2 Joshua Marantz kirjoitti:

My first guess is that the process running Apache doesn't have write-access to /var/cache/ngx_pagespeed. Can you check that before we go any further with analysis?

I would expect, though, that you would see other messages in your logfile to that effect.

On Sat, Mar 18, 2017 at 9:35 PM, Lauri Juusela <lauri....@gmail.com> wrote:

I'm having followed error given in pagespeed message history '[Warning] [1] Failed to read cache clean timestamp /var/cache/ngx_pagespeed/!clean!time!. Doing an extra cache clean to be safe.' multiple times on single request and many times withing minute. I'm having low traffic site. Also i think that cache misses are quite high about 75%. I think this might be reason for poor performance too. Serving pages without pagespeed (300 req/s) and with pagespeed about 30 req/s. I'm not very good with these things so its very probably my mistake somewhere.

--
You received this message because you are subscribed to the Google Groups "ngx-pagespeed-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ngx-pagespeed-discuss+unsub...@googlegroups.com.
Visit this group at https://groups.google.com/group/ngx-pagespeed-discuss.
For more options, visit https://groups.google.com/d/optout.

Lauri Juusela

unread,

Mar 20, 2017, 10:42:03 AM3/20/17

to ngx-pagespeed-discuss

I saw that there was difference in results when using */pagespeed_admin/* - */pagespeed_global_admin/*. Following results are from */pagespeed_global_admin/*.

cache_time_us: 8735162

cache_hits: 6691

cache_misses: 31782

cache_backend_hits: 7561

cache_backend_misses: 30912

cache_fallbacks: 0

cache_expirations: 852

cache_inserts: 2760

cache_deletes: 0

cache_extensions: 465243

not_cacheable: 0

font_service_input_resource_recent_uncacheable_miss: 0

font_service_input_resource_recent_uncacheable_failure: 0

in_place_uncacheable_rewrites: 0

num_local_storage_cache_candidates_found: 0

num_local_storage_cache_stored_total: 0

num_local_storage_cache_stored_images: 0

num_local_storage_cache_stored_css: 0

num_local_storage_cache_candidates_added: 0

num_local_storage_cache_candidates_removed: 0

url_input_resource_recent_uncacheable_miss: 0

url_input_resource_recent_uncacheable_failure: 14

rewrite_cached_output_missed_deadline: 10358

rewrite_cached_output_hits: 1637364

rewrite_cached_output_misses: 75528

resource_fetches_cached: 6078

num_cache_control_rewritable_resources: 9973

num_cache_control_not_rewritable_resources: 1309

ipro_not_in_cache: 28753

downstream_cache_purge_attempts: 0

successful_downstream_cache_purges: 0

cache_batcher_dropped_gets: 0

cache_batcher_coalesced_gets: 0

cache_batcher_queued_gets: 6064

memcache_timeouts: 0

file_cache_bytes_freed_in_cleanup: 0

file_cache_cleanups: 0

file_cache_disk_checks: 0

file_cache_evictions: 0

file_cache_skipped_cleanups: 0

file_cache_started_cleanups: 0

file_cache_write_errors: 0

file_cache_deletes: 0

file_cache_hits: 0

file_cache_inserts: 0

file_cache_misses: 0

lru_cache_deletes: 0

lru_cache_hits: 4842

lru_cache_inserts: 2115

lru_cache_misses: 33631

shm_cache_deletes: 0

shm_cache_hits: 2580854

shm_cache_inserts: 11218

shm_cache_misses: 40810

memcached_async_deletes: 0

memcached_async_hits: 1849

memcached_async_inserts: 13405

memcached_async_misses: 72067

memcached_blocking_deletes: 0

memcached_blocking_hits: 31956

memcached_blocking_inserts: 27908

memcached_blocking_misses: 3540

compressed_cache_corrupt_payloads: 0

compressed_cache_original_size: 52238121

compressed_cache_compressed_size: 11323131

pcache-cohorts-beacon_cohort_deletes: 0

pcache-cohorts-beacon_cohort_hits: 11508

pcache-cohorts-beacon_cohort_inserts: 6780

pcache-cohorts-beacon_cohort_misses: 660

pcache-cohorts-dom_deletes: 0

pcache-cohorts-dom_hits: 11037

pcache-cohorts-dom_inserts: 11415

pcache-cohorts-dom_misses: 627

pcache-cohorts-dependencies_cohort_deletes: 0

pcache-cohorts-dependencies_cohort_hits: 9411

pcache-cohorts-dependencies_cohort_inserts: 9713

pcache-cohorts-dependencies_cohort_misses: 2253

ipro_recorder_inserted_into_cache: 0

ipro_recorder_not_cacheable: 2

cache_flush_count: 0

memcache_last_error_checkpoint_ms: 0

memcache_error_burst_size: 0

cache_flush_timestamp_ms: 0

Joshua Marantz

unread,

Mar 23, 2017, 8:56:22 PM3/23/17

to ngx-pagesp...@googlegroups.com

Apologies -- this fell off my radar earlier. What's the FileCacheCleanIntervalMs set to in your configuration? How many virtual-hosts do you have set up?

It looks like you are using memcached, so nothing is being stored or looked up in the file-cache. We should make sure the file-cache clean interval is set to a fairly high value. Note that you still need a file-cache configured, to handle reads/writes for files that are too large for memcached.

To unsubscribe from this group and stop receiving emails from it, send an email to ngx-pagespeed-discuss+unsubscri...@googlegroups.com.

Visit this group at https://groups.google.com/group/ngx-pagespeed-discuss.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "ngx-pagespeed-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ngx-pagespeed-discuss+unsubscri...@googlegroups.com.

Visit this group at https://groups.google.com/group/ngx-pagespeed-discuss.
For more options, visit https://groups.google.com/d/optout.

Joshua Marantz

unread,

Mar 24, 2017, 8:51:32 AM3/24/17

to ngx-pagesp...@googlegroups.com

I re-read the initial message. Your file-cache clean interval is OK (you should probably make it less frequent but it's OK for now), but the evidence in your first log suggests a permissions problem. Are you sure the nginx child processes run as root, and the root has access to /var/cache/ngx_pagespeed?

I know that in Apache on Debian that when you start the server as root, child processes run either as www-data or www-user or some other synthetic username to avoid potential security issues on your machine.

Lauri Juusela

unread,

Apr 24, 2017, 10:50:15 AM4/24/17

to ngx-pagespeed-discuss

Sorry for informing about this so late. I found out what the problem was. My website is running in a docker container and i had an old data volume for pagespeed cache and in that volume there was no rights to write on it. After i deleted that volume and created new with correct configurations it started to work like it should.

Thanks for help, you gave me the right direction to investigate.

To unsubscribe from this group and stop receiving emails from it, send an email to ngx-pagespeed-discuss+unsub...@googlegroups.com.

Visit this group at https://groups.google.com/group/ngx-pagespeed-discuss.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "ngx-pagespeed-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ngx-pagespeed-discuss+unsub...@googlegroups.com.

Visit this group at https://groups.google.com/group/ngx-pagespeed-discuss.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "ngx-pagespeed-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ngx-pagespeed-discuss+unsub...@googlegroups.com.

Reply all

Reply to author

Forward