Large cache sizes

[mick...@kikuta2008.com]

Hi folks,

Did you test large (say 20-30 GB) cache sizes? Will the cleaning process load the machine?

Can we disable the cache cleaning process? Generally storage is cheap enough today that we wouldn't like to delete anything that was created, even if it sits on the hdd a week or two until the next time it's required.

Thanks for the feedback,

Mickey.

[Jeffrey Crowell]

Clearing a giant cache will likely be pretty slow. The filesystem is walked and old files are removed until the size is under the target size: https://github.com/pagespeed/mod_pagespeed/blob/master/pagespeed/kernel/cache/file_cache.cc#L183-L282

To prevent all cleaning, you can set the variable here

https://developers.google.com/speed/pagespeed/module/system#file_cache

FileCacheInodeLimit to 0

and FileCacheSizeKb to be 9223372036854775807 (LLONG_MAX)

--
You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/ae832116-2b20-4dc3-8ca6-a27f8f54cd92%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Mickey]

Got you, thanks for the quick response.

Another question related to long term cache -- would restarting apache affect image serving (e.g. would mps need to recollect beacon data in order to know which image url to serve)? Or is gathered beacon data saved on files (cache?) as well and survive restarts?

Thanks,
Mickey.

You received this message because you are subscribed to a topic in the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mod-pagespeed-discuss/pbuT_tYiSG0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mod-pagespeed-di...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/CA%2B7dWMEs6wR7%3DkHNN810qnKWWATnFBJ%2Bntiv%3Dhs4OdXmfe3Azw%40mail.gmail.com.

[mick...@kikuta2008.com]

> Another question related to long term cache -- would restarting apache affect image serving (e.g. would mps need to recollect beacon data in order to know which image url to serve)? Or is gathered beacon data saved on files (cache?) as well and survive restarts?

Any update on this?

[Jeffrey Crowell]

Sorry, though I replied to this. Restarting the server will invalidate the propertycache and requires rebeaconing.

--

You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/d409bfa5-290c-4e02-b51a-946baf05fe58%40googlegroups.com.

[Mickey]

NP :-)

I was asking this question since even though we have an unlimited cache size (currently 35 GB and growing) we still see many times pages that were not visited for a day or so include the non optimized version of images urls (without even the simple x).

I thought it might be related to restarts, but we ruled this out. My next assumption is that the number of images we have (several millions) is just too much for the in memory beacon data cache. Is that the reason? If not, what else can it be?

If we manage to at least always show the x version, we can implement the image resize as suggested on the "Images not resized via resize_rendered_image_dimensions" thread
https://groups.google.com/d/msg/mod-pagespeed-discuss/GbntPnwiGTU/0aCnOfAiBgAJ

Thanks for the help and hope you're having a great 4th of July weekend :-)
Mickey.

--------------------------------------------
Q: Why is this email two sentences or less?
A: http://two.sentenc.es

You received this message because you are subscribed to a topic in the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mod-pagespeed-discuss/pbuT_tYiSG0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mod-pagespeed-di...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/CA%2B7dWMEk%3DJ3hHMGfFdeSrYfPy4PSqebWBWZzitP_tcP%3DxCndgQ%40mail.gmail.com.

[mick...@kikuta2008.com]

> I thought it might be related to restarts, but we ruled this out. My next assumption is that the number of images we have (several millions) is just too much for the in memory beacon data cache. Is that the reason? If not, what else can it be?

Any update?

Also, can it be the in memory cache is flushed on httpd reload?

On Tuesday, June 28, 2016 at 6:24:26 PM UTC+3, mick...@kikuta2008.com wrote:

[Jeff Kaufman]

Just looked over the code some, and if your cache is large enough that
we can't walk the whole thing in an hour it's not going to work well.
Filed a bug: https://github.com/pagespeed/mod_pagespeed/issues/1337

It looks like, contrary to what jcrowell wrote above, setting
FileCacheInodeLimit and FileCacheSizeKb isn't enough to disable the
resource-intensive part of cache cleaning, which is the directory
walk. To do that you need to set FileCacheCleanIntervalMs to a large
number, like
630720000000 (20 years).

> can it be the in memory cache is flushed on httpd reload?

Yes, in-memory caches are flushed on an httpd reload, but depending on
your configuration we can be set to either write these through to disk
or not. Specifically, if you set CreateSharedMemoryMetadataCache then
we don't write through, but if you leave it out and we use the default
one then we do write through. We know this is confusing, and at some
point we'd like to switch the default in both cases to be
cache-checkpointing (checkpointing isn't currently available at all
yet).

(Doc: https://developers.google.com/speed/pagespeed/module/system#default_shm_cache)

> --
> You received this message because you are subscribed to the Google Groups
> "mod-pagespeed-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to mod-pagespeed-di...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/mod-pagespeed-discuss/50fa4bb2-7c5d-4fad-aca7-fe75b4774562%40googlegroups.com.

[mick...@kikuta2008.com]

> setting FileCacheInodeLimit and FileCacheSizeKb isn't enough to disable the resource-intensive part of cache cleaning

I think you need add a support (simple) for disabling the cache cleaning completely. Our current cache is 90G and growing. Passing on such larger volumes can be cumbersome even with ionice. 

> Specifically, if you set CreateSharedMemoryMetadataCache then we don't write through, but if you leave it out and we use the default one then we do write through.

In this case, since we didn't change the default (see below), I do not understand why image optimization is completely lost after several hours  (without even the x url version). Obviously some data is lost after several hours and mps needs to recollect it. 

Any idea what can this be? We really need you help here, as image optimization is practically not working on larger sites.

Another idea -- on cases the beacon data is lost and the full size optimized version is available on the cache (disk), you can serve that optimized version. It's not optimal but it's definitely better than nothing, and worth as an extra fallback.

$ diff /etc/httpd/conf.d/pagespeed.conf /vendors/etc/httpd/conf.d/pagespeed.conf

45c45

<     ModPagespeedFileCachePath            "/***/mod_pagespeed/"

---

>     ModPagespeedFileCachePath            "/var/cache/mod_pagespeed/"

89,95c89

<       ModPagespeedEnableFilters collapse_whitespace,elide_attributes,trim_urls

<       ModPagespeedEnableFilters resize_rendered_image_dimensions

<       ModPagespeedEnableFilters remove_quotes,remove_comments

<       ModPagespeedEnableFilters insert_dns_prefetch

<       ModPagespeedEnableFilters dedup_inlined_images

<       ModPagespeedEnableFilters defer_javascript

<       #ModPagespeedEnableFilters debug

---

>     # ModPagespeedEnableFilters collapse_whitespace,elide_attributes

150,153c144,145

<       # Max int

<     ModPagespeedFileCacheSizeKb          2147483647

<     # Once in 20 years

<     ModPagespeedFileCacheCleanIntervalMs 630720000000

---

>     # ModPagespeedFileCacheSizeKb          102400

>     # ModPagespeedFileCacheCleanIntervalMs 3600000

168c160

<     ModPagespeedFileCacheInodeLimit        0

---

>     ModPagespeedFileCacheInodeLimit        500000

On Tuesday, June 28, 2016 at 6:24:26 PM UTC+3, mick...@kikuta2008.com wrote:

[Jeff Kaufman]

On Thu, Jul 7, 2016 at 8:21 AM, <mick...@kikuta2008.com> wrote:
>> setting FileCacheInodeLimit and FileCacheSizeKb isn't enough to disable
>> the resource-intensive part of cache cleaning
> I think you need add a support (simple) for disabling the cache cleaning
> completely. Our current cache is 90G and growing. Passing on such larger
> volumes can be cumbersome even with ionice.

Agreed. This should probably look like:

ModPagespeedFileCacheCleanIntervalMs -1

Filed bug: https://github.com/pagespeed/mod_pagespeed/issues/1343

>
> Any idea what can this be? We really need you help here, as image
> optimization is practically not working on larger sites.

I'm not sure why this is happening. I need to read through the code more.

> --
> You received this message because you are subscribed to the Google Groups
> "mod-pagespeed-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to mod-pagespeed-di...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/mod-pagespeed-discuss/65245721-3c09-4c34-b5db-fa7349d86834%40googlegroups.com.

[Mickey]

> ModPagespeedFileCacheCleanIntervalMs -1
Sounds right.

>> Any idea what can this be? We really need you help here, as image
optimization is practically not working on larger sites.
> I'm not sure why this is happening. I need to read through the code more.

Anything we can do to help you out? Maybe check existence or access time
of data on our disks where the problem occurs?

[mick...@kikuta2008.com]

Is there any update on this issue? As I said our images are in practice not being optimized and we need to find a solution for this issue.

If you need any info from our side (files' last access time, etc.) please let us know.

Thanks for the help :-)
Mickey.

On Tuesday, June 28, 2016 at 6:24:26 PM UTC+3, mick...@kikuta2008.com wrote:

[Otto van der Schaaf]

re: cache cleaner 

I have a patch here which you could try. It contains a hack to disable the cache cleaner:

https://gist.github.com/oschaaf/7a6a8d5197be1d4a598eb01100e62d89

re: In this case, since we didn't change the default (see below), I do not understand why image optimization is completely lost after several hours  (without even the x url version). Obviously some data is lost after several hours and mps needs to recollect it. 

Could you check if the observed interval more or less matches the expiry interval of the original images?

--

You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/7a9ada2a-1dec-45de-abdb-70deef0c2c45%40googlegroups.com.

[mick...@kikuta2008.com]

> I have a patch here which you could try. It contains a hack to disable the cache cleaner:

> https://gist.github.com/oschaaf/7a6a8d5197be1d4a598eb01100e62d89

As this is a production server, I think we'll use Jeff's suggestion and increase the interval to several years, until this patch is officially released.

> Could you check if the observed interval more or less matches the expiry interval of the original images?

Can you please explain this a bit more into details, so we're sure we're checking the right thing? In "observed interval", do you mean the period since last accessed? Also, where do we find the image files on the disk?

On Tuesday, June 28, 2016 at 6:24:26 PM UTC+3, mick...@kikuta2008.com wrote:

[Otto van der Schaaf]

As this is a production server, I think we'll use Jeff's suggestion and increase the interval to several years, until this patch is officially released.

Increasing the cache cleaning interval sounds like a much better idea, sorry. I missed that option.

 

> Could you check if the observed interval more or less matches the expiry interval of the original images?

Can you please explain this a bit more into details, so we're sure we're checking the right thing? In "observed interval", do you mean the period since last accessed? Also, where do we find the image files on the disk?

You mentioned that optimization image optimization was lost after several hours. I assumed that this happens periodically, it that correct?

If so, I am curious if the images in question are being served with Expiry and/or Cache-Control response headers that indicate a cache lifetime of "several hours".

Otto

[Mickey]

> You mentioned that optimization image optimization was lost after several hours. I assumed that this happens periodically, it that correct?

> If so, I am curious if the images in question are being served with Expiry and/or Cache-Control response headers that indicate a cache lifetime of "several hours".

Image cache was set for 3 months, we recently changed that for a year.

See for example
Original served from the site (1 year): http://www.myaquariumclub.com/images/fbfiles/images/DSC04246-eb5b8128728244fd2ac92c90122f6871.JPG
Original served from the CDN (3 month): http://dlgdxii3fgupk.cloudfront.net/myaquariumclub.com/images/fbfiles/images/DSC04246-eb5b8128728244fd2ac92c90122f6871.JPG
Optimized served from the site (5 minutes?? See attached image): http://www.myaquariumclub.com/images/fbfiles/images/xDSC04246-eb5b8128728244fd2ac92c90122f6871.JPG.pagespeed.ic.C_HeKYXySX.webp
Optimized served from the CDN (1 year): http://dlgdxii3fgupk.cloudfront.net/myaquariumclub.com/images/fbfiles/images/xDSC04246-eb5b8128728244fd2ac92c90122f6871.JPG.pagespeed.ic.C_HeKYXySX.webp

My guess is that the CDN has already got the optimized image with a longer cache and serves it, while mps did not identify the image was already optimized (lost the beacon/optimization information) and is re-optimizing it.

From my point of view this is a critical error, as it loads apache with creating optimized images which in turn are never served.

--
You received this message because you are subscribed to a topic in the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mod-pagespeed-discuss/pbuT_tYiSG0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mod-pagespeed-di...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/CAHqmWiMbPCy-%2BbVH9efXhdBLTFw_GPXi_NHozV7vVre%2BZo5ZmA%40mail.gmail.com.

[Otto van der Schaaf]

Thanks. Do you have the admin pages up and running? Especially the console's graphs may be helpful here.

If you have statistics logging set up, and are able to transfer the raw logs to me, I could have a look at that too

to see if I can figure out what happens.

See https://developers.google.com/speed/pagespeed/module/console

Otto

You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/ca70bdde-8504-cebd-5908-361007feadbc%40kikuta2008.com.

[Otto van der Schaaf]

Thanks for the statistics logs. I plotted some of the data to make interpreting easier,  attached is a pdf.  

The graphs all represent the complete timeline from the logs you sent me:  7/14/2016 3:40am till 7/14/2016 3:00pm.

The graphs all periodically drop to zero every few hours, but other then that I do not see something obvious stand out (perhaps someone else has an idea?).

Are you restarting the apache process every few hours? Or is there anything in Apache's logs that indicates this happens?

Otto

[Mickey]

>

Are you restarting the apache process every few hours? Or is there anything in Apache's logs that indicates this happens?

We run a graceful restart every 6 hours. I'm not sure this is the cause though since:

1. The drops do not look periodically.
   
2. I tested a graceful restart and the x optimized version was served. 
3. As much as I understood from previous replies, this should not be a problem, as memory cache information is also saved on the disk. 
4. The insertion graph doesn't look like a cache flush, which usually looks like 1/X function (the most common resources being inserts immediately creating a jump of insertions and than insertion drops asymptotically). Instead you see a steady flow of insertions, with every once in a while a drop and a jump back.

In any case, we have disabled those restarts and I'll let you know if this was the cause.

Thanks for your help Otto, I greatly appreciate it!
Mickey.

To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/CAHqmWiNt3O%3DHZtFEUjQDuMgbiAVhj1B-CJfoOEyhUBxjEYB-Ng%40mail.gmail.com.

[Otto van der Schaaf]

The dips to zero in all the graphs represent the logged statistics being reset to 0. As far as I know this can only happens when a a fresh worker processes is logging them. If the statistics dropping to zero happens at the same time as seeing far less optimized images being served, that may need some more investigation.

By the sound of it  not all the 0 statistics in the logs can be explained by scheduled restarts; but perhaps you have MaxConnectionsPerChild configured (or some other form of automatic conditional worker-process restarting)? 

One more question: I just filed https://github.com/pagespeed/mod_pagespeed/issues/1352

I filed that for the resize_rendered_image_dimensions filter, do you have that enabled? Regardless, if you have a problematic page with unoptimized images in your browser, could you add "?foo=randomnumber" to the querystring and perform a few page reloads - while looking at the network requests (for example in Chrome's dev tools)  to see if .pagespeed. images are (at least temporary) being served?

Otto

To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/1d54f2f1-d005-8e46-f468-84032595ef7f%40kikuta2008.com.

[Mickey]

>

The dips to zero in all the graphs represent the logged statistics being reset to 0. As far as I know this can only happens when a a fresh worker processes is logging them. If the statistics dropping to zero happens at the same time as seeing far less optimized images being served, that may need some more investigation.

If I understand the terminology correctly, I think you can see that on the graphs you have sent. Take a look at general cache graph and image stats graph.

> By the sound of it  not all the 0 statistics in the logs can be explained by scheduled restarts; but perhaps you have MaxConnectionsPerChild configured (or some other form of automatic conditional worker-process restarting)?

You are right. There were also configuration updates.

We disabled our routine graceful restart and avoided conf upgrades, and the graphs look much better. There was still one graceful made by the log rotation scripts. I'm sending you the logfile. One thing I really don't understand -- how come "file cache misses" is increasing over time? Or is this an accumulated number, rather than the misses per X requests?

As for the fix -- here are some more use cases to think about when you decide of a solution:

1. Log rotation scripts.
   
2. People updating their apache config (happens every few hours on our case).
   
3. People updating their servers' OS (Amazon issues an AMI update every 1-2 weeks).
4. People running on the cloud, having several dynamic servers starting and stopping, all connecting to a central cache.
   
5. People using MaxConnectionsPerChild, as you mentioned.

> I filed that for the resize_rendered_image_dimensions filter, do you have that enabled?

Yes, though I really doubt it's related to the specific filter. I think this is more a general cache issue.

> see if .pagespeed. images are (at least temporary) being served?
Yes, the optimized images are being served, and than after several hours apache "forgets" the optimized images exist, and start the optimization process again for the next few hours (serves an unoptimized version, gather the beacon data, and after 2-3 requests start serving again the optimized version). This "amnesia" happens every few hours.

Again thanks for all the help!
Mickey.

--------------------------------------------
Q: Why is this email two sentences or less?
A: http://two.sentenc.es

To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/CAHqmWiPOAy4ytxFwZoj16UOJsLgvk-ZZUHjLuHRuGtu9oXg51A%40mail.gmail.com.

[Otto van der Schaaf]

If I understand the terminology correctly, I think you can see that on the graphs you have sent. Take a look at general cache graph and image stats graph.

> By the sound of it  not all the 0 statistics in the logs can be explained by scheduled restarts; but perhaps you have MaxConnectionsPerChild configured (or some other form of automatic conditional worker-process restarting)?
You are right. There were also configuration updates.

We disabled our routine graceful restart and avoided conf upgrades, and the graphs look much better. There was still one graceful made by the log rotation scripts. I'm sending you the logfile. One thing I really don't understand -- how come "file cache misses" is increasing over time? Or is this an accumulated number, rather than the misses per X requests?

All the statistics are logged as incrementing numbers. So to compute rates you have to look at the delta's (I have attached the processed stats as a pdf again). Without the extra restarts all the dips to zero are gone indeed except one which we can account for. Did restarting less help with more consistent image optimization? 

 

As for the fix -- here are some more use cases to think about when you decide of a solution:

1. Log rotation scripts.
   
2. People updating their apache config (happens every few hours on our case).
   
3. People updating their servers' OS (Amazon issues an AMI update every 1-2 weeks).
4. People running on the cloud, having several dynamic servers starting and stopping, all connecting to a central cache.
   
5. People using MaxConnectionsPerChild, as you mentioned.

> I filed that for the resize_rendered_image_dimensions filter, do you have that enabled?
Yes, though I really doubt it's related to the specific filter. I think this is more a general cache issue.

I tested prioritize critical css, which also relies on beacon data, and after restarting the server no beaconing was required to serve inlined critical css in the html. 

But I can reproduce a problem with resize_rendered_image_dimensions , so I do think it's worth trying out what happens when you disable it.

 

> see if .pagespeed. images are (at least temporary) being served?
Yes, the optimized images are being served, and than after several hours apache "forgets" the optimized images exist, and start the optimization process again for the next few hours (serves an unoptimized version, gather the beacon data, and after 2-3 requests start serving again the optimized version). This "amnesia" happens every few hours.

There's also this default in the code:

const int64 RewriteOptions::kDefaultFinderPropertiesCacheExpirationTimeMs = 2 * Timer::kHourMs;

I think you can configure  that with:

ModPagespeedFinderPropertiesCacheExpirationTimeMs

Maybe worth a shot bumping that value to a short time to see if that allows you to reproduce the problem rapidly?

If it does, perhaps setting it to a large value may help you.

 

Otto

To unsubscribe from this group and all its topics, send an email to mod-pagespeed-discuss+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/CAHqmWiMbPCy-%2BbVH9efXhdBLTFw_GPXi_NHozV7vVre%2BZo5ZmA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-discuss+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/ca70bdde-8504-cebd-5908-361007feadbc%40kikuta2008.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mod-pagespeed-discuss/pbuT_tYiSG0/unsubscribe.

To unsubscribe from this group and all its topics, send an email to mod-pagespeed-discuss+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/CAHqmWiNt3O%3DHZtFEUjQDuMgbiAVhj1B-CJfoOEyhUBxjEYB-Ng%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-discuss+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/1d54f2f1-d005-8e46-f468-84032595ef7f%40kikuta2008.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mod-pagespeed-discuss/pbuT_tYiSG0/unsubscribe.

To unsubscribe from this group and all its topics, send an email to mod-pagespeed-discuss+unsub...@googlegroups.com.

[Mickey]

> Did restarting less help with more consistent image optimization?

No. Image optimizations still did not occur properly. We're still testing to see if we consistently get the simple x optimized version (jpeg2webp, etc, without resizing) or we loose image optimization entirely. Will get back to you later on about that.

> I do think it's worth trying out what happens when you disable it.

We actually need it, so I'd really like to try different configurations before we  take it off (and use jmarantz's suggested prototype to reduce image sizes).

>There's also this default in the code: const int64 RewriteOptions::kDefaultFinderPropertiesCacheExpirationTimeMs = 2 * Timer::kHourMs; I think you can configure  that with: ModPagespeedFinderPropertiesCacheExpirationTimeMs

Definitely worth a shot. I'll wait for our above mentioned test to finish and have this change.

Question -- why have this parameter at all and not use the served resources cache? Obviously, this is the period the admin think the resource does not change. It does sound like this parameter is interfering with long term cache...

Again, thanks for the help!
Mickey.

To unsubscribe from this group and all its topics, send an email to mod-pagespeed-di...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/7eee2b20-7dd6-4285-b449-0ded741af6a2%40googlegroups.com.

[mick...@kikuta2008.com]

> > Did restarting less help with more consistent image optimization?

> No. Image optimizations still did not occur properly. We're still testing to see if we consistently get the simple x optimized version (jpeg2webp, etc, without resizing) or we loose image optimization entirely. Will get back to you later on about that.

Restarts do not make a change in terms of actual image caching.

> >There's also this default in the code: const int64 RewriteOptions::kDefaultFinderPropertiesCacheExpirationTimeMs = 2 * Timer::kHourMs; I think you can configure  that with: ModPagespeedFinderPropertiesCacheExpirationTimeMs
> Definitely worth a shot. I'll wait for our above mentioned test to finish and have this change.

We're getting the following error when we try it:

Syntax error on line *** of /etc/httpd/conf.d/pagespeed.conf:
Invalid command 'ModPagespeedFinderPropertiesCacheExpirationTimeMs', perhaps misspelled or defined by a module not included in the server configuration

Any idea how can we configure it? It does look like this parameter is causing expiration to occur... Maybe this is what expiring our images.

> I can reproduce a problem with resize_rendered_image_dimensions

Any progress on your side?

On Tuesday, June 28, 2016 at 6:24:26 PM UTC+3, mick...@kikuta2008.com wrote:

[Otto van der Schaaf]

Re: ModPagespeedFinderPropertiesCacheExpirationTimeMs

I'm sorry about that, it seems that option is not completely wired through.

What I can do is offer you a patch that does wire it through so you could try it out?

Re: resize_rendered_image_dimensions progress

I looked into it, I just commented here (you can subscribe to the issue to get updated):

https://github.com/pagespeed/mod_pagespeed/issues/1352#issuecomment-234740866

One thing that would help ruling out if this issue is important for you is disabling the filter temporarily and

seeing how that affects image optimization.

Otto

--

You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/0ed31585-bac9-40e8-96ce-38e58b32f8fa%40googlegroups.com.

[Mickey]

> What I can do is offer you a patch that does wire it through so you could try it out?

Thanks a lot for the offer! Still, as this is a production server we can not install patched versions.

You said you managed to reproduce the problem using resize_rendered_image_dimensions -- can you see if changing the value of kDefaultFinderPropertiesCacheExpirationTimeMs fixes the issue?

> One thing that would help ruling out if this issue is important for you is disabling the filter temporarily and seeing how that affects image optimization.

The x optimized version always appear. It's the image sizes that are lost after a while, hence removing resize_rendered_image_dimensions will just serve the x optimized version that is served anyway. Or am I missing something?

Thanks for the help!

Mickey.

--------------------------------------------
Q: Why is this email two sentences or less?
A: http://two.sentenc.es

You received this message because you are subscribed to a topic in the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mod-pagespeed-discuss/pbuT_tYiSG0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mod-pagespeed-di...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/CAHqmWiMc%2B4LFiha3T-Br3A6QfmY%3DmQOrpyMApHccDdTS7%2B-SQA%40mail.gmail.com.

[Otto van der Schaaf]

On Sunday, July 24, 2016 at 7:49:59 AM UTC+2, Mickey wrote:

> What I can do is offer you a patch that does wire it through so you could try it out?
Thanks a lot for the offer! Still, as this is a production server we can not install patched versions.

You said you managed to reproduce the problem using resize_rendered_image_dimensions -- can you see if changing the value of kDefaultFinderPropertiesCacheExpirationTimeMs fixes the issue?

Well,I could reproduce a problem, but I'm not 100% sure that it is the problem you are running into. It may be (part of it). Bumping kDefaultFinderPropertiesCacheExpirationTimeMs is not going to fix https://github.com/pagespeed/mod_pagespeed/issues/1352, as the problem seems to be of a different nature:

When beaconing back image dimensions, at some point the resized dimensions are being reported as if they are the original ones. Which in turn makes resize_rendered_image_dimensions think that no resizing is needed. However, I can reproduce that with a couple (say 5) of full page refreshes in Chrome though, this does not takes hours to show up. 

 

> One thing that would help ruling out if this issue is important for you is disabling the filter temporarily and seeing how that affects image optimization.
The x optimized version always appear. It's the image sizes that are lost after a while, hence removing resize_rendered_image_dimensions will just serve the x optimized version that is served anyway. Or am I missing something?

I was under the assumption that "image optimization was completely lost" after a couple of hours. However, I think both things that may affect your website (short beacon data cacheTTL and possibly #1352) are going to require code changes to fix, so I'm not sure how to help you further in the short term.

Otto

 

To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-discuss+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/0ed31585-bac9-40e8-96ce-38e58b32f8fa%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mod-pagespeed-discuss/pbuT_tYiSG0/unsubscribe.

To unsubscribe from this group and all its topics, send an email to mod-pagespeed-discuss+unsub...@googlegroups.com.

[Mickey]

> Well,I could reproduce a problem, but I'm not 100% sure that it is the problem you are running into.

I think you are right.

Here is the scenario that we experience:

1. Go to a lower traffic page including an image, on a busy site.
   
2. Refresh the page 5-6 times until you get the resized optimized image as the image url (note that after the initial 5-6 refreshes we do get the resized optimized version on following refreshes, no matter how many).
3. Leave the page for several hours.
4. Refresh the page
5. Issue: The image url is the x optimized version without the sizes. Only after further 5-6 requests to the page, the image url is resized again.
6. You can see the problem occurring here (a page with less traffic): http://www.myaquariumclub.com/growing-and-breeding-ghost-shrimp-7872.html

Note that this is occurring although we set the cache size to be extremely large, and its limit was never reached.

> I think both things that may affect your website (short beacon data cacheTTL and possibly #1352) are going to require code changes to fix, so I'm not sure how to help you further in the short term.

From your experience, how much time would it take until those are fixed and a version including those fixes is released?

I'm asking as I wonder if we should wait for you guys (say up to 2-3 months) or patch it ourselves (on the html level, using jmarantz suggestion for resizing images and dropping the resize_rendered_image_dimensions filter)

Many thanks for the help :thumb-up:
Mickey.

To unsubscribe from this group and all its topics, send an email to mod-pagespeed-di...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/46299c4c-9b1c-4f94-b44a-5a8b0f23fa9c%40googlegroups.com.

[Otto van der Schaaf]

Thanks for linking an example. I cannot reproduce https://github.com/pagespeed/mod_pagespeed/issues/1352, so that only leaves the two-hour beacon expiration time as the only possible explanation I can think of.

As far as I can tell making FinderPropertiesCacheExpirationTimeMs configurable would be a very small change, and could help you. 

But there may be arguments against it that I don't know of, so I'll ask around and post here when I know more.

Otto

To unsubscribe from this group and all its topics, send an email to mod-pagespeed-discuss+unsub...@googlegroups.com.

[Mickey]

If you guys can officially release just this fix, that would be awesome!

BTW, why have this parameter instead of using the resource expiration time?

To unsubscribe from this group and all its topics, send an email to mod-pagespeed-di...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/e40d24ef-4d1c-45a7-bd79-25f2d66a0d68%40googlegroups.com.

[mick...@kikuta2008.com]

> As far as I can tell making FinderPropertiesCacheExpirationTimeMs configurable would be a very small change, and could help you.

Do you have any guess as for when this could be officially released? If there's no point waiting for it, we may try another workaround.

[Otto van der Schaaf]

I do not know about the release schedule, but it looks like the change itself was already committed:

https://github.com/pagespeed/mod_pagespeed/commit/099329f5619dd4b3cfa9bbf6dc82b7d37a9075d6

Otto

--

You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/1621e705-8dac-4b31-82a4-a075c6364030%40googlegroups.com.

[Mickey]

Got you, thanks :-)

You received this message because you are subscribed to a topic in the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mod-pagespeed-discuss/pbuT_tYiSG0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mod-pagespeed-di...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/CAHqmWiOhU%3DXK3EyU_7xX3Bh66_6hBiKi63-ebWL%3DvJUWcp9AOQ%40mail.gmail.com.