Re: Large cache sizes

[Otto van der Schaaf]

-mps-discuss, +pagespeed-dev 

Currently, FinderPropertiesCacheExpirationTimeMs is defaulted to a short TTL of two hours.

- Would there be anything against wiring through FinderPropertiesCacheExpirationTimeMs so people can configure it?

- I'm wondering why an expiry of two hours was chosen, is two hours the best default?

A stale-while-revalidate (-like) behaviour would be really nice, I think, but would probably be a lot more work.

Having FinderPropertiesCacheExpirationTimeMs configurable may help some in the short(ish) term.

Otto

On Sun, Jul 24, 2016 at 11:15 PM Otto van der Schaaf <osc...@we-amp.com> wrote:

Thanks for linking an example. I cannot reproduce https://github.com/pagespeed/mod_pagespeed/issues/1352, so that only leaves the two-hour beacon expiration time as the only possible explanation I can think of.

As far as I can tell making FinderPropertiesCacheExpirationTimeMs configurable would be a very small change, and could help you. 

But there may be arguments against it that I don't know of, so I'll ask around and post here when I know more.

Otto

On Sunday, July 24, 2016 at 4:31:10 PM UTC+2, Mickey wrote:

> Well,I could reproduce a problem, but I'm not 100% sure that it is the problem you are running into.
I think you are right.

Here is the scenario that we experience:

1. Go to a lower traffic page including an image, on a busy site.
   
2. Refresh the page 5-6 times until you get the resized optimized image as the image url (note that after the initial 5-6 refreshes we do get the resized optimized version on following refreshes, no matter how many).
3. Leave the page for several hours.
4. Refresh the page
5. Issue: The image url is the x optimized version without the sizes. Only after further 5-6 requests to the page, the image url is resized again.
6. You can see the problem occurring here (a page with less traffic): http://www.myaquariumclub.com/growing-and-breeding-ghost-shrimp-7872.html

Note that this is occurring although we set the cache size to be extremely large, and its limit was never reached.

> I think both things that may affect your website (short beacon data cacheTTL and possibly #1352) are going to require code changes to fix, so I'm not sure how to help you further in the short term.
From your experience, how much time would it take until those are fixed and a version including those fixes is released?

I'm asking as I wonder if we should wait for you guys (say up to 2-3 months) or patch it ourselves (on the html level, using jmarantz suggestion for resizing images and dropping the resize_rendered_image_dimensions filter)

Many thanks for the help :thumb-up:
Mickey.

On 24/07/2016 13:01, Otto van der Schaaf wrote:

On Sunday, July 24, 2016 at 7:49:59 AM UTC+2, Mickey wrote:

> What I can do is offer you a patch that does wire it through so you could try it out?
Thanks a lot for the offer! Still, as this is a production server we can not install patched versions.

You said you managed to reproduce the problem using resize_rendered_image_dimensions -- can you see if changing the value of kDefaultFinderPropertiesCacheExpirationTimeMs fixes the issue?

Well,I could reproduce a problem, but I'm not 100% sure that it is the problem you are running into. It may be (part of it). Bumping kDefaultFinderPropertiesCacheExpirationTimeMs is not going to fix https://github.com/pagespeed/mod_pagespeed/issues/1352, as the problem seems to be of a different nature:

When beaconing back image dimensions, at some point the resized dimensions are being reported as if they are the original ones. Which in turn makes resize_rendered_image_dimensions think that no resizing is needed. However, I can reproduce that with a couple (say 5) of full page refreshes in Chrome though, this does not takes hours to show up. 

 

> One thing that would help ruling out if this issue is important for you is disabling the filter temporarily and seeing how that affects image optimization.
The x optimized version always appear. It's the image sizes that are lost after a while, hence removing resize_rendered_image_dimensions will just serve the x optimized version that is served anyway. Or am I missing something?

I was under the assumption that "image optimization was completely lost" after a couple of hours. However, I think both things that may affect your website (short beacon data cacheTTL and possibly #1352) are going to require code changes to fix, so I'm not sure how to help you further in the short term.

Otto

 

--------------------------------------------
Q: Why is this email two sentences or less?
A: http://two.sentenc.es

On 24/07/2016 00:22, Otto van der Schaaf wrote:

Re: ModPagespeedFinderPropertiesCacheExpirationTimeMs

I'm sorry about that, it seems that option is not completely wired through.

What I can do is offer you a patch that does wire it through so you could try it out?

Re: resize_rendered_image_dimensions progress

I looked into it, I just commented here (you can subscribe to the issue to get updated):

https://github.com/pagespeed/mod_pagespeed/issues/1352#issuecomment-234740866

One thing that would help ruling out if this issue is important for you is disabling the filter temporarily and

seeing how that affects image optimization.

Otto

On Fri, Jul 22, 2016 at 6:26 PM <mick...@kikuta2008.com> wrote:

> > Did restarting less help with more consistent image optimization?

> No. Image optimizations still did not occur properly. We're still testing to see if we consistently get the simple x optimized version (jpeg2webp, etc, without resizing) or we loose image optimization entirely. Will get back to you later on about that.

Restarts do not make a change in terms of actual image caching.

> >There's also this default in the code: const int64 RewriteOptions::kDefaultFinderPropertiesCacheExpirationTimeMs = 2 * Timer::kHourMs; I think you can configure  that with: ModPagespeedFinderPropertiesCacheExpirationTimeMs
> Definitely worth a shot. I'll wait for our above mentioned test to finish and have this change.

We're getting the following error when we try it:

Syntax error on line *** of /etc/httpd/conf.d/pagespeed.conf:
Invalid command 'ModPagespeedFinderPropertiesCacheExpirationTimeMs', perhaps misspelled or defined by a module not included in the server configuration

Any idea how can we configure it? It does look like this parameter is causing expiration to occur... Maybe this is what expiring our images.

> I can reproduce a problem with resize_rendered_image_dimensions

Any progress on your side?

On Tuesday, June 28, 2016 at 6:24:26 PM UTC+3, mick...@kikuta2008.com wrote:

Hi folks,

Did you test large (say 20-30 GB) cache sizes? Will the cleaning process load the machine?

Can we disable the cache cleaning process? Generally storage is cheap enough today that we wouldn't like to delete anything that was created, even if it sits on the hdd a week or two until the next time it's required.

Thanks for the feedback,

Mickey.

--
You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/0ed31585-bac9-40e8-96ce-38e58b32f8fa%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mod-pagespeed-discuss/pbuT_tYiSG0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mod-pagespeed-di...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/CAHqmWiMc%2B4LFiha3T-Br3A6QfmY%3DmQOrpyMApHccDdTS7%2B-SQA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mod-pagespeed-discuss/pbuT_tYiSG0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mod-pagespeed-di...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/46299c4c-9b1c-4f94-b44a-5a8b0f23fa9c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/e40d24ef-4d1c-45a7-bd79-25f2d66a0d68%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.