[page-speed-discuss] leverage proxy caching citation fails to properly report existing "Cache-Control: public" header

[piran]

Please fix your page-speed product. Have spent a great deal of time
troubleshooting why my server is not sending out the desired headers
only to find that it is Google's page-speed v.1.7.1 addon within
Firefox (FF3.6.3) that is apparently faulty. When the headers are
checked by an apparently more competent organisation (askapache dot
com) I find that my server *IS* producing its headers ...pretty much
as desired.

--
You received this message because you are subscribed to the Google Groups "page-speed-discuss" group.
To post to this group, send email to page-spee...@googlegroups.com.
To unsubscribe from this group, send email to page-speed-disc...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/page-speed-discuss?hl=en.

[Libo Song]

Could you please file a bug at http://code.google.com/p/page-speed/issues/list,
or state the details of your problem -- what headers are missing?

[piran]

Will file a bug if this thread eventually finds it necessary.
Can't imagine that I am the only person to see this problem
so I assumed there was something amiss my end.

No, none of my headers are missing (from my server).
Example from one of my site's - main banner JPG file
(sorry about the lack of formatting - my first post here)

Google page-speed:
Response Headersview source
Date Tue, 18 May 2010 13:52:48 GMT
Server Apache/2.0.52
Last-Modified Tue, 27 Apr 2010 12:56:36 GMT
Etag "6d4d-6c72c900"
Accept-Ranges bytes
Content-Length 27981
Cache-Control max-age=604800
Content-Type image/jpeg
Expires Tue, 25 May 2010 13:52:48 GMT

AskApache http online-tools:
HTTP/1.1 200 OK
Date: Tue, 18 May 2010 13:55:19 GMT
Server: Apache/2.0.52
Last-Modified: Tue, 27 Apr 2010 12:56:36 GMT
ETag: "6d4d-6c72c900"
Accept-Ranges: bytes
Content-Length: 27981
Cache-Control: public, no-transform, max-age=31536000
Expires: Tue, 25 May 2010 13:55:19 GMT
Connection: close
Content-Type: image/jpeg

It is the AskApache result that is the more
correct report of what is configured here.

Please fix your product, I have actually completed that which was
demanded by page-speed and implemented 'leveraged proxy caching'.


On May 18, 1:51 pm, Libo Song <ls...@google.com> wrote:
> Could you please file a bug athttp://code.google.com/p/page-speed/issues/list,

> or state the details of your problem -- what headers are missing?
>
>
>
>
>
> On Tue, May 18, 2010 at 3:47 AM, piran <goo...@dreckly.org> wrote:
> > Please fix your page-speed product. Have spent a great deal of time
> > troubleshooting why my server is not sending out the desired headers
> > only to find that it is Google's page-speed v.1.7.1 addon within
> > Firefox (FF3.6.3) that is apparently faulty. When the headers are
> > checked by an apparently more competent organisation (askapache dot
> > com) I find that my server *IS* producing its headers ...pretty much
> > as desired.
>
> > --
> > You received this message because you are subscribed to the Google Groups "page-speed-discuss" group.
> > To post to this group, send email to page-spee...@googlegroups.com.
> > To unsubscribe from this group, send email to page-speed-disc...@googlegroups.com.

> > For more options, visit this group athttp://groups.google.com/group/page-speed-discuss?hl=en.

>
> --
> You received this message because you are subscribed to the Google Groups "page-speed-discuss" group.
> To post to this group, send email to page-spee...@googlegroups.com.
> To unsubscribe from this group, send email to page-speed-disc...@googlegroups.com.

> For more options, visit this group athttp://groups.google.com/group/page-speed-discuss?hl=en.

[Richard Rabbat]

hmm

Your expires directive tells the browser to expire the cached resource in 7 days

while your cache-control directive is telling the browser to expire the resource in 1 year

[piran]

hmmm me too. I've had a lot of trouble trying to get this to work.
The browser 7day expire is a default (just in case the code fails).
The cache-control directive is what Google specified. I can go with
1year old JPGs, once pictures they go out they don't change.
Have been trying variations to see where it goes wrong...

Follows is the relevant section of my httpd.conf template...

Header unset Pragma
#header unset Cache-Control
# 1 YEAR 31536000
# 3 MONTHS 7884000
# 1 MONTH 2628000
# 1 WEEK 604800
# 2 DAYS 172800
# 1 MIN 60

ServerTokens Min
ExpiresActive On
ExpiresDefault A604800

ExpiresByType application/pdf A31536000
ExpiresByType application/x-javascript A31536000
ExpiresByType image/png A31536000
ExpiresByType image/gif A31536000
ExpiresByType image/jpg A31536000
ExpiresByType image/jeg A31536000
ExpiresByType image/x-icon A31536000
ExpiresByType image/cur A31536000
ExpiresByType text/pdf A31536000
ExpiresByType text/css A31536000
ExpiresByType text/plain A31536000
ExpiresByType text/javascript A31536000

<FilesMatch "\.(xml|txt|css|js)$">
Header set Cache-Control "max-age=7884000, public, must-revalidate"
</FilesMatch>

#works# ...but apparently not in google page-speed 1.7.1
<FilesMatch "\.(png|gif|jpg|jpeg)$">
Header set Cache-Control "public, no-transform, max-age=31536000"
</FilesMatch>

#works# ...but apparently not in google page-speed 1.7.1
<FilesMatch "\.(ico|pdf|cur)$">
Header set Cache-Control "public, max-age=31536000"
</FilesMatch>

#works#
<FilesMatch "\.(html|htm|php)$">
Header set Cache-Control "max-age=2628000, public, must-revalidate"
</FilesMatch>

[Libo Song]

Seems there is a typo.
ExpiresByType image/jeg A31536000

[Richard Rabbat]

I think case is solved :)

--
Richard Rabbat | Product Manager | Google, Inc.
E: rab...@google.com | M: +1-650-714-7618 | Google Voice: +1-617-213-0819

[piran]

Typo agreed. Amended, thank you. Situation unchanged.

[piran]

The Google page-speed Cache-control element still
does NOT show the 'public' or 'no-transform' switches
but, yes, it now shows the same 1yr timescale at least.
It's not reporting properly.

> > page-speed-disc...@googlegroups.com<page-speed-discuss%2Bunsubs cr...@googlegroups.com>
> > .

> > >> For more options, visit this group athttp://
> > groups.google.com/group/page-speed-discuss?hl=en.
>

> > > --
> > > You received this message because you are subscribed to the Google Groups
> > "page-speed-discuss" group.
> > > To post to this group, send email to page-spee...@googlegroups.com
> > .
> > > To unsubscribe from this group, send email to

> > page-speed-disc...@googlegroups.com<page-speed-discuss%2Bunsubs cr...@googlegroups.com>
> > .

> > > For more options, visit this group at
> >http://groups.google.com/group/page-speed-discuss?hl=en.
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "page-speed-discuss" group.
> > To post to this group, send email to page-spee...@googlegroups.com.
> > To unsubscribe from this group, send email to

> > page-speed-disc...@googlegroups.com<page-speed-discuss%2Bunsubs cr...@googlegroups.com>
> > .

> > For more options, visit this group at
> >http://groups.google.com/group/page-speed-discuss?hl=en.
>
> --
> Richard Rabbat | Product Manager | Google, Inc.
> E: rab...@google.com | M: +1-650-714-7618 | Google Voice: +1-617-213-0819
>
> --
> You received this message because you are subscribed to the Google Groups "page-speed-discuss" group.
> To post to this group, send email to page-spee...@googlegroups.com.
> To unsubscribe from this group, send email to page-speed-disc...@googlegroups.com.

[piran]

As above. Situation unchanged. Google's page-speed is not
reflecting the content of the header, well not in the same way
as that by the parser at askapache which seems correct.
Page-speed not reading or reporting properly?

> > page-speed-disc...@googlegroups.com<page-speed-discuss%2Bunsubs cr...@googlegroups.com>
> > .

> > >> For more options, visit this group athttp://
> > groups.google.com/group/page-speed-discuss?hl=en.
>

> > > --
> > > You received this message because you are subscribed to the Google Groups
> > "page-speed-discuss" group.
> > > To post to this group, send email to page-spee...@googlegroups.com
> > .
> > > To unsubscribe from this group, send email to

> > page-speed-disc...@googlegroups.com<page-speed-discuss%2Bunsubs cr...@googlegroups.com>
> > .

> > > For more options, visit this group at
> >http://groups.google.com/group/page-speed-discuss?hl=en.
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "page-speed-discuss" group.
> > To post to this group, send email to page-spee...@googlegroups.com.
> > To unsubscribe from this group, send email to

> > page-speed-disc...@googlegroups.com<page-speed-discuss%2Bunsubs cr...@googlegroups.com>
> > .

> > For more options, visit this group at
> >http://groups.google.com/group/page-speed-discuss?hl=en.
>
> --
> Richard Rabbat | Product Manager | Google, Inc.
> E: rab...@google.com | M: +1-650-714-7618 | Google Voice: +1-617-213-0819
>
> --
> You received this message because you are subscribed to the Google Groups "page-speed-discuss" group.
> To post to this group, send email to page-spee...@googlegroups.com.
> To unsubscribe from this group, send email to page-speed-disc...@googlegroups.com.

[Richard Rabbat]

Piran,

what is your objective here? if your objective is to improve the performance of your website, you just did; congratulations. I'm glad we helped you find the bug in your conf file.

can you send one url for one of the resources to check?

richard.

[Libo Song]

A quick testing on my side cannot reproduce the issue.

Here is my result:
Response Headersview source
Date Tue, 18 May 2010 15:54:07 GMT
Server Apache/2.2.14 (Unix) DAV/2
Last-Modified Tue, 18 May 2010 15:36:45 GMT
Etag "af85-30c003-486e01bdb5d40"
Accept-Ranges bytes
Cache-Control public, no-transform, max-age=31536000
MyHeader OK
Content-Length 3183617
Content-Type image/jpeg

[piran]

URL sent by email quoted in thread responder.
Please keep my URLs offline.
Objective: fix page-speed's report. it isn't showing the
required public element and it isn't showing the no-tranform
element and it isn't acknowledging that I have carried out
what it cited was advised? It's not working properly!?

> > > > page-speed-disc...@googlegroups.com<page-speed-discuss%2Bunsubs cr...@googlegroups.com><page-speed-discuss%2Bunsubs

> > cr...@googlegroups.com>
> > > > .
> > > > >> For more options, visit this group athttp://
> > > > groups.google.com/group/page-speed-discuss?hl=en.
>
> > > > > --
> > > > > You received this message because you are subscribed to the Google
> > Groups
> > > > "page-speed-discuss" group.
> > > > > To post to this group, send email to
> > page-spee...@googlegroups.com
> > > > .
> > > > > To unsubscribe from this group, send email to

> > > > page-speed-disc...@googlegroups.com<page-speed-discuss%2Bunsubs cr...@googlegroups.com><page-speed-discuss%2Bunsubs

> > cr...@googlegroups.com>
> > > > .
> > > > > For more options, visit this group at
> > > >http://groups.google.com/group/page-speed-discuss?hl=en.
>
> > > > --
> > > > You received this message because you are subscribed to the Google
> > Groups
> > > > "page-speed-discuss" group.
> > > > To post to this group, send email to
> > page-spee...@googlegroups.com.
> > > > To unsubscribe from this group, send email to

> > > > page-speed-disc...@googlegroups.com<page-speed-discuss%2Bunsubs cr...@googlegroups.com><page-speed-discuss%2Bunsubs

> > cr...@googlegroups.com>
> > > > .
> > > > For more options, visit this group at
> > > >http://groups.google.com/group/page-speed-discuss?hl=en.
>
> > > --
> > > Richard Rabbat | Product Manager | Google, Inc.
> > > E: rab...@google.com | M: +1-650-714-7618 | Google Voice:
> > +1-617-213-0819
>
> > > --
> > > You received this message because you are subscribed to the Google Groups
> > "page-speed-discuss" group.

> > > To post to this group, send email to page-spee...@googlegroups.com
> > .
> > > To unsubscribe from this group, send email to
> > page-speed-disc...@googlegroups.com<page-speed-discuss%2Bunsubs cr...@googlegroups.com>
> > .
> > > For more options, visit this group athttp://
> > groups.google.com/group/page-speed-discuss?hl=en.
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "page-speed-discuss" group.

[Richard Rabbat]

No big deal.

Page Speed tells me

Response Headersview source

Date	Tue, 18 May 2010 16:25:10 GMT

Server	Apache/2.0.52
Last-Modified	Tue, 27 Apr 2010 12:56:36 GMT
Etag	"6d4d-6c72c900"
Accept-Ranges	bytes
Content-Length	27981

Cache-Control	public, no-transform, max-age=31536000
Expires	Wed, 18 May 2011 16:25:10 GMT
Keep-Alive	timeout=15, max=100
Connection	Keep-Alive
Content-Type	image/jpeg

Your max age is fixed, and I can also see the other cache-control directives.
Most likely issue: you're behind a proxy

[piran]

That has EXACTLY reproduced the issue!?
My workstation's FF browser's page-speed addon does not
show that stuff - it misses out the public and no-transform.
What do you want a screenshot from my workstation?
The page-speed product is not reporting the header properly.
Everything else, including your test above, reports it OK.

[piran]

Back to hmmm. So the upshot is that I cannot trust page-speed's
reports?
My server acts as a transparent proxy (SME7.4 contribs dot org).
It's never let me down. Looks like I must purchase more pinches of
salt;~)

On May 18, 5:28 pm, Richard Rabbat <rab...@google.com> wrote:
> No big deal.
>
> Page Speed tells me
>
> Response Headersview source

> DateTue, 18 May 2010 16:25:10 GMTServerApache/2.0.52Last-ModifiedTue, 27 Apr

> 2010 12:56:36 GMTEtag"6d4d-6c72c900"Accept-RangesbytesContent-Length27981

> Cache-Controlpublic, no-transform, max-age=31536000ExpiresWed, 18 May 2011

[Steve Souders]

Hi, Piran.

Thanks for the intriguing bug report. I love debugging problems like this. 

The crux of the issue here, as Richard discovered, is that Page Speed gives you the headers from your browser's perspective. Whereas, askapache.com is giving you the headers they see. Since you're behind a proxy, that intermediary is messing with the headers. 

This is fairly common. An amazing discovery from Tony Gentilcore in my 2nd book was that ~15% of users don't get compressed responses (see http://www.stevesouders.com/blog/2009/11/11/whos-not-getting-gzip/ ). The reason is that many proxies and anti-virus software products remove or mask the Accept-Encoding header (because it's easier to parse the plain text response).

One approach Page Speed could take is to hit the URLs from a backend server that we know is not going through a proxy. I personally think it's better to give the web developer the true status of what they're seeing right now - so they can know, for example, if their proxy is stripping the Accept-Encoding header. Or in your case, if the proxy is altering the caching headers.

I don't think there's a right or wrong here. It's important for web developers to see what happened when they just loaded the page - and Page Speed does that. And askapache.com has a great way for a web developer to confirm whether the headers were altered by an intermediary. 

Perhaps the takeaway from this is to integrate into Page Speed a feature to compare the headers as seen by the browser to the headers seen by a backend server. Such a "diff" would be helpful IMO (since it's kinda painful to compare them manually by eye).

What do you think of that feature?

-Steve

[piran]

Steve----

Looks like it's my day for saying hmmm.
The crux is, from my POV, that I cannot trust what the tool
page-speed is apparently telling me. A tool that cannot be
trusted (for whatever reason) is a tool that needs 'fixing'.

Caching is new to me. I am used to running my own server
but I've ignored caching for a good many years until now.
Testing my own web pages look'n'feel has always been
a problem for me because the server just loops GETs
back without ever going on-line so to speak. Usually I
use loop back sites to have a look a what my site
'looks' like from the internet. Also the highly annoying
google webmaster stuff has a recent loop back feature.

My server goes straight out to line, there's no other
proxy that I know of getting in the way. I am my own
ISP so to speak. I've done the gzip/deflate thing and
the server also runs a PHP accelerator daemon. There
are caches EVERYWHERE;~| That's why I've stayed
away from the things for so long. I want a life;~)

Anything that fixes page-speed would be appreciated.
Let me know if you want it tested. My workstation is
W7/64bit and I run a brace of browsers including the
necessary FF3.6.3 with Firebug 1.5.4. across my
own site-wide intranet. Lots of firewalls and some
of my own customised masquerading blackholes;~0

----best wishes, Robert

On May 18, 5:44 pm, Steve Souders <soud...@google.com> wrote:
> Hi, Piran.
>
> Thanks for the intriguing bug report. I love debugging problems like this.
>
> The crux of the issue here, as Richard discovered, is that Page Speed gives
> you the headers from your browser's perspective. Whereas, askapache.com is
> giving you the headers they see. Since you're behind a proxy, that
> intermediary is messing with the headers.
>
> This is fairly common. An amazing discovery from Tony Gentilcore in my 2nd

> book was that ~15% of users don't get compressed responses (seehttp://www.stevesouders.com/blog/2009/11/11/whos-not-getting-gzip/). The

> ...
>
> read more »

[Brian]

It may also be worth considering if an additional component could be
altering the request/response headers. For example, I thought there
were some "security" desktop products (anti virus, anti phishing, etc)
that could alter the request/response (acting between the browser and
the OS on the client machine). Their reason for doing so would be
that they could more easily scan/filter uncompressed content.

Are you able to test from an alternate client machine? As you are
using Firefox you could also try installing the Web Developer add-on.
It too is capable of reporting Response Headers. Using it would be
another way to narrow-in if the issue is Page Speed or something
else. For example, if Web Developer also reports the header missing
the cache-control public value, it is quite likely the issue exists
above the Page Speed stack either in the browser, OS, or some
intermediary as Steve was suggesting.

Brian

[piran]

Brian----

I considered local issues before reaching out to Google...
Forgive me but I should decline to specifically answer your
question/suggestion on the grounds of reasonable security.

As the Subject line indicates, the gist to my thread starter
was to present page-speed's inability to properly report
the very status of the works it cited was desirable to
achieve better responses. I did the work but the
page-speed tool is inadequately and improperly reporting
results and issues which are no longer in existence.

The consensus of the thread's participants is that that
is apparently due to a proxy probably of my own server.
The purpose of my thread was to have this weakness
addressed somehow or otherwise ameliorated. IOW
my issue is with the tool known as page-speed being
inappropriate for the task admittedly in localised or
special circumstances. The product should be fixed
now that its inherent weakness has been highlighted.

Other mechanisms exist to test back into my sites
from out there in the internet. I am reasonably sure
that my server is pushing out halfway appropriate
headers of some compromise (cache=compromise).

Using another Response Header addon in parallel
is not going to fix page-speed - the thread's Subject.
Indeed it is more likely to confuse or make erratic
any given result. It's not the header data that's at
issue, it's that page-speed is reporting improperly
and needs fixing. That fix or any deeper investigation
is likely to tax my both experience level and more
importantly my very will to live;~) IOW I didn't have
this 'issue' before page-speed came along, though
I will make my site available for use testing any fix.

I have just now rebooted server AND workstation,
cleared just about what I could of available cache.
Page-speed is still unwilling to properly report the
actual existence of the public and no-transform
switches resident in the cache-control header.
The tool cannot be trusted to report properly
and it should be fixed or have one of its
many weaknesses clearly declared.

Thank you for putting some thought to my thread,
I await an announcement of the product's fix.

----best wishes, Robert

[Brian]

Hi Robert,

I can appreciate where you are coming from. One thing we noticed when
working with Apache was that the Expires directive manipulates the
Cache-Control header (setting max-age). Additionally, in your
configuration you are trying to set the Header (Header set Cache-
Control "public, no-transform, max-age=31536000"). This looks like
multiple Apache directives are attempting to "set" the same header.

Looking at your configuration and comments, the one that "works" is
also the one that does not have any conflicting other statements
overlapping with the Expires directives:

#works# ...but apparently not in google page-speed 1.7.1
<FilesMatch "\.(png|gif|jpg|jpeg)$">
Header set Cache-Control "public, no-transform, max-age=31536000"
</FilesMatch>

#works# ...but apparently not in google page-speed 1.7.1
<FilesMatch "\.(ico|pdf|cur)$">
Header set Cache-Control "public, max-age=31536000"
</FilesMatch>

#works#
<FilesMatch "\.(html|htm|php)$">
Header set Cache-Control "max-age=2628000, public, must-revalidate"
</FilesMatch>

However, the ones that do not work are ones who also would hit Expires
directives earlier in the config. What we did to work around this was
instead leverage Apache's Header append directive for any subsequent
"add" to the header beyond the Expires directive.

As an example:
FileETag None
Header append Cache-Control "public"
ExpiresActive On
ExpiresDefault "access plus 1 year"


What happens if you change your configuration to be something like:

<FilesMatch "\.(png|gif|jpg|jpeg)$">

Header append Cache-Control "public"
Header append Cache-Control "no-transform"
</FilesMatch>

<FilesMatch "\.(ico|pdf|cur)$">
Header append Cache-Control "public"
</FilesMatch>

That may be a more reliable way to have Apache set your headers.

~ Brian