strange not found issue

[Pascal Dreissen]

A site is going nuts with mod_pagespeed enabled. A lot of not found
script etc...

In the HTML of the site this rewritten script is going to be loaded:
http://www.fitnessdelivery.nl/js/scriptaculous/controls.js.pagespeed.jm.0KKfyWlgZ0.js

However this is the output:

Apache server with mod_pagespeed

OK
NOT FOUND:http://fitnessdelivery.nl/js/scriptaculous/
controls.js.pagespeed.jm.0KKfyWlgZ0.js

In the error_log this is the output:

[Mon Dec 06 17:16:47 2010] [error] [mod_pagespeed 0.9.0.0-287] Fetch
failed for http://fitnessdelivery.nl/js/scriptaculous/controls.js.pagespeed.jm.0KKfyWlgZ0.js,
status=0

In the cache directory the file does exists however not in the
http://fitnessdelivery dir but in http://www.fitnessdelivery dir:


total 216
drwxr-xr-x 2 httpd httpd 240 Dec 6 17:16 .
drwxr-xr-x 7 httpd httpd 140 Dec 6 17:14 ..
-rw------- 1 httpd httpd 5100 Dec 6 17:13 builder.js,
-rw------- 1 httpd httpd 3238 Dec 6 17:13
builder.js.pagespeed.jm.Db9-pU4lGa.js,
-rw------- 1 httpd httpd 30828 Dec 6 17:13 controls.js,
-rw------- 1 httpd httpd 20529 Dec 6 17:13 controls.js.pagespeed.jm.
0KKfyWlgZ0.js,
-rw------- 1 httpd httpd 31307 Dec 6 17:13 dragdrop.js,
-rw------- 1 httpd httpd 23013 Dec 6 17:13
dragdrop.js.pagespeed.jm.haIz-QGikF.js,
-rw------- 1 httpd httpd 38553 Dec 6 17:13 effects.js,
-rw------- 1 httpd httpd 29243 Dec 6 17:13
effects.js.pagespeed.jm.gdA98aGflX.js,
-rw------- 1 httpd httpd 10626 Dec 6 17:13 slider.js,
-rw------- 1 httpd httpd 7912 Dec 6 17:13
slider.js.pagespeed.jm.bzavegquXO.js,
[root@amslnx04 scriptaculous]# pwd
/var/www/mod_pagespeed/cache/http,3A/,/www.fitnessdelivery.nl/js/
scriptaculous

I have no clue what is going on, other sites have no issue what so
ever, it looks like mod_pagespeed is looking in the /var/www/
mod_pagespeed/cache/http,3A/,/fitnessdelivery.nl/js/scriptaculous
instead of the /var/www/mod_pagespeed/cache/http,3A/,/
www.fitnessdelivery.nl/js/scriptaculous directory, can this be
resolved or is this a bug i am facing. Running the latest build
(mod_pagespeed 0.9.0.0-287)

This is the config for this site:

SetOutputFilter MOD_PAGESPEED_OUTPUT_FILTER
ModPagespeed on
ModPagespeedFileCachePath "/var/www/mod_pagespeed/cache/"
ModPagespeedGeneratedFilePrefix "/var/www/mod_pagespeed/files/"
ModPagespeedRewriteLevel PassThrough
ModPagespeedEnableFilters add_head
ModPagespeedEnableFilters combine_heads
ModPagespeedEnableFilters move_css_to_head
ModPagespeedEnableFilters combine_css
ModPagespeedEnableFilters rewrite_css,rewrite_javascript
ModPagespeedEnableFilters inline_css,inline_javascript
ModPagespeedEnableFilters insert_img_dimensions
ModPagespeedEnableFilters remove_comments
ModPagespeedEnableFilters collapse_whitespace
ModPagespeedEnableFilters elide_attributes
ModPagespeedEnableFilters extend_cache
ModPagespeedEnableFilters remove_quotes

ModPagespeedDomain fitnessdelivery.nl
ModPagespeedDomain d13465.u25.applemooz.nl
ModPagespeedDomain fitnessdelivery.be
ModPagespeedDomain www.fitnessdelivery.be
ModPagespeedDomain fitnessdelivery.de
ModPagespeedDomain www.fitnessdelivery.de
ModPagespeedDomain fitnessdelivery.pl
ModPagespeedDomain www.fitnessdelivery.pl
ModPagespeedDomain fitnessdelivery.fr
ModPagespeedDomain www.fitnessdelivery.fr
ModPagespeedDomain outdoordelivery.be
ModPagespeedDomain www.outdoordelivery.be
ModPagespeedDomain outdoordelivery.nl
ModPagespeedDomain www.outdoordelivery.nl
ModPagespeedDomain outdoordelivery.com
ModPagespeedDomain www.outdoordelivery.com
ModPagespeedDomain sportdelivery.be
ModPagespeedDomain www.sportdelivery.be
ModPagespeedDomain sportdelivery.de
ModPagespeedDomain www.sportdelivery.de
ModPagespeedDomain www.fitnessdelivery.nl

[Joshua Marantz]

Hi Pascal,

I'm looking into this.  Can you describe your network setup a little?  Do you have multiple Apache servers behind a load balancer?

[Pascal Dreissen]

Nope single server setup. Configuration per vhost.

On Dec 6, 5:50 pm, Joshua Marantz <jmara...@google.com> wrote:
> Hi Pascal,
>
> I'm looking into this.  Can you describe your network setup a little?  Do
> you have multiple Apache servers behind a load balancer?
>

> On Mon, Dec 6, 2010 at 11:25 AM, Pascal Dreissen <pdreis...@gmail.com>wrote:
>
>
>
> > A site is going nuts with mod_pagespeed enabled. A lot of not found
> > script etc...
>
> > In the HTML of the site this rewritten script is going to be loaded:
>

> >http://www.fitnessdelivery.nl/js/scriptaculous/controls.js.pagespeed....

>
> > However this is the output:
>
> > Apache server with mod_pagespeed
>
> > OK
> > NOT FOUND:http://fitnessdelivery.nl/js/scriptaculous/
> > controls.js.pagespeed.jm.0KKfyWlgZ0.js
>
> > In the error_log this is the output:
>
> > [Mon Dec 06 17:16:47 2010] [error] [mod_pagespeed 0.9.0.0-287] Fetch
> > failed for

> >http://fitnessdelivery.nl/js/scriptaculous/controls.js.pagespeed.jm.0...
> > ,

> >www.fitnessdelivery.nl/js/scriptaculousdirectory, can this be

[Joshua Marantz]

I can think of two other potential causes for this problem:  VirtualHost and mod_rewrite.

1. VirtualHost

Is it possible that you have configured a VirtualHost for domain a.com, which is rewriting resources and putting them in a file-cache that's unique to a.com, but then also configured VirtualHost for domain b.com, using a different file-cache?  Could it be that, when fetching resources requested by the browser interpreting the rewritten page, Apache is assigning the resource to a VirtualHost which has mod_pagespeed that's looking at the wrong cache?

If that's how you have it set up, would it be possible to use the same cache-prefix for all your mod_pagespeed instances?

2. mod_rewrite:

Do you have mod_rewrite enabled, or some other filter in your Apache path that could be rewriting URLs between fitnessdelivery.* and www.fitnessdelivery.*?

mod_pagespeed can do that itself, of course, if you use ModPagespeedMapRewriteDomain but I didn't see that in your config snippet.

There isn't any obvious place in our code where we would change javascript pathnames like that, except to make relative paths absolute.

Thanks,

-Josh

[Pascal Dreissen]

On Dec 6, 6:41 pm, Joshua Marantz <jmara...@google.com> wrote:
> I can think of two other potential causes for this problem:  VirtualHost and
> mod_rewrite.
>
> 1. VirtualHost
>
> Is it possible that you have configured a VirtualHost for domain a.com,
> which is rewriting resources and putting them in a file-cache that's unique
> to a.com, but then also configured VirtualHost for domain b.com, using a
> different file-cache?  Could it be that, when fetching resources requested
> by the browser interpreting the rewritten page, Apache is assigning the
> resource to a VirtualHost which has mod_pagespeed that's looking at the
> wrong cache?
>
> If that's how you have it set up, would it be possible to use the same
> cache-prefix for all your mod_pagespeed instances?

Can you please elaborate what you exactly mean, if you mean the
CachePath ? This is the same over all domains:

ModPagespeedFileCachePath "/var/www/mod_pagespeed/
cache/"
ModPagespeedGeneratedFilePrefix "/var/www/mod_pagespeed/
files/"

>
> 2. mod_rewrite:
>
> Do you have mod_rewrite enabled, or some other filter in your Apache path
> that could be rewriting URLs between fitnessdelivery.* andwww.fitnessdelivery.*?
>
> mod_pagespeed can do that itself, of course, if you use
> ModPagespeedMapRewriteDomain but I didn't see that in your config snippet.
>
> There isn't any obvious place in our code where we would change javascript
> pathnames like that, except to make relative paths absolute.

This can be the case, in .htaccess all non www is rewritten to www,
but to me this seems ok because all the content is fetched from
browser site as www.

RewriteCond %{HTTP_HOST} ^fitnessdelivery\.nl$
RewriteRule ^(.*)$ http://www.fitnessdelivery.nl/$1 [R=301,L]

So probably this could be the problem i am facing ?

Thanks again for you help!
>
> Thanks,
> -Josh

> > > >http://fitnessdeliverydirbut inhttp://www.fitnessdeliverydir:

[Joshua Marantz]

Thanks for investigating, Pascal.  Responses are inline.

On Tue, Dec 7, 2010 at 9:36 AM, Pascal Dreissen <pdre...@gmail.com> wrote:

Can you please elaborate what you exactly mean, if you mean the
CachePath ? This is the same over all domains:

       ModPagespeedFileCachePath            "/var/www/mod_pagespeed/
cache/"
       ModPagespeedGeneratedFilePrefix      "/var/www/mod_pagespeed/
files/"

Yes that's what I meant.  We can eliminate this as a potential cause.

> 2. mod_rewrite:
>

> Do you have mod_rewrite enabled, or some other filter in your Apache path

> that could be rewriting URLs between fitnessdelivery.* andwww.fitnessdelivery.*?

>
> mod_pagespeed can do that itself, of course, if you use
> ModPagespeedMapRewriteDomain but I didn't see that in your config snippet.
>
> There isn't any obvious place in our code where we would change javascript
> pathnames like that, except to make relative paths absolute.

This can be the case, in .htaccess all non www is rewritten to www,
but to me this seems ok because all the content is fetched from
browser site as www.

RewriteCond %{HTTP_HOST} ^fitnessdelivery\.nl$
RewriteRule ^(.*)$ http://www.fitnessdelivery.nl/$1 [R=301,L]

So probably this could be the problem i am facing ?

Yes this looks like this is related to the problem.  It looks like mod_rewrite is running after mod_pagespeed and thus the mod_pagespeed cache is confused.  Nevertheless I will boldly suggest a remedy.  Please try adding this to your pagespeed.conf:

This is the boldest remedy, mapping your origin domain and mapping your rewrite domain in mod_pagespeed:

    ModPagespeedMapOriginDomain localhost *.fitnessdelivery.*

    ModPagespeedMapRewriteDomain www.fitnessdelivery.nl *.fitnessdelivery.*

This says:

1. anytime mod_pagespeed needs to fetch a resource from the origin, map the URL's hostname to "localhost" before fetching it.

2. anytime mod_pagespeed rewrites a resource, map it to hostname www.fitnessdlievery.nl

I am not sure if both of these are necessary.

An alternative approach is to somehow force mod_rewrite to run *before* mod_pagespeed, but I'm not sure if it's possible for you to control that from an .htaccess file.  I'm investigating a similar issue raised today on this discussion forum about the INCLUDES filter to process .shtml files so hopefully I'll be able to answer both qauestions.

-Josh

[Pascal Dreissen]

Ok,

I just did:

ModPagespeedMapOriginDomain localhost *.fitnessdelivery.*
ModPagespeedMapRewriteDomain www.fitnessdelivery.nl
*.fitnessdelivery.*

But nothing changes, it's exactly the same issue which is happening as
before. So basicly i do not see anything changing:

url: http://www.fitnessdelivery.nl/js/prototype/prototype.js.pagespeed.jm.uoIySJIEVd.js

output:
Apache server with mod_pagespeed

OKNOT FOUND:http://fitnessdelivery.nl/js/prototype/
prototype.js.pagespeed.jm.uoIySJIEVd.js

error_log:
[Tue Dec 07 16:28:02 2010] [error] [mod_pagespeed 0.9.0.0-287] Fetch
failed for http://fitnessdelivery.nl/js/prototype/prototype.js.pagespeed.jm.uoIySJIEVd.js,
status=0

cache dir:
[root@amslnx04 cache]# find |grep uoIySJIEVd
./http,3A/,/www.fitnessdelivery.nl/js/prototype/
prototype.js.pagespeed.jm.uoIySJIEVd.js,


Strangly enough, there are a few which go ok:

url: http://www.fitnessdelivery.nl/js/prototype/validation.js.pagespeed.jm.Zk6B7xuq_E.js

in the cache dir i am seeing this:

[root@amslnx04 cache]# find |grep Zk6B7xuq
./http,3A/,/fitnessdelivery.nl/js/prototype/
validation.js.pagespeed.jm.Zk6B7xuq_E.js,
./http,3A/,/www.fitnessdelivery.nl/js/prototype/
validation.js.pagespeed.jm.Zk6B7xuq_E.js,

On Dec 7, 3:58 pm, Joshua Marantz <jmara...@google.com> wrote:
> Thanks for investigating, Pascal.  Responses are inline.
>

> On Tue, Dec 7, 2010 at 9:36 AM, Pascal Dreissen <pdreis...@gmail.com> wrote:
>
> > Can you please elaborate what you exactly mean, if you mean the
> > CachePath ? This is the same over all domains:
>
> >        ModPagespeedFileCachePath            "/var/www/mod_pagespeed/
> > cache/"
> >        ModPagespeedGeneratedFilePrefix      "/var/www/mod_pagespeed/
> > files/"
>
> Yes that's what I meant.  We can eliminate this as a potential cause.
>
>
>
> > 2. mod_rewrite:
>
> > > Do you have mod_rewrite enabled, or some other filter in your Apache path
> > > that could be rewriting URLs between fitnessdelivery.*
> > andwww.fitnessdelivery.*?
>
> > > mod_pagespeed can do that itself, of course, if you use
> > > ModPagespeedMapRewriteDomain but I didn't see that in your config
> > snippet.
>
> > > There isn't any obvious place in our code where we would change
> > javascript
> > > pathnames like that, except to make relative paths absolute.
>
> > This can be the case, in .htaccess all non www is rewritten to www,
> > but to me this seems ok because all the content is fetched from
> > browser site as www.
>
> > RewriteCond %{HTTP_HOST} ^fitnessdelivery\.nl$

> > RewriteRule ^(.*)$http://www.fitnessdelivery.nl/$1[R=301,L]

>
> > So probably this could be the problem i am facing ?
>
> Yes this looks like this is related to the problem.  It looks like
> mod_rewrite is running after mod_pagespeed and thus the mod_pagespeed cache
> is confused.  Nevertheless I will boldly suggest a remedy.  Please try
> adding this to your pagespeed.conf:
>
> This is the boldest remedy, mapping your origin

> domain<http://code.google.com/speed/page-speed/docs/using_mod.html#Mapping
> Origin
> Domains> and mapping your rewrite
> domain<http://code.google.com/speed/page-speed/docs/using_mod.html#Mapping
> Rewrite
> Domains> in mod_pagespeed:

[Joshua Marantz]

The fact that you said "a few which go OK" makes me think this is a cache problem at some level.

1. If you have mod_cache or mod_diskcache enabled, can you clear those caches?

2. Can you stop the server, delete the mod_pagespeed cache directory, then restart the server?

Do you have any other sort of caching infrastructure in place, e.g. WP Total Cache?

-Josh

[Pascal Dreissen]

Op 7 dec 2010, om 17:11 heeft Joshua Marantz het volgende geschreven:

The fact that you said "a few which go OK" makes me think this is a cache problem at some level.

1. If you have mod_cache or mod_diskcache enabled, can you clear those caches?

Not enabled 

2. Can you stop the server, delete the mod_pagespeed cache directory, then restart the server?

Just done that, exactly same issue as before. However with restarting http (i am normally using reload) i saw an error:

Starting httpd: [Tue Dec 07 17:19:33 2010] [warn] [mod_pagespeed 0.9.0.0-287] AddDomain of domain already in map: http://www.fitnessdelivery.nl/

Do you have any other sort of caching infrastructure in place, e.g. WP Total Cache?

Nope is not wordpress, site is magento.

[Joshua Marantz]

On Tue, Dec 7, 2010 at 11:22 AM, Pascal Dreissen <pdre...@gmail.com> wrote:

Just done that, exactly same issue as before. However with restarting http (i am normally using reload) i saw an error:

Starting httpd: [Tue Dec 07 17:19:33 2010] [warn] [mod_pagespeed 0.9.0.0-287] AddDomain of domain already in map: http://www.fitnessdelivery.nl/

This is not a problem.  You likely had this order:

   ModPagespeedMapOriginDomain localhost *.fitnessdelivery.*
   ModPagespeedMapRewriteDomain www.fitnessdelivery.nl *.fitnessdelivery.*

   ModPagespeedDomain  http://www.fitnessdelivery.nl/

   

You don't really need the last ModPagespeedDomain given the fact that you have the MapRewrite call above it.

But it's not a problem and the warning will only appear on startup, not per request.

Do you have any other sort of caching infrastructure in place, e.g. WP Total Cache?

Nope is not wordpress, site is magento.

I made an error in my wildcard suggestion -- an extra '.'.  Please use these commands:

   ModPagespeedMapOriginDomain localhost *fitnessdelivery.*
   ModPagespeedMapRewriteDomain www.fitnessdelivery.nl *fitnessdelivery.*

Sorry about that,

-Josh

[Pascal Dreissen]

Op 7 dec 2010, om 17:34 heeft Joshua Marantz het volgende geschreven:

On Tue, Dec 7, 2010 at 11:22 AM, Pascal Dreissen <pdre...@gmail.com> wrote:

Just done that, exactly same issue as before. However with restarting http (i am normally using reload) i saw an error:

Starting httpd: [Tue Dec 07 17:19:33 2010] [warn] [mod_pagespeed 0.9.0.0-287] AddDomain of domain already in map: http://www.fitnessdelivery.nl/

This is not a problem.  You likely had this order:

   ModPagespeedMapOriginDomain localhost *.fitnessdelivery.*
   ModPagespeedMapRewriteDomain www.fitnessdelivery.nl *.fitnessdelivery.*
   ModPagespeedDomain  http://www.fitnessdelivery.nl/

   

You don't really need the last ModPagespeedDomain given the fact that you have the MapRewrite call above it.

Ok i understand!

But it's not a problem and the warning will only appear on startup, not per request.

Do you have any other sort of caching infrastructure in place, e.g. WP Total Cache?

Nope is not wordpress, site is magento.

I made an error in my wildcard suggestion -- an extra '.'.  Please use these commands:

   ModPagespeedMapOriginDomain localhost *fitnessdelivery.*
   ModPagespeedMapRewriteDomain www.fitnessdelivery.nl *fitnessdelivery.*

Sorry about that,

No problem, however just changes it, but no change what so ever!

It looks like these configuration settings don't change anything!

-Josh

[Pascal Dreissen]

Ok,

I did some more investigation and it gets stranger and stranger

http://www.fitnessdelivery.nl/?ModPagespeed=on&ModPagespeedFilters=rewrite_javascript

Seems to be ok, javascript is rewritten correctly. When i add a filter
for example rewrite_images

http://www.fitnessdelivery.nl/?ModPagespeed=on&ModPagespeedFilters=rewrite_javascript,rewrite_images

No problem, adding a couple of others is no problem at all:

http://www.fitnessdelivery.nl/?ModPagespeed=on&ModPagespeedFilters=rewrite_javascript,rewrite_images,collapse_whitespace,remove_comments

However, when adding rewrite_css:

http://www.fitnessdelivery.nl/?ModPagespeed=on&ModPagespeedFilters=rewrite_javascript,rewrite_images,collapse_whitespace,remove_comments,rewrite_css

Things are going to break down (please reload a second time to
initiate rewriting).

It seems that only the filters which are rewriting CSS in any way or
the ones that break.

Even if i add extend_cache, things go nuts...

http://www.fitnessdelivery.nl/?ModPagespeed=on&ModPagespeedFilters=rewrite_javascript,rewrite_images,collapse_whitespace,remove_comments,extend_cache

Any idea ?

Thanks in advance for your help!

On Dec 7, 5:51 pm, Pascal Dreissen <pdreis...@gmail.com> wrote:
> Op 7 dec 2010, om 17:34 heeft Joshua Marantz het volgende geschreven:
>

[Pascal Dreissen]

Ok forget my last message, it seemed that my cache was primed, that's
why i did not see the 404... so it does even not work... sorry for
this!

On Dec 7, 9:17 pm, Pascal Dreissen <pdreis...@gmail.com> wrote:
> Ok,
>
> I did some more investigation and it gets stranger and stranger
>

> http://www.fitnessdelivery.nl/?ModPagespeed=on&ModPagespeedFilters=re...

>
> Seems to be ok, javascript is rewritten correctly. When i add a filter
> for example rewrite_images
>

> http://www.fitnessdelivery.nl/?ModPagespeed=on&ModPagespeedFilters=re...

>
> No problem, adding a couple of others is no problem at all:
>

> http://www.fitnessdelivery.nl/?ModPagespeed=on&ModPagespeedFilters=re...
>
> However, when adding rewrite_css:
>
> http://www.fitnessdelivery.nl/?ModPagespeed=on&ModPagespeedFilters=re...

>
> Things are going to break down (please reload a second time to
> initiate rewriting).
>
> It seems that only the filters which are rewriting CSS in any way or
> the ones that break.
>
> Even if i add extend_cache, things go nuts...
>

> http://www.fitnessdelivery.nl/?ModPagespeed=on&ModPagespeedFilters=re...