mod_caucho + mod_pagespeed

[kishore]

Hi,

I use mod_pagespeed for my html pages and thought of extending it to
my jsp pages also with the help of ModPagespeedAllow , but it seems to
be not working with mod_caucho.

Any ideas?

TIA
Kishore

[Joshua Marantz]

I haven't heard of mod_caucho before but I just took a quick look.  Could you describe what's not working?  Is mod_pagespeed enabled, the pages it severs do not have the "X-Mod-Pagespeed" header?

Can you give the exact text of your 'ModPagespeedAllow' directive, along with any 'ModPagespeedDisallow' directives you may have added?

What content-type is served by pages generated by mod_caucho?

-Josh

On Mon, May 16, 2011 at 3:34 AM, kishore <ero...@gmail.com> wrote:

mod_caucho

[kishore]

Hi Josh,

Thanks for the quick reply.

Yes mod_pagespeed is enabled in my server, when i access a html page
filters are working properly (collapse_whitespace,remove_comments) and
m able to see X-Mod-Pagespeed: 0.9.16.9-576 header.

But the same is not happening for jsp pages.

I dont use 'ModPagespeedDisallow' and this is the exact text for
ModPagespeedAllow,

ModPagespeedAllow /*.jsp*

Content-Type: text/html

Thanks,
Kishore

[Joshua Marantz]

Just to clarify, for the '.jsp' pages they are being served without a X-Mod-Pagespeed header?

I think your ModPagespeedAllow will not have any effect.  The default setup is that nothing is disallowed based on just name.  If it were, your ModPagespeedAllow would not work because the full URL is used for comparison and your pattern starts with a "/".  You probably just want "ModPagespeedAllow *.jsp*".  But I don't think that's the problem.

I think it's more likely that mod_pagespeed is not seeing the pages created by mod_caucho at all because of filter ordering. in Apache or some other inter-module compatibility issue.

I dug a little deeper looking for mod_caucho.c on the net to figure out how it interacts with the Apache filter chain, but all I could find was what looked like a version that was built for Apache 1.*.  If you have the Apache 2.* version of of mod_caucho.c handy could you attach it or send a link where I could find it?

[kishore]

Josh,

Yes you are right m not getting the X-Mod-Pagespeed header for the jsp
pages.

I don't find any option to attach mod_caucho.so file, so i have sent
it to your Email Address.

Thanks,
Kishore

[Joshua Marantz]

I took a look at mod_caucho.c and it seems OK to me.  Can you increase the logging level?  As of yesterday's release, 0.9.17.6, mod_pagespeed prints the reason that it skips a rewrite (e.g. content doesn't look like HTML, content-type is not text/html, content is already gzipped, module is disabled, etc).  It does this at loglevel 'debug', so just put:

  loglevel debug

in httpd.conf and restart apache.  Refresh your jsp page from your browser and then take a look at the log.  You don't want to leave it running with 'loglevel debug' for a long time or the log file will fill your disk.

The log is usually some place like /var/log/apache2/error.log depending on your distro.

-Josh

[kishore]

Josh,

I upgraded the version to 0.9.17.6 and tried, but still no success.

Nothing gets printed in the error logs, when i access the jsp page
even after setting the log level to debug.

Also i have changed pagespeedallow to "ModPagespeedAllow *.jsp*"

Thanks,
Kishore

[Shawn Ligocki]

mod_pagespeed only gets called by Apache for files which serve with "Content-Type: text/html", do you think these have a different Content-Type?

The relavent line in pagespeed.conf is:

AddOutputFilterByType MOD_PAGESPEED_OUTPUT_FILTER text/html

-Shawn

[kishore]

Shawn,

But Content-Type seems to be 'text/html' only.

Thanks,
Kishore

[Joshua Marantz]

FYI

I have a new theory: that the content coming out of the JVM is gzipped, but the content-encoding flag is not set by the time mod_pagespeed processes the data.  Apache's mod_headers will add the content-encoding flag via content detection, but that is too late for mod_pagespeed.

mod_pagespeed will silently ignore already-gzipped content that lacks the content-encoding flag.  This is an open issue tracked in http://code.google.com/p/modpagespeed/issues/detail?id=8 .

Does this seem plausible to you?  Could the content coming out of the JVM be gzipped?  If so, is it possible to configure the Java infrastructure to turn that off and let Apache do the gzipping?  That will work better with mod_pagespeed.

At a minimum we should print a debug message when mod_pagespeed detects content that doesn't look like HTML.

-Josh

[kishore]

Josh,

Content coming out of JVM is not gzipped and it is Apache which does
the gzipping.

I have pasted the log of my app server, which says that the Content-
Type returned is text/html.


[18:28:26.057] [id:19] start request
[18:28:26.057] [id:19] P: dispatch protocol
[18:28:26.057] [id:19] etag: /9p0NDHEL3x
[18:28:26.057] [id:19] host: www.testing.com
[18:28:26.057] [id:19] query:
[18:28:26.058] [id:19] host 'Host[www.testing.com]' no change
[18:28:26.058] [id:19] n
[18:28:26.058] [id:19] Q: end of request
[18:28:26.059] [19] keepalive (thread)
[18:28:26.059] [id:19] start request
[18:28:26.059] [id:19] P: dispatch protocol
[18:28:26.059] [id:19] etag: /9p0NDHEL3x
[18:28:26.059] [id:19] host: www.testing.com
[18:28:26.059] [id:19] query:
[18:28:26.059] [id:19] host 'Host[www.testing.com]' no change
[18:28:26.059] [id:19] n
[18:28:26.059] [id:19] Q: end of request
[18:28:26.059] [19] keepalive (thread)
[18:28:26.059] [id:19] start request
[18:28:26.060] [id:19] U:uri /HelloWorld.jsp
[18:28:26.060] [id:19] m:method GET
[18:28:26.060] [id:19] c protocol: HTTP/1.1
[18:28:26.060] [id:19] v server-host: www.testing.com
[18:28:26.060] [id:19] g server-port: 80
[18:28:26.060] [id:19] h 10.0.102.21
[18:28:26.060] [id:19] i 10.0.102.21
[18:28:26.060] [id:19] j remote-port: 15771
[18:28:26.060] [id:19] H Host=www.testing.com
[18:28:26.060] [id:19] H User-Agent=Mozilla/5.0 (X11; U; Linux i686;
en-US; rv:1.9.2.17) Gecko/20110422 Ubuntu/10.04 (lucid) Firefox/3.6.17
[18:28:26.060] [id:19] H Accept=text/html,application/xhtml
+xml,application/xml;q=0.9,*/*;q=0.8
[18:28:26.060] [id:19] H Accept-Language=en-us,en;q=0.5
[18:28:26.060] [id:19] H Accept-Encoding=gzip,deflate
[18:28:26.060] [id:19] H Accept-Charset=ISO-8859-1,utf-8;q=0.7,*;q=0.7
[18:28:26.060] [id:19] H Keep-Alive=115
[18:28:26.060] [id:19] H Connection=keep-alive
[18:28:26.060] [id:19] H Cookie=JSESSIONID=abcJlYGHbe1AP6_hj3cat;
_jsuid=394352885719168048
[18:28:26.060] [id:19] H If-Modified-Since=Wed, 18 May 2011 12:53:40
GMT
[18:28:26.060] [id:19] H If-None-Match="AAAATADKzAg"
[18:28:26.060] [id:19] H Cache-Control=max-age=0
[18:28:26.060] [id:19] H SCRIPT_URL=/HelloWorld.jsp
[18:28:26.060] [id:19] H SCRIPT_URI=http:/www.testing.com/
HelloWorld.jsp
[18:28:26.060] [id:19] Q: end of request

[18:28:26.492] [id:19] s 200 OK
[18:28:26.492] [id:19] H ETag
[18:28:26.492] [id:19] S "AAAATADL41Q"
[18:28:26.492] [id:19] H Last-Modified
[18:28:26.492] [id:19] S Wed, 18 May 2011 12:58:26 GMT
[18:28:26.492] [id:19] H Content-Type
[18:28:26.492] [id:19] S text/html
[18:28:26.492] [id:19] G
[18:28:26.492] [id:19] D:data 391
[18:28:26.493] [id:19] data <<!DOCTYPE html PUBLIC "-/W3C/DTD XHTML
1.1/EN"
[18:28:26.493] "http:/www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
[18:28:26.493] <html xmlns="http:/www.w3.org/1999/xhtml"
xml:lang="en">
[18:28:26.493] <head>
[18:28:26.493]
[18:28:26.493]
[18:28:26.493] <title>Hello world</title>
[18:28:26.493] <meta http-equiv="Content-Type" content="text/html;
charset=utf-8" />
[18:28:26.493]
[18:28:26.493] </head>
[18:28:26.493]
[18:28:26.493]
[18:28:26.493] <body>
[18:28:26.493] <h1>Hello World</h1
[18:28:26.493] <h2>Hello World</h2>
[18:28:26.493] <h3>Hello World</h3>
[18:28:26.493] <h4>Hello World</h4>
[18:28:26.493]
[18:28:26.493]
[18:28:26.493] </body>
[18:28:26.493] </html>
[18:28:26.493]
[18:28:26.493] >
[18:28:26.500] [id:19] Q: quit channel


Thanks,
Kishore

[kishore]

Josh,

Did u get chance to look at this?

Thanks,
Kishore

[Joshua Marantz]

Hi Kishore,

I didn't understand the log that you pasted. I'm a little surprised that you found no mod-pagespeed-related debug messages  in your log.  Scanning through our code I think that at 'debug' level we print quite a bit of information on every request.

It must be that mod_pagespeed is not waking up.  But I looked at mod_caucho.c and could not find anything in there that looked like it would force mod_pagespeed to be bypassed.  However, that's my current guess: it's something in mod_caucho.c that prevents mod_pagespeed from processing the its output.

-Josh

[Hari Selvarajan]

Josh,

I work with Kishore and I've been following this thread. Thanks for
looking into this. I think I've found a workaround (I'm still not sure
of the cause):

AddOutputFilterByType doesn't seem to work with JSP files served by
mod_caucho. If I add an explicit AddOutputFilter for JSP files, I can
get it to work:

# Direct Apache to send all HTML output to the
mod_pagespeed
# output
handler.

# This doesn't work for JSP, just for HTML
AddOutputFilterByType MOD_PAGESPEED_OUTPUT_FILTER text/html

# This works!
AddOutputFilter MOD_PAGESPEED_OUTPUT_FILTER jsp

So I do think we have the solution to our problem. I wonder why the
installation guide for mod_pagespeed recommends AddOutputFilterByType
when it's deprecated by Apache:

http://httpd.apache.org/docs/2.2/mod/core.html#addoutputfilterbytype

"Enabling filters with AddOutputFilterByType may fail partially or
completely in some cases. For example, no filters are applied if the
MIME-type could not be determined and falls back to the DefaultType
setting, even if the DefaultType is the same.

However, if you want to make sure, that the filters will be applied,
assign the content type to a resource explicitly, for example with
AddType or ForceType. Setting the content type within a (non-nph) CGI
script is also safe."

Thanks again for looking at this.

- Hari

[Joshua Marantz]

Great analysis Hari!  Thanks!

On Wed, May 25, 2011 at 8:33 AM, Hari Selvarajan <hari.se...@gmail.com> wrote:

So I do think we have the solution to our problem. I wonder why the
installation guide for mod_pagespeed recommends AddOutputFilterByType
when it's deprecated by Apache:

http://httpd.apache.org/docs/2.2/mod/core.html#addoutputfilterbytype

This is a great question.  The reason we used this is that the alternatives suggested, AddOutputFitler and SetOutputFilter, don't seem to solve the problem.

AddOutputFilter works based on file extension, which seems problematic.  Much HTML is served without a ".html" extension but with a content-type of "text/html".

SetOutputFilter is non-cooperative.  You'd have to specify the entire filter-chain.  We tried using this a long time ago before we released, and we found that mod_pagespeed would be enabled but it would kill mod_deflate.  So you'd have to write

  SetOutputFilter MOD_PAGESPEED_OUTPUT_FILTER;DEFLATE

this is non-modular.  At least that was our experience.

The real problem, I'm guessing, is that mod_caucho doesn't set the content-type to "text/html" itself, but relies on mod_headers to do it much later in the filter-chain; too late for mod_pagespeed.

But the solution you propose looks great to me; it is restricted to files ending in ".jsp" and would not work for JSP content that lacks that extension (perhaps no such thing exists; I don't know much about jsp).

All that said, it would be great to use a non-deprecated mechanism from Apache, if one existed that met our needs.  Does anyone know why that directive was deprecated?

-Josh