Nginx, Passenger and Caching

[sol]

Hi, just a quick question..

I'm trying to move some rewrite rules for mephisto running on nginx
+mongrel to nginx+passenger.

It basically looks like this:

location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect false;

if (-f $document_root/cache/$host$uri/index.html) {
rewrite (.*) /cache/$host$1/index.html last;
}

if (-f $document_root/cache/$host$uri.html) {
rewrite (.*) /cache/$host$1.html last;
}

if (-f $document_root/cache/$host$uri) {
rewrite (.*) /cache/$host$1 last;
}

if (!-f $request_filename) {
proxy_pass http://mongrel;
break;
}

location ~ ^/assets/\d+/ {
rewrite ^/assets/(\d+)/(.+)$ /assets/$host/$1/$2 last;
}


The rules are straight forward, just add .html etc if the files exist.
The problem is the proxy_pass part, I don't know how to pass the
request to passenger if no cached files are found... does anyone have
a hint for me?

[Hongli Lai]

On Mon, Apr 27, 2009 at 10:50 PM, sol <ch.b...@gmail.com> wrote:
> The rules are straight forward, just add .html etc if the files exist.
> The problem is the proxy_pass part, I don't know how to pass the
> request to passenger if no cached files are found... does anyone have
> a hint for me?

You don't need to. If there's a corresponding .html file, then Phusion
Passenger will serve it automatically without special rewrite rules.
If there's a corresponding file with the same name as the URI, then
that will be served automatically. Everything else is automatically
passed to the application.

--
Phusion | The Computer Science Company

Web: http://www.phusion.nl/
E-mail: in...@phusion.nl
Chamber of commerce no: 08173483 (The Netherlands)

[sol]

Thanks for the quick reply :)

Ok just one question to clarify this, if I have a request like '/' or
'/about'
what exactly does passenger do?

Does it check for '/index.html' and 'about.html'?

In that case I would only have to add the '/cache/' directory prefix I
guess..
Is there any documentation about it?

On Apr 27, 10:53 pm, Hongli Lai <hon...@phusion.nl> wrote:

> On Mon, Apr 27, 2009 at 10:50 PM, sol <ch.bl...@gmail.com> wrote:
> > The rules are straight forward, just add .html etc if the files exist.
> > The problem is the proxy_pass part, I don't know how to pass the
> > request to passenger if no cached files are found... does anyone have
> > a hint for me?
>
> You don't need to. If there's a corresponding .html file, then Phusion
> Passenger will serve it automatically without special rewrite rules.
> If there's a corresponding file with the same name as the URI, then
> that will be served automatically. Everything else is automatically
> passed to the application.
>
> --
> Phusion | The Computer Science Company
>
> Web:http://www.phusion.nl/

> E-mail: i...@phusion.nl

[Hongli Lai]

On Mon, Apr 27, 2009 at 10:58 PM, sol <ch.b...@gmail.com> wrote:
>
> Thanks for the quick reply :)
>
> Ok just one question to clarify this, if I have a request like '/' or
> '/about'
> what exactly does passenger do?
>
> Does it check for '/index.html' and 'about.html'?

No, it checks for $DOCUMENT_ROOT/about.html, and if that doesn't
exist, $DOCUMENT_ROOT/about. If that file doesn't exist either then it
will be passed to the Rails app.

So page caching works as expected. Static assets serving works as
expected. Only if your page cache files do not follow the standard
file naming will you have to use rewrite rules. Everything else is
take care of automatically.

--
Phusion | The Computer Science Company

Web: http://www.phusion.nl/
E-mail: in...@phusion.nl

[sol]

Hi again,

Thanks for the clarification Hongli!

However, I'm using multisite mephisto for this app, which has the
cache under public/cache/domain.tld/.
And somehow I can't get passenger to work with that correctly.

I use these rewrite rules inside the server {} block:

rewrite ^/$ /cache/$host/index.html break; # / -> /cache/
example.com/index.html
rewrite ^([^.]+?)/?$ /cache/$host$1.html break; # /about/ -> /
cache/example.com/about.html

location ~ ^/(images|javascripts|stylesheets)/ {
if (!-e $request_filename) {
rewrite ^/(images|javascripts|stylesheets)/(.+)$ /cache/$host/
$1/$2 break;
}
}

# /images/fubar.jpg -> /cache/example.com/images/fubar.jpg

The first two rules are working fine, however, the problem is the last
one for my assets.
It only works if the file exists, if it doesn't exist, the request is
not forwarded to mephisto, and the cache file not generated.

Can anyone give me a hint please why this happens? I can't figure out
how to pass it to passenger...




On Apr 27, 11:29 pm, Hongli Lai <hon...@phusion.nl> wrote:

> On Mon, Apr 27, 2009 at 10:58 PM, sol <ch.bl...@gmail.com> wrote:
>
> > Thanks for the quick reply :)
>
> > Ok just one question to clarify this, if I have a request like '/' or
> > '/about'
> > what exactly does passenger do?
>
> > Does it check for '/index.html' and 'about.html'?
>
> No, it checks for $DOCUMENT_ROOT/about.html, and if that doesn't
> exist, $DOCUMENT_ROOT/about. If that file doesn't exist either then it
> will be passed to the Rails app.
>
> So page caching works as expected. Static assets serving works as
> expected. Only if your page cache files do not follow the standard
> file naming will you have to use rewrite rules. Everything else is
> take care of automatically.
>
> --
> Phusion | The Computer Science Company
>
> Web:http://www.phusion.nl/

> E-mail: i...@phusion.nl

[Hongli Lai]

On Tue, Apr 28, 2009 at 3:46 PM, sol <ch.b...@gmail.com> wrote:
>
> Hi again,
>
> Thanks for the clarification Hongli!
>
> However, I'm using multisite mephisto for this app, which has the
> cache under public/cache/domain.tld/.
> And somehow I can't get passenger to work with that correctly.
>
> I use these rewrite rules inside the server {} block:
>
> rewrite ^/$ /cache/$host/index.html break;             # / -> /cache/
> example.com/index.html
> rewrite ^([^.]+?)/?$ /cache/$host$1.html break;        # /about/ -> /
> cache/example.com/about.html
>
> location ~ ^/(images|javascripts|stylesheets)/ {
>  if (!-e $request_filename) {
>    rewrite ^/(images|javascripts|stylesheets)/(.+)$ /cache/$host/
> $1/$2 break;
>  }
> }
>
> # /images/fubar.jpg -> /cache/example.com/images/fubar.jpg
>
> The first two rules are working fine, however, the problem is the last
> one for my assets.
> It only works if the file exists, if it doesn't exist, the request is
> not forwarded to mephisto, and the cache file not generated.
>
> Can anyone give me a hint please why this happens? I can't figure out
> how to pass it to passenger...

Try removing the "break". I think "break" means "let Nginx's default
handler handle this URI instead of whatever third party module might
be active".

--
Phusion | The Computer Science Company

Web: http://www.phusion.nl/
E-mail: in...@phusion.nl

[sol]

I've tried 'last' instead of 'break' which basically doesn't stop and
return it but looks for other rules, and leaving it empty.
But all options return the same result... as if passenger isn't
working inside a location block.

I'll try asking in the nginx forum..

On Apr 28, 6:47 pm, Hongli Lai <hon...@phusion.nl> wrote:

> E-mail: i...@phusion.nl

[lardawge]

Hongli Lai,

"Everything else is automatically passed to the application."

Can you clarify "everything else"? So is there no longer a need to
include any of this location block?

location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect false;
if (-f $document_root/cache/$host$uri/index.html) {
rewrite (.*) /cache/$host$1/index.html last;
}
if (-f $document_root/cache/$host$uri.html) {
rewrite (.*) /cache/$host$1.html last;
}
if (-f $document_root/cache/$host$uri) {
rewrite (.*) /cache/$host$1 last;
}
if (!-f $request_filename) {
proxy_pass http://mongrel;
break;
}

Can we assume that some magic between "nginx" and passenger is
happening? If not what is the setting for proxy_pass?

Thanks,
Larry

On Apr 27, 4:53 pm, Hongli Lai <hon...@phusion.nl> wrote:

> On Mon, Apr 27, 2009 at 10:50 PM, sol <ch.bl...@gmail.com> wrote:
> > The rules are straight forward, just add .html etc if the files exist.
> > The problem is the proxy_pass part, I don't know how to pass the
> > request to passenger if no cached files are found... does anyone have
> > a hint for me?
>
> You don't need to. If there's a corresponding .html file, then Phusion
> Passenger will serve it automatically without special rewrite rules.
> If there's a corresponding file with the same name as the URI, then
> that will be served automatically. Everything else is automatically
> passed to the application.
>
> --
> Phusion | The Computer Science Company
>
> Web:http://www.phusion.nl/

> E-mail: i...@phusion.nl

[Hongli Lai]

On Sat, May 2, 2009 at 12:59 PM, lardawge <lard...@gmail.com> wrote:
>
> Hongli Lai,
>
> "Everything else is automatically passed to the application."
> Can you clarify "everything else"? So is there no longer a need to
> include any of this location block?

Static assets and page cache files. If the URI maps to a filename
which exists, then that file is automatically served without hitting
the Rails app. If there's a corresponding page file .html file, then
that file is automatically served without hitting the Rails app. All
this happens automatically without the need for rewrite rules.

So the location block is not necessary.

--
Phusion | The Computer Science Company

Web: http://www.phusion.nl/
E-mail: in...@phusion.nl

[lardawge]

Thanks for all your answers! It is really useful info and could be
added to the docs for passenger-nginx (unless it is and i'm not seeing
it).
Last thing, what header info is passed? Is it still necessary to pass
it manually or does passenger pick it up auto-magically?

Thanks again for all your work on passenger, it rocks!

Larry

On May 3, 2:54 pm, Hongli Lai <hon...@phusion.nl> wrote:

> On Sat, May 2, 2009 at 12:59 PM, lardawge <larda...@gmail.com> wrote:
>
> > Hongli Lai,
>
> > "Everything else is automatically passed to the application."
> > Can you clarify "everything else"? So is there no longer a need to
> > include any of this location block?
>
> Static assets and page cache files. If the URI maps to a filename
> which exists, then that file is automatically served without hitting
> the Rails app. If there's a corresponding page file .html file, then
> that file is automatically served without hitting the Rails app. All
> this happens automatically without the need for rewrite rules.
>
> So the location block is not necessary.
>
> --
> Phusion | The Computer Science Company
>
> Web:http://www.phusion.nl/

> E-mail: i...@phusion.nl

[sol]

There is still a problem with this magic though, see
http://groups.google.com/group/phusion-passenger/browse_thread/thread/52fb0c28ad61cc45

I can't get my above location block to work and pass the request back
to passenger.
So any clarification how passenger works with location blocks would be
very useful...
I guess although not necessary, it should not limit the nginx
configuration in any way..

On May 3, 8:54 pm, Hongli Lai <hon...@phusion.nl> wrote:

> On Sat, May 2, 2009 at 12:59 PM, lardawge <larda...@gmail.com> wrote:
>
> > Hongli Lai,
>
> > "Everything else is automatically passed to the application."
> > Can you clarify "everything else"? So is there no longer a need to
> > include any of this location block?
>
> Static assets and page cache files. If the URI maps to a filename
> which exists, then that file is automatically served without hitting
> the Rails app. If there's a corresponding page file .html file, then
> that file is automatically served without hitting the Rails app. All
> this happens automatically without the need for rewrite rules.
>
> So the location block is not necessary.
>
> --
> Phusion | The Computer Science Company
>
> Web:http://www.phusion.nl/

> E-mail: i...@phusion.nl