Problems running site and CMS behind nginx

[msim...@gmail.com]

Hi again everyone.

I've made previous threads with some nginx questions, but I feel it's better to create a new thread with the questions I have. FYI, we're running Hippo 7.9.3. And apologies for loads of text, but I feel some context is required.

Background

We have a fairly standard setup. One site webapp (running in the "site" context), and one cms webapp. I'm now trying to figure out the appropriate nginx configurations to run:

www.example.com   (site.war)
cms.example.com   (cms.war)

And of course also handling things like cms.example.com/console.

I have read and used the following pages:
http://www.onehippo.org/library/deployment/configuring/configure-apache-httpd-web-server-for-sites.html
http://www.onehippo.org/library/deployment/configuring/configure-apache-httpd-web-server-for-cms-and-sites.html

Current Status

I've got all of the various (sub)domains working on nginx, so www.example.com, cms.example.com, and cms.example.com/console etc.

However, I can't get the Channel Manager working properly, as cms.example.com/site/_cmsinternal 404s. It just 'hourglasses', and doesn't load. I have included a /site proxy_pass for the cms subdomain, but clearly I'm missing something. I have set the hst.cmslocation for a newly configured hosts entry for the subdomain.

Also, all of the links generated using hst.link include the site context, so clicking any link results in the likes of www.example.com/site/page. This displays fine, but has some side-effects.

Finally, submitting a form on the homepage of our website goes to www.example.com/site?u_u_i_d... which throws a nginx error of "the rewritten URI has a zero length", as my URL rewrite line must be a bit dodgy.

Questions

1. How can I stop generated links including the site context? (hst.link , actionURLs)
2. What have I done wrong with the Channel Manager configuration? (see below)

I think fixing #1 will solve a lot of my problems. Or, at least, a better understanding of it would help.


My current nginx config

This is running on my windows PC at the moment, so I've set up localhost and cms.localhost hosts entries. And clearly I'm unhappy with some of it...

   

    server {
        listen       80;
        server_name  localhost;
       
        location / {
            proxy_pass http://tomcat/site/;
            proxy_cookie_path /site /;
            proxy_set_header Host $host;
        }
       
        location /site {
            proxy_pass http://tomcat/site/;
            proxy_cookie_path /site /;
            proxy_set_header Host $host;
            rewrite ^/site(.*)$ $1 last;
        }
    }
   
    server {
        listen       80;
        server_name  cms.localhost;

        location /site {
            proxy_pass http://tomcat/site/;
        }
       
        location /console {
            proxy_cookie_path /cms /console;
            proxy_pass http://tomcat/cms/console;
            proxy_intercept_errors on;
        }
       
        location / {
            proxy_cookie_path /cms /;
            proxy_pass http://tomcat/cms/;
        }
       
        # don't like this..! (fixes image previews in cms)
        location /binaries {
            proxy_pass http://tomcat/site/binaries;
        }
    }

    upstream tomcat {
       server localhost:8080;
    }

Any assistance would be appreciated, I feel I'm going around in circles here..!

Thanks again,
Mark.

[msim...@gmail.com]

Ok, I've rewritten the config trying to adhere to the example given on http://www.onehippo.org/library/deployment/configuring/configure-apache-httpd-web-server-for-cms-and-sites.html more closely, and this is what I've come up with.

While it's neater, it doesn't fix the problems:

* Channel Manager won't load
* "site" is still prefixed to all generated links

    server {
        listen       80;
        server_name  cms.localhost;

        location /site {
            proxy_pass http://tomcat/site/;
        }

       
        location / {

            proxy_cookie_path /cms /;
            proxy_pass http://tomcat/cms/;

            proxy_set_header X-Forwarded-Host $host;
            proxy_set_header X-Forwarded-Server $host;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        }
           
        location /console {
            proxy_pass http://tomcat/cms/console;
            proxy_cookie_path /cms /console;

        }
    }
   
    server {
        listen       80;
        server_name  localhost;
       
        location / {
            proxy_pass http://tomcat/site/;
            proxy_cookie_path /site /;

            proxy_set_header X-Forwarded-Host $host;
            proxy_set_header X-Forwarded-Server $host;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

[msim...@gmail.com]

Nope, the new config above is worse, so stick with the first one. Told you I was going around in circles!

Assistance would be appreciated.

[Jeroen Reijn]

Hi Mark,

since nginx seems to be quite a popular topic on the forum these days,  I thought it was worth exploring a little and took a shot at it while commuting to work today.

I have a working example for the CMS with a working channel manager, which you could copy. I'm not saying it's 100% perfect, but it comes close to the existing example we have for apache httpd.

Here is what I did in short. (I'll create a documentation or blog post about it later on, with some further details and also include a configuration for the site.)

This all works locally, so for a remote server it might be slightly different.

1. Locally I created an entry in my /etc/hosts file:

127.0.0.1 localhost cms.localhost

2. In nginx I've created a server configuration in the sites-enabled folder for the cms called cms.localhost with the following content:

server {


  listen       80;
  server_name "cms.localhost";

  location / {

    proxy_set_header X-Forwarded-Host $host;
    proxy_set_header X-Forwarded-Server $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

    proxy_pass http://localhost:8080/cms/;
    proxy_redirect default;
    proxy_cookie_path /cms/ /;
  }


  location /site/ {
    proxy_pass http://localhost:8080/site/;
  }


}

AFAIK that's all there is to it. I guess the important part is  The nginx documentation has quite a good reference when it comes to mimicking apache httpd.

See http://wiki.nginx.org/LikeApache

I guess in your case the redirect instruction was the only thing missing. I also think you don't need to explicitly set the /console location, since this is part of the CMS context.

Let me know if this works for you. The /site context part which is shown on the site is due to the hst:hosts configuration. You can disable showing the context path (and port) of the application in links.

See http://www.onehippo.org/library/deployment/configuring/configure-apache-httpd-web-server-for-sites.html

Cheers,

Jeroen

--
Hippo Community Group: The place for all discussions and announcements about Hippo CMS (and HST, repository etc. etc.)
 
To post to this group, send email to hippo-c...@googlegroups.com
RSS: https://groups.google.com/group/hippo-community/feed/rss_v2_0_msgs.xml?num=50
---
You received this message because you are subscribed to the Google Groups "Hippo Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hippo-communi...@googlegroups.com.
Visit this group at http://groups.google.com/group/hippo-community.
For more options, visit https://groups.google.com/d/optout.

--

Jeroen Reijn
Hippo

Amsterdam office - Oosteinde 11, 1017 WT Amsterdam

Boston office - 745 Atlantic Ave, Eight Floor, Boston MA 02111, United states of America.

US +1 877 414 4776 (toll free)
Europe +31(0)20 522 4466
www.onehippo.com

http://blog.jeroenreijn.com | @jreijn | http://about.me/jeroenreijn

[msim...@gmail.com]

Hi Jeroen,

Thanks for this, this does feel better to me. More works with less config, nice. This is what I see and know now.

* Channel Manager now errors with "Unable to change to composer mode. Please check if the site is online."

> Listing the available channels shows the URL as "http://localhost/site";

This path breaks because of my "rewrite ^/site(.*)$ $1 last;" line in localhost /site nginx config. Without this, blog images don't display on localhost/blog/.../article pages, because of the "site" context.

> The Preview link goes to "http://cms.localhost/site/_cmsinternal/" which displays a 403 error.

* Generated URLs and preview links still include the "site" context - is there a way to stop this without going back to running it as root?

Cheers,
Mark.

[Jeroen Reijn]

Well I think let's first focus on the CMS and channel manager part. The site part will probably take me a commute back home to create the config and do some tests.

I think I know why you get a "unable to change to composer mode". The Nginx configuration is really sensitive.

From your original post I noticed you had the following config:

  location /site {

    proxy_pass http://localhost:8080/site/;

  }

but it should be

  location /site/ {

    proxy_pass http://localhost:8080/site/;

  }

Note the additional slash at the end of the location, which I also use in my example. This needs to be very accurate otherwise it does not work properly. I noticed you have quite some location definitions without the ending '/', but do proxy to and endpoint that ends with '/'. This will probably give you unexpected errors.

Jeroen

[Jeroen Reijn]

BTW you should not need to add the /binaries location in your cms vhost config. This should work without setting it explicitly.

[msim...@gmail.com]

Hi Jeroen,

Yep, I've removed the /binaries path now, and it still works.

I've also added trailing slashes as you recommended (both locations and cookie paths), and that fixes the 500 when accessing localhost/site directly, thanks! It meant I could also remove that rewrite line I mentioned.

I'm still getting that Channel Manager error though. It's printing this out to the logs whenever it does:

[INFO] [talledLocalContainer] 15.04.2015 11:05:46 WARN  http-bio-8080-exec-16 [HstFilter.doFilter:526] ContainerException for '/site/_rp/cafebabe-cafe-babe-cafe-babecafebabe/composermode/localhost/': org.hippoecm.hst.core.container.ContainerException: CXFJaxrs
HstConfigService expects a 'uuid' as pathInfo but was 'cafebabe-cafe-babe-cafe-babecafebabe/composermode/localhost'. Cannot process REST call

Here's my complete nginx config, for reference:

    server {
        listen       80;
        server_name  cms.localhost;

        location / {
            proxy_set_header X-Forwarded-Host $host;
            proxy_set_header X-Forwarded-Server $host;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

           
            proxy_pass http://tomcat/cms/;

            proxy_redirect default;
            proxy_cookie_path /cms/ /;
          }

          location /site/ {

            proxy_pass http://tomcat/site/;

          }
    }
   
    server {
        listen       80;
        server_name  localhost;
       
        location / {
            proxy_pass http://tomcat/site/;

            proxy_cookie_path /site/ /;
        }
       
        # shouldn't need this at all
        location /site/ {
            proxy_pass http://tomcat/site/;
            proxy_cookie_path /site/ /;

        }
    }    
    upstream tomcat {
       server localhost:8080;
    }

Thanks again for your help.

[Jeroen Reijn]

Hi Mark,

well on the way back yesterday I hooked up the site. For nginx this was all I needed.

server {

  listen       80;

  server_name "site.local.dev";

  location / {

    proxy_set_header X-Forwarded-Host $host;

    proxy_set_header X-Forwarded-Server $host;

    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

    proxy_pass http://localhost:8080/site/;

    proxy_redirect default;

    proxy_cookie_path /site/ /;

  }

}

Now do keep in mind that I've not put my nginx config to match the localhost domain (like in your example), because that way you will have the some issues with the context path and port configuration, which you want to keep in case of local development without nginx.

Based on a simple essentials based archetype, I've created a new virtual host group that matches my nginx configured domain: dev-local

I've attached my virtualhostgroup as an attachment. You can import it locally in the console under the path: /hst:hst/hst:hosts

Now after importing it you will notice that at /hst:hst/hst:hosts/dev-local/dev/local/site I've set two specific properties (this will remove the /site context path in links, etc).

        <sv:property sv:name="hst:showcontextpath" sv:type="Boolean">

          <sv:value>false</sv:value>

        </sv:property>

        <sv:property sv:name="hst:showport" sv:type="Boolean">

          <sv:value>false</sv:value>

        </sv:property>

That should solve the site issues you are having. Also for this new host I've configured the correct url to the cms. In my case (I changed it a little since last time) http://cms.local.dev in the node at path /hst:hst/hst:hosts/dev-local

  <sv:property sv:name="hst:cmslocation" sv:type="String">

    <sv:value>http://cms.local.dev</sv:value>

  </sv:property>

It's important that this is set up correctly. With regards to the channel manager issues you're having, my gut feeling says there is something wrong in your hst (host) configuration. This is a little hard to figure out right now.

For some more debugging options you might want to check: http://www.onehippo.org/library/concepts/channels/channel-manager-troubleshooting.html

I'll write this all up in an upcoming labs[1] post. Will ping you when it's published.

[1]http://www.onehippo.org/labs

[msim...@gmail.com]

Hi Jeroen,

Thanks for spending your time helping me, it's much appreciated! I think the new documentation would be really useful.

I wasn't too concerned about using localhost as a domain, as it was only temporary development before moving to a server with a proper domain, but for consistency (and in case there were other side-effects) I moved to your suggested local.dev host. Makes sense.

I had to make a few tweaks to the nginx config you suggested (which I'll list below for your reference), but these combined with the hst:hosts parameters have almost got it fully working! (almost) It's so nice to see no "site" context :).

• I added "proxy_set_header Host $host;" to site.local.dev / path. We have a form on the homepage of the site, and it wasn't passing postdata back properly without this.
• I made the "/site/" location for cms.local.dev exactly the same as "/" for site.local.dev. Otherwise, the URL link in the Channel Manager list view doesn't render/behave properly.
  

However, I'm still having the same Channel Manager problems. I've looked through that troubleshooting page you linked me, with no joy. (For example, the http://cms.local.dev/site/_rp/cafebabe-cafe-babe-cafe-babecafebabe./composermode/cms.local.dev/?FORCE_CLIENT_HOST=true&_dc=1429179283902 link in firebug 500's, not 404s. It looks to be in a different format to the example on that troubleshooting page...). I can't see anything obviously wrong with my host configurations, and enabling diagnostics didn't show anything more that helped me understand the problem.

I'm still getting this printed out to the logs each time:

[INFO] [talledLocalContainer] 16.04.2015 11:15:31 WARN  http-bio-8080-exec-14 [HstFilter.doFilter:526] ContainerException for '/site/_rp/cafebabe-cafe-babe-cafe-babecafebabe/composermode/cms.local.dev/': org.hippoecm.hst.core.container.ContainerException: CXFJaxrsHstConfigService expects a 'uuid' as pathInfo but was 'cafebabe-cafe-babe-cafe-babecafebabe/composermode/cms.local.dev'. Cannot process REST call

Almost there!

Thanks again,
Mark.

...

[msim...@gmail.com]

Slight update - I realised I had a hosts configuration for cms.local.dev that I didn't need, so when I removed that, I could simplify the nginx configuration to match exactly what you provided, except for two minor areas:

• The addition of "proxy_set_header Host $host;" in the "/" location for site.local.dev
• Using a simple upstream definition rather than hardcoding "localhost:8080" in multiple places

I've tried making the config match your example perfectly, and everything works except the homepage form submit (hence the proxy_set_header), and the Channel Manager still fails in the same way.

I've also tried playing with the CMS console to fix this, but if I'm honest I'm out of ideas...

...

[Ard Schrijvers]

Hey,

On Mon, Apr 20, 2015 at 9:36 AM, <msim...@gmail.com> wrote:
> Slight update - I realised I had a hosts configuration for cms.local.dev
> that I didn't need, so when I removed that, I could simplify the nginx
> configuration to match exactly what you provided, except for two minor
> areas:
>
> The addition of "proxy_set_header Host $host;" in the "/" location for
> site.local.dev
> Using a simple upstream definition rather than hardcoding "localhost:8080"
> in multiple places
>
> I've tried making the config match your example perfectly, and everything
> works except the homepage form submit (hence the proxy_set_header), and the
> Channel Manager still fails in the same way.
>
> I've also tried playing with the CMS console to fix this, but if I'm honest
> I'm out of ideas...

perhaps you can contact Hippo via sales (or perhaps infra does read
this and can refer to some troubleshooting) that you require some
infra support for settings up nginx. Most likely they can very easily
help you. Of course you are free to ask on this list, but I am afraid
that it is mainly difficult because we do not have your complete setup
and the developers on this list are more experienced with httpd.

Regards Ard

> --
> Hippo Community Group: The place for all discussions and announcements about
> Hippo CMS (and HST, repository etc. etc.)
>
> To post to this group, send email to hippo-c...@googlegroups.com
> RSS:
> https://groups.google.com/group/hippo-community/feed/rss_v2_0_msgs.xml?num=50
> ---
> You received this message because you are subscribed to the Google Groups
> "Hippo Community" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to hippo-communi...@googlegroups.com.
> Visit this group at http://groups.google.com/group/hippo-community.
> For more options, visit https://groups.google.com/d/optout.



--

Hippo Netherlands, Oosteinde 11, 1017 WT Amsterdam, Netherlands
Hippo USA, Inc.- 745 Atlantic Ave, Eight Floor, Boston MA 02111,

[Jeroen Reijn]

Hi Mark,

this really has to do with the way the CMS is configured and your NGINX config. Via this medium it's quite hard to figure out why it's not working. Like Ard suggested it might be usefull to contact us (Hippo) for further assistence.

Anyway I've created a new Hippo labs post[2] and blog entry[1] on how to configure nginx as a reverse proxy for Hippo CMS, which should be reproducable. Maybe you can start clean without your existing project and see if you can get it to work and from there on figure out what's wrong in your particular situation.

[1] http://www.onehippo.com/en/resources/blogs/facet/how-to-configuring-nginx-as-reverse-proxy-for-hippo-cms.html

[2] http://www.onehippo.org/labs/configuring-nginx-as-a-reverse-proxy-for-hippo-cms.html

Jeroen  

On Mon, Apr 20, 2015 at 9:36 AM, <msim...@gmail.com> wrote:

--

Hippo Community Group: The place for all discussions and announcements about Hippo CMS (and HST, repository etc. etc.)
 
To post to this group, send email to hippo-c...@googlegroups.com
RSS: https://groups.google.com/group/hippo-community/feed/rss_v2_0_msgs.xml?num=50
---
You received this message because you are subscribed to the Google Groups "Hippo Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hippo-communi...@googlegroups.com.
Visit this group at http://groups.google.com/group/hippo-community.
For more options, visit https://groups.google.com/d/optout.

[msim...@gmail.com]

Hi Jeroen, Ard,

Thanks for your help on this matter. I've now solved the final Channel Manager error, so this is for the benefit of anyone else having the same issues.

Basically... the problem is Windows. I was testing my nginx configuration on my Windows development PC, using a Windows port of nginx (which is a beta after all), and try as I might, I just couldn't get it working. Even with a fresh clone of the Hippo CMS.

Wondering if it was something to do with Windows, I decided to try again on our linux test servers, and following the advice here and in the previously mentioned blog posts, I got it working fairly easily.

So, yes, if you have this same problem, try it on a linux based OS instead!

Thanks again, and all the best.

Mark.

[tbuck...@gmail.com]

Hi Jeroen,

I recently followed your nginx configuration guide, and had to make a few tweaks to get everything working correctly. I thought I'd share my changes here in case they are useful to anyone else.

I'm running NGINX and tomcat on separate ubuntu linux servers. Hippo is deployed as a standard project distribution on the tomcat server (site.war and cms.war). When I follow the example NGINX configuration exactly, most things work correctly but the Template Composer (channel manager) in the CMS application is unable to enter edit mode, unable to retrieve the sitemap, etc. The changes I had to make to get the template composer working were:

Send "Host" header when proxying to the /site/ location of the cms:

location /site/ {

    proxy_set_header Host $host;

    proxy_pass http://hippo/site/;

}

Put this directive into the main http configuration section in nginx.conf (NGINX removes HTTP headers with names that contain underscores in proxied requests by default. This caused the CMS app some problems)

http {
    underscores_in_headers on;
}

I see that you have mentioned setting the Host header in this stack overflow post, but it's not currently in the documentation on onehippo.org. Might be worth adding that point as well as the underscores_in_headers directive to the doc on the site to save others some digging. 

Tyler

[Bart van der Schans]

Hi Tyler,

Thanks for reporting back!

We have addressed the underscore header issue and new releases will no
longer use headers with underscores in the name:

https://issues.onehippo.com/browse/HSTTWO-3820

So that is at least one thing less to remember ;-)

Regards,
Bart

> Visit this group at https://groups.google.com/group/hippo-community.

>
> For more options, visit https://groups.google.com/d/optout.



--

Hippo B.V. - Oosteinde 11, 1017 WT Amsterdam
Hippo USA, Inc. - 71 Summer Street, 2nd Floor, Boston, MA 02110

US +1 877 414 47 76 (toll free)
NL +31 20 522 44 66
UK +44 20 35 14 99 60
DE +49 69 80 88 40 67

http://www.onehippo.com
http://www.onehippo.org