SEVERE: Error deploying web application directory /opt/railo/tomcat/webapps/host-manager

[Adam Stetzer]

Hi all.

I've been reading, reading and re-reading about Railo4 / Tomcat and can't seem to figure out how to solve an issue (running on Centos 5.9). Hoping you can help.

We run a multi-site application (over 400 domain names) in a single Railo instance / single context. It actually has been running fine in a load-balanced production environment for over a year. But my logs are dirty dirty dirty. I'd like to clean them up. And I can't help but wonder if there is overhead from what I'm seeing in the Catalina.out files.

The logs are full of these:

Apr 22, 2014 8:15:14 AM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deploying web application directory /opt/railo/tomcat/webapps/host-manager
Apr 22, 2014 8:15:14 AM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deploying web application directory /opt/railo/tomcat/webapps/tomcat-manager
Apr 22, 2014 8:15:14 AM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deploying web application directory /opt/railo/tomcat/webapps/tomcat-docs
Apr 22, 2014 8:15:14 AM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deploying web application directory /opt/railo/tomcat/webapps/host-manager

And then occasionally this bad one:

Apr 22, 2014 8:15:14 AM org.apache.catalina.startup.HostConfig deployDirectory
SEVERE: Error deploying web application directory /opt/railo/tomcat/webapps/host-manager
java.lang.IllegalArgumentException: addChild:  Child name '/host-manager' is not unique
	at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:887)
	at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:877)
	at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:633)
	at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1114)
	at org.apache.catalina.startup.HostConfig$DeployDirectory.run(HostConfig.java:1673)
	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
	at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
	at java.util.concurrent.FutureTask.run(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)
After about 6 hours of reading and research, it seems that I have a miss-match between the code base I've defined in Railo (which I must have done at install) and where the Tomcat config is looking for it. I guess. I tried changing the default webapp / codebase settings in Server.xml - but only ended up breaking Railo.
In other posts I've read that these errors are actually benign. Would love some comfirmation.
Still other posts say I can turn off the host-manager and tomcat-docs functions by moving the folders out. That also broke Railo.
Any help would be appreciated. I'm happy to provide more technical details. Keep in mind, with the number of unique domain names coming into this application, explicitly defining them one-by-one in the Server.xml is off-the-table. 

[Igal @ getRailo.org]

yes, this information is only revealed in the 3rd re-read and you only re-read it twice ;)

are you using a web server?  what connector are you using between the web server and Tomcat?  how are your hosts generated (i.e. hardwired in server.xml or dynamically created on the fly)?

you can disable the host-manager and tomcat-docs, and if you're not using the host-manager then you should disable it in production environment.  if that breaks your Tomcat then that means that they are referenced in another location which should also be updated.

actually I see in your email that you tried to hardwire the hosts in server.xml and got errors.  your best solution would be to figure out these errors and fix it that way.

--
Did you find this reply useful? Help the Railo community and add it to the Railo Server wiki at https://github.com/getrailo/railo/wiki
---
You received this message because you are subscribed to the Google Groups "Railo" group.
To view this discussion on the web visit https://groups.google.com/d/msgid/railo/79a8d8fc-3cea-4f63-a550-f52545109313%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

-- 
Igal Sapir
Railo Core Developer
http://getRailo.org/

[Adam Stetzer]

Thanks so much for your reply... I have been reading, researching, and testing for days. Can't really understand this very well (probably my thickness).

I used the Railo4 installer on CentOS 5.0. Web server is Apache2. With Tomcat and Catalina. I see reference to mod_cfml for dynamic creation of the hosts. Based on my architecture, I can NOT hard-code the hosts in server.xml - there are hundreds and more are added every day (see above). 

"your best solution would be to figure out these errors and fix it that way." - YES - This is where I'm stuck... 

I guess my question is, how do you set up Railo / Apache / Tomcat / Catalina for a multi-site but SINGLE context environment, without having these errors?  I can't seem to find a clear answer. One thread got close when it said to "make your context the default context" - but it didn't work for me. 

[Igal @ getRailo.org]

maybe Jordan will be able to help.  he's actually the one who built that installer and I think he also wrote mod_cfml.

To view this discussion on the web visit https://groups.google.com/d/msgid/railo/7216036b-8cf3-4484-b590-36d5673d43b2%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Adam Stetzer]

That would be awesome. It seems the installer expects that there will be a context for each website. This is certainly logical, but it is not our architecture. I have seen the Mura folks having the same problem - I don't know that product, but suspect from the posts that it is a large CMS architecture where the code base handles the URL and many domain names run from the single code base (same as us).

The conflict seems to be that Tomcat wants to write all sorts of files based on the website url, as a generically-named folder in the same place. The Railo instance seems stable, but the catalina.out log is a mess. And I am worried about the overhead.

I have tuned a few settings in mod_cfml related to 

1) the number of contexts permitted within 24 hour period

2) The wait time between context creation

This helped clean up a few errors, but I am still getting the "SEVERE: Error deploying web application directory /opt/railo/tomcat/webapps/host-manager"

Any help would be greatly appreciated. 

[Adam Stetzer]

Still struggling with this.. The performance is ok, but every now and then a domain responds with the default code base rather than the directory defined in the Apache default / catch-all setting.

I see that Catalina is building folder for every context (domain name) even though I use the same code base for all contexts...  

Sure wish I could figure this one out... 

[Judith Barnett]

I'd be interested in the solution for this as well, as I am about to launch a batch of sites that use the same code base but load different dynamic content based on the domain name.

To view this discussion on the web visit https://groups.google.com/d/msgid/railo/aec2240d-8591-44a1-94f8-4f071c3dbc45%40googlegroups.com.

[Jordan Michaels]

> Still struggling with this.. The performance is ok, but every now and
> then a domain responds with the default code base rather than the
> directory defined in the Apache default / catch-all setting.

This can happen if tomcat hasn't had enough time to finish creating a
context for a specific domain. Either give subsequent requests more time
or give your server more memory/CPU/IO so that new contexts are created
faster.

If *every* request comes through to the default code, then that means
the initial request that created the context didn't get passed properly.
The only way to fix it is to reload Tomcat or use the Tomcat host
manager to remove it so it can be re-created. The host manager is not
available by default and must be installed manually.

> I see that Catalina is building folder for every context (domain
> name) even though I use the same code base for all contexts...

This is normal. Contexts are part of the J2EE spec. The brilliant minds
behind Railo allow you to customize where your domain-specific WEB-INF
goes, but one needs to be created for each context no matter what.

You can customize where the directory is created by modifying your
{railo}/tomcat/conf/web.xml file.

Uncomment this:

<init-param>
<param-name>railo-web-directory</param-name>

<param-value>/var/Railo/config/web/{web-context-label}/</param-value>
<description>Railo Web Directory (for Website-specific
configurations, settings, and libraries)</description>
</init-param>

and customize the <param-value> to point to wherever you want your
context directory to go:
<param-value>C:\railo\webcontexts\{web-context-label}/</param-value>

... or wherever you want to place your WEB-INF directories.

Warm Regards,
Jordan Michaels

>> you're not using the host-manager then you /should/

>>> <https://groups.google.com/d/msgid/railo/79a8d8fc-3cea-4f63-a550-f52545109313%40googlegroups.com?utm_medium=email&utm_source=footer>.

>>> For more options, visit
>>> https://groups.google.com/d/optout

>>> <https://groups.google.com/d/optout>.

>>
>> --
>> Igal Sapir
>> Railo Core Developer
>> http://getRailo.org/
>>
>> --
>> Did you find this reply useful? Help the Railo community and
>> add it to the Railo Server wiki at
>> https://github.com/getrailo/railo/wiki

>> <https://github.com/getrailo/railo/wiki>
>> ---
>> You received this message because you are subscribed to the
>> Google Groups "Railo" group.
>> To view this discussion on the web visit

>> https://groups.google.com/d/msgid/railo/7216036b-8cf3-4484-b590-36d5673d43b2%40googlegroups.com
>> <https://groups.google.com/d/msgid/railo/7216036b-8cf3-4484-b590-36d5673d43b2%40googlegroups.com?utm_medium=email&utm_source=footer>.

>> For more options, visit https://groups.google.com/d/optout

>> <https://groups.google.com/d/optout>.

>
> --
> Igal Sapir
> Railo Core Developer
> http://getRailo.org/
>
> --
> Did you find this reply useful? Help the Railo community and add it to
> the Railo Server wiki at https://github.com/getrailo/railo/wiki
> ---
> You received this message because you are subscribed to the Google
> Groups "Railo" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/railo/aec2240d-8591-44a1-94f8-4f071c3dbc45%40googlegroups.com
> <https://groups.google.com/d/msgid/railo/aec2240d-8591-44a1-94f8-4f071c3dbc45%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Adam Stetzer]

 

Thanks so much Jordan.. I think your analysis is correct. I have been playing with the timeout settings and wait settings in server.xml. My machines are VERY powerful and CPU usage is 0.01 - 32 gigs of RAM - load balanced on 4 machines (each 32 gig and low CPU) - so I don't think resources are the issue. The CONTEXT LIMIT in SERVER.XML was a possible a problem - which I recently increased)

I guess we might be talking past each other though. Using a single code base for ALL requests (many domain names, all managed in a single context) why do we need a WEB-INF PER domain name when there is only 1 context. I can tell you, I don't have a unique WEB-INF per domain name. They are probably over-writing each other. Who knows. 

It seems silly that Tomcat / Cataline has to build a ROOT.xml in /opt/railo/tomcat/conf/Catalina for each and every domain name request when they are all exactly the same. And you are correct, if one does not get built correctly, the domain will never render (until a restart of Tomcat).

POSSIBLE BUG: If a ROOT.xml file / folder is not build correctly in /opt/railo/tomcat/conf/Catalina for a domain name (or I manually delete it), Tomcat will default to showing it's default codebase directory REGARDLESS of what you define as the docbase in server.xml.  This seems like a bug to me. For example:

I defined this in server.xml: 

<Context path="" docBase="/var/html/wwwroor" />

However, if Tomcat can't find the ROOT.xml for a requested domain name in /opt/railo/tomcat/conf/Catalina it throws an error saying "page not found in /opt/railo/tomcat/webapps/ROOT/" .  I think the work-around is to ALWAYS put your code in the default location - but that seems buggy. 

Regardless, there are many applications that need a single code-base architecture. Forcing admins to manually build the HOST ENTRY manually in server.xml is severely limiting (especially for large applications). We have over 400 domains and growing daily - so we need the Host Names managed dynamically...  These are the very applications I would think Railo would want (as opposed to forcing them back to CF10 - which managed this with no problems). 

I'm a big fan of Railo - but the Management Team here is saying "they said it would work JUST like ACF" - and in the case of large single code-base applications with MANY domain names using a single context, it really is not. 

Thoughts? Work-arounds?

[Jordan Michaels]

First, let's get realistic with our expectations. Railo is not ACF, nor
does Railo intend to be an exact duplicate of ACF - that would be both
impossible and terrible business. Having said that, the team behind
Railo has done a remarkable and frankly *awe-inspiring* job and getting
Railo to be extremely compatible with ACF. Even then, the compatibility
was driven more by user request than Railo desiring to to be a
replacement for ACF.

Next, you have to understand that in order to support some of the
functionality you're wanting from Railo, Railo would need to break away
from the Servlet Context specification, which, if you're interested,
states the following:

SRV.3.6 Multiple Hosts and Servlet Contexts
Web servers may support multiple logical hosts sharing one IP address on
a server. This capability is sometimes referred to as "virtual hosting".
In this case, each logical host must have its own servlet context or set
of servlet contexts. Servlet contexts can not be shared across virtual
hosts.

Railo's default configuration follows this specification, and each
"logical host" utilises its own context. ACF, for better or worse, does
not follow this specification, and instead uses a single context to
render all CFML templates regardless of the domain name being requested.
Logical file system paths are interpreted dynamically. As you've seen,
there are some advantages to this, so I, personally, don't fault Adobe
for doing this; it's just different.

Warm Regards,
Jordan Michaels

[G. Dillenberg]

Am Mittwoch, 23. April 2014 16:51:43 UTC+2 schrieb Adam Stetzer:
> That would be awesome. It seems the installer expects that there will be a context for each website. This is certainly logical, but it is not our architecture. I have seen the Mura folks having the same problem - I don't know that product, but suspect from the posts that it is a large CMS architecture where the code base handles the URL and many domain names run from the single code base (same as us).
>
> The conflict seems to be that Tomcat wants to write all sorts of files based on the website url, as a generically-named folder in the same place. The Railo instance seems stable, but the catalina.out log is a mess. And I am worried about the overhead.
>
>
> I have tuned a few settings in mod_cfml related to 
>
>
> 1) the number of contexts permitted within 24 hour period
> 2) The wait time between context creation
>
>
> This helped clean up a few errors, but I am still getting the "SEVERE: Error deploying web application directory /opt/railo/tomcat/webapps/host-manager"
>

Delete the "<Valve className="mod_cfml.core"...." container in server.xml and all wil work fine.

[Adam Stetzer]

"Delete the "<Valve className="mod_cfml.core"...." container in server.xml and all wil work fine."

Wow. Really? If that is true, that would be awesome. I was under the impression I HAD to have the mod_cfml running to make Apache and Tomcat work together.

I will try this in a test environment and report back.

[Jordan Michaels]

Hi Adam,

Since you're using only a single context and a single webapp to handle
all domain requests, what you could do is modify your Tomcat config to
utilize your webapp location as its default webapp.

To do this, first create a host that points to your default webapp
location. You can do this by editing the {railo}/tomcat/conf/server.xml
file and adding a <Host> to the bottom of the file. The comments include
a fill-in-the-blank template that might help.

Once you've added your context, you can edit this line:

<Engine name="Catalina" defaultHost="127.0.0.1">

...and change "defaultHost" to point to the context of your webapp you
just created. After that, all domains that resolve to your server and
are passed to Tomcat by Apache will be handed off to your "defaultHost"
without any new contexts being created for each new domain name.

If your code handles the domain name interpretation, then this would be
a simple, low-overhead way of managing multiple domains with a single
context.

Warm Regards,
Jordan Michaels

[Adam Stetzer]

Thanks Jordan. I greatly appreciate your time and thoughts.

I will give this a go. I assume the actual name I select will not matter, since it will become the defaultHost for all requests. The example has 127.0.0.1, but I assume I can/should use an English host name (something defaulthostname.com).

Thx

[Jordan Michaels]

Exactly right my friend.

It's the "name" attribute of the <Host> and according to that should
technically be something resolvable like a domain name or IP. Even so,
you can probably make it whatever you want. ;)

Warm Regards,
Jordan Michaels

> --
> Did you find this reply useful? Help the Railo community and add it to
> the Railo Server wiki at https://github.com/getrailo/railo/wiki
> ---
> You received this message because you are subscribed to the Google
> Groups "Railo" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/railo/0af2e254-cc99-4720-9e82-d5a044503439%40googlegroups.com
> <https://groups.google.com/d/msgid/railo/0af2e254-cc99-4720-9e82-d5a044503439%40googlegroups.com?utm_medium=email&utm_source=footer>.