DNS lookup failure when executing scheduled task in Railo 4

248 views
Skip to first unread message

Martijn van der Woud

unread,
Jan 9, 2013, 6:35:32 AM1/9/13
to ra...@googlegroups.com
Hello group,

After upgrading to Railo 4, a scheduled task that was running fine under Railo 3 stopped working. When I log the output of the task execution to a file I get the following:

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>502 Proxy Error</title>
</head><body>
<h1>Proxy Error</h1>
<p>The proxy server received an invalid
response from an upstream server.<br />
The proxy server could not handle the request <em><a href="/index.cfm/sendmail-to-admins-to-authorize-ikhoach-requests">GET&nbsp;/index.cfm/sendmail-to-admins-to-authorize-ikhoach-requests<$
Reason: <strong>DNS lookup failure for: test.ikhono.nl:80</strong></p></p>
<hr>
<address>Apache/2.2.14 (Ubuntu) Server at test.ikhono.nl Port 80</address>
</body></html>

The IP address for test.ikhono.nl is defined in /etc/hosts and resolves to 127.0.0.1 for this machine.
When I copy the URL for the scheduled task from the Raio admin into a terminal, calling it with the curl command, the task executes without errors. 

Does Railo somehow fail to take /etc/hosts into account when performing a DNS lookup? Or is there some setting I could tweak? Does is somehow include the port in the hostname where it should not (it seems funny to do a DNS lookup for test.ikhono.nl:80, my gut feeling is that it should be test.ikhono.nl)?






Geoff Parkhurst

unread,
Jan 9, 2013, 8:32:36 AM1/9/13
to ra...@googlegroups.com
Can't remember if /etc/hosts is used, but Java does have its own dns
lookup cache behaviour. Here's a bit of a writeup:

http://java-monitor.com/forum/showthread.php?t=181

Best,
Geoff

On 9 January 2013 11:35, Martijn van der Woud

Martijn van der Woud

unread,
Jan 9, 2013, 8:38:30 AM1/9/13
to ra...@googlegroups.com
Thanks Geoff,

The article seems to deal with DNS caching behavior. It seems unlikely that the problem is caused by a change in JVM behavior, since the task executed fine on Railo 3 with the same JVM on the same machine.

Op woensdag 9 januari 2013 schreef Geoff Parkhurst (geoff.p...@gmail.com) het volgende:

Martijn van der Woud

unread,
Jan 10, 2013, 4:19:37 AM1/10/13
to ra...@googlegroups.com
It seems that my hunch about the portnumber was right. When I remove the port number from the scheduled task configuration, the task executes fine. IMO this is a bug. Anyone see this differently?

Bruce Kirkpatrick

unread,
Jan 10, 2013, 11:07:20 AM1/10/13
to ra...@googlegroups.com
There is a separate field for the port number when you edit the task.  Maybe the mistake occurred because the first page for adding a task doesn't show a port field.   Port 80 is the default, so you'd only need this for 443 or non-standard ports.

Martijn van der Woud

unread,
Jan 11, 2013, 6:19:16 AM1/11/13
to ra...@googlegroups.com
Thanks Bruce for taking a look at this. The issue almost certainly has something to do with a change in the HTTP host header sent by Railo when executing a scheduled task. In my Apache config, I used to proxy CFML requests to Tomcat like this:

RewriteRule ^(.+\.cf[cm])(/.*)?$ ajp://%{HTTP_HOST}:8009$1$2 [P]

This worked fine in Railo 3, where the %{HTTP_HOST} resolved to the hostname specified in the scheduled task, WITHOUT the port number: test.ikhono.nl
It seems that Railo 4 now includes the port number in the host header: test.ikhono.nl:80, so that the proxy request now fails.

I worked around this by replacing %{HTTP_HOST} with the hard-coded value test.ikhono.nl and now the task executes as before. 
RewriteRule ^(.+\.cf[cm])(/.*)?$ ajp://test.ikhono.nl:8009$1$2 [P]

Although I can work around it, my opinion is that this new behavior is a bug. Do you agree with that?

Chris Blackwell

unread,
Jan 11, 2013, 2:54:04 PM1/11/13
to ra...@googlegroups.com
I see this behaviour too. 

If you set up a page that dumps getHttpRequestData() and saves it to a file, then hit it with your browser and via a scheduled task the host headers are different. From the browser its local.mysite.com but from the scheduler its local.mysite.com:80

I don't believe this is a bug, as the host header can contain the post, it just usually doesn't. From http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html

The Host request-header field specifies the Internet host and port number of the resource being requested

Do you need to proxy with the correct host?  I use http proxying instead of ajp but it works fine by just proxying to localhost

<Proxy balancer://local>
BalancerMember http://127.0.0.1:8080
</Proxy>
ProxyPreserveHost on
ProxyPassMatch ^/(.+\.cf[cm])(/.*)?$ balancer://local/$1$2

The servlet container must look at x-forwarded-host which gets added by apache and has always just worked for me.

Cheers, Chris

Bruce Kirkpatrick

unread,
Jan 12, 2013, 7:36:00 PM1/12/13
to ra...@googlegroups.com
It is possible to rewrite the environment variables with regular expressions as well.   It is normal for ports in http_host to carry through to the web app.  I think PHP (and other languages) do the same.  If you omit :80, then it would also omit :80.  I'm not sure why :80 is being used anywhere since that is the default.   I also rewrote all my references to cgi.http_host to refer to a custom variable that has the port number removed.

I extract parts of the host name with reg-ex to work around this since the domain maps to the filesystem on my configuration.  So i can convert something like this:   www.mydomain.com.test.com:8888 to be just mydomain.com.

RewriteCond can have section with parenthesis and then you can refer to them in RewriteRule with % instead of $.  I believe the syntax is close to this, but I have not tested this in apache right now.
RewriteCond %{HTTP_HOST} ((.*)(:(.*)|))
RewriteRule ^/(.*)$ ajp://%2:8009/$1 [L,P]

This one should be able to handle hosts with or without the port the same way.  You can also use variables to store values in the web server config (setenv directive i think), and reuse them in multiple rewriterule statements.

If you are having trouble with rewrites in the future, may I suggest a debugging strategy where you temporarily rewrite all urls into a query string so that you can verify what values they have and then dump the URL scope until the values are what you want.

RewriteRule ^(.*)$ ajp://yourdomain:8009/index.cfm?theurl=$1&host=%{HTTP_HOST} [P,L]

You should probably consider if [P] by itself is correct for your application.  I've usually always had rules where L is also defined (last) to avoid the web server processing other rules below it.

I'm currently using nginx to do this, so that's why I'm not sure about apache.

Martijn van der Woud

unread,
Jan 13, 2013, 6:45:26 AM1/13/13
to ra...@googlegroups.com
OK Chris thank you! Seeing what the RFC says I have to say I was wrong and this is not a bug in Railo. Just something I have to watch out for when upgrading from 3 to 4.

Martijn van der Woud

unread,
Jan 13, 2013, 6:47:49 AM1/13/13
to ra...@googlegroups.com
Wow, thank you Bruce this is really helpful. I will try your regex approach as soon as I can find a spare minute! 

Martijn van der Woud

unread,
Jan 15, 2013, 5:09:17 AM1/15/13
to ra...@googlegroups.com
Hi Bruce,

I got around to playing with your proposed solution today. It turns out that %2 in your example still includes the port name. I guess this is because .* will always match the entire string because the * matcher in regex is greedy.

I modified your example a bit and I've got it working now for HTTP headers with and without port names. I also kept the pattern in the RewriteRule restricting it to .cfm and .cfc request only (I want to serve static files with Apache HTTPD, not Tomcat). My solution:

    # If it's a CFML (*.cfc or *.cfm) request, just proxy it to Tomcat:
    RewriteCond %{HTTP_HOST} (^.+?)(:[0-9]*)?$
    RewriteRule ^(.+\.cf[cm])(/.*)?$ ajp://%1:8009$1$2 [P]

Thanks again for taking the time to explain the possibility of matching sections from the RewriteCond pattern with %







On Sunday, January 13, 2013 1:36:00 AM UTC+1, Bruce Kirkpatrick wrote:

Chris Blackwell

unread,
Jan 15, 2013, 6:04:48 AM1/15/13
to ra...@googlegroups.com
apache's docs say mod_rewrite should be considered a last resort where other simpler, and likely more performant solutions are available.

Bruce Kirkpatrick

unread,
Jan 15, 2013, 11:16:55 AM1/15/13
to ra...@googlegroups.com
I used to have dozens of rewrite rules on apache per virtualhost and 100+ virtualhosts like that and the overhead was well under 1 ms per request.   On nginx, the overhead is even less.   You don't need to worry about performance with a single rewrite rule.  My current app can do nearly all of the rewrite rules with string functions in CFML, so it is a valid point that you don't necessarily need rewrite rules.   Rewrites are important for legacy apps that are using a variety of new/old urls and variety of languages.

Peter Boughton

unread,
Jan 15, 2013, 11:30:39 AM1/15/13
to ra...@googlegroups.com
Martijn wrote:
> It turns out that %2 in your example still includes the port name.
> I guess this is because .* will always match the entire string
> because the * matcher in regex is greedy.

Better than changing a greedy qualifier to a lazy one is to not use . when you don't mean it.

In this instance, you can simply use [^:] instead, since hostnames can't contain colons...

    RewriteCond %{HTTP_HOST} ([^:]+)(?::[0-9]+)?

Martijn van der Woud

unread,
Jan 16, 2013, 5:12:21 AM1/16/13
to ra...@googlegroups.com
Good point, thanks!
Reply all
Reply to author
Forward
0 new messages