Boncode problem (I think)

[EoisM]

Hello, I'm getting

Connection to Tomcat has been closed. Tomcat may be restarting. Please retry later.

for last 2-3 weeks on Railo 4.2.1.008 W2008 server R2 Standard SP1 16G Ram, 64 bit OS, 1T free disk space.

I read posts, made dev area, loaded latest BonCode AJP13_v1019 (good stuff!) to only run in dev, set <LogLevel>2</LogLevel> and got

12/9/2014 4:30:32 PM 1.0.16 ERROR
TCP Client level -- Server/Port:localhost/8009
Unable to read data from the transport connection: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
12/9/2014 4:30:32 PM One Connection raised an error

I thought it was time problem but I'm new to this. Can anybody help?

Thanks,
E

[Igal Sapir]

And you confirmed that Tomcat is running by hitting it directly?

--
Did you find this reply useful?  Help the Railo community and add it to the Railo Server wiki at https://github.com/getrailo/railo/wiki
---
You received this message because you are subscribed to the Google Groups "Railo" group.
To view this discussion on the web visit https://groups.google.com/d/msgid/railo/cc30e117-3db8-40d4-a2ca-6be24c85318e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[EoisM]

Thanks Igal,
I don't know what "hitting it directly" means or how to do it.
All the pages on all other sites work fine.
BoncodeAJP13ConnectionLog.txt has hundreds of lines like "date  time  New Connection to tomcat : [::1]:8009 ID:nnnn"
http://127.0.0.1:8888/ gets "Welcome to Railo World" page
http://127.0.0.1:8009/ get "The connection was reset" page

Did I hit it directly?
E

[Igal @ getRailo.org]

hitting Tomcat directly means going to the port that Tomcat listens on, in your example 8888, rather than port 80 which is the IIS port.  so yes, you hit it directly and it seems to be alive.

I've used IIS with BonCode until about a couple of years ago, when I switched to nginx, so my memory is not that fresh about that setup.  perhaps Bilal or Jordan will see this post in a timely manner and chime in.


Igal

--

Did you find this reply useful? Help the Railo community and add it to the Railo Server wiki at https://github.com/getrailo/railo/wiki
---
You received this message because you are subscribed to the Google Groups "Railo" group.

To view this discussion on the web visit https://groups.google.com/d/msgid/railo/ffba96dc-25c0-4200-82f4-7f5755f2b855%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

-- 
Igal Sapir
Railo Core Developer
http://getRailo.org/

[Jordan Michaels]

Add a connectionTimeout parameter to your AJP connector in your Tomcat
server.xml file. This issue has been coming up a *lot* in recent weeks.

Basically, the problem I've been seeing is when an AJP connection is
created it waits for an indefinite amount of time for a response. If
something happens and the response process dies (like, for example, a DB
connection dies) then the AJP connector will keep that connection open
and waiting for a response that it will never receive.

Adding a timeout to the Connector will force Tomcat to recycle those
dead connections.

By default, Tomcat's server.xml file is located here on Windows:
c:\railo\tomcat\conf\server.xml

You'll find a line like this:
<Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />

and you add a timeout using the "connectionTimeout" parameter:

<Connector port="8009" protocol="AJP/1.3" redirectPort="8443"
connectionTimeout="1200000" />

This will set the timeout for roughly 20 minutes - which should be
plenty of time for the vast majority of services.

Hope this helps!

Warm Regards,
Jordan Michaels

>> <https://groups.google.com/d/msgid/railo/ffba96dc-25c0-4200-82f4-7f5755f2b855%40googlegroups.com?utm_medium=email&utm_source=footer>.

>> For more options, visit https://groups.google.com/d/optout.
>
> --
> Igal Sapir
> Railo Core Developer
> http://getRailo.org/
>

> --
> Did you find this reply useful? Help the Railo community and add it to
> the Railo Server wiki at https://github.com/getrailo/railo/wiki
> ---
> You received this message because you are subscribed to the Google
> Groups "Railo" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/railo/54888040.5030306%40getrailo.org
> <https://groups.google.com/d/msgid/railo/54888040.5030306%40getrailo.org?utm_medium=email&utm_source=footer>.

[EoisM]

Thanks Jordan
I'm doing something wrong.
I opened that server.xml file in the c:\railo\tomcat\conf
I found the line you said <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />
I added the connectionTimeout as you wrote <Connector port="8009" protocol="AJP/1.3" redirectPort="8443"
connectionTimeout="1200000" />
I saved the file. I use Sublime2 for editing.

I restarted Railo services and IIS. Is this not right? Maybe I don't have to restart these?

Because then the only site that worked was http://127.0.0.1:8888/railo-context/admin/server.cfm
All other sites got "The page isn't redirecting properly"

If I return the line to <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />
and restart Railo and IIS everything's still bad.

I fixed it by uninstalling Railo and reinstalling.
Then I restart IIS and everything is okay.

Thank you again,

E



On Tuesday, December 9, 2014 4:47:43 PM UTC-5, EoisM wrote:

[Jordan Michaels]

I don't think whatever was happening to you was related to your adding the connectionTimeout. 1) A simple connectionTimeout param shouldn't cause that kind of error 2) you said returning the line to normal had no effect: "everything's still bad".

Glad you got it sorted though.

-Jordan

--
Did you find this reply useful? Help the Railo community and add it to the Railo Server wiki at https://github.com/getrailo/railo/wiki
---
You received this message because you are subscribed to the Google Groups "Railo" group.

To view this discussion on the web visit https://groups.google.com/d/msgid/railo/8e4ea473-d2de-4152-b1a5-8f5b63b562e3%40googlegroups.com.

[Igal @ getRailo.org]

I agree with Jordan.

if Railo did not start up (which is really a Tomcat service in this
case) then it's most likely due to some illegal construct or data in the
server.xml file -- you can find more information in the Tomcat log files
(which are most likely in c:\railo\tomcat\logs but I'm not sure as I
never really use the installer myself).

again, the best thing is to isolate the problem by trying to hit Tomcat
directly (via port 8888 in your case) so that any issues with IIS or the
connector are ruled out. once you confirm that Tomcat is running
properly you can test the other components in the mix.

[EoisM]

Helllo Jordan

I took a breathe and did it again. Including restarting Railo and IIS.

I kept one eye open. AND HEY IT WORKED!

All the pages are getting good. BonCodeAJP13ConnectionLog.txt logs new connections.

I opened the server.xml with Notepad to make sure and yes my edits are there!

Don't know what I did wrong don't want to do it again.

Now I'll test. Fingers crossed.

Thank you,

E

On Tuesday, December 9, 2014 4:47:43 PM UTC-5, EoisM wrote:

[EoisM]

Hi Igal
I just wrote Jordan that it worked for me this time.
Now to test!
Exciting stuff!

E

On Tuesday, December 9, 2014 4:47:43 PM UTC-5, EoisM wrote:

[Igal @ getRailo.org]

I kept one eye open.

it's the closing of the other eye that did the trick.  we should add it to the wiki...

--

Did you find this reply useful? Help the Railo community and add it to the Railo Server wiki at https://github.com/getrailo/railo/wiki
---
You received this message because you are subscribed to the Google Groups "Railo" group.

To view this discussion on the web visit https://groups.google.com/d/msgid/railo/dcd7b12e-ba1a-41df-a497-3bd3538f977c%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[EoisM]

Hi Igal and Jordan
Everything is working, all pages are working, all sites are working
and I still get

12/10/2014  8:44:21 PM  1.0.16 ERROR

TCP Client level -- Server/Port:localhost/8009
Unable to read data from the transport connection: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.

12/10/2014  8:44:21 PM One Connection raised an error

when I test.
I wrote the line to connectionTimeout="120000" and still there's an error.
I wrote the line to connectionTimeout="240000" and same.
I wrote the line to connectionTimeout="480000" and same.

I am bumming.
Any ideas, guys?

Everything else, all other sites work. Bummer.

Thanks,

E
On Tuesday, December 9, 2014 4:47:43 PM UTC-5, EoisM wrote:

[Igal @ getRailo.org]

everything is working and yet you get that where?  in log files?  please clarify...

--

Did you find this reply useful? Help the Railo community and add it to the Railo Server wiki at https://github.com/getrailo/railo/wiki
---
You received this message because you are subscribed to the Google Groups "Railo" group.

To view this discussion on the web visit https://groups.google.com/d/msgid/railo/75b757f4-ae6f-4f22-b9c7-6502ff1bdfd9%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[EoisM]

Hello again,
Sorry, yes, in BoncodeAJP13ConnectionLog.txt is error message.
And more craziness.
Railo always finishes the job its doing. Even before I found this board Railo would finish the job.
The end of the job is a cfmail to me to let me know its done its work.
I did cfmail because I knew the job would run long.

I always get cfmail saying job is done, here is result and result is right result for job.
But page in browser is always

Connection to Tomcat has been closed. Tomcat may be restarting. Please retry later.

Administrator: please check your log file for detail on this error if needed.

Boy I could curse at this.
E

On Tuesday, December 9, 2014 4:47:43 PM UTC-5, EoisM wrote:

[Igal @ getRailo.org]

a few thoughts in random order:

if the request is handled by Railo and completes -- then that's a very good start.

IIS has a timeout of its own, by default 2 minutes IIRC.  I'm not sure what the error message would be though.  the one you posted suggest that that is not the issue but it's worth checking still.

you can use cflog instead of cfmail which would probably be less prone to errors (go figure why an email was not received...)

the BonCode logging slows down the process by quite a bit so I recommend not keeping it on once you're done with it.

you should check the Tomcat logs to get more information (catalina-out and more importantly catalina-err)

--

Did you find this reply useful? Help the Railo community and add it to the Railo Server wiki at https://github.com/getrailo/railo/wiki
---
You received this message because you are subscribed to the Google Groups "Railo" group.

To view this discussion on the web visit https://groups.google.com/d/msgid/railo/e516815f-7e2d-4c86-9a63-03ef06156efd%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Jordan Michaels]

Where are you hitting it from? A browser on the server itself or from
your home desktop or... where?

This message seems to indicate that the client (your browser) is closing
the connection early.

Are you hitting your server through a security appliance or some other
kind of proxy? Is your server behind a load balancer or VPN? Maybe a
device in between you and your server is causing issues.

Warm Regards,
Jordan Michaels

> --
> Did you find this reply useful? Help the Railo community and add it to
> the Railo Server wiki at https://github.com/getrailo/railo/wiki
> ---
> You received this message because you are subscribed to the Google
> Groups "Railo" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/railo/75b757f4-ae6f-4f22-b9c7-6502ff1bdfd9%40googlegroups.com
> <https://groups.google.com/d/msgid/railo/75b757f4-ae6f-4f22-b9c7-6502ff1bdfd9%40googlegroups.com?utm_medium=email&utm_source=footer>.

[EoisM]

Hi guys,
Here's what I did.
Igal - yes, good start. Railo completes!
I up IIS timeout to 60minutes. Pretty big, huh? I restart IIS.
I add cflog <cflog file="EoisLOG" type="debug" text="any">. i can't see examples. Are there examples I can copy? Don't know if I used right tags in cflog.
Got result - "Severity","ThreadID","Date","Time","Application","Message"
"DEBUG","ajp-bio-8009-exec-283","12/11/2014","13:32:08","","any"
 
No idea what that means. I'm newbie. Sorry.
Keep cfmail just for safety and yes, Railo still completes. Sends "Done, Eois!" email with all results. good good.

Boncode now <LogLevel>0</LogLevel>. I wait until night to restart though. No problems then.

Look for catalina-err and catalina-out files on all disks. Searched and can't find. Where hides it?

Jordan - good idea, I call page on server browser, not office, not home. Same same.

Connection to Tomcat has been closed. Tomcat may be restarting. Please retry later.

Administrator: please enable logging with log level 1 or greater for detail problem capture if needed.

BUT no errors in Boncat log now! Still posts from all other business, no errors! Hmm! And still Railo completes and I get "Done, Eois" with all results. Hmm!

Now I try Chrome, Opera, IE, all I can find. Same same. No Boncat errors, no cflog errors, always "Done, Eois".

I close office browsers, take lunch, breathe. Okay I come back, open Firefox, call up dev site to test again and WHAT? There's results page! Like Railo should give me! But I close office browsers for lunch. What?

Now I am testing. What if Railo does half? Hmm. Not all to process, only half? Hmm. I send half.

AND IT WORKS! No errors and final page shows! Excited!

No ideas now. Maybe I do 3/4 to process and let you know.

Thanks guys you are great!
E

On Tuesday, December 9, 2014 4:47:43 PM UTC-5, EoisM wrote:

[EoisM]

I tested 3/4 today and I have my problem back.
1/2 is okay, 3/4 is too much.
Any ideas?

E

On Tuesday, December 9, 2014 4:47:43 PM UTC-5, EoisM wrote:

[Igal @ getRailo.org]

you will get much better responses if you would clarify your questions.

I, for one, have no idea what 1/2 and/or 3/4 mean.

--

Did you find this reply useful? Help the Railo community and add it to the Railo Server wiki at https://github.com/getrailo/railo/wiki
---
You received this message because you are subscribed to the Google Groups "Railo" group.

To view this discussion on the web visit https://groups.google.com/d/msgid/railo/01669114-69d2-431c-a484-31d49067a990%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Jordan Michaels]

I was thinking the same thing. Don't really know what you're talking about E. ;)

-Jordan



----- Original Message -----

> <https://groups.google.com/d/msgid/railo/01669114-69d2-431c-a484-31d49067a990%40googlegroups.com?utm_medium=email&utm_source=footer>.

> For more options, visit https://groups.google.com/d/optout.

--
Igal Sapir
Railo Core Developer
http://getRailo.org/

--
Did you find this reply useful? Help the Railo community and add it to the Railo Server wiki at https://github.com/getrailo/railo/wiki
---
You received this message because you are subscribed to the Google Groups "Railo" group.

To view this discussion on the web visit https://groups.google.com/d/msgid/railo/548B3CA2.4040803%40getrailo.org.

[EoisM]

Sorry sorry.
Code does this-> 1) You upload file from browser 2)CF code analyse 3)Show and send result.
Test with little 1k upload. Good good.
Test with little 5k upload. Good good. I am happy.
Test with 50k upload. Good good.
100K good
300k bad news everybody.  (joined Railo group here)
Igal and Jordan help and with good stuff. You guys are teachers!
But still trouble. I am bumming.
I test 150k (1/2) file. Good good.
I test 165k (not 3/4, but careful) and Hey bad news everybody.
Code was CF8  w2002 server. Could put 1000K+. Took time and worked, send "Done!" email with results.

I am digger! I figure out!

but need help.

E

On Tuesday, December 9, 2014 4:47:43 PM UTC-5, EoisM wrote:

[Igal @ getRailo.org]

how long does it take to upload the larger files?  maybe the request times out somewhere...

--

Did you find this reply useful? Help the Railo community and add it to the Railo Server wiki at https://github.com/getrailo/railo/wiki
---
You received this message because you are subscribed to the Google Groups "Railo" group.

To view this discussion on the web visit https://groups.google.com/d/msgid/railo/4ff85ffe-3abb-48fe-a22f-e55f2e227041%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Rani]

Hi EoisM,

Not sure what 1/2, 3/4 means but i have a suggestion. You could put an generic error message on Application.cfc ( from your website ) into onError() function, maybe will help:


<!--- OnError routine --->
    <cffunction
            name="onError"
            returnType="void"
            output="false"
            hint="Initialized when error occurs">

            <cfargument name="exception" required="true">

            <cflog type="error" file="{your-error-log-file-name}" text="An error has occurred: #Arguments.exception.message# | #arguments.exception#" >

            Oops. An error has occured but stay calm an detailed e-mail has been sent to website's webmaster to be solved ASAP.

            <cfmail to="{your-email}" from="{no-r...@your.domain}" subject="myWebsite :: An error has occurred: #Arguments.exception.message#" type="html">

                 An error occurred in webpage: http://#cgi.server_name##cgi.script_name#?#cgi.query_string#<br />
                 Time: #dateFormat(now(), "short")# #timeFormat(now(), "short")#<br />

                 <cfdump var="#arguments.exception#" label="Error">
                 <cfdump var="#session#" label="Session variables">
                 <cfdump var="#form#" label="Form">
                 <cfdump var="#url#" label="URL">
                 <cfdump var="#cgi#"  label="CGI">
            </cfmail>

            <cfreturn />

    </cffunction>

P.S.

YOu could verify mail.log file to see if there are errors regarding sending e-mails: {railo-root}/{your-website-root}/WEB-INF/railo/logs/mail.log.

kind regards,

[Bilal]

E:

This does smell like you hit some sort of timeout. The question only remains which timeout you are hitting (whether it is IIS or Tomcat)?

Do you have a sense (e.g. using a trace code) how long until you hit this, this would help pinpoint it.

Also a general architectural thinking point.

If you are loading large files, lets say you are inserting from file into database, it may be beneficial to look at multi-threading the process. E.g. insert 10 records at a time.

Best,

Bilal

[EoisM]

Hi vineri thank you.
I got code in there like that. No errors sent. I see if your code works different. Thanks.

Hi Igal, I testing your idea. Good one, thanks.

E

On Tuesday, December 9, 2014 4:47:43 PM UTC-5, EoisM wrote:

[EoisM]

Hi guys,
Igal - everything uploads fast, like eye blinks. Big pipes here. Click upload at browser, check server, file saved. I can time if you want to see it. Railo processes big file, just no confirming browser page "Thank you, file is done, here are results". Instead, Tomcat error shows up.

Rani, got your name wrong, sorry. I got code like you wrote and I add yours this week to be sure.

Bilal, big files are broken up into small parts already. Had that idea, too, did not help. You wrote to use trace code. Is example somewhere? If I see it I can do it but need to see it first to understand.

Thanks guys,

E



On Tuesday, December 9, 2014 4:47:43 PM UTC-5, EoisM wrote: