So I finally bit the bullet and updated our production server to Lucee. We updated our test server to Lucee a few weeks ago and haven’t had any issue. The software between our test and production environment is identical. The only difference is the volume of traffic. Before upgrading from Railo to Lucee, our install was rock solid. We could go months without a single reboot of Tomcat/Railo or IIS. However, that’s all changed since upgrading to Lucee. After we upgraded, we made sure all of our settings from our server.xml were merged with Lucee’s install of the new Server.xml. TomCat doesn't seem to be crashing as we have FusionReactor running in the background and we can still serve up requests to any website via port 8888.
I believe the core of the issue is with the BonCode connector. Sporadically we receive the following message.
Generic Connector Communication Error:
Please check and adjust your setup:
Ensure that Tomcat is running on given host and port.
If this is a timeout error consider adjusting IIS timeout by changing executionTimeout attribute in web.config (see manual).
An immediate refresh of the page/website and it loads right up. There is no way to reproduce the error. It occurs on just about all of our roughly 85 sites randomly throughout the day. Prior to upgrading to Lucee we never had this issue. In case its asked, all of our IIS application pools have the IIS default 20min timeout. Some higher traffic sites are set to "AlwaysRunning"
We enabled logging on the boncode connector, and see this in just about all of the boncode logs:
2015-04-14 07:27:15 1.0.21 ERROR
TCP Client level -- Server/Port:localhost/8009
Unable to write data to the transport connection: An established connection was aborted by the software in your host machine.
2015-04-14 07:27:15 One Connection raised an error
I’ve included some settings below that I believe are relevant to diagnosing the issue.
BonCode Version:
BonCodeAJP Connector Version 1.0.21
using setting file in C:\Windows\BonCodeAJP13.settings
Windows Server 2012 R1
IIS 8
Lucee 4.5.1.000 final
Apache Tomcat/7.0.61 <-- I tried several versions of tomcat
Java 1.7.0_76 (Oracle Corporation) 64bit
4096mb Ram dedicated to Lucee (32gigs server total)
BonCodeAJP13.settings
<Settings>
<Server>localhost</Server><Port>8009</Port><MaxConnections>0</MaxConnections><EnableRemoteAdmin>True</EnableRemoteAdmin><EnableHeaderDataSupport>True</EnableHeaderDataSupport><ForceSecureSession>False</ForceSecureSession><AllowEmptyHeaders>False</AllowEmptyHeaders><LogLevel>3</LogLevel><LogDir>C:\Lucee\BonCodeLogs</LogDir></Settings>
A note about the above: I had <MaxConnections> set equal to what I had in the server.xml, but that didn't help with the error.
Connector settings in TomCat Server.xml
<Connector port="8009"
acceptorThreadCount="6"
maxThreads="1500"
minSpareThreads="150"
maxSpareThreads="500"
tcpNoDelay="true"
connectionTimeout="180000"
enableLookups="false"
protocol="AJP/1.3"
redirectPort="8443"
/>
Would appreciate any help anyone can offer.