Very slow startup and errors on startup.

321 views
Skip to first unread message

Dustin Martin

unread,
Mar 30, 2011, 3:34:14 PM3/30/11
to col...@googlegroups.com
I've been working on a new Coldbox app and when I start it on my development laptop after I start CF the app takes a very very long time to startup and is always interrupted with errors. Do I have something setup wrong or is there anything I can do to avoid the errors (listed below)? I get the following errors each time I start up. After I get each one, I just refresh the browser and it continues to try to start up until it gives me the next error. After getting 2-4 errors it finally finishes and is ready to go. I can reinit and even reboot the server without casuing issues but if I completely shut down ColdFusion and then start it again I can expect to get errors on the next startup of my Coldbox app.

Note: I'm running CF 9.0.1 and Coldbox 3.0 and using the ORM.

------------------------------------------------------------------------------
ERROR #1
------------------------------------------------------------------------------
The request has exceeded the allowable time limit Tag: CFLOOP
 
The error occurred in C:\WebRoot\Development\CF\Projects\BH\frameworks\coldbox\3.0\system\logging\appenders\DBAppender.cfc: line 154
Called from C:\WebRoot\Development\CF\Projects\BH\frameworks\coldbox\3.0\system\logging\appenders\DBAppender.cfc: line 88
Called from C:\WebRoot\Development\CF\Projects\BH\frameworks\coldbox\3.0\system\logging\LogBox.cfc: line 243
Called from C:\WebRoot\Development\CF\Projects\BH\frameworks\coldbox\3.0\system\logging\LogBox.cfc: line 86
Called from C:\WebRoot\Development\CF\Projects\BH\frameworks\coldbox\3.0\system\web\services\LoaderService.cfc: line 57
Called from C:\WebRoot\Development\CF\Projects\BH\frameworks\coldbox\3.0\system\Coldbox.cfc: line 71
Called from C:\WebRoot\Development\CF\Projects\BH\Application.cfc: line 65
152 : <cfif qTables.table_name eq getProperty("table")>
153 : <cfset tableFound = true>
154 : <cfbreak>
155 : </cfif>
156 : </cfloop>



------------------------------------------------------------------------------
ERROR #2
------------------------------------------------------------------------------
The request has exceeded the allowable time limit Tag: cfdirectory

 
The error occurred in C:\WebRoot\Development\CF\Projects\BH\frameworks\coldbox\3.0\system\web\services\HandlerService.cfc: line 479
Called from C:\WebRoot\Development\CF\Projects\BH\frameworks\coldbox\3.0\system\web\services\HandlerService.cfc: line 395
Called from C:\WebRoot\Development\CF\Projects\BH\frameworks\coldbox\3.0\system\web\services\HandlerService.cfc: line 39
Called from C:\WebRoot\Development\CF\Projects\BH\frameworks\coldbox\3.0\system\web\services\LoaderService.cfc: line 81
Called from C:\WebRoot\Development\CF\Projects\BH\frameworks\coldbox\3.0\system\Coldbox.cfc: line 71
Called from C:\WebRoot\Development\CF\Projects\BH\Application.cfc: line 65
477 : 
478 : <!--- List Handlers --->
479 : <cfdirectory action="list" recurse="true" name="files" directory="#arguments.directory#" filter="*.cfc"/>
480 : 
481 : <cfscript>



------------------------------------------------------------------------------
ERROR #3
------------------------------------------------------------------------------
The request has exceeded the allowable time limit Tag: cfoutput

 
The error occurred in C:\WebRoot\Development\CF\Projects\BH\frameworks\coldbox\3.0\system\web\context\ExceptionBean.cfc: line 158
Called from C:\WebRoot\Development\CF\Projects\BH\frameworks\coldbox\3.0\system\plugins\Logger.cfc: line 124
Called from C:\WebRoot\Development\CF\Projects\BH\frameworks\coldbox\3.0\system\web\services\ExceptionService.cfc: line 64
Called from C:\WebRoot\Development\CF\Projects\BH\frameworks\coldbox\3.0\system\Coldbox.cfc: line 321
Called from C:\WebRoot\Development\CF\Projects\BH\Application.cfc: line 94
156 : <cfif structkeyExists(instance.exceptionStruct, "TagContext") and tagContextLength>
157 : <cfloop from="1" to="#tagContextLength#" index="i">
158 :  <cfsavecontent variable="entry"><cfoutput>ID: <cfif not structKeyExists(arrayTagContext[i], "ID")>N/A<cfelse>#arrayTagContext[i].ID#</cfif>; LINE: #arrayTagContext[i].LINE#; TEMPLATE: #arrayTagContext[i].Template# #chr(13)#</cfoutput></cfsavecontent>
159 :  <cfset rtnString = rtnString & entry>
160 : </cfloop>


br...@bradwood.com

unread,
Mar 30, 2011, 3:47:23 PM3/30/11
to col...@googlegroups.com
Did you try pushing the "turbo" button on the front of your PC?  :)

How quickly does a ColdBox Sample App load?  What are the specs on your machine?  More specifically, what is your heap size set to, how much RAM is on the box, and how much is free.  Is Jrun racking up an extremely large amount of page faults?  (You can add that column to the process list in task manager)


Thanks!

~Brad
--
You received this message because you are subscribed to the Google Groups "ColdBox Platform" group.
To post to this group, send email to col...@googlegroups.com
To unsubscribe from this group, send email to coldbox-u...@googlegroups.com
For more options, visit this group at http://groups-beta.google.com/group/coldbox
For News, visit http://blog.coldbox.org
For Documentation, visit http://wiki.coldbox.org

Dustin Martin

unread,
Mar 30, 2011, 4:46:06 PM3/30/11
to col...@googlegroups.com
Ha, oh ya, the turbo button! Why didn’t I think of that? Is that next to the “any” key? :-)

- The sample app ( I generated one with the CF Builder extension) took about 5+ minutes before throwing a cfdirectory timeout error like I normally get. I refreshed and got the same error and then one more time and it finally ran.

- My machine has 4Gb of RAM, about 600Mb is free right now, Core 2 Duo CPU @ 2.24GHz

- I left the defaults for the JVM config so the heap size (Xmx) is set to 512m.

- JRun has had 320,000+ page faults.

Luis Majano

unread,
Mar 30, 2011, 6:26:28 PM3/30/11
to col...@googlegroups.com
In the cf admin remove report execution times in the debugger section

> --
> You received this message because you are subscribed to the Google Groups "ColdBox Platform" group.
> To post to this group, send email to col...@googlegroups.com
> To unsubscribe from this group, send email to coldbox-u...@googlegroups.com
> For more options, visit this group at http://groups-beta.google.com/group/coldbox
> For News, visit http://blog.coldbox.org
> For Documentation, visit http://wiki.coldbox.org
>

--
Luis F. Majano
President
Ortus Solutions, Corp

ColdBox Platform: http://www.coldbox.org
Linked In: http://www.linkedin.com/pub/3/731/483
Blog: http://www.luismajano.com
IECFUG Manager: http://www.iecfug.com

Dustin Martin

unread,
Mar 30, 2011, 6:36:47 PM3/30/11
to col...@googlegroups.com
Luis, I already tried that but it didn't help. Thanks for the suggestion though.

br...@bradwood.com

unread,
Mar 30, 2011, 6:42:55 PM3/30/11
to col...@googlegroups.com
Hmm, something definitely isn't right then.  The sample apps should load up in a matter of seconds.  Is your CPU usage pegged while the app is initting?  If you take a stack trace, is there anything in particular that the JVM is getting stuck on?  (You can snag a stack trace using SeeFusion, Fusion-Reactor, or the ColdFusion Server Monitor snapshot feature.

Thanks!

~Brad

-------- Original Message --------
Subject: Re: [coldbox:9058] Very slow startup and errors on startup.
From: Dustin Martin <d...@dustinmartin.net>

Andrew Scott

unread,
Mar 30, 2011, 7:28:33 PM3/30/11
to col...@googlegroups.com
Have you got the server monitor running? That can, not always though, slow
down CF and cause these types of errors.

Regards,
Andrew Scott
http://www.andyscott.id.au/

Dustin Martin

unread,
Mar 31, 2011, 2:17:18 PM3/31/11
to col...@googlegroups.com
@Andrew  - I checked but the server monitor has been disabled

@Brad - I turned on the monitor to get a stack trace but I'm not entirely sure what to look for. I've included it below if you don't mind looking at it.

Thanks!


Snapshot generated on : Thu Mar 31 12:22:12 EDT 2011 Caused by : User request
Profiling : Disabled    
Memory Tracking : Disabled    

*Total Running requests - 2
*Total Queued requests - 0
Template Running requests - 1
Template Queued requests - 0
Flash Remoting Running requests - 1
Flash Remoting Queued requests - 0
CFC method Running requests - 0
CFC method Queued requests - 0
Web Service Running requests - 0
Web Service Queued requests - 0

JVM Memory Stats
Free Memory : 46200832 bytes
Used Memory : 117442560 bytes

Throttle Stats [Throttle Queue Size : Throttle Memory Used]
            0  : 0.0 megabytes

Query Cache Stats - 
[Hit Ratio : Size : Count]
   0.0 : 0 bytes : 0

Cached Queries - 
[Query Name : DSN : Size : TIme Executed : Time Taken : Line No : Function Name : Template Path]

DB Pool Stats - 
[DSN :Open Conn Count : Total Conn Count : Max Con Count : Avg Open Conn Count : Avg Total Conn Count]
   dsnFOG_DAT2 : 0 : 0 : Unlimited : 0 : 0
   cfbookclub : 0 : 0 : Unlimited : 0 : 0
   DB2PTCP : 0 : 0 : Unlimited : 0 : 0
   projects1_DAT1 : 0 : 0 : Unlimited : 0 : 0
   cfdocexamples : 0 : 0 : Unlimited : 0 : 0
   DB20TCP : 0 : 0 : Unlimited : 0 : 0
   BH : 0 : 0 : Unlimited : 0 : 0
   cfcodeexplorer : 0 : 0 : Unlimited : 0 : 0
   dsnLogon : 0 : 0 : Unlimited : 0 : 0
   dsnCommon : 0 : 0 : Unlimited : 0 : 0
   Backhauls : 0 : 0 : Unlimited : 0 : 0
   BH_Local : 1 : 1 : Unlimited : 0 : 0
   custoserv_DAT2 : 0 : 0 : Unlimited : 0 : 0
   backhauls_DAT2 : 0 : 0 : Unlimited : 0 : 0
   dsnHr : 0 : 0 : Unlimited : 0 : 0
   cfartgallery : 0 : 0 : Unlimited : 0 : 0


Thread  web-3 request type - TEMPLATE REQUEST
*Template Path - C:\WebRoot\Development\CF\Projects\BH\index.cfm
*Request Parameters - {}
*Request Method - GET
*Client IP address - 127.0.0.1
*Thread elapsed time - 41079 milliseconds
*Application Name - project



Java stack trace - 
Full thread dump Java 1.6.0_17

"scheduler-2" prio=5 tid=135 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at coldfusion.scheduling.Scheduler.createRunnable(Scheduler.java:132)
     at coldfusion.scheduling.ThreadPool.createRunnable(ThreadPool.java:128)
     at coldfusion.scheduling.WorkerThread.run(WorkerThread.java:68)

"jndi-3" prio=5 tid=25 BLOCKED
     at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:388)
     at java.net.ServerSocket.implAccept(ServerSocket.java:453)
     at java.net.ServerSocket.accept(ServerSocket.java:421)
     at jrun.servlet.network.NetworkService.accept(NetworkService.java:357)
     at jrun.naming.NamingNetworkService.accept(NamingNetworkService.java:115)
     at jrun.naming.NamingNetworkService.createRunnable(NamingNetworkService.java:141)
     at jrunx.scheduler.ThreadPool$ThreadThrottle.createRunnable(ThreadPool.java:349)
     at jrunx.scheduler.WorkerThread.run(WorkerThread.java:62)

"Thread-8" prio=5 tid=68 TIMED_WAITING
     at java.lang.Thread.sleep(Native Method)
     at org.apache.log4j.helpers.FileWatchdog.run(FileWatchdog.java:95)

"clock" prio=10 tid=34 TIMED_WAITING
     at java.lang.Thread.sleep(Native Method)
     at jrunx.tyrex.services.Clock.run(Clock.java:202)

"obj-skimmer-DB20TCP" prio=5 tid=85 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at coldfusion.server.j2ee.pool.PoolSkimmerThread.run(PoolSkimmerThread.java:47)
     at java.lang.Thread.run(Thread.java:619)

"RMI TCP Accept-0" prio=5 tid=11 RUNNABLE
     at java.net.PlainSocketImpl.socketAccept(Native Method)
     at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)
     at java.net.ServerSocket.implAccept(ServerSocket.java:453)
     at java.net.ServerSocket.accept(ServerSocket.java:421)
     at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:369)
     at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
     at java.lang.Thread.run(Thread.java:619)

"tyrex.util.daemonMaster" prio=10 tid=27 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at jrunx.tyrex.services.DaemonMaster.run(DaemonMaster.java:269)
     at java.lang.Thread.run(Thread.java:619)

"worker #4" prio=5 tid=110 WAITING
     at java.lang.Object.wait(Native Method)
     at java.lang.Object.wait(Object.java:485)
     at com.jnbridge.jnbcore.server.b.c.run(Unknown Source)
     at java.lang.Thread.run(Thread.java:619)

"scheduler-15" prio=5 tid=46 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at jrunx.scheduler.SchedulerService.createRunnable(SchedulerService.java:188)
     at jrunx.scheduler.ThreadPool$ThreadThrottle.createRunnable(ThreadPool.java:349)
     at jrunx.scheduler.WorkerThread.run(WorkerThread.java:62)

"worker #3" prio=5 tid=109 WAITING
     at java.lang.Object.wait(Native Method)
     at java.lang.Object.wait(Object.java:485)
     at com.jnbridge.jnbcore.server.b.c.run(Unknown Source)
     at java.lang.Thread.run(Thread.java:619)

"jndi-6" prio=5 tid=60 BLOCKED
     at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:388)
     at java.net.ServerSocket.implAccept(ServerSocket.java:453)
     at java.net.ServerSocket.accept(ServerSocket.java:421)
     at jrun.servlet.network.NetworkService.accept(NetworkService.java:357)
     at jrun.naming.NamingNetworkService.accept(NamingNetworkService.java:115)
     at jrun.naming.NamingNetworkService.createRunnable(NamingNetworkService.java:141)
     at jrunx.scheduler.ThreadPool$ThreadThrottle.createRunnable(ThreadPool.java:349)
     at jrunx.scheduler.WorkerThread.run(WorkerThread.java:62)

"Signal Dispatcher" prio=9 tid=4 RUNNABLE

"jms-fifo-5" prio=5 tid=18 WAITING
     at java.lang.Object.wait(Native Method)
     at java.lang.Object.wait(Object.java:485)
     at jrun.jms.core.threads.FIFOQueue.queueHasElements(FIFOQueue.java:45)
     at jrun.jms.core.threads.FIFOQueue.get(FIFOQueue.java:65)
     at jrun.jms.core.threads.ThreadPoolWorker.runWorker(ThreadPoolWorker.java:91)
     at jrun.jms.core.threads.ThreadPoolWorker$1.run(ThreadPoolWorker.java:49)
     at java.lang.Thread.run(Thread.java:619)

"Transaction Domain PrivateJmsDomain" prio=5 tid=29 WAITING
     at java.lang.Object.wait(Native Method)
     at java.lang.Object.wait(Object.java:485)
     at jrunx.tyrex.tm.impl.TransactionDomainImpl.run(TransactionDomainImpl.java:1300)
     at java.lang.Thread.run(Thread.java:619)

"scheduler-13" prio=5 tid=44 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at jrunx.scheduler.SchedulerService.createRunnable(SchedulerService.java:188)
     at jrunx.scheduler.ThreadPool$ThreadThrottle.createRunnable(ThreadPool.java:349)
     at jrunx.scheduler.WorkerThread.run(WorkerThread.java:62)

"web-2" prio=5 tid=127 RUNNABLE
     at java.lang.Thread.getStackTrace(Thread.java:1436)
     at coldfusion.monitor.stack.J5StackTrace.getAllStackTraces(J5StackTrace.java:39)
     at coldfusion.monitor.active.Snapshot.getJavaStackTrace(Snapshot.java:583)
     at coldfusion.monitor.active.Snapshot.getSnapshotData(Snapshot.java:149)
     at coldfusion.monitor.active.Snapshot.dumpSnapshot(Snapshot.java:572)
     at coldfusion.monitor.active.Snapshot.dumpSnapshot(Snapshot.java:559)
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
     at java.lang.reflect.Method.invoke(Method.java:597)
     at coldfusion.runtime.StructBean.invoke(StructBean.java:508)
     at coldfusion.runtime.CfJspPage._invoke(CfJspPage.java:2393)
     at cfservermonitoring2ecfc1181200266$funcDUMPSNAPSHOT.runFunction(E:\cf9_updates_rc\cfusion\wwwroot\CFIDE\adminapi\servermonitoring.cfc:1992)
     at coldfusion.runtime.UDFMethod.invoke(UDFMethod.java:472)
     at coldfusion.runtime.UDFMethod$ReturnTypeFilter.invoke(UDFMethod.java:405)
     at coldfusion.runtime.UDFMethod$ArgumentCollectionFilter.invoke(UDFMethod.java:368)
     at coldfusion.filter.FunctionAccessFilter.invoke(FunctionAccessFilter.java:55)
     at coldfusion.runtime.UDFMethod.runFilterChain(UDFMethod.java:321)
     at coldfusion.runtime.UDFMethod.invoke(UDFMethod.java:220)
     at coldfusion.runtime.TemplateProxy.invoke(TemplateProxy.java:491)
     at coldfusion.runtime.TemplateProxy.invoke(TemplateProxy.java:437)
     at coldfusion.flash.filter.CFCInvokeFilter.invokeMethod(CFCInvokeFilter.java:319)
     at coldfusion.flash.filter.CFCInvokeFilter.invoke(CFCInvokeFilter.java:199)
     at coldfusion.flash.filter.CFCInvokeFilter.invoke(CFCInvokeFilter.java:77)
     at coldfusion.filter.ApplicationFilter.invoke(ApplicationFilter.java:381)
     at coldfusion.filter.MonitoringFilter.invoke(MonitoringFilter.java:40)
     at coldfusion.filter.LicenseFilter.invoke(LicenseFilter.java:27)
     at coldfusion.flash.filter.CFCInvokePathFilter.invoke(CFCInvokePathFilter.java:70)
     at coldfusion.filter.ClientScopePersistenceFilter.invoke(ClientScopePersistenceFilter.java:28)
     at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:38)
     at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)
     at coldfusion.flash.messaging.ColdFusionAdapter.invoke(ColdFusionAdapter.java:333)
     at flex.messaging.services.RemotingService.serviceMessage(RemotingService.java:183)
     at flex.messaging.MessageBroker.routeMessageToService(MessageBroker.java:1400)
     at flex.messaging.endpoints.AbstractEndpoint.serviceMessage(AbstractEndpoint.java:1005)
     at flex.messaging.endpoints.amf.MessageBrokerFilter.invoke(MessageBrokerFilter.java:103)
     at flex.messaging.endpoints.amf.LegacyFilter.invoke(LegacyFilter.java:158)
     at flex.messaging.endpoints.amf.SessionFilter.invoke(SessionFilter.java:44)
     at flex.messaging.endpoints.amf.BatchProcessFilter.invoke(BatchProcessFilter.java:67)
     at flex.messaging.endpoints.amf.SerializationFilter.invoke(SerializationFilter.java:166)
     at flex.messaging.endpoints.BaseHTTPEndpoint.service(BaseHTTPEndpoint.java:291)
     at coldfusion.flash.messaging.CFAMFEndPoint.service(CFAMFEndPoint.java:295)
     at flex.messaging.MessageBrokerServlet.service(MessageBrokerServlet.java:353)
     at coldfusion.flex.ColdFusionMessageBrokerServlet.service(ColdFusionMessageBrokerServlet.java:114)
     at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
     at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:89)
     at jrun.servlet.FilterChain.doFilter(FilterChain.java:86)
     at coldfusion.filter.FlashRequestControlFilter.doFilter(FlashRequestControlFilter.java:71)
     at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46)
     at jrun.servlet.FilterChain.doFilter(FilterChain.java:94)
     at jrun.servlet.FilterChain.service(FilterChain.java:101)
     at jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:106)
     at jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42)
     at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java:286)
     at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java:543)
     at jrun.servlet.http.WebService.invokeRunnable(WebService.java:172)
     at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.java:428)
     at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)

"Thread-4" prio=5 tid=55 WAITING
     at java.lang.Object.wait(Native Method)
     at java.lang.Object.wait(Object.java:485)
     at jrun.jms.core.messagemgr.Scheduler.next(Scheduler.java:298)
     at jrun.jms.core.messagemgr.Scheduler.run(Scheduler.java:233)
     at java.lang.Thread.run(Thread.java:619)

"scheduler-10" prio=5 tid=41 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at jrunx.scheduler.SchedulerService.createRunnable(SchedulerService.java:188)
     at jrunx.scheduler.ThreadPool$ThreadThrottle.createRunnable(ThreadPool.java:349)
     at jrunx.scheduler.WorkerThread.run(WorkerThread.java:62)

"scheduler-0" prio=5 tid=30 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at jrunx.scheduler.SchedulerService.createRunnable(SchedulerService.java:188)
     at jrunx.scheduler.ThreadPool$ThreadThrottle.createRunnable(ThreadPool.java:349)
     at jrunx.scheduler.WorkerThread.run(WorkerThread.java:62)

"obj-skimmer-dsnFOG_DAT2" prio=5 tid=79 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at coldfusion.server.j2ee.pool.PoolSkimmerThread.run(PoolSkimmerThread.java:47)
     at java.lang.Thread.run(Thread.java:619)

"obj-skimmer-projects1_DAT1" prio=5 tid=82 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at coldfusion.server.j2ee.pool.PoolSkimmerThread.run(PoolSkimmerThread.java:47)
     at java.lang.Thread.run(Thread.java:619)

"Java2D Disposer" prio=10 tid=104 WAITING
     at java.lang.Object.wait(Native Method)
     at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
     at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
     at sun.java2d.Disposer.run(Disposer.java:125)
     at java.lang.Thread.run(Thread.java:619)

"jndi-1" prio=5 tid=23 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at jrunx.scheduler.JSemaphore.acquire(JSemaphore.java:74)
     at jrun.servlet.network.NetworkService.accept(NetworkService.java:349)
     at jrun.naming.NamingNetworkService.accept(NamingNetworkService.java:115)
     at jrun.naming.NamingNetworkService.createRunnable(NamingNetworkService.java:141)
     at jrunx.scheduler.ThreadPool$ThreadThrottle.createRunnable(ThreadPool.java:349)
     at jrunx.scheduler.WorkerThread.run(WorkerThread.java:62)

"jms-fifo-4" prio=5 tid=17 WAITING
     at java.lang.Object.wait(Native Method)
     at java.lang.Object.wait(Object.java:485)
     at jrun.jms.core.threads.FIFOQueue.queueHasElements(FIFOQueue.java:45)
     at jrun.jms.core.threads.FIFOQueue.get(FIFOQueue.java:65)
     at jrun.jms.core.threads.ThreadPoolWorker.runWorker(ThreadPoolWorker.java:91)
     at jrun.jms.core.threads.ThreadPoolWorker$1.run(ThreadPoolWorker.java:49)
     at java.lang.Thread.run(Thread.java:619)

"GC Daemon" prio=2 tid=13 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at sun.misc.GC$Daemon.run(GC.java:100)

"jndi-2" prio=5 tid=24 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at jrunx.scheduler.JSemaphore.acquire(JSemaphore.java:74)
     at jrun.servlet.network.NetworkService.accept(NetworkService.java:349)
     at jrun.naming.NamingNetworkService.accept(NamingNetworkService.java:115)
     at jrun.naming.NamingNetworkService.createRunnable(NamingNetworkService.java:141)
     at jrunx.scheduler.ThreadPool$ThreadThrottle.createRunnable(ThreadPool.java:349)
     at jrunx.scheduler.WorkerThread.run(WorkerThread.java:62)

"web-5" prio=5 tid=142 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at jrunx.scheduler.JSemaphore.acquire(JSemaphore.java:74)
     at jrun.servlet.network.NetworkService.accept(NetworkService.java:349)
     at jrun.servlet.http.WebService.createRunnable(WebService.java:104)
     at jrunx.scheduler.ThreadPool$ThreadThrottle.createRunnable(ThreadPool.java:349)
     at jrunx.scheduler.WorkerThread.run(WorkerThread.java:62)

"obj-skimmer-cfcodeexplorer" prio=5 tid=86 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at coldfusion.server.j2ee.pool.PoolSkimmerThread.run(PoolSkimmerThread.java:47)
     at java.lang.Thread.run(Thread.java:619)

"scheduler-1" prio=5 tid=116 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at coldfusion.scheduling.Scheduler.createRunnable(Scheduler.java:132)
     at coldfusion.scheduling.ThreadPool.createRunnable(ThreadPool.java:128)
     at coldfusion.scheduling.WorkerThread.run(WorkerThread.java:68)

"RMI Reaper" prio=5 tid=12 WAITING
     at java.lang.Object.wait(Native Method)
     at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
     at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
     at sun.rmi.transport.ObjectTable$Reaper.run(ObjectTable.java:333)
     at java.lang.Thread.run(Thread.java:619)

"obj-skimmer-custoserv_DAT2" prio=5 tid=91 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at coldfusion.server.j2ee.pool.PoolSkimmerThread.run(PoolSkimmerThread.java:47)
     at java.lang.Thread.run(Thread.java:619)

"Timer-2" prio=5 tid=65 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at java.util.TimerThread.mainLoop(Timer.java:509)
     at java.util.TimerThread.run(Timer.java:462)

"ServerThread" prio=5 tid=105 RUNNABLE
     at java.net.PlainSocketImpl.socketAccept(Native Method)
     at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)
     at java.net.ServerSocket.implAccept(ServerSocket.java:453)
     at java.net.ServerSocket.accept(ServerSocket.java:421)
     at com.jnbridge.jnbcore.server.b.b.if(Unknown Source)
     at com.jnbridge.jnbcore.JNBMain$a.run(Unknown Source)

"RMI Scheduler(0)" prio=5 tid=20 TIMED_WAITING
     at sun.misc.Unsafe.park(Native Method)
     at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
     at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963)
     at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
     at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:583)
     at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:576)
     at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
     at java.lang.Thread.run(Thread.java:619)

"web-3" prio=5 tid=128 RUNNABLE
     at java.io.FileInputStream.open(Native Method)
     at java.io.FileInputStream.<init>(FileInputStream.java:106)
     at coldfusion.runtime.TemplateClassLoader.getClassBytes(TemplateClassLoader.java:257)
     at coldfusion.runtime.TemplateClassLoader.findClass(TemplateClassLoader.java:239)
     at java.lang.ClassLoader.loadClass(ClassLoader.java:303)
     at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
     at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:316)
     at java.lang.Class.getDeclaredConstructors0(Native Method)
     at java.lang.Class.privateGetDeclaredConstructors(Class.java:2389)
     at java.lang.Class.getConstructor0(Class.java:2699)
     at java.lang.Class.newInstance0(Class.java:326)
     at java.lang.Class.newInstance(Class.java:308)
     at coldfusion.runtime.TemplateClassLoader.newInstance(TemplateClassLoader.java:570)
     at coldfusion.runtime.TemplateClassLoader.newInstance(TemplateClassLoader.java:541)
     at coldfusion.runtime.TemplateProxyFactory.getCFCInstance(TemplateProxyFactory.java:271)
     at coldfusion.runtime.TemplateProxyFactory.resolveName(TemplateProxyFactory.java:174)
     at coldfusion.runtime.TemplateProxyFactory.resolveName(TemplateProxyFactory.java:159)
     at coldfusion.runtime.TemplateProxyFactory.resolveName(TemplateProxyFactory.java:149)
     at coldfusion.cfc.ComponentProxyFactory.getProxy(ComponentProxyFactory.java:62)
     at coldfusion.runtime.CFPage.CreateObject(CFPage.java:4827)
     at coldfusion.runtime.CFPage.CreateObject(CFPage.java:4841)
     at cfController2ecfc1957137711$funcINIT.runFunction(C:\WebRoot\Development\CF\Projects\BH\frameworks\coldbox\3.0\system\web\Controller.cfc:40)
     at coldfusion.runtime.UDFMethod.invoke(UDFMethod.java:472)
     at coldfusion.filter.SilentFilter.invoke(SilentFilter.java:47)
     at coldfusion.runtime.UDFMethod$ReturnTypeFilter.invoke(UDFMethod.java:405)
     at coldfusion.runtime.UDFMethod$ArgumentCollectionFilter.invoke(UDFMethod.java:368)
     at coldfusion.filter.FunctionAccessFilter.invoke(FunctionAccessFilter.java:55)
     at coldfusion.runtime.UDFMethod.runFilterChain(UDFMethod.java:321)
     at coldfusion.runtime.UDFMethod.invoke(UDFMethod.java:220)
     at coldfusion.runtime.TemplateProxy.invoke(TemplateProxy.java:491)
     at coldfusion.runtime.TemplateProxy.invoke(TemplateProxy.java:337)
     at coldfusion.runtime.CfJspPage._invoke(CfJspPage.java:2360)
     at cfColdbox2ecfc1765630879$funcLOADCOLDBOX.runFunction(C:\WebRoot\Development\CF\Projects\BH\frameworks\coldbox\3.0\system\Coldbox.cfc:69)
     at coldfusion.runtime.UDFMethod.invoke(UDFMethod.java:472)
     at coldfusion.filter.SilentFilter.invoke(SilentFilter.java:47)
     at coldfusion.runtime.UDFMethod$ReturnTypeFilter.invoke(UDFMethod.java:405)
     at coldfusion.runtime.UDFMethod$ArgumentCollectionFilter.invoke(UDFMethod.java:368)
     at coldfusion.filter.FunctionAccessFilter.invoke(FunctionAccessFilter.java:55)
     at coldfusion.runtime.UDFMethod.runFilterChain(UDFMethod.java:321)
     at coldfusion.runtime.UDFMethod.invoke(UDFMethod.java:220)
     at coldfusion.runtime.TemplateProxy.invoke(TemplateProxy.java:491)
     at coldfusion.runtime.TemplateProxy.invoke(TemplateProxy.java:337)
     at coldfusion.runtime.CfJspPage._invoke(CfJspPage.java:2360)
     at cfApplication2ecfc422500432$funcONAPPLICATIONSTART.runFunction(C:\WebRoot\Development\CF\Projects\BH\Application.cfc:65)
     at coldfusion.runtime.UDFMethod.invoke(UDFMethod.java:472)
     at coldfusion.filter.SilentFilter.invoke(SilentFilter.java:47)
     at coldfusion.runtime.UDFMethod$ReturnTypeFilter.invoke(UDFMethod.java:405)
     at coldfusion.runtime.UDFMethod$ArgumentCollectionFilter.invoke(UDFMethod.java:368)
     at coldfusion.filter.FunctionAccessFilter.invoke(FunctionAccessFilter.java:55)
     at coldfusion.runtime.UDFMethod.runFilterChain(UDFMethod.java:321)
     at coldfusion.runtime.UDFMethod.invoke(UDFMethod.java:220)
     at coldfusion.runtime.TemplateProxy.invoke(TemplateProxy.java:491)
     at coldfusion.runtime.TemplateProxy.invoke(TemplateProxy.java:337)
     at coldfusion.runtime.AppEventInvoker.invoke(AppEventInvoker.java:88)
     at coldfusion.runtime.AppEventInvoker.onApplicationStart(AppEventInvoker.java:211)
     at coldfusion.filter.ApplicationFilter.invoke(ApplicationFilter.java:242)
     at coldfusion.filter.RequestMonitorFilter.invoke(RequestMonitorFilter.java:48)
     at coldfusion.filter.MonitoringFilter.invoke(MonitoringFilter.java:40)
     at coldfusion.filter.PathFilter.invoke(PathFilter.java:94)
     at coldfusion.filter.LicenseFilter.invoke(LicenseFilter.java:27)
     at coldfusion.filter.ExceptionFilter.invoke(ExceptionFilter.java:70)
     at coldfusion.filter.ClientScopePersistenceFilter.invoke(ClientScopePersistenceFilter.java:28)
     at coldfusion.filter.BrowserFilter.invoke(BrowserFilter.java:38)
     at coldfusion.filter.NoCacheFilter.invoke(NoCacheFilter.java:46)
     at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:38)
     at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)
     at coldfusion.filter.CachingFilter.invoke(CachingFilter.java:62)
     at coldfusion.CfmServlet.service(CfmServlet.java:200)
     at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:89)
     at jrun.servlet.FilterChain.doFilter(FilterChain.java:86)
     at coldfusion.monitor.event.MonitoringServletFilter.doFilter(MonitoringServletFilter.java:42)
     at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46)
     at jrun.servlet.FilterChain.doFilter(FilterChain.java:94)
     at jrun.servlet.FilterChain.service(FilterChain.java:101)
     at jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:106)
     at jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42)
     at jrun.servlet.JRunRequestDispatcher.invokeNext(JRunRequestDispatcher.java:586)
     at jrun.servlet.JRunRequestDispatcher.forwardInvoke(JRunRequestDispatcher.java:555)
     at jrun.servlet.JRunRequestDispatcher.forward(JRunRequestDispatcher.java:203)
     at jrun.servlet.file.FileServlet.service(FileServlet.java:323)
     at jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:106)
     at jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42)
     at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java:286)
     at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java:543)
     at jrun.servlet.http.WebService.invokeRunnable(WebService.java:172)
     at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.java:428)
     at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)

"jms-fifo-1" prio=5 tid=14 WAITING
     at java.lang.Object.wait(Native Method)
     at java.lang.Object.wait(Object.java:485)
     at jrun.jms.core.threads.FIFOQueue.queueHasElements(FIFOQueue.java:45)
     at jrun.jms.core.threads.FIFOQueue.get(FIFOQueue.java:65)
     at jrun.jms.core.threads.ThreadPoolWorker.runWorker(ThreadPoolWorker.java:91)
     at jrun.jms.core.threads.ThreadPoolWorker$1.run(ThreadPoolWorker.java:49)
     at java.lang.Thread.run(Thread.java:619)

"scheduler-20" prio=5 tid=57 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at jrunx.scheduler.SchedulerService.createRunnable(SchedulerService.java:188)
     at jrunx.scheduler.ThreadPool$ThreadThrottle.createRunnable(ThreadPool.java:349)
     at jrunx.scheduler.WorkerThread.run(WorkerThread.java:62)

"obj-skimmer-Backhauls" prio=5 tid=89 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at coldfusion.server.j2ee.pool.PoolSkimmerThread.run(PoolSkimmerThread.java:47)
     at java.lang.Thread.run(Thread.java:619)

"mipc-1" prio=5 tid=56 RUNNABLE
     at java.net.PlainSocketImpl.socketAccept(Native Method)
     at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)
     at java.net.ServerSocket.implAccept(ServerSocket.java:453)
     at java.net.ServerSocket.accept(ServerSocket.java:421)
     at jrun.jms.core.kernel.mipc.MultiplexConnectionServer.run(MultiplexConnectionServer.java:155)

"cfthread-0" prio=5 tid=131 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at coldfusion.util.GenericThreadPool$ThreadPoolRunnableFactory.createRunnable(GenericThreadPool.java:177)
     at coldfusion.scheduling.ThreadPool.createRunnable(ThreadPool.java:128)
     at coldfusion.scheduling.WorkerThread.run(WorkerThread.java:68)

"web-6" prio=5 tid=143 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at jrunx.scheduler.JSemaphore.acquire(JSemaphore.java:74)
     at jrun.servlet.network.NetworkService.accept(NetworkService.java:349)
     at jrun.servlet.http.WebService.createRunnable(WebService.java:104)
     at jrunx.scheduler.ThreadPool$ThreadThrottle.createRunnable(ThreadPool.java:349)
     at jrunx.scheduler.WorkerThread.run(WorkerThread.java:62)

"web-0" prio=5 tid=58 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at jrunx.scheduler.JSemaphore.acquire(JSemaphore.java:74)
     at jrun.servlet.network.NetworkService.accept(NetworkService.java:349)
     at jrun.servlet.http.WebService.createRunnable(WebService.java:104)
     at jrunx.scheduler.ThreadPool$ThreadThrottle.createRunnable(ThreadPool.java:349)
     at jrunx.scheduler.WorkerThread.run(WorkerThread.java:62)

"obj-skimmer-cfdocexamples" prio=5 tid=83 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at coldfusion.server.j2ee.pool.PoolSkimmerThread.run(PoolSkimmerThread.java:47)
     at java.lang.Thread.run(Thread.java:619)

"scheduler-11" prio=5 tid=42 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at jrunx.scheduler.SchedulerService.createRunnable(SchedulerService.java:188)
     at jrunx.scheduler.ThreadPool$ThreadThrottle.createRunnable(ThreadPool.java:349)
     at jrunx.scheduler.WorkerThread.run(WorkerThread.java:62)

"scheduler-17" prio=5 tid=48 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at jrunx.scheduler.SchedulerService.createRunnable(SchedulerService.java:188)
     at jrunx.scheduler.ThreadPool$ThreadThrottle.createRunnable(ThreadPool.java:349)
     at jrunx.scheduler.WorkerThread.run(WorkerThread.java:62)

"scheduler-7" prio=5 tid=38 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at jrunx.scheduler.SchedulerService.createRunnable(SchedulerService.java:188)
     at jrunx.scheduler.ThreadPool$ThreadThrottle.createRunnable(ThreadPool.java:349)
     at jrunx.scheduler.WorkerThread.run(WorkerThread.java:62)

"obj-skimmer-dsnLogon" prio=5 tid=87 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at coldfusion.server.j2ee.pool.PoolSkimmerThread.run(PoolSkimmerThread.java:47)
     at java.lang.Thread.run(Thread.java:619)

"scheduler-4" prio=5 tid=35 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at jrunx.scheduler.SchedulerService.createRunnable(SchedulerService.java:188)
     at jrunx.scheduler.ThreadPool$ThreadThrottle.createRunnable(ThreadPool.java:349)
     at jrunx.scheduler.WorkerThread.run(WorkerThread.java:62)

"obj-skimmer-BH_Local" prio=5 tid=90 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at coldfusion.server.j2ee.pool.PoolSkimmerThread.run(PoolSkimmerThread.java:47)
     at java.lang.Thread.run(Thread.java:619)

"scheduler-14" prio=5 tid=45 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at jrunx.scheduler.SchedulerService.createRunnable(SchedulerService.java:188)
     at jrunx.scheduler.ThreadPool$ThreadThrottle.createRunnable(ThreadPool.java:349)
     at jrunx.scheduler.WorkerThread.run(WorkerThread.java:62)

"jndi-0" prio=5 tid=22 BLOCKED
     at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:388)
     at java.net.ServerSocket.implAccept(ServerSocket.java:453)
     at java.net.ServerSocket.accept(ServerSocket.java:421)
     at jrun.servlet.network.NetworkService.accept(NetworkService.java:357)
     at jrun.naming.NamingNetworkService.accept(NamingNetworkService.java:115)
     at jrun.naming.NamingNetworkService.createRunnable(NamingNetworkService.java:141)
     at jrunx.scheduler.ThreadPool$ThreadThrottle.createRunnable(ThreadPool.java:349)
     at jrunx.scheduler.WorkerThread.run(WorkerThread.java:62)

"jndi-5" prio=5 tid=52 BLOCKED
     at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:388)
     at java.net.ServerSocket.implAccept(ServerSocket.java:453)
     at java.net.ServerSocket.accept(ServerSocket.java:421)
     at jrun.servlet.network.NetworkService.accept(NetworkService.java:357)
     at jrun.naming.NamingNetworkService.accept(NamingNetworkService.java:115)
     at jrun.naming.NamingNetworkService.createRunnable(NamingNetworkService.java:141)
     at jrunx.scheduler.ThreadPool$ThreadThrottle.createRunnable(ThreadPool.java:349)
     at jrunx.scheduler.WorkerThread.run(WorkerThread.java:62)

"scheduler-12" prio=5 tid=43 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at jrunx.scheduler.SchedulerService.createRunnable(SchedulerService.java:188)
     at jrunx.scheduler.ThreadPool$ThreadThrottle.createRunnable(ThreadPool.java:349)
     at jrunx.scheduler.WorkerThread.run(WorkerThread.java:62)

"Thread-14" prio=5 tid=103 WAITING
     at java.lang.Object.wait(Native Method)
     at java.lang.Object.wait(Object.java:485)
     at coldfusion.util.SimpleThreadPool.getAssignment(SimpleThreadPool.java:113)
     at coldfusion.util.SimpleWorkerThread.run(SimpleThreadPool.java:207)

"worker #1" prio=5 tid=107 WAITING
     at java.lang.Object.wait(Native Method)
     at java.lang.Object.wait(Object.java:485)
     at com.jnbridge.jnbcore.server.b.c.run(Unknown Source)
     at java.lang.Thread.run(Thread.java:619)

"scheduler-18" prio=5 tid=49 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at jrunx.scheduler.SchedulerService.createRunnable(SchedulerService.java:188)
     at jrunx.scheduler.ThreadPool$ThreadThrottle.createRunnable(ThreadPool.java:349)
     at jrunx.scheduler.WorkerThread.run(WorkerThread.java:62)

"worker #2" prio=5 tid=108 WAITING
     at java.lang.Object.wait(Native Method)
     at java.lang.Object.wait(Object.java:485)
     at com.jnbridge.jnbcore.server.b.c.run(Unknown Source)
     at java.lang.Thread.run(Thread.java:619)

"obj-skimmer-dsnCommon" prio=5 tid=88 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at coldfusion.server.j2ee.pool.PoolSkimmerThread.run(PoolSkimmerThread.java:47)
     at java.lang.Thread.run(Thread.java:619)

"scheduler-9" prio=5 tid=40 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at jrunx.scheduler.SchedulerService.createRunnable(SchedulerService.java:188)
     at jrunx.scheduler.ThreadPool$ThreadThrottle.createRunnable(ThreadPool.java:349)
     at jrunx.scheduler.WorkerThread.run(WorkerThread.java:62)

"scheduler-5" prio=5 tid=36 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at jrunx.scheduler.SchedulerService.createRunnable(SchedulerService.java:188)
     at jrunx.scheduler.ThreadPool$ThreadThrottle.createRunnable(ThreadPool.java:349)
     at jrunx.scheduler.WorkerThread.run(WorkerThread.java:62)

"scheduler-19" prio=5 tid=50 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at jrunx.scheduler.SchedulerService.createRunnable(SchedulerService.java:188)
     at jrunx.scheduler.ThreadPool$ThreadThrottle.createRunnable(ThreadPool.java:349)
     at jrunx.scheduler.WorkerThread.run(WorkerThread.java:62)

"scheduler-21" prio=5 tid=59 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at jrunx.scheduler.SchedulerService.createRunnable(SchedulerService.java:188)
     at jrunx.scheduler.ThreadPool$ThreadThrottle.createRunnable(ThreadPool.java:349)
     at jrunx.scheduler.WorkerThread.run(WorkerThread.java:62)

"Timer-1" prio=5 tid=64 WAITING
     at java.lang.Object.wait(Native Method)
     at java.lang.Object.wait(Object.java:485)
     at java.util.TimerThread.mainLoop(Timer.java:483)
     at java.util.TimerThread.run(Timer.java:462)

"obj-skimmer-backhauls_DAT2" prio=5 tid=93 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at coldfusion.server.j2ee.pool.PoolSkimmerThread.run(PoolSkimmerThread.java:47)
     at java.lang.Thread.run(Thread.java:619)

"obj-skimmer-BH_Local" prio=5 tid=150 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at coldfusion.server.j2ee.pool.PoolSkimmerThread.run(PoolSkimmerThread.java:47)
     at java.lang.Thread.run(Thread.java:619)

"obj-skimmer-BH_Local" prio=5 tid=153 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at coldfusion.server.j2ee.pool.PoolSkimmerThread.run(PoolSkimmerThread.java:47)
     at java.lang.Thread.run(Thread.java:619)

"scheduler-8" prio=5 tid=39 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at jrunx.scheduler.SchedulerService.createRunnable(SchedulerService.java:188)
     at jrunx.scheduler.ThreadPool$ThreadThrottle.createRunnable(ThreadPool.java:349)
     at jrunx.scheduler.WorkerThread.run(WorkerThread.java:62)

"scheduler-0" prio=5 tid=74 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at coldfusion.scheduling.Scheduler.createRunnable(Scheduler.java:132)
     at coldfusion.scheduling.ThreadPool.createRunnable(ThreadPool.java:128)
     at coldfusion.scheduling.WorkerThread.run(WorkerThread.java:68)

"jms-fifo-2" prio=5 tid=15 WAITING
     at java.lang.Object.wait(Native Method)
     at java.lang.Object.wait(Object.java:485)
     at jrun.jms.core.threads.FIFOQueue.queueHasElements(FIFOQueue.java:45)
     at jrun.jms.core.threads.FIFOQueue.get(FIFOQueue.java:65)
     at jrun.jms.core.threads.ThreadPoolWorker.runWorker(ThreadPoolWorker.java:91)
     at jrun.jms.core.threads.ThreadPoolWorker$1.run(ThreadPoolWorker.java:49)
     at java.lang.Thread.run(Thread.java:619)

"jms-fifo-3" prio=5 tid=16 WAITING
     at java.lang.Object.wait(Native Method)
     at java.lang.Object.wait(Object.java:485)
     at jrun.jms.core.threads.FIFOQueue.queueHasElements(FIFOQueue.java:45)
     at jrun.jms.core.threads.FIFOQueue.get(FIFOQueue.java:65)
     at jrun.jms.core.threads.ThreadPoolWorker.runWorker(ThreadPoolWorker.java:91)
     at jrun.jms.core.threads.ThreadPoolWorker$1.run(ThreadPoolWorker.java:49)
     at java.lang.Thread.run(Thread.java:619)

"jndi-4" prio=5 tid=26 RUNNABLE
     at java.net.PlainSocketImpl.socketAccept(Native Method)
     at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)
     at java.net.ServerSocket.implAccept(ServerSocket.java:453)
     at java.net.ServerSocket.accept(ServerSocket.java:421)
     at jrun.servlet.network.NetworkService.accept(NetworkService.java:357)
     at jrun.naming.NamingNetworkService.accept(NamingNetworkService.java:115)
     at jrun.naming.NamingNetworkService.createRunnable(NamingNetworkService.java:141)
     at jrunx.scheduler.ThreadPool$ThreadThrottle.createRunnable(ThreadPool.java:349)
     at jrunx.scheduler.WorkerThread.run(WorkerThread.java:62)

"scheduler-6" prio=5 tid=37 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at jrunx.scheduler.SchedulerService.createRunnable(SchedulerService.java:188)
     at jrunx.scheduler.ThreadPool$ThreadThrottle.createRunnable(ThreadPool.java:349)
     at jrunx.scheduler.WorkerThread.run(WorkerThread.java:62)

"obj-skimmer-dsnHr" prio=5 tid=92 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at coldfusion.server.j2ee.pool.PoolSkimmerThread.run(PoolSkimmerThread.java:47)
     at java.lang.Thread.run(Thread.java:619)

"obj-skimmer-cfartgallery" prio=5 tid=94 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at coldfusion.server.j2ee.pool.PoolSkimmerThread.run(PoolSkimmerThread.java:47)
     at java.lang.Thread.run(Thread.java:619)

"scheduler-2" prio=5 tid=32 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at jrunx.scheduler.SchedulerService.createRunnable(SchedulerService.java:188)
     at jrunx.scheduler.ThreadPool$ThreadThrottle.createRunnable(ThreadPool.java:349)
     at jrunx.scheduler.WorkerThread.run(WorkerThread.java:62)

"Attach Listener" prio=5 tid=5 RUNNABLE

"Reference Handler" prio=10 tid=2 WAITING
     at java.lang.Object.wait(Native Method)
     at java.lang.Object.wait(Object.java:485)
     at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)

"scheduler-3" prio=5 tid=33 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at jrunx.scheduler.SchedulerService.createRunnable(SchedulerService.java:188)
     at jrunx.scheduler.ThreadPool$ThreadThrottle.createRunnable(ThreadPool.java:349)
     at jrunx.scheduler.WorkerThread.run(WorkerThread.java:62)

"obj-skimmer-BH" prio=5 tid=84 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at coldfusion.server.j2ee.pool.PoolSkimmerThread.run(PoolSkimmerThread.java:47)
     at java.lang.Thread.run(Thread.java:619)

"web-4" prio=5 tid=141 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at jrunx.scheduler.JSemaphore.acquire(JSemaphore.java:74)
     at jrun.servlet.network.NetworkService.accept(NetworkService.java:349)
     at jrun.servlet.http.WebService.createRunnable(WebService.java:104)
     at jrunx.scheduler.ThreadPool$ThreadThrottle.createRunnable(ThreadPool.java:349)
     at jrunx.scheduler.WorkerThread.run(WorkerThread.java:62)

"obj-skimmer-DB2PTCP" prio=5 tid=81 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at coldfusion.server.j2ee.pool.PoolSkimmerThread.run(PoolSkimmerThread.java:47)
     at java.lang.Thread.run(Thread.java:619)

"worker #0" prio=5 tid=106 WAITING
     at java.lang.Object.wait(Native Method)
     at java.lang.Object.wait(Object.java:485)
     at com.jnbridge.jnbcore.server.b.c.run(Unknown Source)
     at java.lang.Thread.run(Thread.java:619)

"Timer-0" prio=5 tid=63 WAITING
     at java.lang.Object.wait(Native Method)
     at java.lang.Object.wait(Object.java:485)
     at java.util.TimerThread.mainLoop(Timer.java:483)
     at java.util.TimerThread.run(Timer.java:462)

"obj-skimmer-cfbookclub" prio=5 tid=80 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at coldfusion.server.j2ee.pool.PoolSkimmerThread.run(PoolSkimmerThread.java:47)
     at java.lang.Thread.run(Thread.java:619)

"Transaction Domain DefaultDomain" prio=5 tid=28 WAITING
     at java.lang.Object.wait(Native Method)
     at java.lang.Object.wait(Object.java:485)
     at jrunx.tyrex.tm.impl.TransactionDomainImpl.run(TransactionDomainImpl.java:1300)
     at java.lang.Thread.run(Thread.java:619)

"scheduler-16" prio=5 tid=47 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at jrunx.scheduler.SchedulerService.createRunnable(SchedulerService.java:188)
     at jrunx.scheduler.ThreadPool$ThreadThrottle.createRunnable(ThreadPool.java:349)
     at jrunx.scheduler.WorkerThread.run(WorkerThread.java:62)

"web-1" prio=5 tid=118 RUNNABLE
     at java.net.PlainSocketImpl.socketAccept(Native Method)
     at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)
     at java.net.ServerSocket.implAccept(ServerSocket.java:453)
     at java.net.ServerSocket.accept(ServerSocket.java:421)
     at jrun.servlet.network.NetworkService.accept(NetworkService.java:357)
     at jrun.servlet.http.WebService.createRunnable(WebService.java:104)
     at jrunx.scheduler.ThreadPool$ThreadThrottle.createRunnable(ThreadPool.java:349)
     at jrunx.scheduler.WorkerThread.run(WorkerThread.java:62)

"scheduler-1" prio=5 tid=31 TIMED_WAITING
     at java.lang.Object.wait(Native Method)
     at jrunx.scheduler.SchedulerService.createRunnable(SchedulerService.java:188)
     at jrunx.scheduler.ThreadPool$ThreadThrottle.createRunnable(ThreadPool.java:349)
     at jrunx.scheduler.WorkerThread.run(WorkerThread.java:62)

"DestroyJavaVM" prio=5 tid=117 RUNNABLE

"Finalizer" prio=8 tid=3 WAITING
     at java.lang.Object.wait(Native Method)
     at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
     at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
     at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)

br...@bradwood.com

unread,
Mar 31, 2011, 2:24:41 PM3/31/11
to col...@googlegroups.com
The idea with a stack trace is often to take several in a row and just look for patterns.  I see the one running thread in your stack that you provided was waiting for class loader hard drive access while creating an object.  "java.io.FileInputStream.open"

If all your stack traces seem to be waiting on file I/O, is there anything on your computer that would be slowing down the hard drive a lot?  Also, has ColdBox always been this way, or did the behavior just start?

You mentioned that you have 4 Gigs of RAM installed on your machine.  How much memory is being used normally?  I've seen excessive swap usage really slow a machine down, but that wouldn't necessarily manifest itself in your stack trace as I/O because Java doesn't actually know when the OS swaps out RAM to disk.

Thanks!


~Brad



-------- Original Message --------
Subject: Re: RE: [coldbox:9116] Very slow startup and errors on
startup.
From: Dustin Martin <d...@dustinmartin.net>
Date: Thu, March 31, 2011 1:17 pm
To: col...@googlegroups.com

@Andrew  - I checked but the server monitor has been disabled

@Brad - I turned on the monitor to get a stack trace but I'm not entirely sure what to look for. I've included it below if you don't mind looking at it.

Thanks!

Dustin Martin

unread,
Mar 31, 2011, 4:39:52 PM3/31/11
to col...@googlegroups.com
I took several stack traces in a row and they all had java.io.FileInputStream.open in them. As far as something that might slow down my hard drive, my company requires hard drive encryption on all laptops so that might be it. On an old laptop I had the encryption software installed on I probably saw it slow down by 25%. It is a pain in the rear.

Now if I remember correctly, CF takes all the CFM/CFC files and compiles them to Java class files, correct? I assume that this might be where I'm running into issues? If this is indeed the problem, I have access to an unencrypted network drive that I can try running CF from. Perhaps that might fix my issue. 

As far as memory usage, when I'm running all my dev software (CF, CF Builder, SQL Server, etc.) I use up most of the 4 gig of RAM. I've recently been having CF hit a dev MSSQL box instead of my local MSSQL install just so I can save the memory by having it disabled.

Dustin Martin

unread,
Apr 4, 2011, 3:44:41 PM4/4/11
to col...@googlegroups.com
I figured out how to fix this problem: simply turn off the ColdFusion setting to save class files to disk. Everything is working great now. My guess would be that running off an unencrypted drive might have worked too but this is much easier.

Thank you very very much for the help!

br...@bradwood.com

unread,
Apr 4, 2011, 4:15:00 PM4/4/11
to col...@googlegroups.com
Thanks for reporting back on that.  Since CF can generate thousands of class files that makes sense that it could really slow down on an encrypted drive. 
Since CF also stores the templates in memory, you shouldn't notice any impact until you restart your server and it has to start over again and recompile everything.  I would bump up your "Maximum number of cached templates" setting to ensure you don't surpass it since you turned off disk caching.

phill.nacelli

unread,
Jun 13, 2012, 4:55:42 PM6/13/12
to col...@googlegroups.com
Dustin, be sure to also flush all these thousands of class files that Brad mentioned.  Once you end up thousands of them, you'll be experiencing slow file system crawls which may be the reason why you had problems, add the file system encryption on top of that and as you now know, it's no picnic! ;)

BTW, Thanks for reporting back how you solved the problem. Cheers. 
Reply all
Reply to author
Forward
0 new messages