Very slow ant run-server

[Morris Ford]

As of yesterday every time I run ant run-server, it takes almost
forever. Something changed and I cannot find what it might be. I have
used this system for several months and the entire environment was
copied from another system that I have used for several years. I
checked a build on the old system and it also is exceedingly slow. I
really don't want to have to completely blow away everything to get it
running properly. Is there a security certificate store somewhere that
may have expired?

Morris

[Jonathan Kaplan]

Morris,

Can you run a few stacks on the server process as it is starting up to
see where it is getting stuck? It may be fetching some network
resource that should be local instead.

[Morris Ford]

This is what I am seeing.

2011-03-31 10:45:57

Full thread dump Java HotSpot(TM) 64-Bit Server VM (19.1-b02-334 mixed mode):

"Attach Listener" daemon prio=9 tid=101f7c000 nid=0x1275e9000 waiting on condition [00000000]

   java.lang.Thread.State: RUNNABLE

"Parsing & Indexing Loop (201011082200)" daemon prio=1 tid=10c9b5000 nid=0x128c04000 waiting on condition [128c03000]

   java.lang.Thread.State: WAITING (parking)

at sun.misc.Unsafe.park(Native Method)

- parking to wait for  <7da651ee8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)

at java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:220)

at org.netbeans.modules.parsing.impl.TaskProcessor$CompilationJob.run(TaskProcessor.java:573)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)

at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)

at java.util.concurrent.FutureTask.run(FutureTask.java:138)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

at java.lang.Thread.run(Thread.java:680)

"DestroyJavaVM" prio=5 tid=101ce0000 nid=0x100601000 waiting on condition [00000000]

   java.lang.Thread.State: RUNNABLE

"AWT-EventQueue-1" prio=6 tid=10e20b800 nid=0x128e07000 in Object.wait() [128e06000]

   java.lang.Thread.State: WAITING (on object monitor)

at java.lang.Object.wait(Native Method)

- waiting on <7d807f480> (a org.netbeans.core.TimableEventQueue)

at java.lang.Object.wait(Object.java:485)

at java.awt.EventQueue.getNextEvent(EventQueue.java:521)

- locked <7d807f480> (a org.netbeans.core.TimableEventQueue)

at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:263)

at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:211)

at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:201)

at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:196)

at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:188)

at java.awt.EventDispatchThread.run(EventDispatchThread.java:122)

"TimerQueue" daemon prio=5 tid=102cc2800 nid=0x124ca7000 in Object.wait() [124ca6000]

   java.lang.Thread.State: WAITING (on object monitor)

at java.lang.Object.wait(Native Method)

- waiting on <7d7e32040> (a javax.swing.TimerQueue)

at javax.swing.TimerQueue.run(TimerQueue.java:232)

- locked <7d7e32040> (a javax.swing.TimerQueue)

at java.lang.Thread.run(Thread.java:680)

"Thread-4" daemon prio=5 tid=102e04000 nid=0x127e04000 in Object.wait() [127e03000]

   java.lang.Thread.State: WAITING (on object monitor)

at java.lang.Object.wait(Native Method)

- waiting on <7d84a38c0> (a java.util.LinkedList)

at java.lang.Object.wait(Object.java:485)

at java.util.prefs.AbstractPreferences$EventDispatchThread.run(AbstractPreferences.java:1461)

- locked <7d84a38c0> (a java.util.LinkedList)

"Java2D Disposer" daemon prio=10 tid=10c883000 nid=0x123735000 in Object.wait() [123734000]

   java.lang.Thread.State: WAITING (on object monitor)

at java.lang.Object.wait(Native Method)

- waiting on <7d78be1a8> (a java.lang.ref.ReferenceQueue$Lock)

at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)

- locked <7d78be1a8> (a java.lang.ref.ReferenceQueue$Lock)

at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)

at sun.java2d.Disposer.run(Disposer.java:127)

at java.lang.Thread.run(Thread.java:680)

"AWT-Shutdown" prio=5 tid=1019b2800 nid=0x121a9f000 in Object.wait() [121a9e000]

   java.lang.Thread.State: WAITING (on object monitor)

at java.lang.Object.wait(Native Method)

- waiting on <7d78be148> (a java.lang.Object)

at java.lang.Object.wait(Object.java:485)

at sun.awt.AWTAutoShutdown.run(AWTAutoShutdown.java:265)

- locked <7d78be148> (a java.lang.Object)

at java.lang.Thread.run(Thread.java:680)

"AWT-AppKit" daemon prio=5 tid=1019b1800 nid=0x7fff7080aca0 runnable [00000000]

   java.lang.Thread.State: RUNNABLE

"Poller SunPKCS11-Darwin" daemon prio=1 tid=101a0c000 nid=0x11141a000 waiting on condition [111419000]

   java.lang.Thread.State: TIMED_WAITING (sleeping)

at java.lang.Thread.sleep(Native Method)

at sun.security.pkcs11.SunPKCS11$TokenPoller.run(SunPKCS11.java:692)

at java.lang.Thread.run(Thread.java:680)

"CLI Requests Server" daemon prio=5 tid=10e049000 nid=0x111014000 runnable [111013000]

   java.lang.Thread.State: RUNNABLE

at java.net.PlainSocketImpl.socketAccept(Native Method)

at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:408)

- locked <7d78e25b8> (a java.net.SocksSocketImpl)

at java.net.ServerSocket.implAccept(ServerSocket.java:462)

at java.net.ServerSocket.accept(ServerSocket.java:430)

at org.netbeans.CLIHandler$Server.run(CLIHandler.java:999)

"Active Reference Queue Daemon" daemon prio=1 tid=10291b000 nid=0x10ed04000 in Object.wait() [10ed03000]

   java.lang.Thread.State: WAITING (on object monitor)

at java.lang.Object.wait(Native Method)

- waiting on <7d7832b00> (a java.lang.ref.ReferenceQueue$Lock)

at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)

- locked <7d7832b00> (a java.lang.ref.ReferenceQueue$Lock)

at org.openide.util.lookup.implspi.ActiveQueue$Impl.run(ActiveQueue.java:59)

at java.lang.Thread.run(Thread.java:680)

"Timer-0" daemon prio=5 tid=10e035800 nid=0x10eb01000 in Object.wait() [10eb00000]

   java.lang.Thread.State: TIMED_WAITING (on object monitor)

at java.lang.Object.wait(Native Method)

- waiting on <7d7832b18> (a java.util.TaskQueue)

at java.util.TimerThread.mainLoop(Timer.java:509)

- locked <7d7832b18> (a java.util.TaskQueue)

at java.util.TimerThread.run(Timer.java:462)

"Low Memory Detector" daemon prio=5 tid=10e000000 nid=0x10dc01000 runnable [00000000]

   java.lang.Thread.State: RUNNABLE

"CompilerThread1" daemon prio=9 tid=101816000 nid=0x10d80a000 waiting on condition [00000000]

   java.lang.Thread.State: RUNNABLE

"CompilerThread0" daemon prio=9 tid=101815000 nid=0x10d607000 waiting on condition [00000000]

   java.lang.Thread.State: RUNNABLE

"Signal Dispatcher" daemon prio=9 tid=101814800 nid=0x10d404000 runnable [00000000]

   java.lang.Thread.State: RUNNABLE

"Surrogate Locker Thread (CMS)" daemon prio=5 tid=101813800 nid=0x10d201000 waiting on condition [00000000]

   java.lang.Thread.State: RUNNABLE

"Finalizer" daemon prio=8 tid=1028f1800 nid=0x10ba01000 in Object.wait() [10ba00000]

   java.lang.Thread.State: WAITING (on object monitor)

at java.lang.Object.wait(Native Method)

- waiting on <7d78929b8> (a java.lang.ref.ReferenceQueue$Lock)

at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)

- locked <7d78929b8> (a java.lang.ref.ReferenceQueue$Lock)

at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)

at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)

"Reference Handler" daemon prio=10 tid=1028f1000 nid=0x10af23000 in Object.wait() [10af22000]

   java.lang.Thread.State: WAITING (on object monitor)

at java.lang.Object.wait(Native Method)

- waiting on <7d78be160> (a java.lang.ref.Reference$Lock)

at java.lang.Object.wait(Object.java:485)

at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)

- locked <7d78be160> (a java.lang.ref.Reference$Lock)

"VM Thread" prio=9 tid=1028ec800 nid=0x10ad20000 runnable 

"Gang worker#0 (Parallel GC Threads)" prio=9 tid=102803800 nid=0x102604000 runnable 

"Gang worker#1 (Parallel GC Threads)" prio=9 tid=102804000 nid=0x106a02000 runnable 

"Gang worker#2 (Parallel GC Threads)" prio=9 tid=102805000 nid=0x106c05000 runnable 

"Gang worker#3 (Parallel GC Threads)" prio=9 tid=102805800 nid=0x106e08000 runnable 

"Concurrent Mark-Sweep GC Thread" prio=9 tid=102880800 nid=0x10aa2c000 runnable 

"VM Periodic Task Thread" prio=10 tid=10c80d000 nid=0x10de04000 waiting on condition 

"Exception Catcher Thread" prio=10 tid=102801800 nid=0x102401000 runnable 

JNI global references: 18024

[Jonathan Kaplan]

I think that trace is from NetBeans. Try running the Wonderland server from the command line using ant, so it doesn't run in NetBeans.