Slow system with many threads reporting locked

[eo...@boomi.com]

1. What version of Terracotta Server you are currently using; 4.3.2
2. Paste the configuration you are using; Hibernate 3.6.9.Final
3. Providing JDK and OS versions maybe useful as well. Ubuntu 16.04.2 JDK 1.6

28 threads in thread dump 

   java.lang.Thread.State: WAITING (on object monitor)

        at java.lang.Object.wait(Native Method)

        at java.lang.Object.wait(Object.java:503)

        at com.tc.object.RemoteServerMapManagerImpl.waitUntilRunningAbortable(RemoteServerMapManagerImpl.java:519)

        at com.tc.object.RemoteServerMapManagerImpl.getMappingForKey(RemoteServerMapManagerImpl.java:150)

        - locked <0x00000000c3f743b0> (a com.tc.object.RemoteServerMapManagerImpl)

        at com.tc.object.TCObjectServerMapImpl.getValueForKeyFromServer(TCObjectServerMapImpl.java:678)

        at com.tc.object.TCObjectServerMapImpl.getValueUnlocked(TCObjectServerMapImpl.java:559)

        at com.terracotta.toolkit.util.ExplicitLockingTCObjectServerMapImpl.getValueUnlocked(ExplicitLockingTCObjectServerMapImpl.java:349)

        at com.terracotta.toolkit.collections.map.ServerMap.doLogicalGetValueUnlocked(ServerMap.java:483)

        at com.terracotta.toolkit.collections.map.ServerMap.getSerializedMapValue(ServerMap.java:295)

        at com.terracotta.toolkit.collections.map.ServerMap.doGet(ServerMap.java:263)

        at com.terracotta.toolkit.collections.map.ServerMap.get(ServerMap.java:1274)

        at com.terracotta.toolkit.collections.map.AggregateServerMap.get(AggregateServerMap.java:395)

        at com.terracotta.toolkit.collections.map.ToolkitCacheImpl.get(ToolkitCacheImpl.java:251)

[James House]

Can you please confirm whether you're using the open source version of 4.3.2  (vs. commercial product) ?

Also, can you find a way to share the full thread dump, and preferably a few thread dumps taken a few seconds apart?

Lastly, are you using using explicit locking features (e.g. cache.acquireWriteLockOnKey(Object key)) ?

James

On Thursday, October 19, 2017 at 2:16:45 PM UTC-4,:

[eo...@boomi.com]

It is open source 4.3.2 and not the commercial product

We are not using explicit locking features.

Can I communicate with you directly about the thread dumps? 

[James House]

Yes, you could email me a link to artifacts on something like dropbox, google drive, etc..

[jli...@gmail.com]

I have a very similar situation, can you post some of the possible solutions?