We are having app using ehcache 2.9.0. Lately we have observed deadlocks related to ehcache get/put operations weekly once, and once we clear the cache the issue goes away.
we observe the lock on the RandomAccessFile during cache get operation and ReentrantReadWriteLock during put operation.
Request insight from ehcache experts on what could be possible issue ?
PFB the threads from the thread dumps
"HTTP938" #25950 daemon prio=5 os_prio=0 tid=0x0000000002614000 nid=0x5a64 waiting on condition [0x00007fcd36965000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000005cbbc6768> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:967) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1283) at java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727) at net.sf.ehcache.store.disk.Segment.flush(Segment.java:1010) at net.sf.ehcache.store.disk.DiskStore.flush(DiskStore.java:248) at net.sf.ehcache.store.CacheStore$1.evicted(CacheStore.java:99) at net.sf.ehcache.store.CacheStore$1.evicted(CacheStore.java:96) at net.sf.ehcache.store.cachingtier.OnHeapCachingTier$1.evicted(OnHeapCachingTier.java:86) at net.sf.ehcache.store.cachingtier.CountBasedBackEnd$1.removed(CountBasedBackEnd.java:103) at net.sf.ehcache.util.concurrent.ConcurrentHashMap.internalReplace(ConcurrentHashMap.java:1346) at net.sf.ehcache.util.concurrent.ConcurrentHashMap.removeAndNotify(ConcurrentHashMap.java:2647) at net.sf.ehcache.store.cachingtier.CountBasedBackEnd.remove(CountBasedBackEnd.java:115) at net.sf.ehcache.store.cachingtier.OnHeapCachingTier.remove(OnHeapCachingTier.java:206) at net.sf.ehcache.store.CacheStore.put(CacheStore.java:134) at net.sf.ehcache.Cache.putInternal(Cache.java:1600) at net.sf.ehcache.Cache.put(Cache.java:1526) at net.sf.ehcache.Cache.put(Cache.java:1491) at org.springframework.cache.ehcache.EhCacheCache.put(EhCacheCache.java:82) at org.springframework.cache.interceptor.AbstractCacheInvoker.doPut(AbstractCacheInvoker.java:82) at org.springframework.cache.interceptor.CacheAspectSupport$CachePutRequest.apply(CacheAspectSupport.java:651) at org.springframework.cache.interceptor.CacheAspectSupport.execute(CacheAspectSupport.java:358) at org.springframework.cache.interceptor.CacheAspectSupport.execute(CacheAspectSupport.java:299) at org.springframework.cache.interceptor.CacheInterceptor.invoke(CacheInterceptor.java:61) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:653)
"HTTP927" #25855 daemon prio=5 os_prio=0 tid=0x00007fce685c9000 nid=0x4fb6 waiting for monitor entry [0x00007fcd359e8000] java.lang.Thread.State: BLOCKED (on object monitor) at net.sf.ehcache.store.disk.DiskStorageFactory.read(DiskStorageFactory.java:362) - waiting to lock <0x00000005cbdfb538> (a java.io.RandomAccessFile) at net.sf.ehcache.store.disk.DiskStorageFactory.retrieve(DiskStorageFactory.java:864) at net.sf.ehcache.store.disk.Segment.decode(Segment.java:171) at net.sf.ehcache.store.disk.Segment.remove(Segment.java:644) at net.sf.ehcache.store.disk.DiskStore.remove(DiskStore.java:625) at net.sf.ehcache.store.CacheStore.remove(CacheStore.java:236) at net.sf.ehcache.Cache.removeInternal(Cache.java:2401) at net.sf.ehcache.Cache.tryRemoveImmediately(Cache.java:2162) at net.sf.ehcache.Cache.get(Cache.java:1739) at org.springframework.cache.ehcache.EhCacheCache.get(EhCacheCache.java:65) at org.springframework.cache.interceptor.AbstractCacheInvoker.doGet(AbstractCacheInvoker.java:68) at org.springframework.cache.interceptor.CacheAspectSupport.findInCaches(CacheAspectSupport.java:461) at org.springframework.cache.interceptor.CacheAspectSupport.findCachedItem(CacheAspectSupport.java:432) at org.springframework.cache.interceptor.CacheAspectSupport.execute(CacheAspectSupport.java:333) at org.springframework.cache.interceptor.CacheAspectSupport.execute(CacheAspectSupport.java:299) at org.springframework.cache.interceptor.CacheInterceptor.invoke(CacheInterceptor.java:61) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:653)
Thanks, Nivedita
--
You received this message because you are subscribed to the Google Groups "ehcache-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ehcache-users+unsubscribe@googlegroups.com.
To post to this group, send email to ehcach...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ehcache-users/f837efe3-be17-4c0b-9d5a-d91bcaa2cac0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Thread Name | HTTP762 |
---|---|
State | Runnable |
Monitor | Owns Monitor Lock on 0x00000005cbdfb538 |
Java Stack |
at java.io.RandomAccessFile.readBytes(Native Method)
at java.io.RandomAccessFile.read(RandomAccessFile.java:377) at java.io.RandomAccessFile.readFully(RandomAccessFile.java:436) at java.io.RandomAccessFile.readFully(RandomAccessFile.java:416) at net.sf.ehcache.store.disk.DiskStorageFactory.read(DiskStorageFactory.java:365) - locked [0x00000005cbdfb538] (a java.io.RandomAccessFile) |
at
net.sf.ehcache.store.disk.DiskStorageFactory.retrieve(DiskStorageFactory.java:864)
at net.sf.ehcache.store.disk.Segment.decode(Segment.java:171) |
at net.sf.ehcache.store.disk.Segment.evict(Segment.java:858)
at net.sf.ehcache.store.disk.Segment.evict(Segment.java:830) at net.sf.ehcache.store.disk.Segment.flush(Segment.java:1034) |
at net.sf.ehcache.store.disk.DiskStore.flush(DiskStore.java:248)
at net.sf.ehcache.store.CacheStore$1.evicted(CacheStore.java:99) at net.sf.ehcache.store.CacheStore$1.evicted(CacheStore.java:96) at net.sf.ehcache.store.cachingtier.OnHeapCachingTier$1.evicted(OnHeapCachingTier.java:86) at net.sf.ehcache.store.cachingtier.CountBasedBackEnd$1.removed(CountBasedBackEnd.java:103) at net.sf.ehcache.util.concurrent.ConcurrentHashMap.internalReplace(ConcurrentHashMap.java:1346) at net.sf.ehcache.util.concurrent.ConcurrentHashMap.removeAndNotify(ConcurrentHashMap.java:2647) at net.sf.ehcache.store.cachingtier.CountBasedBackEnd.remove(CountBasedBackEnd.java:115) at net.sf.ehcache.store.cachingtier.OnHeapCachingTier.remove(OnHeapCachingTier.java:206) at net.sf.ehcache.store.CacheStore.put(CacheStore.java:134) at net.sf.ehcache.Cache.putInternal(Cache.java:1600) at net.sf.ehcache.Cache.put(Cache.java:1526) at net.sf.ehcache.Cache.put(Cache.java:1491) at org.springframework.cache.ehcache.EhCacheCache.put(EhCacheCache.java:82) at org.springframework.cache.interceptor.AbstractCacheInvoker.doPut(AbstractCacheInvoker.java:82) at org.springframework.cache.interceptor.CacheAspectSupport$CachePutRequest.apply(CacheAspectSupport.java:651) at org.springframework.cache.interceptor.CacheAspectSupport.execute(CacheAspectSupport.java:358) at org.springframework.cache.interceptor.CacheAspectSupport.execute(CacheAspectSupport.java:299) at org.springframework.cache.interceptor.CacheInterceptor.invoke(CacheInterceptor.java:61) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:653) |
To unsubscribe from this group and stop receiving emails from it, send an email to ehcache-user...@googlegroups.com.
To unsubscribe from this group and stop receiving emails from it, send an email to ehcache-users+unsubscribe@googlegroups.com.
To post to this group, send email to ehcach...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ehcache-users/26ccd570-694e-4269-b110-f28ccdff194a%40googlegroups.com.
To unsubscribe from this group and stop receiving emails from it, send an email to ehcach...@googlegroups.com.
To post to this group, send email to ehcach...@googlegroups.com.
Anybody got around this thread BLOCKED issue.