Are read locks released automatically after some time?

[Mickael Marrache]

Hi,

I'm considering the case of an application that only reads from a map. If the application crashes while reading, read locks may still be retained. If the application starts again, what happens with these locks? May the reading application still read the locked entries? (I guess that a read lock may be acquired by multiple clients and therefore that should not be an issue?)

Now, if there also is a writing application, I guess the entries cannot be updated, right? Is it possible to detect these stale locks?

Thanks,

Mickael

[Roman Leventov]

Chronicle Map's locks are just 64-bit areas in off-heap memory. "Read lock acquisition" is an atomic increment of a value in a certain bits range, while dedicated bit for write lock is zero. "Read lock release" is an atomic decrement of the same value. When application crashes, nobody clears those bits automatically.

So when application starts again, it could still read entries, because it could increment the read count value. Eventually it could become "polluted wth dead read locks", the area of bits for read count value overflows and IllegalStateException is thrown on an attempt to acquire read lock. But this unlikely to happen, currently the capacity of the read count area is 2^30 (subject to change in the future, though).

> Now, if there also is a writing application, I guess the entries cannot be updated, right?

Yes, it will be not possible.

> Is it possible to detect these stale locks?

Currently the only available way to clear stale locks is to perform a recovery procedure on the map: https://github.com/OpenHFT/Chronicle-Map#recovery

As I noted in this message: https://groups.google.com/d/msg/java-chronicle/APHa04f1IlA/4Ivs3X_IBAAJ, a "lightweight" recovery procedure could be added, which only clears locks, but doesn't scan the entire map.

Some way to find and clear stale locks automatically during normal work with a Chronicle Map was suggested here: https://github.com/OpenHFT/Chronicle-Map/issues/79#issuecomment-210227919

On Tue, Aug 16, 2016 at 1:34 AM, Mickael Marrache <mickael...@gmail.com> wrote:

Now, if there also is a writing application, I guess the entries cannot be updated, right?

 

Is it possible to detect these stale locks?

Thanks,

Mickael

--
You received this message because you are subscribed to the Google Groups "Chronicle" group.
To unsubscribe from this group and stop receiving emails from it, send an email to java-chronicle+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Peter Lawrey]

Hello,

One of the design principles of Chronicle Map and Queue is we only utilise the features the OS gives us and don't expect a daemon process or broker to be running.  This makes for a clear outcome if your application crashes.  On the down side, there is no house keeping process running as in other solutions and each process must decide what to do in the event of another process failing in a distributed manner which is tricky.

Regards,

   Peter.

[Roman Leventov]

I would like to add something to what Peter has written:

1) proposed way for auto lock clearing doesn't require introduction of housekeeping processes, though there are other technical difficulties with implementing that.

2) Housekeeping _threads_  (not processes), started in each JVM upon "opening" a Chronicle Map and ceased upon map.close(), are possible -- actually, one kind of such threads already exists, it's so-called "deleted entry cleanup threads", addressing the pollution with deleted entries of replicated chronicle maps: https://higherfrequencytrading.atlassian.net/browse/HCOLL-227. Other kinds of such threads could possibly be introduced, e. g. allocation area defragmentation/compaction threads, repartitioning threads, etc. Such threads should be designed keeping in mind that threads doing the same tasks may exist in different JVMs, so it's not an issue. 

"Housekeeping threads" shift Chronicle Map towards the "db engine" from the "data structure" type of technology, but the user experience is still almost like it's a plain map data structure (no "db management", dedicated processes, etc.)

On Tue, Aug 16, 2016 at 4:24 AM, Peter Lawrey <peter....@gmail.com> wrote:

Hello,

One of the design principles of Chronicle Map and Queue is we only utilise the features the OS gives us and don't expect a daemon process or broker to be running.  This makes for a clear outcome if your application crashes.  On the down side, there is no house keeping process running as in other solutions and each process must decide what to do in the event of another process failing in a distributed manner which is tricky.

Regards,

   Peter.