Hazelcast 3.0 / Lock not released / Current thread is not owner of the lock!

[dan]

Hi --

After upgrading to 3.0, we are encountering errors where our locks are not being released.  This webapp has run for many, many moons on the 2.5/6 versions without issues.  In our app, we have a critical section that acquires a lock and then releases it in a finally clause.  This critical section is entered by from different sources so a many locks might be active at one time.

The webapp runs for some time correctly, and then an error occurs.  Here's the first error we see:

Caused by: java.lang.IllegalMonitorStateException: Current thread is not owner of the lock! -> Owner: a4b73e1c-6d27-4527-bb86-b1070d82dcca, thread-id: 131

at com.hazelcast.concurrent.lock.UnlockOperation.run(UnlockOperation.java:51)

at com.hazelcast.spi.impl.OperationServiceImpl.doRunOperation(OperationServiceImpl.java:222)

at com.hazelcast.spi.impl.OperationServiceImpl.access$800(OperationServiceImpl.java:55)

at com.hazelcast.spi.impl.OperationServiceImpl$RemoteOperationProcessor.run(OperationServiceImpl.java:684)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:722)

at com.hazelcast.spi.impl.OperationServiceImpl$OperationThread.run(OperationServiceImpl.java:731)

 

2013-08-19 06:27:42.228 [hz.BamHazelcast.cached.thread-1] INFO  com.hazelcast.core.LifecycleService - [192.168.50.129]:5701 [TEST] Address[192.168.50.129]:5701 is MERGED 

Any ideas?  Did something change in the way threads are managed for locks? What does the MERGED message above mean?

Thanks,

Dan

[Mehmet Dogan]

It seems you had a split brain issue. MERGED means node 192.168.50.129 was split from other nodes and then it merged back to the original cluster. During split and merge owner of the lock in the cluster changed and after you couldn't unlock it.

@mmdogan

~Sent from mobile

--
You received this message because you are subscribed to the Google Groups "Hazelcast" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hazelcast+...@googlegroups.com.
To post to this group, send email to haze...@googlegroups.com.
Visit this group at http://groups.google.com/group/hazelcast.
For more options, visit https://groups.google.com/groups/opt_out.

[dan]

Thanks, this feedback was right on the money.  I found the cause of the problem -- we were running a test cluster and the ip address of one machine had changed, but we were using tcp discovery.  It was launched as split brain, but I didn't realize it.  Thanks for your help :)

Dan

[Josef Cacek]

Hi,

I had to think twice, what was the problem with Mehmet's answer and
then I realized that the first sentence could sound offensive for
someone who is not familiar with the terminology in distributed
applications world.
The "split brain" just means that your cluster falls apart. The
cluster nodes (members) disjoint for instance due to a network
problem. The cluster parts can rejoin again later - the cluster is
"healed" and the data are "merged".

http://docs.hazelcast.org/docs/latest/manual/html-single/index.html#split-brain-syndrome

Regards,
-- Josef

On Thu, Aug 9, 2018 at 10:19 AM, Mehmet Dogan <meh...@hazelcast.com> wrote:
> @himanisardana85: Why is that? What was wrong with the answer?
>
> On Thu, Aug 9, 2018 at 10:50 AM <himanis...@gmail.com> wrote:
>>
>> Terrible way to answer people.amazingly horrible you are @mmdogan

>> Visit this group at https://groups.google.com/group/hazelcast.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/hazelcast/93546f26-2714-4179-a92d-422e0eb2cbad%40googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>
> --
>
> Mehmet Dogan

>
> --
> You received this message because you are subscribed to the Google Groups
> "Hazelcast" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to hazelcast+...@googlegroups.com.
> To post to this group, send email to haze...@googlegroups.com.

> Visit this group at https://groups.google.com/group/hazelcast.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/hazelcast/CAE%3D6bkVpbCntLxoeAHvg2GJtZoJGjpwDFGmDgKoMkgtyLhvJ7w%40mail.gmail.com.
>
> For more options, visit https://groups.google.com/d/optout.

[himanis...@gmail.com]

My apologies guys.u was nt aware that split brain is a network failure condition.i thought the word was used for the person who placed the question.


Again my sincere apologies!!