Re: 4.0.3 WrongTargetException (WrongTarget! local)

[M. Sancar Koyunlu]

It looks like the raft invocation system can not find any member to invoke on. 
Could it be the case that some of the members failed and left the cluster/and may be rejoined?  
Could you be seeing the same behavior in this one https://github.com/hazelcast/hazelcast/issues/15299 ?
If yes, `.setMissingCPMemberAutoRemovalSeconds(14400)` It could be the case that you are in this period and the majority can not be achieved. 
In that case, the restarted members needs to be removed from the cluster manually. 
See https://docs.hazelcast.org/docs/3.12.1/manual/html-single/index.html#cp-subsystem-management-apis

On Thu, Dec 24, 2020 at 11:31 AM Dzenan Becko <dzena...@gmail.com> wrote:

And these warning messages are displayed in console output:

Dec 24, 2020 9:28:10 AM com.hazelcast.spi.impl.operationservice.impl.Invocation

WARNING: [127.0.0.1]:5701 [dev] [4.0.3] Retrying invocation: Invocation{op=com.hazelcast.cp.internal.operation.DefaultRaftReplicateOp{serviceName='hz:core:raft', identityHash=1066381965, partitionId=0, replicaIndex=0, callId=236, invocationTime=1608798490233 (2020-12-24 09:28:10.233), waitTimeout=-1, callTimeout=60000, groupId=CPGroupId{name='METADATA', seed=0, groupId=0}, op=com.hazelcast.cp.internal.raftop.metadata.AddCPMemberOp{serviceName='hz:core:raft', member=CPMember{uuid=14510156-e10e-48ed-b042-9cc4ea733076, address=[127.0.0.1]:5701}}}, tryCount=250, tryPauseMillis=500, invokeCount=100, callTimeoutMillis=60000, firstInvocationTimeMs=1608798455708, firstInvocationTime='2020-12-24 09:27:35.708', lastHeartbeatMillis=0, lastHeartbeatTime='1970-01-01 01:00:00.000', target=null, pendingResponse={VOID}, backupsAcksExpected=-1, backupsAcksReceived=0, connection=null}, Reason: com.hazelcast.spi.exception.WrongTargetException: WrongTarget! local: Member [127.0.0.1]:5701 - 51462f8b-b718-45cc-8b32-4995f5a69b50 this, expected-target: null, partitionId: 0, replicaIndex: 0, operation: com.hazelcast.cp.internal.operation.DefaultRaftReplicateOp, service: hz:core:raft

Dec 24, 2020 9:28:10 AM com.hazelcast.spi.impl.operationservice.impl.Invocation

WARNING: [127.0.0.1]:5701 [dev] [4.0.3] Retrying invocation: Invocation{op=com.hazelcast.cp.internal.operation.RaftQueryOp{serviceName='hz:core:raft', identityHash=1830841634, partitionId=0, replicaIndex=0, callId=237, invocationTime=1608798490236 (2020-12-24 09:28:10.236), waitTimeout=-1, callTimeout=60000, op=com.hazelcast.cp.internal.raftop.metadata.GetActiveRaftGroupByNameOp{serviceName='hz:core:raft', groupName=default}, groupId=CPGroupId{name='METADATA', seed=0, groupId=0}, policy=LINEARIZABLE}, tryCount=250, tryPauseMillis=500, invokeCount=100, callTimeoutMillis=60000, firstInvocationTimeMs=1608798455710, firstInvocationTime='2020-12-24 09:27:35.710', lastHeartbeatMillis=0, lastHeartbeatTime='1970-01-01 01:00:00.000', target=null, pendingResponse={VOID}, backupsAcksExpected=-1, backupsAcksReceived=0, connection=null}, Reason: com.hazelcast.spi.exception.WrongTargetException: WrongTarget! local: Member [127.0.0.1]:5701 - 51462f8b-b718-45cc-8b32-4995f5a69b50 this, expected-target: null, partitionId: 0, replicaIndex: 0, operation: com.hazelcast.cp.internal.operation.RaftQueryOp, service: hz:core:raft

Dana četvrtak, 24. decembar 2020. u 09:25:42 UTC+1 osoba Dzenan Becko je napisala:

Hi all,
I am trying to implement hazelcast with CPSubsystem and replace usage of ILock with FencedLock. Current claster size is 3 and look like this:

Members {size:3, ver:202}
Member [11.116.118.30]:5701 - dc15874f-1dde-45f3-a2f3-f4907f60e9e2
Member [11.116.115.136]:5701 - c4800b0b-8ab4-43fb-9147-3df8c340f59f
Member [11.116.118.29]:5701 - e21a951b-aaea-4b2f-8350-44354d0a1281 this

Hazelcast configuration look like this:
Config config = new Config();
config.getCPSubsystemConfig()
.setCPMemberCount(3)
.setSessionTimeToLiveSeconds(300)
.setSessionHeartbeatIntervalSeconds(5)
.setMissingCPMemberAutoRemovalSeconds(14400)
.setFailOnIndeterminateOperationState(false);


I have autowired HazelcastInstance in my service and when I try to use FencedLock on this way:

FencedLock hazelcastLock = hazelcastInstance.getCPSubsystem().getLock(hazelcastLockName);

this line gives me a WrongTargetException: WrongTarget! local: Member [11.116.118.29]:5701 - e21a951b-aaea-4b2f-8350-44354d0a1281 this, expected-target: null, partitionId: 0, replicaIndex: 0, operation: com.hazelcast.cp.internal.operation.RaftQueryOp, service: hz:core:raft

What causes that exception?

--
You received this message because you are subscribed to the Google Groups "Hazelcast" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hazelcast+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/hazelcast/012323b9-6ad9-4521-b1dd-db1b2bbf66b6n%40googlegroups.com.

--

Sancar Koyunlu Software Engineer       hazelcast®     2 W 5th Ave, Ste 300 | San Mateo, CA 94402 | USA +1 (650) 521-5453 | hazelcast.com

This message contains confidential information and is intended only for the individuals named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. If verification is required, please request a hard-copy version. -Hazelcast