1 I reproduced it with the snapshot too:
2 run with param 1
3 run without params
4 see
poll got one
poll got two
5 sleep, wake up
6 i got this (process doing the poll):
Thu Jul 09 15:34:54 CEST 2009 FINEST: [com.hazelcast.impl.BaseManager]
Executor core:16, max:64, keepAlive:60
Thu Jul 09 15:34:54 CEST 2009 FINEST:
[com.hazelcast.cluster.ClusterManager] ClusterManager adding Member
[
172.26.0.63:5702] this
Thu Jul 09 15:34:54 CEST 2009 INFO: [com.hazelcast.nio.InSelector]
Started Selector at 5702
Hazelcast 1.6-snapshot-20090709 (20090709) starting at Address
[
172.26.0.63:5702]
Copyright (C) 2009 Hazelcast.com
Thu Jul 09 15:34:54 CEST 2009 FINEST: [com.hazelcast.impl.Node] Address
[
172.26.0.63:5702] master: Address[
172.26.0.63:5701]
Thu Jul 09 15:34:54 CEST 2009 FINEST: [com.hazelcast.impl.Node]
joining... Address[
172.26.0.63:5701]
Thu Jul 09 15:34:54 CEST 2009 INFO: [com.hazelcast.nio.OutSelector]
connect to Address[
172.26.0.63:5701]
Thu Jul 09 15:34:54 CEST 2009 FINEST: [com.hazelcast.nio.OutSelector
$Connector] connecting to Address[
172.26.0.63:5701]
Thu Jul 09 15:34:54 CEST 2009 FINEST: [com.hazelcast.nio.OutSelector
$Connector] connection check. connected: false, Address
[
172.26.0.63:5701]
Thu Jul 09 15:34:54 CEST 2009 FINEST: [com.hazelcast.nio.OutSelector
$Connector] connected to Address[
172.26.0.63:5701]
Thu Jul 09 15:34:54 CEST 2009 FINEST:
[com.hazelcast.impl.ThreadContext] hz.ServiceThread is starting with
cacheSize 100
Thu Jul 09 15:34:54 CEST 2009 FINEST:
[com.hazelcast.impl.ThreadContext] hz.OutThread is starting with
cacheSize 0
Thu Jul 09 15:34:55 CEST 2009 FINEST: [com.hazelcast.impl.Node] Master
connnection Connection [Address[
172.26.0.63:5701]] live=true
Thu Jul 09 15:34:55 CEST 2009 FINEST:
[com.hazelcast.impl.ThreadContext] main is starting with cacheSize 12
Thu Jul 09 15:34:55 CEST 2009 FINEST:
[com.hazelcast.impl.ThreadContext] hz.InThread is starting with
cacheSize 100
Thu Jul 09 15:34:55 CEST 2009 FINEST: [com.hazelcast.impl.Node]
joining... Address[
172.26.0.63:5701]
Thu Jul 09 15:34:55 CEST 2009 FINEST: [com.hazelcast.impl.Node] Master
connnection Connection [Address[
172.26.0.63:5701]] live=true
Thu Jul 09 15:34:56 CEST 2009 FINEST: [com.hazelcast.impl.Node]
joining... Address[
172.26.0.63:5701]
Thu Jul 09 15:34:56 CEST 2009 FINEST: [com.hazelcast.impl.Node] Master
connnection Connection [Address[
172.26.0.63:5701]] live=true
Thu Jul 09 15:34:56 CEST 2009 FINEST: [com.hazelcast.impl.Node]
joining... Address[
172.26.0.63:5701]
Thu Jul 09 15:34:56 CEST 2009 FINEST: [com.hazelcast.impl.Node] Master
connnection Connection [Address[
172.26.0.63:5701]] live=true
Thu Jul 09 15:34:57 CEST 2009 FINEST: [com.hazelcast.impl.Node]
joining... Address[
172.26.0.63:5701]
Thu Jul 09 15:34:57 CEST 2009 FINEST: [com.hazelcast.impl.Node] Master
connnection Connection [Address[
172.26.0.63:5701]] live=true
Thu Jul 09 15:34:57 CEST 2009 FINEST: [com.hazelcast.impl.Node]
joining... Address[
172.26.0.63:5701]
Thu Jul 09 15:34:57 CEST 2009 FINEST: [com.hazelcast.impl.Node] Master
connnection Connection [Address[
172.26.0.63:5701]] live=true
Thu Jul 09 15:34:58 CEST 2009 FINEST: [com.hazelcast.impl.Node]
joining... Address[
172.26.0.63:5701]
Thu Jul 09 15:34:58 CEST 2009 FINEST: [com.hazelcast.impl.Node] Master
connnection Connection [Address[
172.26.0.63:5701]] live=true
Thu Jul 09 15:34:58 CEST 2009 FINEST: [com.hazelcast.impl.Node]
joining... Address[
172.26.0.63:5701]
Thu Jul 09 15:34:58 CEST 2009 FINEST: [com.hazelcast.impl.Node] Master
connnection Connection [Address[
172.26.0.63:5701]] live=true
Thu Jul 09 15:34:59 CEST 2009 FINEST: [com.hazelcast.impl.Node]
joining... Address[
172.26.0.63:5701]
Thu Jul 09 15:34:59 CEST 2009 FINEST: [com.hazelcast.impl.Node] Master
connnection Connection [Address[
172.26.0.63:5701]] live=true
Thu Jul 09 15:34:59 CEST 2009 FINEST: [com.hazelcast.impl.Node]
joining... Address[
172.26.0.63:5701]
Thu Jul 09 15:34:59 CEST 2009 FINEST: [com.hazelcast.impl.Node] Master
connnection Connection [Address[
172.26.0.63:5701]] live=true
Thu Jul 09 15:35:00 CEST 2009 FINEST: [com.hazelcast.impl.Node]
joining... Address[
172.26.0.63:5701]
Thu Jul 09 15:35:00 CEST 2009 FINEST: [com.hazelcast.impl.Node] Master
connnection Connection [Address[
172.26.0.63:5701]] live=true
Thu Jul 09 15:35:00 CEST 2009 FINEST: [com.hazelcast.impl.Node]
joining... Address[
172.26.0.63:5701]
Thu Jul 09 15:35:00 CEST 2009 FINEST: [com.hazelcast.impl.Node] Master
connnection Connection [Address[
172.26.0.63:5701]] live=true
Thu Jul 09 15:35:00 CEST 2009 FINEST:
[com.hazelcast.cluster.ClusterManager] MEMBERS UPDATE!!
Thu Jul 09 15:35:00 CEST 2009 FINEST:
[com.hazelcast.cluster.ClusterManager] ClusterManager adding Member
[
172.26.0.63:5701] *
Thu Jul 09 15:35:00 CEST 2009 FINEST:
[com.hazelcast.cluster.ClusterManager] ClusterManager adding Member
[
172.26.0.63:5702] this
Thu Jul 09 15:35:00 CEST 2009 INFO:
[com.hazelcast.cluster.ClusterManager]
Members [2] {
Member [
172.26.0.63:5701] *
Member [
172.26.0.63:5702] this
}
Thu Jul 09 15:35:00 CEST 2009 FINEST: [com.hazelcast.impl.BaseManager]
q:queryswaiting.maxSizePerJVM=10000
Thu Jul 09 15:35:00 CEST 2009 FINEST: [com.hazelcast.impl.BaseManager]
q:queryswaiting.maxAge=
2147483647
Thu Jul 09 15:35:00 CEST 2009 FINEST:
[com.hazelcast.impl.ConcurrentMapManager] Migration Complete from
Address[
172.26.0.63:5702]
Thu Jul 09 15:35:00 CEST 2009 FINEST:
[com.hazelcast.impl.ConcurrentMapManager$33$1] Migration ended!
Thu Jul 09 15:35:00 CEST 2009 FINEST:
[com.hazelcast.impl.ConcurrentMapManager] Migration Complete from
Address[
172.26.0.63:5701]
Thu Jul 09 15:35:01 CEST 2009 FINEST: [com.hazelcast.impl.Node] Join
DONE
poll got one
poll got two
Thu Jul 09 15:38:43 CEST 2009 FINEST: [com.hazelcast.impl.BaseManager]
Removing Address Address[
172.26.0.63:5701]
Thu Jul 09 15:38:43 CEST 2009 FINEST: [com.hazelcast.impl.BaseManager]
Now Master Address[
172.26.0.63:5702]
Thu Jul 09 15:38:43 CEST 2009 FINEST:
[com.hazelcast.cluster.ClusterManager] removing Address
[
172.26.0.63:5701]
Thu Jul 09 15:38:43 CEST 2009 FINEST: [com.hazelcast.impl.BaseManager]
Address[
172.26.0.63:5701] is dead and its backup was Member
[
172.26.0.63:5702] this *
Thu Jul 09 15:38:43 CEST 2009 INFO:
[com.hazelcast.cluster.ClusterManager]
Members [1] {
Member [
172.26.0.63:5702] this *
}
Thu Jul 09 15:38:43 CEST 2009 FINEST: [com.hazelcast.impl.BaseManager]
Removing Address Address[
172.26.0.63:5701]
poll got one
Thu Jul 09 15:38:43 CEST 2009 FINEST: [com.hazelcast.impl.BaseManager]
Address[
172.26.0.63:5701] is dead and its backup was null
Thu Jul 09 15:38:43 CEST 2009 INFO:
[com.hazelcast.cluster.ClusterManager]
Members [1] {
Member [
172.26.0.63:5702] this *
}
Thu Jul 09 15:38:43 CEST 2009 FINEST:
[com.hazelcast.impl.ConcurrentMapManager] Migration Complete from
Address[
172.26.0.63:5702]
Thu Jul 09 15:38:43 CEST 2009 FINEST:
[com.hazelcast.impl.ConcurrentMapManager$33$1] Migration ended!
Thu Jul 09 15:38:43 CEST 2009 FINEST:
[com.hazelcast.impl.ConcurrentMapManager] Migration Complete from
Address[
172.26.0.63:5702]
Thu Jul 09 15:38:43 CEST 2009 FINEST:
[com.hazelcast.impl.ConcurrentMapManager$33$1] Migration ended!
poll got two
the one doing hte put:
Thu Jul 09 15:34:41 CEST 2009 FINEST: [com.hazelcast.impl.BaseManager]
Executor core:16, max:64, keepAlive:60
Thu Jul 09 15:34:41 CEST 2009 FINEST:
[com.hazelcast.cluster.ClusterManager] ClusterManager adding Member
[
172.26.0.63:5701] this
Thu Jul 09 15:34:41 CEST 2009 INFO: [com.hazelcast.nio.InSelector]
Started Selector at 5701
Hazelcast 1.6-snapshot-20090709 (20090709) starting at Address
[
172.26.0.63:5701]
Copyright (C) 2009 Hazelcast.com
Thu Jul 09 15:34:43 CEST 2009 FINEST: [com.hazelcast.impl.Node] Address
[
172.26.0.63:5701] master: null
Thu Jul 09 15:34:43 CEST 2009 FINEST:
[com.hazelcast.cluster.ClusterManager] ClusterManager adding Member
[
172.26.0.63:5701] this
Thu Jul 09 15:34:43 CEST 2009 FINEST: [com.hazelcast.impl.Node] Join
DONE
Thu Jul 09 15:34:43 CEST 2009 INFO: [com.hazelcast.impl.Node]
Members [1] {
Member [
172.26.0.63:5701] this *
}
Thu Jul 09 15:34:43 CEST 2009 FINEST:
[com.hazelcast.impl.ThreadContext] hz.ServiceThread is starting with
cacheSize 100
Thu Jul 09 15:34:43 CEST 2009 FINEST:
[com.hazelcast.impl.ThreadContext] main is starting with cacheSize 12
Thu Jul 09 15:34:43 CEST 2009 FINEST: [com.hazelcast.impl.BaseManager]
q:queryswaiting.maxSizePerJVM=10000
Thu Jul 09 15:34:43 CEST 2009 FINEST: [com.hazelcast.impl.BaseManager]
q:queryswaiting.maxAge=
2147483647
Thu Jul 09 15:34:54 CEST 2009 INFO: [com.hazelcast.nio.InSelector
$Acceptor] 5701 this socket is connected to /
172.26.0.63:62217
Thu Jul 09 15:34:54 CEST 2009 FINEST:
[com.hazelcast.impl.ThreadContext] hz.InThread is starting with
cacheSize 100
Thu Jul 09 15:34:55 CEST 2009 FINEST:
[com.hazelcast.cluster.ClusterManager] false Handling JoinRequest
{nodeType=MEMBER, address=Address[
172.26.0.63:5702], groupName='dev',
groupPassword='dev-pass'}
Thu Jul 09 15:34:55 CEST 2009 FINEST:
[com.hazelcast.impl.ThreadContext] hz.OutThread is starting with
cacheSize 0
Thu Jul 09 15:34:55 CEST 2009 FINEST:
[com.hazelcast.cluster.ClusterManager] false Handling JoinRequest
{nodeType=MEMBER, address=Address[
172.26.0.63:5702], groupName='dev',
groupPassword='dev-pass'}
Thu Jul 09 15:34:55 CEST 2009 FINEST:
[com.hazelcast.cluster.ClusterManager] false Handling JoinRequest
{nodeType=MEMBER, address=Address[
172.26.0.63:5702], groupName='dev',
groupPassword='dev-pass'}
Thu Jul 09 15:34:56 CEST 2009 FINEST:
[com.hazelcast.cluster.ClusterManager] false Handling JoinRequest
{nodeType=MEMBER, address=Address[
172.26.0.63:5702], groupName='dev',
groupPassword='dev-pass'}
Thu Jul 09 15:34:56 CEST 2009 FINEST:
[com.hazelcast.cluster.ClusterManager] false Handling JoinRequest
{nodeType=MEMBER, address=Address[
172.26.0.63:5702], groupName='dev',
groupPassword='dev-pass'}
Thu Jul 09 15:34:57 CEST 2009 FINEST:
[com.hazelcast.cluster.ClusterManager] false Handling JoinRequest
{nodeType=MEMBER, address=Address[
172.26.0.63:5702], groupName='dev',
groupPassword='dev-pass'}
Thu Jul 09 15:34:57 CEST 2009 FINEST:
[com.hazelcast.cluster.ClusterManager] false Handling JoinRequest
{nodeType=MEMBER, address=Address[
172.26.0.63:5702], groupName='dev',
groupPassword='dev-pass'}
Thu Jul 09 15:34:58 CEST 2009 FINEST:
[com.hazelcast.cluster.ClusterManager] false Handling JoinRequest
{nodeType=MEMBER, address=Address[
172.26.0.63:5702], groupName='dev',
groupPassword='dev-pass'}
Thu Jul 09 15:34:58 CEST 2009 FINEST:
[com.hazelcast.cluster.ClusterManager] false Handling JoinRequest
{nodeType=MEMBER, address=Address[
172.26.0.63:5702], groupName='dev',
groupPassword='dev-pass'}
Thu Jul 09 15:34:59 CEST 2009 FINEST:
[com.hazelcast.cluster.ClusterManager] false Handling JoinRequest
{nodeType=MEMBER, address=Address[
172.26.0.63:5702], groupName='dev',
groupPassword='dev-pass'}
Thu Jul 09 15:34:59 CEST 2009 FINEST:
[com.hazelcast.cluster.ClusterManager] false Handling JoinRequest
{nodeType=MEMBER, address=Address[
172.26.0.63:5702], groupName='dev',
groupPassword='dev-pass'}
Thu Jul 09 15:35:00 CEST 2009 FINEST:
[com.hazelcast.cluster.ClusterManager] false Handling JoinRequest
{nodeType=MEMBER, address=Address[
172.26.0.63:5702], groupName='dev',
groupPassword='dev-pass'}
Thu Jul 09 15:35:00 CEST 2009 FINEST:
[com.hazelcast.cluster.ClusterManager] false Handling JoinRequest
{nodeType=MEMBER, address=Address[
172.26.0.63:5702], groupName='dev',
groupPassword='dev-pass'}
Thu Jul 09 15:35:00 CEST 2009 FINEST:
[com.hazelcast.impl.ThreadContext] hz.pool-1-thread-2 is starting with
cacheSize 12
Thu Jul 09 15:35:00 CEST 2009 FINEST:
[com.hazelcast.cluster.ClusterManager] MEMBERS UPDATE!!
Thu Jul 09 15:35:00 CEST 2009 FINEST:
[com.hazelcast.cluster.ClusterManager] ClusterManager adding Member
[
172.26.0.63:5701] this *
Thu Jul 09 15:35:00 CEST 2009 FINEST:
[com.hazelcast.cluster.ClusterManager] ClusterManager adding Member
[
172.26.0.63:5702]
Thu Jul 09 15:35:00 CEST 2009 INFO:
[com.hazelcast.cluster.ClusterManager]
Members [2] {
Member [
172.26.0.63:5701] this *
Member [
172.26.0.63:5702]
}
Thu Jul 09 15:35:00 CEST 2009 FINEST: [com.hazelcast.impl.BaseManager]
Backup changed!!! so backing up to Member [
172.26.0.63:5702]
Thu Jul 09 15:35:00 CEST 2009 FINEST: [com.hazelcast.impl.BaseManager]
Backup changed!!! so backing up to Member [
172.26.0.63:5702]
Thu Jul 09 15:35:00 CEST 2009 FINEST: [com.hazelcast.impl.BaseManager]
Backup changed!!! so backing up to Member [
172.26.0.63:5702]
Thu Jul 09 15:35:00 CEST 2009 FINEST: [com.hazelcast.impl.BaseManager]
Backup changed!!! so backing up to Member [
172.26.0.63:5702]
Thu Jul 09 15:35:00 CEST 2009 FINEST: [com.hazelcast.impl.BaseManager]
Backup changed!!! so backing up to Member [
172.26.0.63:5702]
Thu Jul 09 15:35:00 CEST 2009 FINEST: [com.hazelcast.impl.BaseManager]
Backup changed!!! so backing up to Member [
172.26.0.63:5702]
Thu Jul 09 15:35:00 CEST 2009 FINEST: [com.hazelcast.impl.BaseManager]
Backup changed!!! so backing up to Member [
172.26.0.63:5702]
Thu Jul 09 15:35:00 CEST 2009 FINEST: [com.hazelcast.impl.BaseManager]
Backup changed!!! so backing up to Member [
172.26.0.63:5702]
Thu Jul 09 15:35:00 CEST 2009 FINEST: [com.hazelcast.impl.BaseManager]
Backup changed!!! so backing up to Member [
172.26.0.63:5702]
Thu Jul 09 15:35:00 CEST 2009 FINEST: [com.hazelcast.impl.BaseManager]
Backup changed!!! so backing up to Member [
172.26.0.63:5702]
Thu Jul 09 15:35:00 CEST 2009 FINEST:
[com.hazelcast.impl.ConcurrentMapManager] Migration Complete from
Address[
172.26.0.63:5701]
Thu Jul 09 15:35:00 CEST 2009 FINEST:
[com.hazelcast.impl.ConcurrentMapManager$33$1] Migration ended!
Thu Jul 09 15:35:00 CEST 2009 FINEST:
[com.hazelcast.impl.ConcurrentMapManager] Migration Complete from
Address[
172.26.0.63:5702]
Thu Jul 09 15:38:43 CEST 2009 FINEST:
[com.hazelcast.cluster.ClusterManager] NO HEARTBEAT should remove
Address[
172.26.0.63:5702]
Thu Jul 09 15:38:43 CEST 2009 FINEST: [com.hazelcast.impl.BaseManager]
Removing Address Address[
172.26.0.63:5702]
Thu Jul 09 15:38:43 CEST 2009 INFO: [com.hazelcast.nio.SelectorBase]
Selector exception at hz.InThread, cause=
java.nio.channels.CancelledKeyException
Thu Jul 09 15:38:43 CEST 2009 FINEST:
[com.hazelcast.cluster.ClusterManager] removing Address
[
172.26.0.63:5702]
Thu Jul 09 15:38:43 CEST 2009 FINEST: [com.hazelcast.impl.BaseManager]
Address[
172.26.0.63:5702] is dead and its backup was Member
[
172.26.0.63:5701] this *
java.nio.channels.CancelledKeyException
at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:55)
at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:59)
at com.hazelcast.nio.SelectorBase.run(SelectorBase.java:116)
at java.lang.Thread.run(Thread.java:619)
Thu Jul 09 15:38:43 CEST 2009 INFO:
[com.hazelcast.cluster.ClusterManager]
Members [1] {
Member [
172.26.0.63:5701] this *
}
Thu Jul 09 15:38:43 CEST 2009 FINEST: [com.hazelcast.impl.BaseManager]
Removing Address Address[
172.26.0.63:5702]
Thu Jul 09 15:38:43 CEST 2009 FINEST: [com.hazelcast.impl.BaseManager]
Address[
172.26.0.63:5702] is dead and its backup was null
Thu Jul 09 15:38:43 CEST 2009 INFO:
[com.hazelcast.cluster.ClusterManager]
Members [1] {
Member [
172.26.0.63:5701] this *
}
Thu Jul 09 15:38:43 CEST 2009 FINEST:
[com.hazelcast.impl.ConcurrentMapManager] Migration Complete from
Address[
172.26.0.63:5701]
Thu Jul 09 15:38:43 CEST 2009 FINEST:
[com.hazelcast.impl.ConcurrentMapManager$33$1] Migration ended!
Thu Jul 09 15:38:43 CEST 2009 FINEST:
[com.hazelcast.impl.ConcurrentMapManager] Migration Complete from
Address[
172.26.0.63:5701]
Thu Jul 09 15:38:43 CEST 2009 FINEST:
[com.hazelcast.impl.ConcurrentMapManager$33$1] Migration ended!
> ...
>
> read more »