REFUSE_SERVER_UNAVAILABLE

[Haiway]

When I run the sandbox demo,it did not work,and the logs:
ERROR o.k.k.c.c.i.c.DefaultOperationTcpChannel - Connection for channel [default_operation_tcp_channel] was rejected: REFUSE_SERVER_UNAVAILABLE

 please help me ,thanks

[Viktor Batytskyi]

Haiway,

This message is logged when Java client receives REFUSE_SERVER_UNAVAILABLE response code from operations server's CONNACK message (in response to client's CONNECT). Operations server sends this code when an error during client's request processing occurs. So to help us investigate the reason why you had this message on client please provide us with trace level logs of Operations server.

Thanks,
Viktor.

[jinu paul]

Hi Viktor,

Seems i have the same issue as explained above. Please send your thoughts across how we can solve this.

Your help is greatly appreciated.

Now my event listener log says like this.

Sat Oct 15 2016 12:19:54 PM [pool-6-thread-2] INFO  o.k.k.c.c.i.DefaultOperationDataProcessor - Created Sync request: {"requestId": 19658, "syncReﾢ￁ﾈ#ﾱﾑ﾿￡'"}, "profileHash": {"bytes": "/c￷ￇﾎDﾎￇwￄﾬ@ﾼﾚﾨ.a9"}, "timeout": 60000}, "bootstrapSyncRequest": null, "profileSyncRequest": null, "configurationSyncRequest": {"configurationHash": {"bytes": "JL￢￶#ﾱ%ﾕﾍﾖXﾈﾲﾪ;ﾰAN*"}, "resyncOnly": true}, "notificationSyncRequest": {"topicListHash": 251, "topicStates": [{"topicId": 220, "seqNumber": 7}], "acceptedUnicastNotifications": null, "subscriptionCommands": []}, "userSyncRequest": {"userAttachRequest": {"userVerifierId": "41465538310519253645", "userExternalId": "CIOuserverifier", "userAccessToken": "41465538310519253645"}, "endpointAttachRequests": [], "endpointDetachRequests": []}, "eventSyncRequest": {"eventSequenceNumberRequest": {}, "eventListenersRequests": null, "events": null}, "logSyncRequest": {"requestId": 0, "logEntries": null}, "extensionSyncRequests": null}

Sat Oct 15 2016 12:19:54 PM [pool-6-thread-2] INFO  o.k.k.c.c.i.c.DefaultOperationTcpChannel - Channel [default_operation_tcp_channel] is reading data from stream using [1024] byte buffer

Sat Oct 15 2016 12:19:54 PM [pool-6-thread-2] INFO  o.k.k.c.c.i.c.DefaultOperationTcpChannel - ConnAck (REFUSE_SERVER_UNAVAILABLE) message received for channel [default_operation_tcp_channel]

Sat Oct 15 2016 12:19:54 PM [pool-6-thread-2] ERROR o.k.k.c.c.i.c.DefaultOperationTcpChannel - Connection for channel [default_operation_tcp_channel] was rejected: REFUSE_SERVER_UNAVAILABLE

Sat Oct 15 2016 12:19:54 PM [pool-6-thread-2] INFO  o.k.k.c.c.i.c.DefaultOperationTcpChannel - [default_operation_tcp_channel] has failed

Sat Oct 15 2016 12:19:54 PM [pool-6-thread-2] INFO  o.k.k.c.c.i.c.DefaultOperationTcpChannel - Channel "default_operation_tcp_channel": closing current connection

Sat Oct 15 2016 12:19:54 PM [pool-6-thread-2] INFO  o.k.k.c.c.i.DefaultFailoverManager - Server [OPERATIONS, 161946944] failed

Sat Oct 15 2016 12:19:54 PM [pool-6-thread-2] WARN  o.k.k.c.b.DefaultBootstrapManager - Failed to find server for channel [TransportProtocolId [id=1456013202, version=1]]

Sat Oct 15 2016 12:19:54 PM [pool-6-thread-2] WARN  o.k.k.c.b.DefaultBootstrapManager - Attempt to receive operations server list will be made in 2000 ms, according to failover strategy decision

Sat Oct 15 2016 12:19:54 PM [pool-6-thread-2] INFO  o.k.k.c.c.i.c.DefaultOperationTcpChannel - Read Task is interrupted for channel [default_operation_tcp_channel]

My operation server trace log looks like this.

2016-10-15 12:01:22,739 [EPS-io-dispatcher-7] TRACE o.k.k.s.sync.platform.AvroEncDec - Decoding client sync {"requestId": 18586, "syncRequestMetaDﾢ￁ﾈ#ﾱﾑ﾿￡'"}, "profileHash": {"bytes": "/c￷ￇﾎDﾎￇwￄﾬ@ﾼﾚﾨ.a9"}, "timeout": 60000}, "bootstrapSyncRequest": null, "profileSyncRequest": null, "configurationSyncRequest": {"configurationHash": {"bytes": "JL￢￶#ﾱ%ﾕﾍﾖXﾈﾲﾪ;ﾰAN*"}, "resyncOnly": true}, "notificationSyncRequest": {"topicListHash": 251, "topicStates": [{"topicId": 220, "seqNumber": 7}], "acceptedUnicastNotifications": null, "subscriptionCommands": []}, "userSyncRequest": {"userAttachRequest": {"userVerifierId": "41465538310519253645", "userExternalId": "CIOuserverifier", "userAccessToken": "41465538310519253645"}, "endpointAttachRequests": [], "endpointDetachRequests": []}, "eventSyncRequest": {"eventSequenceNumberRequest": {}, "eventListenersRequests": null, "events": null}, "logSyncRequest": {"requestId": 0, "logEntries": null}, "extensionSyncRequests": null}

2016-10-15 12:01:22,739 [EPS-io-dispatcher-7] TRACE o.k.k.s.sync.platform.AvroEncDec - Decoded client sync ClientSync [requestId=18586, clientSyncMetaData=ClientSyncMetaData [applicationToken=null, sdkToken=0DmQxaMY6JVI0d2fJSh1kCSB80U, endpointPublicKeyHash=java.nio.HeapByteBuffer[pos=0 lim=20 cap=20], profileHash=java.nio.HeapByteBuffer[pos=0 lim=20 cap=20], timeout=60000], bootstrapSync=null, profileSync=null, forceConfigurationSync=false, configurationSync=ConfigurationClientSync [configurationHash=java.nio.HeapByteBuffer[pos=0 lim=20 cap=20], resyncOnly=true], forceNotificationSync=false, notificationSync=NotificationClientSync [topicListHash=251, topicStates=[TopicState [topicId=220, seqNumber=7]], acceptedUnicastNotifications=null, subscriptionCommands=[]], userSync=UserClientSync [userAttachRequest=org.kaaproject.kaa.server.sync.UserAttachRequest@c48c69c1, endpointAttachRequests=[], endpointDetachRequests=[]], eventSync=EventClientSync [seqNumberRequest=true, eventListenersRequests=null, events=null], logSync=LogClientSync [requestId=0]]

2016-10-15 12:01:22,739 [EPS-io-dispatcher-7] DEBUG o.k.k.s.o.s.c.c.ConcurrentCacheService - Fetching result for sdk token: 0DmQxaMY6JVI0d2fJSh1kCSB80U to retrieve application token

2016-10-15 12:01:22,739 [EPS-io-dispatcher-7] DEBUG o.k.k.s.c.d.i.s.HibernateSdkProfileDao - Searching for an SDK profile by token: [0DmQxaMY6JVI0d2fJSh1kCSB80U]

2016-10-15 12:01:22,739 [EPS-io-dispatcher-7] TRACE o.k.k.s.c.d.i.s.HibernateAbstractDao - Searching SdkProfile entity by criterion [token=0DmQxaMY6JVI0d2fJSh1kCSB80U] 

2016-10-15 12:01:22,740 [EPS-io-dispatcher-7] TRACE o.k.k.s.o.s.a.a.i.EncDecActorMessageProcessor - Request processing failed

java.lang.IllegalStateException: Cache Operation Exception

at org.kaaproject.kaa.server.operations.service.cache.concurrent.CacheTemporaryMemorizer.launderThrowable(CacheTemporaryMemorizer.java:106) ~[kaa-node-0.8.0.jar:na]

at org.kaaproject.kaa.server.operations.service.cache.concurrent.CacheTemporaryMemorizer.compute(CacheTemporaryMemorizer.java:79) ~[kaa-node-0.8.0.jar:na]

at org.kaaproject.kaa.server.operations.service.cache.concurrent.ConcurrentCacheService.getAppTokenBySdkToken(ConcurrentCacheService.java:760) ~[kaa-node-0.8.0.jar:na]

at org.kaaproject.kaa.server.operations.service.akka.actors.io.EncDecActorMessageProcessor.getAppToken(EncDecActorMessageProcessor.java:370) ~[kaa-node-0.8.0.jar:na]

at org.kaaproject.kaa.server.operations.service.akka.actors.io.EncDecActorMessageProcessor.addAppTokenToClientSyncMetaData(EncDecActorMessageProcessor.java:362) ~[kaa-node-0.8.0.jar:na]

at org.kaaproject.kaa.server.operations.service.akka.actors.io.EncDecActorMessageProcessor.decodePlatformLevelData(EncDecActorMessageProcessor.java:315) ~[kaa-node-0.8.0.jar:na]

at org.kaaproject.kaa.server.operations.service.akka.actors.io.EncDecActorMessageProcessor.decodeEncryptedRequest(EncDecActorMessageProcessor.java:251) ~[kaa-node-0.8.0.jar:na]

at org.kaaproject.kaa.server.operations.service.akka.actors.io.EncDecActorMessageProcessor.decodeRequest(EncDecActorMessageProcessor.java:238) ~[kaa-node-0.8.0.jar:na]

at org.kaaproject.kaa.server.operations.service.akka.actors.io.EncDecActorMessageProcessor.processSignedRequest(EncDecActorMessageProcessor.java:182) ~[kaa-node-0.8.0.jar:na]

at org.kaaproject.kaa.server.operations.service.akka.actors.io.EncDecActorMessageProcessor.decodeAndForward(EncDecActorMessageProcessor.java:99) ~[kaa-node-0.8.0.jar:na]

at org.kaaproject.kaa.server.operations.service.akka.actors.io.EncDecActor.onReceive(EncDecActor.java:144) [kaa-node-0.8.0.jar:na]

at akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:167) [akka-actor_2.11-2.4.1.jar:na]

at akka.actor.Actor$class.aroundReceive(Actor.scala:480) [akka-actor_2.11-2.4.1.jar:na]

at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:97) [akka-actor_2.11-2.4.1.jar:na]

at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526) [akka-actor_2.11-2.4.1.jar:na]

at akka.actor.ActorCell.invoke(ActorCell.scala:495) [akka-actor_2.11-2.4.1.jar:na]

at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257) [akka-actor_2.11-2.4.1.jar:na]

at akka.dispatch.Mailbox.run(Mailbox.scala:224) [akka-actor_2.11-2.4.1.jar:na]

at akka.dispatch.Mailbox.exec(Mailbox.scala:234) [akka-actor_2.11-2.4.1.jar:na]

at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [scala-library-2.11.7.jar:na]

at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [scala-library-2.11.7.jar:na]

at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [scala-library-2.11.7.jar:na]

at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [scala-library-2.11.7.jar:na]

Caused by: java.util.concurrent.ExecutionException: org.springframework.orm.hibernate4.HibernateSystemException: No Session found for current thread; nested exception is org.hibernate.HibernateException: No Session found for current thread

at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[na:1.8.0_91]

at java.util.concurrent.FutureTask.get(FutureTask.java:192) ~[na:1.8.0_91]

at org.kaaproject.kaa.server.operations.service.cache.concurrent.CacheTemporaryMemorizer.compute(CacheTemporaryMemorizer.java:75) ~[kaa-node-0.8.0.jar:na]

... 21 common frames omitted

Caused by: org.springframework.orm.hibernate4.HibernateSystemException: No Session found for current thread; nested exception is org.hibernate.HibernateException: No Session found for current thread

at org.springframework.orm.hibernate4.SessionFactoryUtils.convertHibernateAccessException(SessionFactoryUtils.java:216) ~[spring-orm-4.0.2.RELEASE.jar:4.0.2.RELEASE]

at org.springframework.orm.hibernate4.HibernateExceptionTranslator.convertHibernateAccessException(HibernateExceptionTranslator.java:57) ~[spring-orm-4.0.2.RELEASE.jar:4.0.2.RELEASE]

at org.springframework.orm.hibernate4.HibernateExceptionTranslator.translateExceptionIfPossible(HibernateExceptionTranslator.java:44) ~[spring-orm-4.0.2.RELEASE.jar:4.0.2.RELEASE]

at org.springframework.dao.support.ChainedPersistenceExceptionTranslator.translateExceptionIfPossible(ChainedPersistenceExceptionTranslator.java:59) ~[spring-tx-4.0.2.RELEASE.jar:4.0.2.RELEASE]

at org.springframework.dao.support.DataAccessUtils.translateIfNecessary(DataAccessUtils.java:213) ~[spring-tx-4.0.2.RELEASE.jar:4.0.2.RELEASE]

at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:147) ~[spring-tx-4.0.2.RELEASE.jar:4.0.2.RELEASE]

at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.0.2.RELEASE.jar:4.0.2.RELEASE]

at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207) ~[spring-aop-4.0.2.RELEASE.jar:4.0.2.RELEASE]

at com.sun.proxy.$Proxy91.findSdkProfileByToken..

It is a cluster setup. I have checked and all services are running on all nodes.

again, any help will be greatly appreciated.

Thanks

JInu

[Sivas Sankar]

I too have the same problem with my application some one please help me

My log shows:

[pool-6-thread-1] INFO org.kaaproject.kaa.client.channel.impl.channels.DefaultOperationTcpChannel - Channel [default_operation_tcp_channel]: opening connection to server IPTransportInfo [host=localhost, port=9997]

[pool-6-thread-1] INFO org.kaaproject.kaa.client.logging.AbstractLogCollector - Adding following bucket id [0] for timeout tracking

[pool-6-thread-1] INFO org.kaaproject.kaa.client.channel.impl.DefaultOperationDataProcessor - Created Sync request: {"requestId": 1, "syncRequestMetaData": {"sdkToken": "xsiLo8jbhEj9cXa0T62edF3TaK8", "endpointPublicKeyHash": {"bytes": "ﾬﾕ,Iﾞ ￛLￊﾒu�ﾠﾬￆtﾼ￴￪"}, "profileHash": {"bytes": ""}, "timeout": 60000}, "bootstrapSyncRequest": null, "profileSyncRequest": {"endpointPublicKey": {"bytes": "0ﾂ "0

*ﾆHﾆ￷

"resyncOnly": true}, "notificationSyncRequest": {"topicListHash": 0, "topicStates": null, "acceptedUnicastNotifications": null, "subscriptionCommands": []}, "userSyncRequest": {"userAttachRequest": null, "endpointAttachRequests": [], "endpointDetachRequests": []}, "eventSyncRequest": {"eventSequenceNumberRequest": {}, "eventListenersRequests": null, "events": null}, "logSyncRequest": {"requestId": 0, "logEntries": [{"data": {"bytes": "

[pool-6-thread-1] INFO org.kaaproject.kaa.client.channel.impl.channels.DefaultOperationTcpChannel - ConnAck (REFUSE_SERVER_UNAVAILABLE) message received for channel [default_operation_tcp_channel]

[pool-6-thread-1] ERROR org.kaaproject.kaa.client.channel.impl.channels.DefaultOperationTcpChannel - Connection for channel [default_operation_tcp_channel] was rejected: REFUSE_SERVER_UNAVAILABLE

[pool-6-thread-1] INFO org.kaaproject.kaa.client.channel.impl.channels.DefaultOperationTcpChannel - Cleaning client state

[pool-6-thread-1] INFO org.kaaproject.kaa.client.channel.impl.channels.DefaultOperationTcpChannel - [default_operation_tcp_channel] has failed

[pool-6-thread-1] INFO org.kaaproject.kaa.client.channel.impl.channels.DefaultOperationTcpChannel - Channel "default_operation_tcp_channel": closing current connection

[pool-6-thread-1] WARN org.kaaproject.kaa.client.channel.impl.channels.DefaultOperationTcpChannel - Loss of connectivity detected

[pool-6-thread-1] WARN org.kaaproject.kaa.client.channel.impl.channels.DefaultOperationTcpChannel - Attempt to reconnect will be made in 5000 ms according to failover strategy decision

[Oleksandr]

Hi!

JInu, Sivas, could you provide more detailed information? For example, what client application you use, full client and Kaa servers logs with TRACE log level, describe all steps to reproduce your issue. Please refer to our Troubleshooting guide for details.

Also, starting from October 1st, Kaa crew stopped supporting google forum and shifted totally to http://stackoverflow.com/questions/tagged/kaa, see - https://groups.google.com/forum/#!topic/kaaproject/tBsBmSGGDBg.

Please, repost your question on StackOverflow.

Best regards,

Oleksandr

On Tuesday, July 21, 2015 at 10:33:55 AM UTC+3, Haiway wrote: