kylo can't connect to nifi cluster

[zhijia lin]

hi,

i have setup a nifi cluster with 3 instances in 3 hosts according the "nifi cluster" guideline and adding  the KyloProvenanceEventReportingTask and MetadataProviderSelectorService  by http://kylo.readthedocs.io/en/latest/how-to-guides/NiFiKyloProvenanceReportingTask.html,  however, seems kylo can't connect to nifi

kylo-service.log

2017-06-10 00:17:48 DEBUG main:ReactorMetadataEventService:83 - Adding event listener: com.thinkbiganalytics.metadata.event.jms.JmsChangeEventDispatcher$PreconditionTriggerDispatcher@39d85aaa

2017-06-10 00:17:49 ERROR pool-5-thread-1:JerseyRestClient:355 - Failed to process request /flow/about

javax.ws.rs.ProcessingException: java.net.ConnectException: Connection refused

        at org.glassfish.jersey.client.internal.HttpUrlConnector.apply(HttpUrlConnector.java:287)

        at org.glassfish.jersey.client.ClientRuntime.invoke(ClientRuntime.java:255)

        at org.glassfish.jersey.client.JerseyInvocation$2.call(JerseyInvocation.java:700)

        at org.glassfish.jersey.internal.Errors.process(Errors.java:315)

        at org.glassfish.jersey.internal.Errors.process(Errors.java:297)

        at org.glassfish.jersey.internal.Errors.process(Errors.java:228)

        at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:444)

        at org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:696)

        at org.glassfish.jersey.client.JerseyInvocation$Builder.method(JerseyInvocation.java:420)

        at org.glassfish.jersey.client.JerseyInvocation$Builder.get(JerseyInvocation.java:316)

        at com.thinkbiganalytics.rest.JerseyRestClient.get(JerseyRestClient.java:349)

        at com.thinkbiganalytics.rest.JerseyRestClient.get(JerseyRestClient.java:331)

        at com.thinkbiganalytics.nifi.v1.rest.client.NiFiRestClientV1.about(NiFiRestClientV1.java:120)

        at com.thinkbiganalytics.feedmgr.nifi.NifiConnectionService.isConnected(NifiConnectionService.java:101)

        at com.thinkbiganalytics.feedmgr.nifi.NifiConnectionService.isConnected(NifiConnectionService.java:95)

        at com.thinkbiganalytics.feedmgr.nifi.NifiConnectionService.checkConnection(NifiConnectionService.java:74)

        at com.thinkbiganalytics.feedmgr.nifi.NifiConnectionService.lambda$initConnectionTimer$2(NifiConnectionService.java:117)

        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)

        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)

        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

        at java.lang.Thread.run(Thread.java:745)

Caused by: java.net.ConnectException: Connection refused

in the 3 nifi instance, the nifi-app.log showed:

2017-06-10 00:26:30,468 INFO [Timer-Driven Process Thread-5] c.t.n.p.r.KyloProvenanceEventReportingTask KyloProvenanceEventReportingTask[id=8ca1d837-015c-1000-ffff-ffffe0d735ef] Reporting Task Aborted.  Last Event Recorded was: -1 

2017-06-10 00:26:34,283 INFO [Clustering Tasks Thread-1] org.apache.nifi.cluster.heartbeat Heartbeat created at 2017-06-10 00:26:34,279 and sent to 10.3.11.11:8078 at 2017-06-10 00:26:34,283; send took 4 millis

2017-06-10 00:26:34,491 INFO [Timer-Driven Process Thread-5] c.t.n.p.r.KyloProvenanceEventReportingTask KyloProvenanceEventReportingTask[id=8ca1d837-015c-1000-ffff-ffffe0d735ef] Kylo is not available to process requests yet.  This task will exit and wait for its next schedule interval.

2017-06-10 00:26:34,491 INFO [Timer-Driven Process Thread-5] c.t.n.p.r.KyloProvenanceEventReportingTask KyloProvenanceEventReportingTask[id=8ca1d837-015c-1000-ffff-ffffe0d735ef] Reporting Task Aborted.  Last Event Recorded was: -1 

2017-06-10 00:26:38,511 INFO [Timer-Driven Process Thread-5] c.t.n.p.r.KyloProvenanceEventReportingTask KyloProvenanceEventReportingTask[id=8ca1d837-015c-1000-ffff-ffffe0d735ef] Kylo is not available to process requests yet.  This task will exit and wait for its next schedule interval.

2017-06-10 00:26:38,511 INFO [Timer-Driven Process Thread-5] c.t.n.p.r.KyloProvenanceEventReportingTask KyloProvenanceEventReportingTask[id=8ca1d837-015c-1000-ffff-ffffe0d735ef] Reporting Task Aborted.  Last Event Recorded was: -1 

kylo-service application.property:

nifi.rest.host=localhost

nifi.rest.port=8079

nifi.service.kylo_metadata_service.rest_client_url=http://10.3.11.10:8400/proxy/v1/metadata

kylo and one of the nifi instance are installed in 10.3.11.10 (localhost)

any hints to resolve it?

thanks

[zhijia lin]

it was fixed after  changing 'nifi.rest.host=localhost'  to 'nifi.rest.host=10.3.11.10'

would anyone help to explain why? 

[Greg Hart]

Hi Zhijia,

Kylo communicates with NiFi through NiFi's REST API. If NiFi is not listening on localhost then you must change the nifi.rest.host property to the address where NiFi is listening.

[zhijia lin]

nifi.rest.host=localhost works when there is only one nifi instance installed along with kylo

while there are 3 nifi instances, as one of the 3 nifi instances and kylo are in the same server, so i am wondering nifi.rest.host=localhost still works, but seems no.