CDAP setup on Kubernetes Error in Preview

75 views
Skip to first unread message

Khurram Noor

unread,
Oct 26, 2022, 5:55:56 AM10/26/22
to CDAP User
Hi Team 


I am trying to deploy cdap on Kubernetes but unable to successfully run the preview feature of CDAP,  i am attaching log of preview pod and app_fabric pod , noticed some errors in it, 
it would be nice if someone help to resolve the issue, 

Pods details 

NAME                                                              READY          STATUS    RESTARTS   AGE
cdap-cdap-appfabric-0                                             1/1     Running               0          141m
cdap-cdap-logs-0                                                  1/1          Running              0          141m
cdap-cdap-messaging-0                                       1/1         Running            0          141m
cdap-cdap-metadata-8656c676d7-572s8             1/1     Running            0          141m
cdap-cdap-metrics-0                                               1/1        Running           0          141m
cdap-cdap-preview-0                                               1/1        Running            0          141m
cdap-cdap-router-7d6fc94c95-ts5qd                   1/1         Running           0          141m
cdap-cdap-runtime-0                                               1/1         Running           0          141m
cdap-cdap-service-system-dataprep-service-60135721-54fe-11kh57v   1/1     Running   0                                                                                                                                            139m
cdap-cdap-service-system-pipeline-studio-6029c552-54fe-11ekwwrt   1/1     Running   0          139m
cdap-cdap-spark-default-test-datastreamssparkstreaming-21dmsbjj   1/1     Running   0          33s
cdap-cdap-userinterface-764b7886b-zvtx2       1/1     Running   0          141m
es-cluster-es-default-0                                           1/1     Running   0          3h1m
hadoop                                                                      1/1     Running   0          167m
postgres-postgresql-0                                             1/1     Running   0          3h5m


i have tried with the flag in config 
    preview.runner.artifact.localizer.enabled: "true" but with no effect , 







Regards
Khurram Shahzad
preview_pod_log.txt
app_fabric_pod_log.txt

Khurram Noor

unread,
Nov 3, 2022, 3:41:32 AM11/3/22
to CDAP User
Hi Team 
In Preview POD i am getting the following error 
2022-11-03 07:26:11,981 - DEBUG [PreviewRunnerService:i.c.c.i.a.w.s.ArtifactLocalizerClient@98] - Sending request to http://localhost:11021/v3Internal/worker/artifact/namespaces/system/artifacts/cdap-data-pipeline/versions/6.9.0-SNAPSHOT?unpack=false
2022-11-03 07:26:11,982 - DEBUG [PreviewRunnerService:i.c.c.i.a.p.DefaultPreviewRunner@296] - Setting preview status for program:default.ca10e071-5b48-11ed-8ac1-be1edaf9bb0a.-SNAPSHOT.workflow.DataPipelineWorkflow to DEPLOY_FAILED
2022-11-03 07:26:11,989 - ERROR [PreviewRunnerService:i.c.c.i.a.p.PreviewRunnerService@123] - Failed to execute preview
java.net.ConnectException: Connection refused (Connection refused)
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:607)
        at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
        at sun.net.www.http.HttpClient.<init>(HttpClient.java:242)
        at sun.net.www.http.HttpClient.New(HttpClient.java:339)
        at sun.net.www.http.HttpClient.New(HttpClient.java:357)
        at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1228)
        at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1162)
        at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1056)
        at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:990)
        at io.cdap.common.http.HttpRequests.execute(HttpRequests.java:65)
        at io.cdap.common.http.HttpRequests.execute(HttpRequests.java:126)
        at io.cdap.cdap.internal.app.worker.sidecar.ArtifactLocalizerClient.sendRequest(ArtifactLocalizerClient.java:103)
        at io.cdap.cdap.internal.app.worker.sidecar.ArtifactLocalizerClient.getArtifactLocation(ArtifactLocalizerClient.java:71)
        at io.cdap.cdap.internal.app.runtime.artifact.RemoteArtifactRepositoryReaderWithLocalization.getArtifactLocation(RemoteArtifactRepositoryReaderWithLocalization.java:50)
        at io.cdap.cdap.internal.app.runtime.artifact.RemoteArtifactRepositoryReader.getArtifactDetails(RemoteArtifactRepositoryReader.java:166)
        at io.cdap.cdap.internal.app.runtime.artifact.RemoteArtifactRepository.getArtifactDetails(RemoteArtifactRepository.java:202)
        at io.cdap.cdap.internal.app.services.ApplicationLifecycleService.deployApp(ApplicationLifecycleService.java:864)
        at io.cdap.cdap.internal.app.preview.DefaultPreviewRunner.startPreview(DefaultPreviewRunner.java:190)
        at io.cdap.cdap.internal.app.preview.PreviewRunnerService.run(PreviewRunnerService.java:105)
        at com.google.common.util.concurrent.AbstractExecutionThreadService$1$1.run(AbstractExecutionThreadService.java:52)
        at java.lang.Thread.run(Thread.java:750)
2022-11-03 07:26:13,169 - WARN  [PreviewDataSubscriberService:i.c.c.c.s.AbstractRetryableScheduledService@207] - Failed to execute task for scheduled service PreviewDataSubscriberService
java.lang.NullPointerException: null
        at io.cdap.cdap.internal.app.preview.PreviewDataSubscriberService.emitRunTimeMetric(PreviewDataSubscriberService.java:186)
        at io.cdap.cdap.internal.app.preview.PreviewDataSubscriberService.access$300(PreviewDataSubscriberService.java:60)
        at io.cdap.cdap.internal.app.preview.PreviewDataSubscriberService$PreviewStatusWriter.processMessage(PreviewDataSubscriberService.java:245)
        at io.cdap.cdap.internal.app.preview.PreviewDataSubscriberService.processMessages(PreviewDataSubscriberService.java:153)
        at io.cdap.cdap.messaging.subscriber.AbstractMessagingSubscriberService.lambda$processMessages$0(AbstractMessagingSubscriberService.java:146)
        at io.cdap.cdap.spi.data.transaction.TransactionRunners.lambda$run$1(TransactionRunners.java:165)
        at io.cdap.cdap.spi.data.nosql.NoSqlTransactionRunner.lambda$run$0(NoSqlTransactionRunner.java:67)
        at io.cdap.cdap.spi.data.nosql.dataset.NoSQLTransactionals$1.finishExecute(NoSQLTransactionals.java:71)
        at io.cdap.cdap.spi.data.nosql.dataset.NoSQLTransactionals$1.execute(NoSQLTransactionals.java:50)
        at io.cdap.cdap.data2.transaction.Transactions$2.executeInternal(Transactions.java:261)
        at io.cdap.cdap.data2.transaction.Transactions$2.execute(Transactions.java:248)
        at io.cdap.cdap.spi.data.nosql.NoSqlTransactionRunner.run(NoSqlTransactionRunner.java:66)
        at io.cdap.cdap.spi.data.transaction.TransactionRunners.run(TransactionRunners.java:165)
        at io.cdap.cdap.messaging.subscriber.AbstractMessagingSubscriberService.processMessages(AbstractMessagingSubscriberService.java:142)
        at io.cdap.cdap.messaging.subscriber.AbstractMessagingPollingService.fetchAndProcessMessages(AbstractMessagingPollingService.java:206)
        at io.cdap.cdap.messaging.subscriber.AbstractMessagingPollingService.runTask(AbstractMessagingPollingService.java:154)
        at io.cdap.cdap.common.service.AbstractRetryableScheduledService.runOneIteration(AbstractRetryableScheduledService.java:162)
        at com.google.common.util.concurrent.AbstractScheduledService$1$1.run(AbstractScheduledService.java:170)
        at com.google.common.util.concurrent.AbstractScheduledService$CustomScheduler$ReschedulableCallable.call(AbstractScheduledService.java:355)
        at com.google.common.util.concurrent.AbstractScheduledService$CustomScheduler$ReschedulableCallable.call(AbstractScheduledService.java:321)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:750)
2022-11-03 07:26:13,170 - WARN  [PreviewDataSubscriberService:i.c.c.m.s.AbstractMessagingPollingService@141] - Failed to get and process notifications. Will retry in next run
java.lang.NullPointerException: null
        at io.cdap.cdap.internal.app.preview.PreviewDataSubscriberService.emitRunTimeMetric(PreviewDataSubscriberService.java:186)
        at io.cdap.cdap.internal.app.preview.PreviewDataSubscriberService.access$300(PreviewDataSubscriberService.java:60)
        at io.cdap.cdap.internal.app.preview.PreviewDataSubscriberService$PreviewStatusWriter.processMessage(PreviewDataSubscriberService.java:245)
        at io.cdap.cdap.internal.app.preview.PreviewDataSubscriberService.processMessages(PreviewDataSubscriberService.java:153)
        at io.cdap.cdap.messaging.subscriber.AbstractMessagingSubscriberService.lambda$processMessages$0(AbstractMessagingSubscriberService.java:146)
        at io.cdap.cdap.spi.data.transaction.TransactionRunners.lambda$run$1(TransactionRunners.java:165)
        at io.cdap.cdap.spi.data.nosql.NoSqlTransactionRunner.lambda$run$0(NoSqlTransactionRunner.java:67)
        at io.cdap.cdap.spi.data.nosql.dataset.NoSQLTransactionals$1.finishExecute(NoSQLTransactionals.java:71)
        at io.cdap.cdap.spi.data.nosql.dataset.NoSQLTransactionals$1.execute(NoSQLTransactionals.java:50)
        at io.cdap.cdap.data2.transaction.Transactions$2.executeInternal(Transactions.java:261)
        at io.cdap.cdap.data2.transaction.Transactions$2.execute(Transactions.java:248)
        at io.cdap.cdap.spi.data.nosql.NoSqlTransactionRunner.run(NoSqlTransactionRunner.java:66)
        at io.cdap.cdap.spi.data.transaction.TransactionRunners.run(TransactionRunners.java:165)
        at io.cdap.cdap.messaging.subscriber.AbstractMessagingSubscriberService.processMessages(AbstractMessagingSubscriberService.java:142)
        at io.cdap.cdap.messaging.subscriber.AbstractMessagingPollingService.fetchAndProcessMessages(AbstractMessagingPollingService.java:206)
        at io.cdap.cdap.messaging.subscriber.AbstractMessagingPollingService.runTask(AbstractMessagingPollingService.java:154)
        at io.cdap.cdap.common.service.AbstractRetryableScheduledService.runOneIteration(AbstractRetryableScheduledService.java:162)
        at com.google.common.util.concurrent.AbstractScheduledService$1$1.run(AbstractScheduledService.java:170)
        at com.google.common.util.concurrent.AbstractScheduledService$CustomScheduler$ReschedulableCallable.call(AbstractScheduledService.java:355)
        at com.google.common.util.concurrent.AbstractScheduledService$CustomScheduler$ReschedulableCallable.call(AbstractScheduledService.java:321)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:750)

look like its trying to access the service on localhost:11021 which is not running in preview pod, 
any idea , i am block due to preview issue on the cdap UI.


Regards

Khurram Shahzad
Reply all
Reply to author
Forward
0 new messages