HDFS connect

57 views

Skip to first unread message

Shannon Ma

unread,

Oct 20, 2016, 5:47:18 PM10/20/16

to Confluent Platform

Hi,

I have a kafka stream and result sends to hdfs, it was working for a while, then i got this, the hdfs side looks ok, not sure how i can fix this, i tried to clear all kakfa under /tmp and restart everything, but this still happens, and looks like i cannot recover.

17:38| ERROR | WorkerSinkTask.java 401 | Task hdfs-traffic-sink-T-0 threw an uncaught and unrecoverable exception

java.lang.NullPointerException

at io.confluent.connect.hdfs.DataWriter.write(DataWriter.java:234)

at io.confluent.connect.hdfs.HdfsSinkTask.put(HdfsSinkTask.java:91)

at org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:381)

at org.apache.kafka.connect.runtime.WorkerSinkTask.poll(WorkerSinkTask.java:227)

at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:170)

at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:142)

at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:140)

at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:175)

at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

at java.util.concurrent.FutureTask.run(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

at java.lang.Thread.run(Unknown Source)

17:38| ERROR | WorkerSinkTask.java 402 | Task is being killed and will not recover until manually restarted

17:38| ERROR | WorkerTask.java 142 | Task hdfs-traffic-sink-T-0 threw an uncaught and unrecoverable exception

org.apache.kafka.connect.errors.ConnectException: Exiting WorkerSinkTask due to unrecoverable exception.

at org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:403)