how to kill old tasks

37 views
Skip to first unread message

Shannon Ma

unread,
Oct 6, 2016, 1:18:18 PM10/6/16
to Confluent Platform
Hi

I am having this issue, and not sure what to do,

i am doing oracle (connect) -> kafka -> transform -> kafka -> opentsdb (connect), 

and i cannot get the opentsdb connect work, so i am trying now

oracle (connect) -> kafka -> process  (calling rest api on opentsdb)


i am doing all these in the code (connect + stream), when i am running the new code



13:02| INFO | WorkerSinkTask.java 261 | WorkerSinkTask{id=tsdb-sink-0} Committing offsets
13:02| DEBUG | ConsumerCoordinator.java 544 | Group connect-tsdb-sink committed offset 0 for partition METRIC_DATA-0
13:02| DEBUG | WorkerSinkTask.java 183 | Finished WorkerSinkTask{id=tsdb-sink-0} offset commit successfully in 16 ms
13:02| ERROR | WorkerTask.java 142 | Task tsdb-sink-0 threw an uncaught and unrecoverable exception
org.apache.kafka.connect.errors.DataException: Failed to deserialize data to Avro:
        at io.confluent.connect.avro.AvroConverter.toConnectData(AvroConverter.java:109)
        at org.apache.kafka.connect.runtime.WorkerSinkTask.convertMessages(WorkerSinkTask.java:357)
        at org.apache.kafka.connect.runtime.WorkerSinkTask.poll(WorkerSinkTask.java:226)
        at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:170)
        at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:142)
        at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:140)
        at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:175)
        at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
        at java.util.concurrent.FutureTask.run(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)




tsdb-sink-0, i dont have tsdb-sink anymore, but somehow it is still there,


any idea?  i even renamed the stream name, a different topic, still the same,


Thanks
Shannon

Gwen Shapira

unread,
Oct 6, 2016, 10:17:24 PM10/6/16
to confluent...@googlegroups.com
I'm assuming you are running connect in distributed mode? In this
case, you can use a REST API to stop a connector:
http://docs.confluent.io/3.0.1/connect/userguide.html#connectors

The REST URI you need is probably:
PUT /connectors/(string:name)/pause
or
DELETE /connectors/(string:name)/
> --
> You received this message because you are subscribed to the Google Groups
> "Confluent Platform" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to confluent-platf...@googlegroups.com.
> To post to this group, send email to confluent...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/confluent-platform/e12ba27b-f46f-4c6c-9875-852a427dc7bf%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--
Gwen Shapira
Product Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog

Shannon Ma

unread,
Oct 7, 2016, 10:50:55 AM10/7/16
to Confluent Platform
Right now i am just testing on a standalone, i ended up with removing the kafka logs and zookeeper data, will try the api next time.
Reply all
Reply to author
Forward
0 new messages