Kafka connect dies with custom query and whitelist setting

[Steven Lowenthal]

I added a custom query to my job (distributed mode), but accidentally left the whitelist setting.  This causes kafka-connect to die, with no way to delete the bad job.

[2016-06-02 17:59:47,506] INFO JdbcSourceConnectorConfig values:

        mode = timestamp+incrementing

        timestamp.column.name = insert_load_datetime

        incrementing.column.name = id

        topic.prefix = ongo_

        poll.interval.ms = 5000

        query = select * from replication where insert_load_datetime < CURRENT_TIMESTAMP AND (insert_load_datetime, id > ? ) ORDER BY insert_load_datetime, id ASC LIMIT 1000

        batch.max.rows = 100

        connection.url = jdbc:postgresql://gn5456ndsrepl.cbv0gfpvcvqy.us-west-2.rds.amazonaws.com:5432/postgres?user=gn5456ndsrepl&password=gn5456ndsrepl&currentSchema=nds_replicate&defaultRowFetchSize=200

        table.blacklist = []

        table.poll.interval.ms = 7500

        table.whitelist = [replication]

 (io.confluent.connect.jdbc.JdbcSourceConnectorConfig)

[2016-06-02 17:59:47,598] ERROR Uncaught exception in herder work thread, exiting:  (org.apache.kafka.connect.runtime.distributed.DistributedHerder)

org.apache.kafka.connect.errors.ConnectException: Connector threw an exception while starting

        at org.apache.kafka.connect.runtime.Worker.addConnector(Worker.java:188)

        at org.apache.kafka.connect.runtime.distributed.DistributedHerder.startConnector(DistributedHerder.java:670)

        at org.apache.kafka.connect.runtime.distributed.DistributedHerder.startWork(DistributedHerder.java:642)

        at org.apache.kafka.connect.runtime.distributed.DistributedHerder.handleRebalanceCompleted(DistributedHerder.java:600)

        at org.apache.kafka.connect.runtime.distributed.DistributedHerder.tick(DistributedHerder.java:184)

        at org.apache.kafka.connect.runtime.distributed.DistributedHerder.run(DistributedHerder.java:159)

        at java.lang.Thread.run(Thread.java:745)

Caused by: org.apache.kafka.connect.errors.ConnectException: query may not be combined with whole-table copying settings.

        at io.confluent.connect.jdbc.JdbcSourceConnector.start(JdbcSourceConnector.java:92)

        at org.apache.kafka.connect.runtime.Worker.addConnector(Worker.java:186)

        ... 6 more

[2016-06-02 17:59:47,599] INFO Kafka Connect stopping (org.apache.kafka.connect.runtime.Connect)

[2016-06-02 17:59:47,614] INFO Stopped ServerConnector@18d90fc4{HTTP/1.1}{0.0.0.0:8083} (org.eclipse.jetty.server.ServerConnector)

[2016-06-02 17:59:47,621] INFO Stopped o.e.j.s.ServletContextHandler@106faf11{/,null,UNAVAILABLE} (org.eclipse.jetty.server.handler.ContextHandler)

[Liquan Pei]

Hi 

Thanks for reporting this, may I ask which version of CP that you are using? 

Liquan

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platf...@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/17a17fdf-8ad9-4834-a99a-c8bdb2b6251c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Liquan Pei | Software Engineer | Confluent | +1 413.230.6855

Download Apache Kafka and Confluent Platform: www.confluent.io/download

[Steven Lowenthal]

We're Using 2.0.1

[George @paytm.com]

Sounds like the issue we are experiencing

[Steven Lowenthal]

I wound up deleting the topic connect-config (I still use the default name) to get rid of this.