debugging JDBC connector

[Imran Akbar]

I'm trying to use the Kafka JDBC connector to read from a Teradata warehouse.

I get no errors from the connector, and my consumer of that topic sees no messages.

How can I debug what's going wrong?  Can I increase the log level or something?

thanks,

imran

[Gwen Shapira]

Yes, increasing the log level should be the first step.

It is usually in a file called: config/connect-log4j.properties

The default is INFO, if you set it all the way to TRACE you'll see
tons of information and hopefully enough to troubleshoot the
connector.

Let us know what log messages turned out to be important and we can
increase them to INFO or maybe higher :)

Gwen

> --
> You received this message because you are subscribed to the Google Groups
> "Confluent Platform" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to confluent-platf...@googlegroups.com.
> To post to this group, send email to confluent...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/confluent-platform/a070a975-51f5-4e96-bd02-618a60b75ee8%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--
Gwen Shapira
Product Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog

[Imran Akbar]

thanks Gwen, that worked great.

I can see that the connector is establishing a connection to the database, and pulling a list of available tables.

I then see a bunch of messages like this:

[2016-08-16 19:51:03,159] TRACE Sending record ProducerRecord(topic=test-tdiaViewDaily, partition=null, key=null, value=[B@7b91bf92, timestamp=null) with callback org.apache.kafka.connect.runtime.WorkerSourceTask$1@58441d3 to topic test-tdiaViewDaily partition 0 (org.apache.kafka.clients.producer.KafkaProducer:464)

[2016-08-16 19:51:03,159] TRACE Appending record with key null, value org.apache.kafka.connect.data.Struct@b89d3447 (org.apache.kafka.connect.runtime.WorkerSourceTask:185)

and like this:

[2016-08-16 19:50:47,124] TRACE Wrote record successfully: topic test-tdiaViewDaily partition 0 offset 6981928

Would I see actual rows of data if it were picking those up properly?

regards,

imran

On Tuesday, August 16, 2016 at 10:40:44 AM UTC-7, Gwen Shapira wrote:

Yes, increasing the log level should be the first step.

It is usually in a file called: config/connect-log4j.properties

The default is INFO, if you set it all the way to TRACE you'll see
tons of information and hopefully enough to troubleshoot the
connector.

Let us know what log messages turned out to be important and we can
increase them to INFO or maybe higher :)

Gwen

On Tue, Aug 16, 2016 at 10:08 AM, Imran Akbar <skun...@gmail.com> wrote:
> I'm trying to use the Kafka JDBC connector to read from a Teradata
> warehouse.
> I get no errors from the connector, and my consumer of that topic sees no
> messages.
>
> How can I debug what's going wrong?  Can I increase the log level or
> something?
>
> thanks,
> imran
>
> --
> You received this message because you are subscribed to the Google Groups
> "Confluent Platform" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to confluent-platform+unsub...@googlegroups.com.

[Imran Akbar]

Finally figured out the issue…

The topic I was supposed to be subscribed to is not just the prefix, but the prefix + the table name!

As soon as I did that, the consumer pulled in all the rows just fine.

PLEASE make this explicit on the page here: http://docs.confluent.io/2.0.0/connect/connect-jdbc/docs/index.html

thanks,

imran

[Gwen Shapira]

I'm glad you figured it out, and we'll fix our docs :)

> email to confluent-platform+unsubscribe@googlegroups.com.

> To post to this group, send email to confluent...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/confluent-platform/a070a975-51f5-4e96-bd02-618a60b75ee8%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--
Gwen Shapira
Product Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.

To post to this group, send email to confluent-platform@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/bb55e454-47b1-4752-9595-90aed7b89800%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.