Kafka Connect JDBC MSSQL In Docker

[Ben Davison]

Hi All,

I'm trying to get Kafka Connect JDBC connector working in Docker, but keep running into:

Error:
2016-01-25 17:23:28,423] ERROR Couldn't open connection to jdbc:sqlserver:/<DB_CONN>:59281;database=<DB>;user=<USER>;password=<PASSWORD>: java.sql.SQLException: No suitable driver found for jdbc:sqlserver://<DB_HOST>:59281;database=<DB>;user=<USER>;password=<PASSWORD> (io.confluent.connect.jdbc.JdbcSourceConnector:76)

(Anything in <> is a placeholder)

I have got this running locally, but am having trouble when putting into docker.

(Please note we upgraded the confluent images to kafka 0.9)

Dockerfile:

# Builds a docker image for the Kafka Connect.

FROM blah/data/platform

ENV CONFLUENT_USER confluent
ENV CONFLUENT_GROUP confluent

ADD sqljdbc_6.0 /tmp/
COPY kafka-connect.sh /usr/local/bin/

COPY sqlserver-partner-data.properties /etc/kafka-connect-jdbc/
COPY connect-standalone.properties /etc/kafka-connect-jdbc/

COPY connect-standalone /usr/bin/

RUN groupadd -r ${CONFLUENT_GROUP} &&\
    useradd -r -g ${CONFLUENT_GROUP} ${CONFLUENT_USER} &&\
    chown -R ${CONFLUENT_USER}:${CONFLUENT_GROUP} /etc/kafka-connect-jdbc /usr/local/bin/kafka-connect.sh &&\
    chown -R ${CONFLUENT_USER}:${CONFLUENT_GROUP} /usr/bin/connect-standalone &&\
    chown -R ${CONFLUENT_USER}:${CONFLUENT_GROUP} /tmp/enu/ &&\
    chmod +x /usr/local/bin/kafka-connect.sh &&\
    chmod 777 /usr/bin/connect-standalone &&\
    chmod 777 -R /tmp/enu/*

USER ${CONFLUENT_USER}

CMD [ "/usr/local/bin/kafka-connect.sh" ]

sqlserver-partner-data.properties:
name=partner-data
connector.class=io.confluent.connect.jdbc.JdbcSourceConnector
tasks.max=1
connection.url=jdbc:sqlserver://DB_HOST:59281;database=DB_USER;user=USER:password=PASS
mode=bulk
query=select a.* from blah
topic.prefix=data-partner-affiliate-data

All I have added to connect-standalone executable is CLASSPATH=$CLASSPATH:"/tmp/enu/sqljdbc42.jar" and an echo at the end.

Here is what my CLASSPATH looks like from the echo: /tmp/enu/sqljdbc42.jar:/usr/bin/../share/java/confluent-common/*:/usr/bin/../share/java/kafka-serde-tools/*:/usr/bin/../share/java/kafka-connect-hdfs/*:/usr/bin/../share/java/kafka-connect-jdbc/*:/tmp/enu/sqljdbc42.jar

I also have an echo to dump out what /tmp/enu looks like.

Content of /tmp/enu:

-rwxrwxrwx 1 confluent confluent 707400 Nov 4 01:12 sqljdbc42.jar

Has anyone tried this before?

Thanks for the help,

Ben

This email, including attachments, is private and confidential. If you have received this email in error please notify the sender and delete it from your system. Emails are not secure and may contain viruses. No liability can be accepted for viruses that might be transferred by this email or any attachment. Any unauthorised copying of this message or unauthorised distribution and publication of the information contained herein are prohibited.

7digital Limited. Registered office: 69 Wilson Street, London EC2A 2BB.
Registered in England and Wales. Registered No. 04843573.

[Gwen Shapira]

Hi Ben,

I think the issue is the sqljdbc version. sqljdbc42 is only used from Java 8. If you are using the Connector we built (rather than compile from sources), it was built with Java 1.7 (and therefore an older jdbc). We've had good experience with sqljdbc.jar (jdbc3), but sqljdbc4 and sqljdbc41 should also work.

Gwen

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platf...@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/CA%2BvjBRgRJ%3DF%2BErEhr10F-CfY9Eo3dOXcgUypAOKYs%3DoTeEYf0A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Ben Davison]

Hi Gwen,

Thanks for the suggestion, that actually fixed my next issue ;)

For the first issue we found if you bung the sqldb4 into

ADD sqljdbc_6.0/enu /usr/lib/jvm/java-1.7.0-openjdk-amd64/jre/lib/ext/

The classpath issue is fixed, the next issue we faced was what you described. We added sqldb4.jar and it worked.

Thanks,

Ben

To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/CAA9QSJLcUyZ-dtL2HiPqL2TUwRrr1maOgzfWCZAxZc%2BBB4kOGA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.