Kafka Connection

[Xiaojing Gong]

I did the test of the Presto instruction for my aws Presto cluster. 

Hive, Cassandra, Mysql connection are working. however, for Kafka , I have problems.

My kafka cluster are working:

kafka@ip-10-94-131-79 kafka_2.11-0.9.0.0]$ bin/kafka-topics.sh --list --zookeeper localhost:2181

TutorialTopic

__consumer_offsets

department

employee

employee_denormalized

orders

sales - marked for deletion

sparkfu

test

topic

tpch.customer

tpch.lineitem

tpch.nation

tpch.orders

tpch.part

tpch.partsupp

tpch.region

tpch.supplier

My Presto Configuration:

connector.name=kafka

kafka.nodes=kafka-spark-cassandra-1:9092,kafka-spark-cassandra-2:9092,kafka-spark-cassandra-3:9092

kafka.table=tpch.customer,tpch.orders,tpch.lineitem,tpch.part,tpch.partsupp,tpch.supplier,tpch.nation,tpch.region

kafka.hide-internal-columns=false

Error message: 

016-02-05T21:54:08.338-0500 ERROR main com.facebook.presto.server.PrestoServer Unable to create injector, see the following errors:

1) Configuration property 'kafka.table=tpch.customer,tpch.orders,tpch.lineitem,tpch.part,tpch.partsupp,tpch.supplier,tpch.nation,tpch.region' was not used

  at io.airlift.bootstrap.Bootstrap.lambda$initialize$2(Bootstrap.java:235)

1 error

com.google.inject.CreationException: Unable to create injector, see the following errors:

1) Configuration property 'kafka.table=tpch.customer,tpch.orders,tpch.lineitem,tpch.part,tpch.partsupp,tpch.supplier,tpch.nation,tpch.region' was not used

  at io.airlift.bootstrap.Bootstrap.lambda$initialize$2(Bootstrap.java:235)

1 error

at com.google.inject.internal.Errors.throwCreationExceptionIfErrorsExist(Errors.java:466)

at com.google.inject.internal.InternalInjectorCreator.initializeStatically(InternalInjectorCreator.java:155)

at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:107)

at com.google.inject.Guice.createInjector(Guice.java:96)

at io.airlift.bootstrap.Bootstrap.initialize(Bootstrap.java:242)

at com.facebook.presto.kafka.KafkaConnectorFactory.create(KafkaConnectorFactory.java:97)

at com.facebook.presto.connector.ConnectorManager.createConnector(ConnectorManager.java:301)

[Anton Petrov]

It looks like the third property in your kafka.properties is incorrect. According to https://prestodb.io/docs/current/connector/kafka-tutorial.html, it should be "kafka.table-names" instead of what you have, which is "kafka.table". Hope this helps!

--
You received this message because you are subscribed to the Google Groups "Presto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to presto-users...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Xiaojing Gong]

Thank you so much. I was looking at https://prestodb.io/docs/current/connector/kafka.html

It gave me the misunderstanding.  it said . But now it is working. Thank you!!

kafka.table-names=table1,table2
kafka.nodes=host1:port,host2:port

connector.name=kafka