cannot be changed from type [text] to [ObjectMapper]]]

[Alex Nogueira]

Hi all,

First, thanks for the opportunnity.

I'm use the kafka with the kafka-connect(ElasticsearchSink).

My solution is to resolve a problem with the messages in Enterprise Service Bus.

The question is that when I receive the message in kafka, on kafka-connect-elasticsearch, appear the error bellow:

#######

ERROR Encountered an illegal document error 'ElasticsearchException[Elasticsearch exception [type=illegal_argument_exception, reason=mapper [fields.type] cannot be changed from type [text] to [ObjectMapper]]]'. To ignore future records like this, change the configuration 'behavior.on.malformed.documents' to 'IGNORE'. (io.confluent.connect.elasticsearch.ElasticsearchClient:435)

#######

Follow my configuration

#connect-standalone.properties

key.converter=org.apache.kafka.connect.storage.StringConverter

key.converter.schemas.enable=false

key.deserializer=org.apache.kafka.common.serialization.JsonDeserializer

value.deserializer=org.apache.kafka.common.serialization.JsonDeserializer

value.converter=org.apache.kafka.connect.storage.StringConverter

value.converter.schemas.enable=false

offset.storage.file.filename=/tmp/connect.offsets

offset.flush.interval.ms=100000

batch.size=2000

max.buffered.records=20000

plugin.path=/opt/plataformas/elastic-kafka/kafka/plugins

#elasticsearch-connect.properties

name=elasticsearch-sink

connector.class=io.confluent.connect.elasticsearch.ElasticsearchSinkConnector

key.converter =org.apache.kafka.connect.storage.StringConverter

key.converter.schemas.enable=false

value.converter=org.apache.kafka.connect.json.JsonConverter

value.converter.schemas.enable=false

type.name=_doc

tasks.max=1

topics=testeosb1

topic.index.map=testeosb1:testeosb1

connection.url=http://localhost:9200

#type.name=log

key.ignore=true

Follow my payloads:

--------------Example 1-----------------

{"type":"record","name":"hotelcrimes","namespace":"com.oyo","fields":[{"name":"_id","type":{"type":"record","name":"id","fields":[{"name":"oid","type":["null","string"]}]}},{"name":"bid","type":["null","int"]},{"name":"biv","type":["null","string"]},{"name":"cca","type":["null","string"]},{"name":"ct","type":["null","string"]},{"name":"hid","type":["null","int"]},{"name":"md","type":{"type":"record","name":"md","fields":[{"name":"feedback_id","type":["null","int"]},{"name":"remove_comment","type":["null","string"]},{"name":"shifting_id","type":["null","int"]},{"name":"ticket_id","type":["null","int"]}]}},{"name":"noc","type":["null","double"]},{"name":"oyo_id","type":["null","string"]},{"name":"r","type":["null","boolean"]},{"name":"st","type":["null","boolean"]},{"name":"ufc","type":["null","boolean"]},{"name":"updated_at","type":["null","string"]},{"name":"created_at","type":["null","string"]}]}

----------------Example 2-----------------

"<?xml version=\"1.0\"?>\n<x:books xmlns:x=\"urn:books\">\n    <book id=\"bk001\">\n        <author>Writer</author>\n        <title>The First Book</title>\n        <genre>Fiction</genre>\n        <price>44.95</price>\n        <pub_date>2000-10-01</pub_date>\n        <review>An amazing story of nothing.</review>\n    </book>\n\n    <book id=\"bk002\">\n        <author>Poet</author>\n        <title>The Poet's First Poem</title>\n        <genre>Poem</genre>\n        <price>24.95</price>\n        <pub_date>2000-10-01</pub_date>\n        <review>Least poetic poems.</review>\n    </book>\n</x:books>"

Thanks a lot