Mongo DB Connector sourcing only actual document-Not meta info

24 views
Skip to first unread message

Oliver Lopes

unread,
Apr 12, 2021, 5:23:42 PM4/12/21
to Confluent Platform
Hi,

I am using Mongo Source connector to source the documents from Mongo DB collection, but observed that, when connector sourcing the data, some un  necessary  or meta info is also coming along with document to kafka topic.

is there way to remove un necessary information and source only actual document what is saved in collection as part of record value.

Document saved in Collection:

    "_id" : ObjectId("6071bf8208d2f585001023bd"), 
    "name" : "bob", 
    "msgCount" : 2.0, 
    "address" : {
        "state" : "NY"
    }
}



Message coming to TOPIC:
[{"topic":"dbName.userAvroTest","partition":1,"offset":0,"timestamp":1618063824374,"timestampType":"CREATE_TIME","headers":[],"key":"\u0000\u0000\u0000\u0000\f�\u0001{\"_id\": {\"_data\": \"826071B1CE0000000129295A10046DBFB060925A4AB6A44F3EED085CCD5546645F696400646071B1CD08D2F585001023BC0004\"}}","value":"\u0000\u0000\u0000\u0000\f�\u0007{\"_id\": {\"_data\": \"826071B1CE0000000129295A10046DBFB060925A4AB6A44F3EED085CCD5546645F696400646071B1CD08D2F585001023BC0004\"}, \"operationType\": \"insert\", \"clusterTime\": {\"$timestamp\": {\"t\": 1618063822, \"i\": 1}}, \"fullDocument\": {\"_id\": {\"$oid\": \"6071b1cd08d2f585001023bc\"}, \"name\": \"bob\", \"msgCount\": 1.0, \"address\": {\"state\": \"NY\"}}, \"ns\": {\"db\": \"dbName\", \"coll\": \"userAvroTest\"}, \"documentKey\": {\"_id\": {\"$oid\": \"6071b1cd08d2f585001023bc\"}}}","__confluent_index":0}]



Oliver Lopes

unread,
Apr 12, 2021, 5:33:42 PM4/12/21
to Confluent Platform

correcting the subject.

Mongo DB Connector sourcing unnecessary meta info along with actual document

Reply all
Reply to author
Forward
0 new messages