MongoDB and "java.io.IOException: Is a directory"

[Paolo Giacometti]

Hi, I'm trying to configure Debezium with docker and a MongoDB source but I get a Java exception "Is a directory" every time.

Here the commands to replicate the error:

# Start Apache Pulsar
docker run -d --name pulsar -p 6650:6650 -p 7080:8080 apachepulsar/pulsar:2.5.2 bin/pulsar standalone

# wait the startup
docker logs -f pulsar

# Start MongoDB 5 standalone
docker run -p 27017:27017  --name mongodb -e MONGO_INITDB_ROOT_USERNAME=admin -e MONGO_INITDB_ROOT_PASSWORD=pass -d mongodb/mongodb-community-server:5.0-ubuntu2004

# connect with mongosh and create database and sample collection (could be an optional step)
mongosh
test> use admin
switched to db admin

admin> db.auth('admin','pass')
{ ok: 1 }
admin> use db1
switched to db db1
db1> db.test_cdc.insertOne({ f1: 'start'})
{
  acknowledged: true,
  insertedId: ObjectId('66a250b57ab87dd879482f8b')
}
db1> show collections;
test_cdc
db1> exit

# Create a basic Debezium configuration file
mkdir {data,conf}; chmod 777 {data,conf}
cat <<-EOF > conf/application.properties
debezium.sink.type=pulsar
debezium.sink.pulsar.client.serviceUrl=pulsar://pulsar:6650
debezium.source.connector.class=io.debezium.connector.mongodb.MongoDbConnector
debezium.source.mongodb.connection.string=mongodb://admin:pass@mongodb
debezium.source.database.include.list=db1
debezium.source.capture.scope=database
debezium.source.topic.prefix=db1
EOF

# Run the server
docker run -it --name debezium -p 8080:8080 -v $PWD/conf:/debezium/conf -v $PWD/data:/debezium/data --link mongodb --link pulsar debezium/server

Then I see Debezium succesfully connects to Pulsar and MongoDB but I get this error:

{"timestamp":"2024-07-25T13:19:36.504Z","sequence":88,"loggerClassName":"org.slf4j.impl.Slf4jLogger","loggerName":"io.debezium.server.ConnectorLifecycle","level":"ERROR","message":"Connector completed: success = 'false', message = 'Unable to configure and start the 'org.apache.kafka.connect.storage.FileOffsetBackingStore' offset backing store', error = 'org.apache.kafka.connect.errors.ConnectException: java.io.IOException: Is a directory'","threadName":"pool-7-thread-1","threadId":25,"mdc":{},"ndc":"","hostName":"6486836d2749","processName":"io.debezium.server.Main","processId":1,"exception":{"refId":1,"exceptionType":"org.apache.kafka.connect.errors.ConnectException","message":"java.io.IOException: Is a directory","frames":[{"class":"org.apache.kafka.connect.storage.FileOffsetBackingStore","method":"load","line":92},{"class":"org.apache.kafka.connect.storage.FileOffsetBackingStore","method":"start","line":64},{"class":"io.debezium.embedded.EmbeddedEngine","method":"initializeOffsetStore","line":561},{"class":"io.debezium.embedded.EmbeddedEngine","method":"run","line":434},{"class":"io.debezium.embedded.ConvertingEngineBuilder$2","method":"run","line":229},{"class":"io.debezium.server.DebeziumServer","method":"lambda$start$1","line":170},{"class":"java.util.concurrent.ThreadPoolExecutor","method":"runWorker","line":1128},{"class":"java.util.concurrent.ThreadPoolExecutor$Worker","method":"run","line":628},{"class":"java.lang.Thread","method":"run","line":829}],"causedBy":{"exception":{"refId":2,"exceptionType":"java.io.IOException","message":"Is a directory","frames":[{"class":"sun.nio.ch.FileDispatcherImpl","method":"read0"},{"class":"sun.nio.ch.FileDispatcherImpl","method":"read","line":48},{"class":"sun.nio.ch.IOUtil","method":"readIntoNativeBuffer","line":276},{"class":"sun.nio.ch.IOUtil","method":"read","line":245},{"class":"sun.nio.ch.FileChannelImpl","method":"read","line":223},{"class":"sun.nio.ch.ChannelInputStream","method":"read","line":65},{"class":"sun.nio.ch.ChannelInputStream","method":"read","line":107},{"class":"sun.nio.ch.ChannelInputStream","method":"read","line":101},{"class":"java.io.ObjectInputStream$PeekInputStream","method":"read","line":2893},{"class":"java.io.ObjectInputStream$PeekInputStream","method":"readFully","line":2909},{"class":"java.io.ObjectInputStream$BlockDataInputStream","method":"readShort","line":3406},{"class":"java.io.ObjectInputStream","method":"readStreamHeader","line":932},{"class":"java.io.ObjectInputStream","method":"<init>","line":375},{"class":"org.apache.kafka.connect.util.SafeObjectInputStream","method":"<init>","line":48},{"class":"org.apache.kafka.connect.storage.FileOffsetBackingStore","method":"load","line":76},{"class":"org.apache.kafka.connect.storage.FileOffsetBackingStore","method":"start","line":64},{"class":"io.debezium.embedded.EmbeddedEngine","method":"initializeOffsetStore","line":561},{"class":"io.debezium.embedded.EmbeddedEngine","method":"run","line":434},{"class":"io.debezium.embedded.ConvertingEngineBuilder$2","method":"run","line":229},{"class":"io.debezium.server.DebeziumServer","method":"lambda$start$1","line":170},{"class":"java.util.concurrent.ThreadPoolExecutor","method":"runWorker","line":1128},{"class":"java.util.concurrent.ThreadPoolExecutor$Worker","method":"run","line":628},{"class":"java.lang.Thread","method":"run","line":829}]}}}}

Do you have any ideas?
Thanks


 

[Chris Cranford]

Hi -

The problem is that your configuration is incomplete. It's missing `debezium.source.offset.storage.file.filename` [1].
Please see the example configuration in the documentation [2].

Thanks,
-cc

[1]: https://debezium.io/documentation/reference/3.0/operations/debezium-server.html#debezium-source-offset-storage-file-filename
[2]: https://debezium.io/documentation/reference/3.0/operations/debezium-server.html#_configuration

--
You received this message because you are subscribed to the Google Groups "debezium" group.
To unsubscribe from this group and stop receiving emails from it, send an email to debezium+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/debezium/cd3c572d-6fed-4463-8602-18ab77af3acbn%40googlegroups.com.

[Paolo Giacometti]

Thank you. I didn't realise that the parameter "debezium.source.offset.storage.file.filename" was mandatory.
The message "is a directory" was unclear to me.

Paolo.