Is it possible to start the mysql binlog connector from user specified binlog position.

Ritesh Singh

unread,

Jul 27, 2020, 1:18:54 PM7/27/20

to debezium

Hi,

I want to start a binlog connector from specific binlog position.

My scenario is that I want to bypass the snapshot phase and directly start the binlog phase from specific binlog position.

Thanks

Ritesh

Gunnar Morling

unread,

Jul 30, 2020, 8:34:03 AM7/30/20

to debezium

You can use snapshot mode "schema_only". It will start streaming from the current binlog position. then.

--Gunnar

Ritesh Singh

unread,

Jul 30, 2020, 8:58:51 AM7/30/20

to debe...@googlegroups.com

When I start the connector for first time with "schema_only" , I get an error message:

org.apache.kafka.connect.errors.ConnectException: Could not find existing binlog information while attempting schema only recovery snapshot
at io.debezium.connector.mysql.AbstractReader.wrap(AbstractReader.java:230)
at io.debezium.connector.mysql.AbstractReader.failed(AbstractReader.java:208)
at io.debezium.connector.mysql.SnapshotReader.execute(SnapshotReader.java:721)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: Could not find existing binlog information while attempting schema only recovery snapshot
at io.debezium.connector.mysql.SnapshotReader.readBinlogPosition(SnapshotReader.java:738)
at io.debezium.connector.mysql.SnapshotReader.execute(SnapshotReader.java:370)
... 3 more

And if I manually insert an entry in the connect-offsets topic, and then start a connector, then it works.

Also If I start the connector for the first time with "inital" or "when needed", and then restart the connector with "schema_only", then also it works.

My scenario is that I want to start the connector for the first time with "schema_only" snapshot mode.

Thanks

Ritesh

--
You received this message because you are subscribed to the Google Groups "debezium" group.
To unsubscribe from this group and stop receiving emails from it, send an email to debezium+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/debezium/e240a07a-8cb4-494f-b3db-b1481d077c25n%40googlegroups.com.

Gunnar Morling

unread,

Jul 30, 2020, 10:11:26 AM7/30/20

to debe...@googlegroups.com

Are you using "schema_only", or is it actually "schema_only_recovery"?
In that case, please try the former. If you indeed get this error with
"schema_only", can you please log a Jira issue, as this shouldn't be
the case. Thanks.

--Gunnar

Am Do., 30. Juli 2020 um 14:58 Uhr schrieb Ritesh Singh
<ritesh...@gmail.com>:

> To view this discussion on the web visit https://groups.google.com/d/msgid/debezium/CABtDoTnwehFHRrwz_k%3DhJAaqU28G6DHEXyDZD1KQf2u3j1yYyw%40mail.gmail.com.

Ritesh Singh

unread,

Jul 31, 2020, 12:00:22 PM7/31/20

to debe...@googlegroups.com

I confirmed that schema_only is working as desired.

It was my mistake that I was using schema_only_recovery.

Thanks

Ritesh

To view this discussion on the web visit https://groups.google.com/d/msgid/debezium/CADGJaX-3iRiHy%3Dy2HazqqVtw6_jteLy5UsJwo36L0iCYQ8o5Wg%40mail.gmail.com.

V主宰

unread,

Aug 20, 2020, 10:19:44 AM8/20/20

to debezium

My scenario is that debuzium meet a exception and connector restart cannot be resolved, so want specified binlog position to skip a event, how to do ?

jiri.p...@gmail.com

unread,

Aug 21, 2020, 3:31:54 AM8/21/20

to debezium

See https://debezium.io/documentation/faq/#how_to_change_the_offsets_of_the_source_database

Please next time open a new topic instead of reusing a resolved one, thanks!

Reply all

Reply to author

Forward