Debezium Server no response to Redis

[Ravinder Singh]

Hi

I am using Debezium Server with SQL Server as source and Redis as sink. I have Redis-Comander to view the messages written to Redis. All these are running in Docker containers.

The issue is that when I perform a change in the table, Redis (or Redis Commander) does not see this change. 

However, when I close down all container and restart them, I see the changes coming up in Redis Commander. Now every time I perform  docker compose down and up again, these changes get appended to the Redis Queue creating duplicates. 

Please see attached config and docker compose file which I use. Any help is highly appreciated.

Thanks

Ravinder

[Oren Elias]

I suspect that you are using the File based offset writer. This means that every time you restart the debezium.server the container is recreated and the initial snapshot starts again. 

To resolve, you can either create a persistent volume for the offsets file or use the Redis Offset adapter to store the offsets in Redis as well. 

Oren. 

--
You received this message because you are subscribed to the Google Groups "debezium" group.
To unsubscribe from this group and stop receiving emails from it, send an email to debezium+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/debezium/27799a1d-ecb7-460c-9bc6-abd77129d7f6n%40googlegroups.com.

[Ravinder Singh]

Hi Oren

Yes it was File based offset. I have now persisted that by adding a volume in debezium-server  service (./deb/data:/debezium/data) and I do not get duplicates.

However I still do not see any synchronous change in Redis. Whenever I add or update the data in the table I do not see that in Redis. After creating the above volume and performing add/updates in DB I see no change at all.

Please help.

Thanks

[Oren Elias]

please share the debezium server log

To view this discussion on the web visit https://groups.google.com/d/msgid/debezium/64761939-632a-4509-b320-2cd66df216b4n%40googlegroups.com.

[Ravinder Singh]

Hi 

I am sharing two logs one INFO level and one DEBUG level.

Both logs are produced after creating a change.

Do let me know if you find anything. I will look at my end too.

Thanks

Ravinder

[jiri.p...@gmail.com]

Hi,

this looks like an issue on input not output. Could you please take a look into change table for the captured table (cdc schema) to see if the change is recorded there?

Thanks

J.

[Ravinder Singh]

Hi 

Thank you for the help. I looked at the CDC, they are working fine. 

I actually found the issue. This was due to Filegroup in the SQL Server, which when corrected is all sorted. 

Thank you for your help and time.

Thanks

Ravi