message tombstoning with kafka cli

[Anya Cranmer]

I'm trying to test tombstoning messages using the kafka cli. These are some of my kafka server properties:

log.cleaner.enable=true
log.cleaner.min.cleanable.ratio=0.1
log.segment.bytes=4000
log.cleanup.policy=compact
log.cleaner.delete.retention.ms=10000

Log compaction is working fine; however, the log cleaner is not removing tombstoned messages. Here's how I'm generating the null message

kafka-console-producer --broker-list local.docker:9092 --topic CONSTITUENT --property parse.key=true --property key.separator=###
testkey###


After log compaction runs (and after when I believe that tombstone entries should have been deleted), I still see this empty message in the logs (among other log entries)

kafka-console-consumer --zookeeper local.docker:2181 --topic CONSTITUENT --from-beginning  --property print.key=true
testkey

Is this the correct way to produce a tombstone using the kafka cli? If not, I'm looking for feedback on how to do this.

thanks!!

[Ticean Bennett]

I'm seeing the same behavior where the messages are compacted but the tombstones aren't removed. I've used settings similar to those above and also set `delete.retention.ms=10000`.

Is this the correct way to produce tombstone markers from the CLI?

Thanks!

[Ewen Cheslack-Postava]

Are the tombstones still in the active log segment? The active segment will not be compacted, so you would still see the tombstones until a new segment had been rolled and compaction had run again.

-Ewen

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent-platform@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/bd4e0e6e-be9f-4303-9814-90aed7d4320e%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.