Global Ktable with compacted topic

[Sathish K]

Dear Team,

In a Kstreams application, im streaming from a topic to GlobalKtable for querying from it later. I understood the whole concept of GlobalKtable and compacted topics, my question is what is the difference between using GlobalKtable with compacted topic and non-compact topic.

I want to make sure the key with value null has to be deleted in Globalktable.. if i use compacted topic how the deletion will be probagated to GlobalKtable?

Thanks in Advance.

[Matthias J. Sax]

A <key,null> record does have delete semantics in a GlobalKTable similar
to KTable and the underlying changelog topic.

> my question is what is the difference between using GlobalKtable with compacted topic and non-compact topic.

You should always use a compacted topic as input of a GlobalKTable. It's
technically possible (and not necessarily wrong) to read a non-compacted
topic, but it's semantically questionable.

>> I want to make sure the key with value null has to be deleted in Globalktable.. if i use compacted topic how the deletion will be probagated to GlobalKtable?

If the GlobalKTable processes a <key:null> record it will delete the
corresponding entry in it's internal state. Note that GlobalKTable
bootstraps at beginning (ie, reads everything to "endOffsets") _and_
also fetches newly written records of the input topic during regular
processing. Thus, if a <key:null> is written while the application is
running, it will read it and thus delete the corresponding record.


-Matthias

[Sathish K]

Thank u so much. Pretty clear now..
Just a quick question, while creating the compacted topic, do i need to give --config retention.ms=-1

In short, is the default retention.ms applicable for compacted topic?

I need my compacted records forever.. thats why im asking this. Thank u again

[Matthias J. Sax]

If topic policy is "compaction" config parameter retention.ms is ignored.

-Matthias

[Sathish K]

Ok. Thank you so much for your reply