Automatic segment compaction segment size not matching targetCompactionSizeBytes

[xiangqun li]

Hi,

I am trying to use the "Automatic segment compaction". My auto-compaction config is as follows:

{

  "dataSource": ds,

  "inputSegmentSizeBytes":524288000,

  "targetCompactionSizeBytes": 524288000,

  "skipOffsetFromLatest": "PT3H",

  "keepSegmentGranularity": false

}

However, after the compact task finished, I can see that the segment(shard) size is around 130+ MB. Those segments will also be involved in the next round compaction tasks and result in the same-size segments as before.

I am suspecting that tuningConfig("maxRowsPerSegment": 7000000) of original supervisor affected the size of the segment after compaction. Please advise on how to proceed.

Thanks.

[Rommel Garcia]

Your target compaction size is 524MB. If segment(s) that’s got a total size of less than 524MB then your segment size will be smaller and the same size. 

Rommel Garcia

--
You received this message because you are subscribed to the Google Groups "Druid User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to druid-user+...@googlegroups.com.
To post to this group, send email to druid...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/druid-user/845ba0a0-2e81-49be-a42e-afc6efa4e81a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.