I have deployed pinot in a kubernetes cluster. When I start ingesting data in kafka, the RAM keeps on piling up even after segment creation and eventually pod crashes due to OOM.
I am keeping the segment threshold size as 1000000 and ingesting about 10 million entries in kafka, memory limit is 4GB for server.
Thanks!