Hi Guys,
I am seeing this error in my task logs:
==========
Exception in thread "plumber_merge_0" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:3181)
at java.util.ArrayList.grow(ArrayList.java:265)
at java.util.ArrayList.ensureExplicitCapacity(ArrayList.java:239)
at java.util.ArrayList.ensureCapacityInternal(ArrayList.java:231)
at java.util.ArrayList.add(ArrayList.java:462)
at io.druid.segment.CompressedVSizeIndexedSupplier.fromIterable(CompressedVSizeIndexedSupplier.java:116)
at io.druid.segment.IndexIO$DefaultIndexIOHandler.convertV8toV9(IndexIO.java:677)
at io.druid.segment.IndexMerger.makeIndexFiles(IndexMerger.java:898)
at io.druid.segment.IndexMerger.merge(IndexMerger.java:437)
at io.druid.segment.IndexMerger.mergeQueryableIndex(IndexMerger.java:241)
at io.druid.segment.IndexMerger.mergeQueryableIndex(IndexMerger.java:213)
at io.druid.segment.realtime.plumber.RealtimePlumber$2.doRun(RealtimePlumber.java:415)
at io.druid.common.guava.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
==============
I have attached the complete task logs.
One thing to note is not all tasks fail for this datasource. Some are successful but most of them fail.
I have other datasources also on this cluster and all of them work without problems.
I have played around the maxRowsInMemory and reduced it to 25k, but still no success. However, I feel this is happening during the segment packaging stage so maxRowsInMemory should not have any effect.
Any help is appreciated.
Thanks