Main thread profile blob becoming too big

30 views
Skip to first unread message

Pavani Manthana

unread,
Nov 20, 2019, 4:13:02 PM11/20/19
to Glowroot
Hi Trask - We are using 0.13.1 version of central collector and noticing that for certain rollups the aggregation is resulting in these errors in cass db.

WARN  [MutationStage-6] 2019-11-20 13:16:16,176 AbstractLocalAwareExecutorService.java:167 - Uncaught exception on thread Thread[MutationStage-6,5,main]: {}
java.lang.IllegalArgumentException: Mutation of 18.948MiB is too large for the maximum size of 16.000MiB
        at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:255) ~[apache-cassandra-3.11.1.jar:3.11.1]
        at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:585) ~[apache-cassandra-3.11.1.jar:3.11.1]
        at org.apache.cassandra.db.Keyspace.applyFuture(Keyspace.java:425) ~[apache-cassandra-3.11.1.jar:3.11.1]
        at org.apache.cassandra.db.Mutation.applyFuture(Mutation.java:222) ~[apache-cassandra-3.11.1.jar:3.11.1]
        at org.apache.cassandra.db.MutationVerbHandler.doVerb(MutationVerbHandler.java:68) ~[apache-cassandra-3.11.1.jar:3.11.1]

Upon further analysis, deleting aggregate_tn_main_thread_profile_rollup & aggregate_tt_main_thread_profile_rollup for the rollup fixed the issue. 

So my question what can we do to not let the blob size grow too large. Can we setup some kind of limit on it?

Thanks,
Pavani
 
Reply all
Reply to author
Forward
0 new messages