Kafka producer record-size-avg metric

396 views
Skip to first unread message

Zack Kobza

unread,
Jun 22, 2020, 12:11:27 PM6/22/20
to kafka-clients
The Kafka documentation states that the producer's record-size-avg metric is "The average record size", but this does not seem to be the case. The client code calculates the average maximum record size across producer batches for record-size-avg.  I am misinterpreting this metric? Should the documentation be updated to reflect this?

Here are code snippets from version 2.6 that show the creation of the sensor and how it is populated:

Sender.java:
    811: this.maxRecordSizeSensor = metrics.sensor("record-size");
    812: this.maxRecordSizeSensor.add(metrics.recordSizeMax, new Max());
    813: this.maxRecordSizeSensor.add(metrics.recordSizeAvg, new Avg());
    889: this.maxRecordSizeSensor.record(batch.maxRecordSize, now);

ProducerBatch.java:
    108 & 134: this.maxRecordSize = Math.max(this.maxRecordSize, AbstractRecords.estimateSizeInBytesUpperBound(magic(), recordsBuilder.compressionType(), key, value, headers));

Zack Kobza

unread,
Jun 22, 2020, 12:15:35 PM6/22/20
to kafka-clients
To add more flavor, I have been using this value for performance testing and was mislead because the record-size-avg is not the average record size the client was sending. I calculated the average record size separately and it did not equal the value reported by the metric.
Reply all
Reply to author
Forward
0 new messages