Tracking memory allocations

[Iskander Sharipov]

Hello.

I have two main metrics for dynamic memory allocations:
1. Total memory allocations per operation.
2. Number of allocations per operation.

Straightforward approach is to track both of them using 2 kinds of benchmarks.
One shows count dynamics, the other shows size dynamics.

The disadvantage is that those measurements do not change independently.
95% of the time if one benchmark has no changes, the other will not have changes as well.

But one can't simple merge two into single metric using coefficients. This is wrong in some scenarios.

I want to avoid excessive (95% duplication) data, benchmarks and plots without
information loss.

Are there any practices existing for this?

My initial idea was to propose tuple-like results: (bytes allocated, allocations count).
Then use Python tuple comparison rules.
This way, allocation count is more significant in the way it
determines the primary value. If primary value is equal,
then secondary is used to resolve better/worse relations.
This still leaves Y value calculations an open question though.
For occasions where number of allocations is more important,
tuple elements can be swapped prior to plot rendering.
Because this seems quite complicated (and have missing pieces in the design),
no real feature proposal/request is done yet.

[Iskander Sharipov]

When both metrics go down - good.
When one metric goes down and the other unchanged - still good.
It's generally hard to make a decision when they go in different directions (one improves, other degrades).
When both metrics increase - bad.
When one metric goes up and the other unchanged - still bad.

So, only 2 out of 8 states (outcomes) are worth separate analysis.
6/8 can be classified as regression or improvement.
I find this as an argument in favor of working out on aggregated view.