Benchmarking when hacking on Kryo

[Jo Durchholz]

Hi all,

I want to so some hacking on Kryo, and need to make sure this does not
reduce performance or memory usage.
(I feel confident about unit testing, so no questions on that for now.)

Question 1: What's the best incantation for benchmarking this?
I guess I should create baseline measurements, then benchmark my
changes, but
a) What are the best options to do the benchmarking? I see various
suggestions in KryoBenchmarks.java but don't know which of them or which
combination would be best.
b) Is there a good way to compare old and new measurements, other than
ogling the CSV files?

Question 2: What is the best way to get allocation behaviour out of a
test run? I guess the benchmarks are fine for that, but I didn't see (or
maybe just missed?) the allocation information.

Regards,
Jo

[Nate]

Hi Jo,

You can base your own benchmarks on the JMH benchmarks in Kryo. Customize them with your own data structures and serialization settings. Run before and after tests to compare.

It's not useful to compare benchmarks others have done on different hardware, even if the data is the same.

For allocations, read up on JMH. Don't try to write benchmarks without JMH.

Cheers,

-Nate

--
You received this message because you are subscribed to the "kryo-users" group.
http://groups.google.com/group/kryo-users
---
You received this message because you are subscribed to the Google Groups "kryo-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kryo-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kryo-users/b0ff1ad3-23fc-4807-9d01-9cd1925df26a%40durchholz.org.

[j...@durchholz.org]

Hi Nate,

On 21.04.24 20:00, Nate wrote:
> Hi Jo,
>
> You can base your own benchmarks on the JMH benchmarks in Kryo.
> Customize them with your own data structures and serialization settings.
> Run before and after tests to compare.

What's the easiest way to do the comparisons?

I (now, yay!) know how to run the JMH benchmarks, and where the results
live.
But I don't know how to look at the data, compare it with old data, and
spot what's interesting.
What's best - write code that translates the differences to percentages?
Graph everything and do visual comparison? Something else?

> It's not useful to compare benchmarks others have done on different
> hardware, even if the data is the same.

Definitely :-)

> For allocations, read up on JMH. Don't try to write benchmarks without JMH.

Ah. I didn't notice JMH has a GC profiler as well.
That's good.

Regards,
Jo