Any plans to use the OpenJDK JMH framework

[Abhinav Rau]

According to the OpenJDK JMH website "JMH is a Java harness for building, running, and analyzing nano/micro/milli/macro benchmarks written in Java and other languages targeting the JVM."

I have been reading up on it and it looks like it is a very comprehensive tool which provides very advanced features. More details here: http://psy-lob-saw.blogspot.com/p/jmh-related-posts.html

Would this tool be a good candidate to use for the jvm-serializer benchmark? The reasons to use JMH would be:

1. More accurate results since it is written by same guys at Oracle who wrote the JIT
2. Cleaner code - All benchmarking is driven by annotations
3. Multi-language support
4. Programmable API
5. Pluggable profilers which can measure different aspects of the JVM 

• Name	Description
  CL	Classloader profiling via standard MBeans
  COMP	JIT compiler profiling via standard MBeans
  GC	GC profiling via standard MBeans
  HS_CL	HotSpot ™ classloader profiling via implementation-specific MBeans
  HS_COMP	HotSpot ™ JIT compiler profiling via implementation-specific MBeans
  HS_GC	HotSpot ™ memory manager (GC) profiling via implementation-specific MBeans
  HS_RT	HotSpot ™ runtime profiling via implementation-specific MBeans
  HS_THR	HotSpot ™ threading subsystem via implementation-specific MBeans
  STACK	Simple and naive Java stack profiler

   
  

Any thoughts and feedback?

Abhinav

[Kannan Goundan]

I think switching to JMH is a good idea; I don't think anyone is happy our current quick-and-dirty benchmark runner.

My only advice would be to really limit the scope of the initial diff.  There are many things that could be improved about the benchmark, but the initial switch to JMH will be easier for others to digest/review if it's minimal.

--
You received this message because you are subscribed to the Google Groups "java-serialization-benchmarking" group.
To unsubscribe from this group and stop receiving emails from it, send an email to java-serialization-be...@googlegroups.com.
To post to this group, send email to java-serializat...@googlegroups.com.
Visit this group at https://groups.google.com/group/java-serialization-benchmarking.
For more options, visit https://groups.google.com/d/optout.

[Tatu Saloranta]

Same for me -- there are things that would be really useful, mostly just basic things like isolation of test runs into different JVMs. So no one is married to existing set up. It would be nice to have similar post-processing for diagrams.

Also, there have been desires for some types of parameterization beyond just (slightly) difference input documents, for example whether to include GC overhead as part of the test; or to allow use of certain optimizations. These would probably be easier pass via jmh.

And personally I really like jmh based on my experiences. It's just matter of someone having enough itch to scratch it. :)

-+ Tatu +-

[Abhinav Rau]

Thanks for the response. I will try to do it with minimal changes so we can compare JMH and the current benchmark runner.

There are lot more things I would like to do (like use Maven) but I understand that gradual changes is the best way to go.

Abhinav

To unsubscribe from this group and stop receiving emails from it, send an email to java-serialization-benchmarking+unsubscribe@googlegroups.com.
To post to this group, send email to java-serialization-benchm...@googlegroups.com.

[Abhinav Rau]

Let me take a crack at it and I will post my experiences here. As Kannan stated, small incremental changes would be the way to go.


Thanks.

Abhinav

To unsubscribe from this group and stop receiving emails from it, send an email to java-serialization-benchmarking+unsubscribe@googlegroups.com.
To post to this group, send email to java-serialization-benchm...@googlegroups.com.

Visit this group at https://groups.google.com/group/java-serialization-benchmarking.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "java-serialization-benchmarking" group.

To unsubscribe from this group and stop receiving emails from it, send an email to java-serialization-benchmarking+unsubscribe@googlegroups.com.
To post to this group, send email to java-serialization-benchm...@googlegroups.com.

[Abhinav Rau]

Let me take a crack at it and I will post my experiences here. As Kannan stated, small incremental changes would be the way to go.


Thanks.

Abhinav


On Monday, January 25, 2016 at 1:19:58 PM UTC-5, cowtowncoder wrote:

To unsubscribe from this group and stop receiving emails from it, send an email to java-serialization-benchmarking+unsubscribe@googlegroups.com.
To post to this group, send email to java-serialization-benchm...@googlegroups.com.

Visit this group at https://groups.google.com/group/java-serialization-benchmarking.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "java-serialization-benchmarking" group.

To unsubscribe from this group and stop receiving emails from it, send an email to java-serialization-benchmarking+unsubscribe@googlegroups.com.
To post to this group, send email to java-serialization-benchm...@googlegroups.com.

[Tatu Saloranta]

Either that, or alternatively create a new project ("jvm-serializers-ng"), using some parts (POJOs) and dropping others. I don't think there's absolute need to change things in place, or in separate branch; sometimes rewrite makes sense.

-+ Tatu +-

To unsubscribe from this group and stop receiving emails from it, send an email to java-serialization-be...@googlegroups.com.
To post to this group, send email to java-serializat...@googlegroups.com.

Visit this group at https://groups.google.com/group/java-serialization-benchmarking.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "java-serialization-benchmarking" group.

To unsubscribe from this group and stop receiving emails from it, send an email to java-serialization-be...@googlegroups.com.
To post to this group, send email to java-serializat...@googlegroups.com.

Visit this group at https://groups.google.com/group/java-serialization-benchmarking.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "java-serialization-benchmarking" group.

To unsubscribe from this group and stop receiving emails from it, send an email to java-serialization-be...@googlegroups.com.
To post to this group, send email to java-serializat...@googlegroups.com.

[Kannan Goundan]

Just to clarify, by "minimal" I didn't mean the size of the diff (in terms of lines of code).

For example, here are some changes that have been discussed.
- Changing whether we use byte arrays vs streams.

- Changing whether we include an extra copy operation in the timings.

I'd prefer to not change that stuff in the initial diff.  However, as far as the benchmarking framework is concerned, I'm fine with you rewriting as much as you want.  (I'm just happy you're taking this on :-)