Benchmarking Moo(se)?

[Aran Deltac]

Hey all, I put together some benchmarks:

https://github.com/bluefeet/Moo-se-Bench

Let me know if you have any ideas/thoughts/criticisms.

Aran

[Kent Fredric]

On 25 July 2015 at 12:00, Aran Deltac <blue...@gmail.com> wrote:
>
> Let me know if you have any ideas/thoughts/criticisms.

I would be interested in a time-sequence chart showing each major
libraries performance against its version so we can see if either are
getting better/slower over time.

Also, the baseline perl version for tests should be included.

It would perhaps also be nice to do a comparison of said libraries on
different perl versions ( 5.22 is expected to be faster than 5.20 for
instance )

--
Kent

KENTNL - https://metacpan.org/author/KENTNL

[Aran Deltac]

Thanks Kent, good points, I'll add these to the TODO.  I was also thinking 5.22 would be an interesting test with the COW changes.  :)

[Dave Rolsky]

You didn't make your Moose classes immutable, which means everything is
going to be very, very, very slow.

Moose makes attribute accessors inline by default, which is why it matches
Moose. If you make the class immutable, constructors are inlined, and I
suspect you'll see the two come much closer together in performance.

That said, immutabilizing makes loading the class even slower.


-dave

/*============================================================
http://VegGuide.org http://blog.urth.org
Your guide to all that's veg House Absolute(ly Pointless)
============================================================*/

[Aran Deltac]

Oh my, hard to believe I forgot make_immutable! I'll update it ASAP, thanks Dave. There is a friend of mine who created an issue on the project where he is showing some odd performance differences between different versions of perl, if anyone is interested in checking it out.

[Aran Deltac]

I've updated the benchmarks to use make_immutable.  There was definite improvements for Moose, but none of them were enough to change the conclusions.

[Kent Fredric]

On 25 July 2015 at 18:36, Aran Deltac <blue...@gmail.com> wrote:
> I've updated the benchmarks to use make_immutable. There was definite
> improvements for Moose, but none of them were enough to change the
> conclusions.

Would you be able to have a seperate category for immutablizing,
especially in the "compile time" test?

Its not a big value, but given `use Moose' is not *only* loading
modules, but defining a class in the callers context, and Moo is doing
the same, having the comparison for "compile time to define a single
class" , both with and without immutability seems like relevant data.

Also, for all the other tests, side by side comparison of
non-immutable moose with immutable moose seem valuable for people to
decide if their startup overheads ( which should be marginal ) justify
the better run time ( which should be less marginal )

[Dave Rolsky]

On Sat, 25 Jul 2015, Aran Deltac wrote:

> I've updated the benchmarks to use make_immutable.  There was definite improvements for Moose, but none of them were enough to change the
> conclusions.

Well, the difference between Moo and Moose went from orders of magnitude
to percentages in most cases. This is a pretty huge change.

That all said, my experience is that in most applications performance is
not dominated by object creation speed or accessor speed, but rather by
the speed of algorithms and database access.

You'd probably get more performance improvement by changing a particluar
section of code to create fewer objects rather than worrying about Moo vs
moose.

[Chris Marshall]

Interesting. Two observations on the measurements:

- In the 0.2 sec it takes a Moose program to load, a Moo one
could complete >100,000 operations (i.e. could finish)

- Moo has the capability of Moosification. It would be nice
to have some measurements of Moo+sification vs Moose
to understand the overhead there.

Thanks for the timings and code.

--Chris

[Graham Knop]

It would be worth comparing Moo with and without Class::XSAccessor
installed. Moo will automatically use it for simple accessors if it's
available, and then will easily outperform Moose.

[Chris Prather]

​Which makes me wonder if we shouldn't look at something similar for Moose itself.

​

​-Chris