"latest" meditation

18 views
Skip to first unread message

Steffen Schwigon

unread,
Jan 23, 2012, 5:39:09 AM1/23/12
to codespeed
Hi!

Just asking to understand. I'm interested in the meaning of “latest” in
the “Comparison” tab.

I maybe misuse “executables”. From the same project (“perl”), I create
different executables by separating them by version which covers about a
year each; i.e., I have these executables:

perl-5.8
perl-5.10
perl-5.11
perl-5.12
perl-5.13
perl-5.14
perl-5.15

Now they all use the same repository, which means they share the same
space of git commits.

I seems that in the “Comparison” there is only 2 possible values, one
“baseline” and one “latest” - probably because “latest” will always be
from the perl-5.15 executable I can only compare benchmarks of 5.15 and
5.8 (the baseline).

Is this observation correct? Or does it work different?

What could I do to compare my other versions?

Ideas so far:

- assign a different project for each version (still all with the same
git repo)? I don't like that idea.

- assign tags for each versions “latest” to override the default
“latest”?

- assign *other* tags than “baseline or “latest”?


Any comments?

Thanks for listening.

Kind regards,
Steffen
--
Steffen Schwigon <s...@renormalist.net>
http://speed.perlformance.net

Miquel Torres

unread,
Jan 23, 2012, 2:40:07 PM1/23/12
to code...@googlegroups.com
Hi Steffen,

I am trying to fully understand your use case. The first question is: is the history for each executable static or does it still keep evolving?

Logic says that, because they are versions from the past, history should be fixed. Which means that only the last version (5.15) may see newer commits. Is it like that?

In that case, you don't need to have different executables for that. Why should you? Are you doing that just to be able to compare them?

If that is the case, you should use Codespeed's executables as intended: you only need one executable for a binary which is compiled with particular settings. Then, as history evolves in master, every time a new 5.x version is released you tag that commit. Then, even though you only have a single executable, all versions will appear in the comparison view, *in addition* to "latests". which is only meant to reflect performance of master/trunk.

Does that make sense to you?


2012/1/23 Steffen Schwigon <s...@renormalist.net>

Steffen Schwigon

unread,
Jan 24, 2012, 6:40:11 AM1/24/12
to code...@googlegroups.com
Miquel Torres <tob...@googlemail.com> writes:
> Hi Steffen,
>
> I am trying to fully understand your use case. The first question is:
> is the history for each executable static or does it still keep
> evolving?

Yes and No. :-)

Simplified answer: generally each version can evolve separately.

The full truth is this:

- the “odd” versions (5.11, 5.13, *5.15*) are the development
versions. Only the latest of them is actually changing. So, 5.11 and
5.13 are closed, 5.15 is currently the active master (aka. “blead”).

- Every year it gets a stable “even” release: the next will 5.16,
before we had 5.14 (current “stable”) and 5.12 (“maint”).

The “even” releases get new changes until their maintenance ends -
usually backports but in their own commits in separate branches
(often cherry-picked).


> Logic says that, because they are versions from the past, history
> should be fixed. Which means that only the last version (5.15) may see
> newer commits. Is it like that?

So in fact only “some” histories are fixed:

- the old “odds” and
- the old out-of-maintenance “evens”.

It's a valid simplification to assume all can change.

Additionally to this assumption I like to visually mark the different
life-phases by colors, because each year has its own “topic”: refactor
the core syntax plugin api, generalize/plugin'ify the regex engine, etc.


> In that case, you don't need to have different executables for
> that. Why should you? Are you doing that just to be able to compare
> them?

Traditional questions are:

- compare modern versions (5.12, 5.14, 5.15+)
to the “veteran” version 5.8.

- supervise the current development version for speed regressions
(against itself)

- compare multiple but comparable benchmarks (e.g., several Fibonacci
variants [functional + builtin_OO + alternative_OO) in several
Perl versions.


> If that is the case, you should use Codespeed's executables as
> intended: you only need one executable for a binary which is compiled
> with particular settings.

I would have agreed before this my own answer. Now, as I read it, I
think I'm doogin godd in separating the versions.

However, I'm not sure how the “branch support” works which got added to
codespeed some months ago. Maybe that would be the right thing...

> Then, as history evolves in master, every time a new 5.x version is
> released you tag that commit.

You mean one “perl” executable and tags “5.12”, “5.14”, “5.16”?

Maybe I can do that additionally to already used “stable”, “latest”?

It might get confusing because for all the versioned executables I have
(I get a lot of tagged proposals for each exe against each tag).

What about assigning the tag “latest” manually? Or is it *always*
assigned implicitely? Then every versioned executable would have its own
“latest” tag.


> Then, even though you only have a single executable, all versions will
> appear in the comparison view, *in addition* to "latests". which is
> only meant to reflect performance of master/trunk.
>
> Does that make sense to you?

It becomes better. :-)

1.) I will try to experiment with manually set “latest” tags.
2.) And I need to learn about codespeed's branches mechanism.

I hope one of those two approaches will “suit my demanding taste”. :-)

Kind regards,
Steffen

--
Steffen Schwigon <s...@renormalist.net>

Steffen Schwigon

unread,
Jan 24, 2012, 8:17:54 AM1/24/12
to code...@googlegroups.com
Steffen Schwigon <s...@renormalist.net> writes:
> I think I'm doogin godd in separating the versions.

I meant: “doing good”.

Steffen

Reply all
Reply to author
Forward
0 new messages