Perf testing on release branches?

[George Burgess]

Happy Monday,

When I commit a change to Chromium's master branch, our perf bots generally do an excellent job of telling me if my change regressed metrics we care about. Though, It's not clear to me what our perf testing story is for release branches. Do we have a similar level of coverage for commits on release branches? Or is the expectation that commits to release branches should generally be performance-neutral/already vetted on master?

Better yet, might there be docs/webpages on all of this that I can't seem to be able to find? :)

Thanks!

George

[Ned]

We currently don't have coverage for commits on release branches. For now, one can still evaluate the impact of their commits to release branch using pert trybot tool (+Dave Tu to confirm)

+Ushesh Desai +Ben Henry +Annie Sullivan +Tim Dresser  for thoughts about how important it's to address this gap.

--
You received this message because you are subscribed to the Google Groups "Chrome benchmarking" group.
To unsubscribe from this group and stop receiving emails from it, send an email to benchmarking-d...@chromium.org.
To post to this group, send email to benchmar...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/benchmarking-dev/CA%2BrzOEmWygTXyLR%2B2JoxgrqnzC_f%3Dgn-O_KyB4jRT7ASHUP%2BjQ%40mail.gmail.com.

[Tim Dresser]

We don't have test coverage in general for release branches, right? My intuition is that if it's good enough for correctness testing, it's probably good enough for us, but I don't have any data to back that up.

[Annie Sullivan]

+amineer: do you know the status of release branch testing in Chrome? My guess is that we'd want perf testing on the roadmap at some point, but not until we're happy with correctness testing?

On Tue, Jul 10, 2018 at 8:08 AM, Tim Dresser <tdre...@google.com> wrote:

We don't have test coverage in general for release branches, right? My intuition is that if it's good enough for correctness testing, it's probably good enough for us, but I don't have any data to back that up.

On Tue, Jul 10, 2018 at 7:09 AM Ned <nedn...@google.com> wrote:

We currently don't have coverage for commits on release branches. For now, one can still evaluate the impact of their commits to release branch using pert trybot tool (+Dave Tu to confirm)

+Ushesh Desai +Ben Henry +Annie Sullivan +Tim Dresser  for thoughts about how important it's to address this gap.

On Mon, Jul 9, 2018 at 8:23 PM George Burgess <gb...@chromium.org> wrote:

Happy Monday,

When I commit a change to Chromium's master branch, our perf bots generally do an excellent job of telling me if my change regressed metrics we care about. Though, It's not clear to me what our perf testing story is for release branches. Do we have a similar level of coverage for commits on release branches? Or is the expectation that commits to release branches should generally be performance-neutral/already vetted on master?

Better yet, might there be docs/webpages on all of this that I can't seem to be able to find? :)

Thanks!

George

--
You received this message because you are subscribed to the Google Groups "Chrome benchmarking" group.

To unsubscribe from this group and stop receiving emails from it, send an email to benchmarking-dev+unsubscribe@chromium.org.

[Ben Henry]

There are some discussions ongoing currently about improving the effectiveness and efficacy of testing on release branches. +Michael Moss 

Ben

On Tue, Jul 10, 2018 at 7:05 AM Annie Sullivan <sull...@chromium.org> wrote:

+amineer: do you know the status of release branch testing in Chrome? My guess is that we'd want perf testing on the roadmap at some point, but not until we're happy with correctness testing?

On Tue, Jul 10, 2018 at 8:08 AM, Tim Dresser <tdre...@google.com> wrote:

We don't have test coverage in general for release branches, right? My intuition is that if it's good enough for correctness testing, it's probably good enough for us, but I don't have any data to back that up.

On Tue, Jul 10, 2018 at 7:09 AM Ned <nedn...@google.com> wrote:

We currently don't have coverage for commits on release branches. For now, one can still evaluate the impact of their commits to release branch using pert trybot tool (+Dave Tu to confirm)

+Ushesh Desai +Ben Henry +Annie Sullivan +Tim Dresser  for thoughts about how important it's to address this gap.

On Mon, Jul 9, 2018 at 8:23 PM George Burgess <gb...@chromium.org> wrote:

Happy Monday,

When I commit a change to Chromium's master branch, our perf bots generally do an excellent job of telling me if my change regressed metrics we care about. Though, It's not clear to me what our perf testing story is for release branches. Do we have a similar level of coverage for commits on release branches? Or is the expectation that commits to release branches should generally be performance-neutral/already vetted on master?

Better yet, might there be docs/webpages on all of this that I can't seem to be able to find? :)

Thanks!

George

--
You received this message because you are subscribed to the Google Groups "Chrome benchmarking" group.

To unsubscribe from this group and stop receiving emails from it, send an email to benchmarking-d...@chromium.org.

[Dave Tu]

Perf trybots on release branches: Pinpoint supports it in the backend, but there's just no UI for it yet, since nobody has asked for it.