Tracking Chromium's build time

[Hans Wennborg]

We've set up a simple system that tracks Chromium's build time here:
https://commondatastorage.googleapis.com/chromium-browser-clang/build-time.html
(go/chrome-build-time for Googlers).

The graph shows the CPU time required to do a clean build of the
'chrome' target with default gn args on Linux (that is, a debug build
without goma).

Builds are done on a virtual machine and revisions are only built
once, so there is some noise, but large changes and overall trends
should be visible.

For example, it shows that while build time has increased
significantly over the years, it actually decreased in 2019, and we
hope to achieve that in 2020 also.

Some interesting revisions are highlighted with red dots. If you land
such a revision, please let me know and I can add it to the highlights
list.

If someone wants to do further analysis, the raw data (build time,
build log, .ninja_log, and .ninja_deps) for each build is publicly
available in the gs://chromium-browser-clang/build-time-data/ GCS
bucket.

Please let me know what you think!

Thanks,
Hans

[Aaron Gable]

Cool! Do you have a link to the system producing this data? Is this simply a compile-only builder on the perf waterfall with a scheduler that doesn't let coalesce build requests so it builds every revision?

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
    http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAB8jPhe8zyxD17Ot7weqNgLh4r0NfPwB5oe9eT83hsADvYSQ3g%40mail.gmail.com.

[Hans Wennborg]

It's just some shell scripts running on a VM (I can send you a link
off-list); it's not using any Chrome infrastructure.

If Chrome infra could build something more proper to track compile
time, that would be amazing, but for now we opted for something quick
and dirty to get it running.

The machine can't keep up with every revision, it just builds as fast
as it can which currently means one build roughly every 40 minutes.

[Dean Berris]

I'm keen to have this data in the Chromeperf Dashboard so we can also do anomaly detection on this. I suspect we can get this data even from the current builders, like how the binary size data is also being uploaded (and alerting on).

Thanks for sharing, Hans!

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAB8jPhdsDUowER%3Dq3nBKrkqSiDjXobWU4e0jvSZ9-oHgeqPa1w%40mail.gmail.com.

[Nico Weber]

(FWIW, we asked to get this on the perf bots for a long time (eg https://bugs.chromium.org/p/chromium/issues/detail?id=224814). Eventually we figured having something simplistic is better than not having something fancy. If this motivates build time tracking on the perf bots, that'd be great though! Please do make sure to track a bot that does local non-goma clobber builds.)

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CABFSQmW8JYFe7_VaNaySpXkuEE8L3PnWUq5g3KgQdNSFxnWk7Q%40mail.gmail.com.

[Dean Berris]

Thanks Nico for pointing that out -- I'll work with +Benjamin Pastene on a way to get builders that don't use GOMA to upload timings to the Dashboard.

[Hans Wennborg]

Sounds great! One deficiency of my approach is that it only does
Linux, but getting numbers on other platforms would be very valuable
too. And running it on proper infra is probably also a good idea :-)

[Hans Wennborg]

Did anything ever come of this?

My hacky system caught a large build-time regression last week
(https://commondatastorage.googleapis.com/chromium-browser-clang/build-time.html#from:2020-11-30;to:2020-12-07).

It would be amazing to have proper infrastructure to track and flag
those kinds of regressions.

On Tue, May 12, 2020 at 4:54 AM Dean Berris <dbe...@chromium.org> wrote:
>

[Yuke Liao]

+Gary Tong who is also interested in solving this problem.

[Bruce Dawson]

I'm glad that your system caught that regression. What was the cause?

I hope that we can get a more official system.

[Hans Wennborg]

On Tue, Dec 8, 2020 at 2:34 AM Bruce Dawson <bruce...@google.com> wrote:
>
> I'm glad that your system caught that regression. What was the cause?

It was a seemingly innocuous change
(https://chromium-review.googlesource.com/c/chromium/src/+/2544938)
that caused v8.h to be included in more files.

[Erik Staab]

(resending from my chromium.org acct)

On Wed, Dec 9, 2020 at 8:58 AM Erik Staab <est...@google.com> wrote:

On Mon, Dec 7, 2020 at 5:36 PM 'Bruce Dawson' via Chromium-dev <chromi...@chromium.org> wrote:

I'm glad that your system caught that regression. What was the cause?

I hope that we can get a more official system.

Gary brought this up in our CQ speedup meeting and we were trying to decide what to do since the local build is a proxy for what we care most about (the remote build in CQ).

I think we can help with the infra piece but will probably need help from someone else to monitor it. I see there's discussion about using the perf dashboard to track this, which I know very little about at the moment.

Let's go back to tracking this in crbug.com/224814.

Thanks,

Erik

 

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/9e869a8d-aa55-4fa9-ad9c-d5829a10280an%40chromium.org.