The build failures email flood is finally over

6 views
Skip to first unread message

Raphael Kubo da Costa

unread,
Feb 17, 2023, 4:29:52 AM2/17/23
to chromium-...@chromium.org
Hi all,

You've probably noticed that there's been a huge flood of emails from
luci-notify lately with reports of build failures in the
"publish_tarball" and "Build From Tarball" bots. Thankfully, they're
finally over now.

This ended up happening for a few separate, somewhat independent reasons:

* The notification settings were changed recently
(https://crrev.com/c/4226079) and this list was set to receive emails
when the "Build From Tarball" bot failed.

* The "Build From Tarball" bot was not totally green back then though. I
landed a few changes to fix some of the failures
(https://crrev.com/c/4231332, https://crrev.com/c/4239837,
https://crrev.com/c/4236459), but it took a while for some of the
changes to be rolled in due to some other, unrelated issues on the
Chromium infra side.

* On the "publish_tarball" side, https://crrev.com/c/4228579 landed to
fix https://crbug.com/1413633, but it ended up uncovering a lot of older
releases (since M74) that had not been processed in the past due to
limitations of the old API the bot was using. This caused several
hundreds of new builds to be queued, including builds for very old
releases which the "publish_tarball" infrastructure no longer supports
and which caused the bot to fail. I kind of fixed this in
https://crrev.com/c/4235127 but by then a lot of ancient builds had
already been queued. thomasanderson@ also landed
https://crrev.com/c/4237587 to process only Linux releases rather than
every non-iOS release.

After about a week and all the changes above, both bots seem to have
processed all queued builds and everything looks calm and green again,
and I can finally change my membership settings to start receiving
emails from this list again :-)

sL1pKn07 SpinFlo

unread,
Feb 17, 2023, 4:35:35 AM2/17/23
to Raphael Kubo da Costa, chromium-...@chromium.org
--
You received this message because you are subscribed to the Google Groups "chromium-packagers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-packag...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-packagers/df03bd5a-7e05-552c-ecfb-e49c8a356089%40intel.com.

THANKS!

greetings
Reply all
Reply to author
Forward
0 new messages