code review 38720044: go.tools/dashboard/builder: do not fatal on subrepo fet... (issue 38720044)

[js...@google.com]

Reviewers: adg,

Message:
Hello a...@golang.org (cc: golan...@googlegroups.com),

I'd like you to review this change to
https://code.google.com/p/go.tools


Description:
go.tools/dashboard/builder: do not fatal on subrepo fetch errors

When fetching repos it is not uncommon for a 500 or 503 to be returned
from code.google.com. When this happens, log the error and continue so
that we try again later, rather than treating this as fatal.

Please review this at https://codereview.appspot.com/38720044/

Affected files (+2, -1 lines):
M dashboard/builder/main.go


Index: dashboard/builder/main.go
===================================================================
--- a/dashboard/builder/main.go
+++ b/dashboard/builder/main.go
@@ -492,7 +492,8 @@
for _, pkg := range dashboardPackages("subrepo") {
pkgmaster, err := vcs.RepoRootForImportPath(pkg, *verbose)
if err != nil {
- log.Fatalf("Error finding subrepo (%s): %s", pkg, err)
+ log.Printf("Error finding subrepo (%s): %s", pkg, err)
+ continue
}
pkgroot := &Repo{
Path: filepath.Join(*buildroot, pkg),

[dvy...@google.com]

I would add some backoff, because it's usually either rate limiting or
the repo is down.

https://codereview.appspot.com/38720044/

[js...@google.com]

On 2013/12/10 13:32:22, dvyukov wrote:
> I would add some backoff, because it's usually either rate limiting or
the repo
> is down.

We already sleep once we've tried all of the subrepos, so I'm not sure
that we gain much by adding an additional backoff and/or retries. In
most cases it seems that we fail the occasional subrepo case, then sleep
and pick it up next time around. If we wanted to implement backoff then
it would probably make sense to do it for the existing sleep case.

https://codereview.appspot.com/38720044/

[a...@golang.org]

LGTM

I'm happy with the simple approach.

https://codereview.appspot.com/38720044/

[js...@google.com]

*** Submitted as
https://code.google.com/p/go/source/detail?r=efda98edbd15&repo=tools ***

go.tools/dashboard/builder: do not fatal on subrepo fetch errors

When fetching repos it is not uncommon for a 500 or 503 to be returned
from code.google.com. When this happens, log the error and continue so
that we try again later, rather than treating this as fatal.

R=adg, dvyukov
CC=golang-dev
https://codereview.appspot.com/38720044


https://codereview.appspot.com/38720044/