regular failure of go.googlesource.com pulls

Dan Kortschak

unread,

Aug 27, 2015, 6:46:30 PM8/27/15

to golang-nuts

I have seen over the past few weeks failures of go get with golang.org
packages due to go.googlesource.com 500 errors. e.g.

```
Cloning into '/home/daniel/Development/src/golang.org/x/exp'...
fatal: unable to access 'https://go.googlesource.com/exp/': The requested URL returned error: 500
package golang.org/x/exp/shiny/font: exit status 128
```

What is likely the cause of this and can it be either stopped or worked
around?

Brad Fitzpatrick

unread,

Aug 28, 2015, 10:27:15 AM8/28/15

to Dan Kortschak, golang-nuts

I've pinged the Gerrit hosting folk and asked.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Dan Kortschak

unread,

Aug 28, 2015, 6:38:33 PM8/28/15

to Brad Fitzpatrick, golang-nuts

Thanks.

Dan Kortschak

unread,

Sep 13, 2015, 8:44:26 AM9/13/15

to Brad Fitzpatrick, golang-nuts

Any news on this. It's still happening.

And what's worse, I can't just pull from github because if I do, I get
this

```
package golang.org/x/image/math/fixed: golang.org/x/image is a custom
import path for https://go.googlesource.com/image,
but /home/daniel/Development/src/golang.org/x/image is checked out from
https://github.com/golang/image.git
```

This seems unnecessarily strict. Why is the source of the code blocking
the build? Shouldn't it just block if the package path does not match
the import comment?

Brad Fitzpatrick

unread,

Sep 13, 2015, 12:29:47 PM9/13/15

to Dan Kortschak, golang-nuts

What matters is where it is in your $GOPATH, not where you pulled it from.

The problem is not pulling from Github, but using `go get` and it placing it in the correct spot (for github) but you want it to be in the spot where for golang.org/x/*.

Dan Kortschak

unread,

Sep 13, 2015, 5:30:33 PM9/13/15

to Brad Fitzpatrick, golang-nuts

It was in the same location in my GOPATH and my file system globally. That error is a result of editing the .git/config so it pulls from github instead of googlesource - because the latter is regularly broken.

Dan Kortschak

unread,

Sep 13, 2015, 6:36:01 PM9/13/15

to Brad Fitzpatrick, golang-nuts

To demonstrate this more clearly. Here I've deleted the the x/image repo
and then cloned from github. GOPATH is ~/Development.

~/Development/src/golang.org/x $ git clone
https://github.com/golang/image.git
Cloning into 'image'...
remote: Counting objects: 920, done.
remote: Total 920 (delta 0), reused 0 (delta 0), pack-reused 920
Receiving objects: 100% (920/920), 2.36 MiB | 610.00 KiB/s, done.
Resolving deltas: 100% (499/499), done.
Checking connectivity... done.
~/Development/src/golang.org/x $ go get -u golang.org/x/image/...
package golang.org/x/image/bmp: golang.org/x/image is a custom import path for https://go.googlesource.com/image, but /home/daniel/Development/src/golang.org/x/image is checked out from https://github.com/golang/image.git
package golang.org/x/image/math/f64: golang.org/x/image is a custom import path for https://go.googlesource.com/image, but /home/daniel/Development/src/golang.org/x/image is checked out from https://github.com/golang/image.git
package golang.org/x/image/font: golang.org/x/image is a custom import path for https://go.googlesource.com/image, but /home/daniel/Development/src/golang.org/x/image is checked out from https://github.com/golang/image.git
package golang.org/x/image/font/plan9font: golang.org/x/image is a custom import path for https://go.googlesource.com/image, but /home/daniel/Development/src/golang.org/x/image is checked out from https://github.com/golang/image.git
package golang.org/x/image/math/f32: golang.org/x/image is a custom import path for https://go.googlesource.com/image, but /home/daniel/Development/src/golang.org/x/image is checked out from https://github.com/golang/image.git

package golang.org/x/image/math/fixed: golang.org/x/image is a custom import path for https://go.googlesource.com/image, but /home/daniel/Development/src/golang.org/x/image is checked out from https://github.com/golang/image.git

On Sun, 2015-09-13 at 21:30 +0000, Dan Kortschak wrote:
> It was in the same location in my GOPATH and my file system globally.
> That error is a result of editing the .git/config so it pulls from
> github instead of googlesource - because the latter is regularly
> broken.
>
> On 14/09/2015, at 1:59 AM, "Brad Fitzpatrick"

David Symonds

unread,

Sep 13, 2015, 7:26:51 PM9/13/15

to Dan Kortschak, Brad Fitzpatrick, golang-nuts

On 14 September 2015 at 07:30, Dan Kortschak

<dan.ko...@adelaide.edu.au> wrote:

> It was in the same location in my GOPATH and my file system globally. That
> error is a result of editing the .git/config so it pulls from github instead
> of googlesource - because the latter is regularly broken.

Yes, this is the http://golang.org/cmd/go/#hdr-Remote_import_paths
mechanism working as it was designed, though clearly not as you'd
prefer. It's enforcing that your upstream is what is declared at the
package path. I proposed https://github.com/golang/go/issues/9532 to
permit multiple upstreams, but it didn't get any traction.

Your best bet at this stage might be to only run
cd $GOPATH/src/golang.org/x/image && git pull

Dan Kortschak

unread,

Sep 13, 2015, 7:29:15 PM9/13/15

to David Symonds, Brad Fitzpatrick, golang-nuts

On Mon, 2015-09-14 at 09:26 +1000, David Symonds wrote:
> Yes, this is the http://golang.org/cmd/go/#hdr-Remote_import_paths
> mechanism working as it was designed, though clearly not as you'd
> prefer. It's enforcing that your upstream is what is declared at the
> package path. I proposed https://github.com/golang/go/issues/9532 to
> permit multiple upstreams, but it didn't get any traction.
>
> Your best bet at this stage might be to only run
> cd $GOPATH/src/golang.org/x/image && git pull
>

In this particular case, that's really not feasible. The go get -u was
on a downstream package to pull all the dependencies.

Now, if googlesource were not consistently broken here. This would not
be a problem, but I think the notion of making a hard dep on a single
point of failure is not a good idea.

David Symonds

unread,

Sep 13, 2015, 7:54:32 PM9/13/15

to Dan Kortschak, Brad Fitzpatrick, golang-nuts

On 14 September 2015 at 09:28, Dan Kortschak

<dan.ko...@adelaide.edu.au> wrote:

> Now, if googlesource were not consistently broken here. This would not
> be a problem, but I think the notion of making a hard dep on a single
> point of failure is not a good idea.

I agree that it'd be better if go.googlesource.com were more reliable.
I'm not sure there's a better option here, though. The whole point of
having a "canonical" upstream is that it's a single source of truth,
which necessarily implies that it is a single point of failure.
Multiple upstreams would help there (my #9532 bug), though as that bug
says you should be able to do "go get -u -f" and make the update
happen anyway.

Dan Kortschak

unread,

Sep 13, 2015, 8:16:43 PM9/13/15

to David Symonds, Brad Fitzpatrick, golang-nuts

On Mon, 2015-09-14 at 09:54 +1000, David Symonds wrote:
> though as that bug
> says you should be able to do "go get -u -f" and make the update
> happen anyway.

That's helpful. Thank you.

s...@google.com

unread,

Sep 14, 2015, 2:43:56 PM9/14/15

to golang-nuts, brad...@golang.org

On Sunday, September 13, 2015 at 5:44:26 AM UTC-7, kortschak wrote:

Any news on this. It's still happening.

On Fri, 2015-08-28 at 07:26 -0700, Brad Fitzpatrick wrote:
> I've pinged the Gerrit hosting folk and asked.
>
>
> On Thu, Aug 27, 2015 at 3:45 PM, Dan Kortschak <
> dan.ko...@adelaide.edu.au> wrote:
>
> > I have seen over the past few weeks failures of go get with golang.org
> > packages due to go.googlesource.com 500 errors. e.g.
> >
> > ```
> > Cloning into '/home/daniel/Development/src/golang.org/x/exp'...
> > fatal: unable to access 'https://go.googlesource.com/exp/': The requested
> > URL returned error: 500

Can you reproduce this reliably?

If so setting GIT_CURL_VERBOSE=1 in your environment before running the offending operation should give us a bit more detail about the HTTP transactions involved.

s...@google.com

unread,

Sep 14, 2015, 5:41:21 PM9/14/15

to golang-nuts, brad...@golang.org, s...@google.com

OP sent me a trace privately. From the symptoms and the trace, it seems to have been caused by a damaged authentication token. I recommended the OP generate a new one.

Brad Fitzpatrick

unread,

Sep 14, 2015, 5:43:20 PM9/14/15

to Shawn Pearce, golang-nuts

Why is that a 500 and not a 403?

Why do you need auth for a public repo?

Shawn Pearce

unread,

Sep 14, 2015, 5:48:38 PM9/14/15

to Brad Fitzpatrick, golang-nuts

On Mon, Sep 14, 2015 at 2:43 PM, Brad Fitzpatrick <brad...@golang.org> wrote:

Why is that a 500 and not a 403?

Its stupid. The authentication API our servers are using is failing with its own internal error rather than a more specific reason like "expired", "revoked", "invalid".

Why do you need auth for a public repo?

You don't. Presumably to contribute through Gerrit, the OP has setup authentication using the recommended path of:

- Create ~/.gitcookies with authentication token

- git config --global http.cookiefile $HOME/.gitcookies

and since its a HTTP cookie, Git sends it on every HTTP request to the go.googlesource.com server.

Reply all

Reply to author

Forward