Proposal: Restore original meaning of "go build -a", add new flag instead

[Ian Lance Taylor]

Issues 9344 and 9369 are about the fact that for Go 1.4 we changed the
behaviour of the -a flag to go build/install/etc. Previously the -a
flag meant that a package would be rebuilt (reinstalled, etc.) even if
it were up to date. In Go 1.4 we changed that so that it did not
apply to the standard packages. This meant that the previously common
idiom "go build -a -tags netgo" no longer rebuilt the standard library
with the netgo tag.

The reason for the change was so that it would be possible to rebuild
and reinstall all packages with a new release of Go, as discussed in
issues 4106, 6424, and 8290.

I think that the change to the -a option makes the option harder to
understand. It prevents people from rebuilding the standard library
even though that is clearly something that people sometimes want to
do. I think most people would find it hard to predict the current
interaction between -a and an explicit "std" on the command line.

I propose that we revert -a back to the behaviour it had before Go
1.4. I propose that we instead add a new boolean option, -buildstd,
meaning to rebuild the standard library as appropriate. With
-buildstd, standard library packages act like any other package. With
-buildstd=false, standard library package dependencies are not built.
We can preserve an approximation of the current 1.4 behaviour by
having -buildstd default to true during development and to false for a
release.

I propose that this change be made for the Go 1.4.1 release, so that
people who want to build a pure Go net package can use "go build -a
-tags netgo -buildstd".

Advantages: 1) Fixes issues 9344 and 9369 by giving people a way to
rebuild the standard library with the netgo tag; 2) retains the fix
for issues 4106, 6424, and 8290 by giving people a way to rebuild and
reinstall all packages except the standard library; 3) simplifies the
meaning of the -a option back to what it used to be.

Disadvantages: Adds Yet Another Flag to the go tool.

Ian

[minux]

I have a counter-proposal:

add support for general package negation to cmd/go. e.g.

go install -a all -std

should do the right thing (actually, not quite, see below).

yes, this is more work, but it's also a new general capability that could

be useful in other scenarios, not limited to just ignoring std packages.

And it avoids adding new options to the Go tool. The only problem

besides implementation complexity is what if a package import starts

with a dash. And in that case, we can use !, which is allowed to be

reserved, according to the spec: http://golang.org/ref/spec#Import_declarations

(Choosing ! has the unpleasant effect that it collides with shell history

expansion meta-character, but that's not a big issue, just quote/escape

the ! character.)

And then we can restore the -a flag, and advise people of binary distribution

to use go install -a all -std to rebuild all their packages.

Rationale for this is that no only std packages need special treatment,

some packages (commands) in x/tools also need to be skipped because

they are also intended to be installed into $GOROOT (godoc, cover, vet,

for example), so the full command line should be:

go install -a all -std -golang.org/x/tools/cmd/{godoc,cover,vet}

Having this general mechanism will save us the trouble should we introduce

more special commands into the tools sub-repo in the future.

[Ian Lance Taylor]

On Tue, Jan 6, 2015 at 5:58 PM, minux <mi...@golang.org> wrote:
> I have a counter-proposal:
>
> add support for general package negation to cmd/go. e.g.
> go install -a all -std
> should do the right thing (actually, not quite, see below).

I'm not clear on how this interacts with "go build -i". The go tool
right now deals with explicitly specified packages and with package
dependencies. Your proposal seems to be mainly about explicitly
specific packages. It's not clear how it affects package
dependencies.

Ian

[Dave Cheney]

I support Ian's proposal.

I would like to see the existing (Go 1.3 and prior) behaviour of -a
restored, and shipped in Go 1.4.1. This should be done before, and
independent of any other proposals to address the issues which
prompted the change to -a in Go 1.4

> --
> You received this message because you are subscribed to the Google Groups "golang-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[minux]

Yeah, this point is debatable, as I can see arguments from both sides.

my original plan is that only explicitly listed packages (or set of packages,

std, ./..., etc) are excluded. 

[minux]

On Tue, Jan 6, 2015 at 9:22 PM, Dave Cheney <da...@cheney.net> wrote:

I support Ian's proposal.

I would like to see the existing (Go 1.3 and prior) behaviour of -a
restored, and shipped in Go 1.4.1. This should be done before, and
independent of any other proposals to address the issues which
prompted the change to -a in Go 1.4

Once the new flag to cmd/go is added, removing it in Go 1.5 and replacing

it with a new mechanism might be even worse for the user than the current

situation:

Users of Go 1.4.1 will need to learn a new flag that is due to be removed

in 1.5. I think that's worse than not being to reinstall std packages.

If we want a safe solution for 1.4.1, just revert the -a restriction commit

as it's not really working anyway (go1.4 install -a std will still build and

install a few packages and programs.)

[Dave Cheney]

On Wed, Jan 7, 2015 at 1:27 PM, minux <mi...@golang.org> wrote:
>
> On Tue, Jan 6, 2015 at 9:22 PM, Dave Cheney <da...@cheney.net> wrote:
>>
>> I support Ian's proposal.
>>
>> I would like to see the existing (Go 1.3 and prior) behaviour of -a
>> restored, and shipped in Go 1.4.1. This should be done before, and
>> independent of any other proposals to address the issues which
>> prompted the change to -a in Go 1.4
>
> Once the new flag to cmd/go is added, removing it in Go 1.5 and replacing
> it with a new mechanism might be even worse for the user than the current
> situation:
> Users of Go 1.4.1 will need to learn a new flag that is due to be removed
> in 1.5. I think that's worse than not being to reinstall std packages.

Users of Go 1.4.1 will have to learn nothing, their pre existing
knowledge of how -a worked (and did not work), from Go 1.3.x and
earlier will be correct.

[minux]

On Tue, Jan 6, 2015 at 9:32 PM, Dave Cheney <da...@cheney.net> wrote:

On Wed, Jan 7, 2015 at 1:27 PM, minux <mi...@golang.org> wrote:
>
> On Tue, Jan 6, 2015 at 9:22 PM, Dave Cheney <da...@cheney.net> wrote:
>>
>> I support Ian's proposal.
>>
>> I would like to see the existing (Go 1.3 and prior) behaviour of -a
>> restored, and shipped in Go 1.4.1. This should be done before, and
>> independent of any other proposals to address the issues which
>> prompted the change to -a in Go 1.4
>
> Once the new flag to cmd/go is added, removing it in Go 1.5 and replacing
> it with a new mechanism might be even worse for the user than the current
> situation:
> Users of Go 1.4.1 will need to learn a new flag that is due to be removed
> in 1.5. I think that's worse than not being to reinstall std packages.

Users of Go 1.4.1 will have to learn nothing, their pre existing
knowledge of how -a worked (and did not work), from Go 1.3.x and
earlier will be correct.

Ian's proposal also introduces a new flag -buildstd, so actually the correct

way for 1.4.1 user to reinstall net package is:

go install -tags netgo -a -buildstd net

This is in contrast to how Go 1.3 and earlier worked, where simply:

go install -tags netgo -a net

should do.

For 1.4.1, my position is to just revert https://golang.org/cl/151730045.

For 1.5, we are free to discuss whatever new mechanism we want to

solve #8290.

To solve #8290, perhaps the easiest solution would be to introduce

a new package set called gopath or something. However, the problem

of golang.org/x/tools/cmd/{godoc,vet,cover} being installed into $GOROOT

is still not solved. (Perhaps it's a just a bad idea to make those commands

go into $GOROOT, I see no problem of installing them into $GOPATH/bin)

[Dave Cheney]

I apologise, I mistook your position for something more involved. I
agree, just revert CL 151730045 for 1.4.1

> For 1.5, we are free to discuss whatever new mechanism we want to
> solve #8290.

I don't see #8290 as a bug. Why can't the user just do

cd $GOPATH
rm -rf pkg
cd src
go install ./...

[minux]

On Tue, Jan 6, 2015 at 9:43 PM, Dave Cheney <da...@cheney.net> wrote:

On Wed, Jan 7, 2015 at 1:40 PM, minux <mi...@golang.org> wrote:

> For 1.5, we are free to discuss whatever new mechanism we want to

> solve #8290.

I don't see #8290 as a bug. Why can't the user just do

cd $GOPATH
rm -rf pkg
cd src
go install ./...

Right. In the shell command sequence is not acceptable, just write a

go gettable command to do that for you, and problem solved.

[glyn.no...@gmail.com]

Reverting to 1.3 behaviour in 1.4.1 sounds good to me. Saves people having to unlearn static linking techniques which worked in 1.3.

[Russ Cox]

I think the Go 1.4 behavior of -a is far better than the old behavior
for the majority of users (just not the majority of people on
golang-dev). The fact is that few users rebuild the standard library
compared to rebuilding all their other code.

If we need to provide a way to force rebuild of the standard library,
why not make *that* the new flag? -rebuildstd

Russ

[Ian Lance Taylor]

The effect of that is equivalent to my proposal. In both cases the
behaviour remains the same as 1.4 unless a new additional flag is
specified.

I think the change and additional flag that I'm proposing is easier to
understand than an exception on an exception.

Ian

[Russ Cox]

On Wed, Jan 7, 2015 at 1:31 PM, Ian Lance Taylor <ia...@golang.org> wrote:

On Wed, Jan 7, 2015 at 8:50 AM, Russ Cox <r...@golang.org> wrote:
>
> I think the Go 1.4 behavior of -a is far better than the old behavior
> for the majority of users (just not the majority of people on
> golang-dev). The fact is that few users rebuild the standard library
> compared to rebuilding all their other code.
>
> If we need to provide a way to force rebuild of the standard library,
> why not make *that* the new flag? -rebuildstd

The effect of that is equivalent to my proposal.  In both cases the
behaviour remains the same as 1.4 unless a new additional flag is
specified.

I'm sorry, I missed something. Your mail includes the sentence "I propose that we revert -a back to the behaviour it had before Go 1.4." But that paragraph is a bit confusing to me, rereading it. Is there a 'not' missing anywhere?

Can you restate your proposed change?

Thanks.

Russ

[Ian Lance Taylor]

* Revert -a to the behaviour it had before Go 1.4.

* Add a new option -buildstd.
-buildstd=true means that the standard library packages act like any
other package.
-buildstd=false means that standard library packages are never
considered to be stale.
In a release build, the default is -buildstd=false.
In a development build, the default is -buildstd=true.

The effect is that in a release build, "go build -a ..." rebuilds all
packages except the standard library packages.

The old effect of "go build -a -tags netgo" is now achieved via "go

build -a -tags netgo -buildstd".

Ian