Re: Proposals for Go 1.15

[Daniel Martí]

I've just given the latest blog post a read - it's great that more
language proposals keep on moving forward. Nice work!

I want to bring up one idea that covers two of the three proposed
changes, so I thought I'd bring it up here instead of having to choose
either of their issue threads.

> Since removing this conversion is not a backwards-compatible change,
> we propose to start with a vet error instead.

> Reporting a compiler error in this case is not a backwards-compatible
> change, thus we also propose to start with a vet error instead.

Two years ago, when the discussion for these issues took place, I'd 100%
agree. However, as of today, we do have a mechanism in place for this;
the language version defined in a module's go.mod file.

The implementation would be pretty straightforward. If a module's
language version is 1.15 or later, both of those edge cases would start
erroring. As far as I understand, this would be fully backwards
compatible, and would not require the extra vet work at all.

To my mind, the only problem with this approach is that there is still a
bit of confusion out there about the purpose of the language version
line in go.mod. We can fix that, however - if the docs are lacking, we
can improve them. And we can make this plan abundantly clear in the
changelog pages for 1.14 and 1.15.

If this method was considered and discarded, I'd like to hear the
background behind that. I did skim the threads after reading the blog
post, but could not find any mention of it.

Thanks!

[thepud...@gmail.com]

Hello Daniel,

If you or anyone else interested in this topic has not read this recently, you might want to re-read the 2-3 pages of the "Go 2 Transition" document starting at the "Language Changes" section through "Language Redefinitions" section:

   https://github.com/golang/proposal/blob/master/design/28221-go2-transitions.md#language-changes

That "Go 2 transition" document is 15 pages, but the majority of the document is a discussion of other languages & rejected alternative approaches, and those 2-3 pages referenced above cover how removals from the language can be done.

My understanding is that the intent of the `go` language version directive in `go.mod` files is intended to include the ability to remove features from the language (along the lines of what you describe, if I followed).

Also, you'll note one of the examples discussed there is removing the conversion string(i) for an integer.

That said, I think there also might be a view that using the `go` language version directive in `go.mod` to remove a feature of the language is not a tool to be wielded lightly, which might explain the vet-first thinking for those two issues, but I am not sure.

In any event, I'm sure others will have more illuminating things to say -- I mainly just wanted to provide the reference above in case useful to the discussion.

Best,

thepudds

[Daniel Martí]

> Also, you'll note one of the examples discussed there is removing the
> conversion string(i) for an integer.

Yes - thanks for helping provide a link! This is indeed the idea I was
referring to.

> That said, I think there also might be a view that using the `go` language
> version directive in `go.mod` to remove a feature of the language is not a
> tool to be wielded lightly, which might explain the vet-first thinking for
> those two issues, but I am not sure.

I have wondered this as well. But I don't think that would make sense,
because if such a vet check is enforced by 'go test', then it's
practically the same as having the compiler reject the code. And
otherwise, it's not really a significant step towards removing the
feature.

Like you, I'd like more context here :) That was the main purpose of the
email.

[Bryan C. Mills]

A vet check is a much more reversible starting point: if we decide that removing the feature was a bad idea, we can revert the vet change and not need to maintain any variations in the compiler going forward.

On the other hand, a language version (strictly speaking) is forever: if we made a program invalid for `go 1.15`, that program must remain invalid for `go 1.15` when compiled with Go 1.16. Otherwise, folks on the 1.16 release won't know that they are breaking users on 1.15. (A vet check has no effect on outside users of a package, so a 1.16 user introducing a vet warning for 1.15 users is not nearly as big a deal as actually breaking the build under 1.15.)

So I tend to agree that adding the vet check is a better starting point — it's easy to ramp up to an actual language change from there once we get more experience with the full implications of the change.

--
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-dev/20200129171729.iivrpg3jduzn6e7s%40carbon.localhost.

[thepud...@gmail.com]

Separately, one nuance for using the `go` language version directive in `go.mod` is I think (but am not sure) that code without an explicit `go.mod` defaults to having its Go language version treated as the version of the toolchain. This would mean that old code that has not yet created a `go.mod` would be treated as the Go 1.15 language when using the Go 1.15 toolchain. This could cause things to break for older direct or indirect dependencies. As more calendar time goes by, presumably the percentage of projects with a `go.mod` file will go up, and hence hopefully the impact of older code not declaring its Go language version would hopefully trend downward over time.

There is also the frequency at which the vet check would trigger, vs. how often code is compiled.

For example, I suspect most gophers do not routinely run "go test all" to run tests for all of the direct and indirect dependencies of their project, or at least not once a moderate number of dependencies are used.

A related question is if the two new vet checks would be part of the "high-confidence subset" of vet checks that are used by default by 'go test'. My guess is the answer would be "no", at least to start, but I don't know if that has been stated explicitly somewhere for these two changes.

Best,

thepudds

To unsubscribe from this group and stop receiving emails from it, send an email to golan...@googlegroups.com.

[Robert Griesemer]

Thanks, Dan, for your question.

Apologies for not laying out the rationale clearly in the blog post; I can see that that would have been really good to spell out more explicitly.

We definitely want to make those language changes and we do plan to use the language version directive in the future. And, as you mentioned, two years ago, we did think that we would be there at this point.

The main reasons why we decided to start with the vet checks are:

- Not everybody has switched to modules yet. We do know from surveys that the vast majority has (we estimate some 70-80%), but it is not the default mechanism. And even if everybody was using modules, introducing these changes would require .mod files to be adjusted to avoid breakages.

- We really don't want to break people's code (including .mod files) if we don't absolutely have to. The proposed changes are what I'd consider fine-tuning of the language, they are neither urgent nor very important (as in, say, fixing a security issue). The vet checks are a heads-up that change is coming soon; the grace period allows users to adjust their code as they see fit and on their schedule.

Generally, this last point of introducing breaking language changes gradually, starting with a vet check if at all possible, seems like a good process to follow. It is a gentle way to ease users into a change, on their time frame. It also allows us to back-step if a problem comes up that we couldn't foresee before testing the change across large amounts of code.

The overarching principle is that we do not want to break people's code - because that cost is huge. But if we do break code, we should do it with plenty of warning and time to adjust. For the proposed changes, there is no rush.

Hope that clarifies things.

- gri

--
You received this message because you are subscribed to the Google Groups "golang-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-dev/20200129163738.hy6c6d2d54hezpun%40carbon.localhost.

[roger peppe]

On Wed, 29 Jan 2020 at 18:32, Robert Griesemer <g...@golang.org> wrote:

The vet checks are a heads-up that change is coming soon; the grace period allows users to adjust their code as they see fit and on their schedule.

Is the plan to enable this vet check by default when running go test?

[Robert Griesemer]

My vote would be yes, as those checks would be on by default. And it's trivially possible to run go test -vet=off if necessary.

- gri

[Daniel Martí]

Thanks for the detailed explanation! I'm still not quite sure I agree,
though.

If an author has been avoiding modules on purpose on previous Go
versions, and avoided adding a go.mod file, it's unlikely modules would
suddenly turn on by themselves with Go 1.15. For the vast majority of
that 20% of users, I'm think we'd only surprise them if we switched the
default to GO111MODULE=on.

On the other hand, if they already have a go.mod file, previous Go
versions (1.12, 1.13, 1.14) would already have added a version line. So
they would only jump to 'go 1.15' if they upgraded straight from
1.11 to 1.15. If they did that, I imagine they'd have bigger problems :)

And, in either of those cases, a truly simple fix is available;
downgrade the language version to 1.14 or earlier. This is the kind of
thing I meant by better documentation, particularly when it comes to the
changelogs. I'd even argue that this is an easier fix than -vet=off, as
it can be used once and it's valid forever. Telling entire teams of
developers to always use -vet=off or set up GOFLAGS with it doesn't seem
like a solution, in comparison.

So I don't think vet would ease the transition or avoid breakage. I do
agree that reverting a released vet check is easier than reverting a
language feature removal, but that doesn't seem like a strong argument
to not use language versions for their intended purpose.

If we want to go ahead with removing language features, I'd hope that
we're sure that it's the right thing to do, and that a large corpus of
Go code has already been inspected. There are also the betas and RCs, so
we could strongly encourage users (particularly enterprises) to give
them a try.

If anyone refuses to try modules, or try Go 1.15, and they want to
complain a year later when they realise they didn't want either feature
to be removed... I'm not sure that giving them an extra six months to
provide feedback would help.

I realise I'm being stubborn about not having vet fill this role, so
I'll stop here :)

[Bryan C. Mills]

In thinking about this thread, I discovered a couple of remaining issues with our incremential-migration and vendoring support for `go` version directives in `go.mod` files.

I've filed https://golang.org/issue/36876 and https://golang.org/issue/36875 for discussion.

--
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-dev/20200129204232.pi3lld5wccyb43pc%40carbon.localhost.

[thepud...@gmail.com]

 > And, in either of those cases, a truly simple fix is available; downgrade 

 > the language version to 1.14 or earlier.

Hello Daniel,

I might have misunderstood, but one thing I didn't see in your description is the impact of dependencies (direct or indirect) that do not yet have a `go.mod` file.

You likely know this already, but as far as I understand, the `go` language version directive in `go.mod` only controls the language version for the code within that module itself, and a `go.mod` file does not control the language version for that module's dependencies. 

That means of course that the impact of something like 20% of projects not yet having a `go.mod` is compounded when a medium-sized project can have >100 direct and indirect dependencies. And while it is possible to add fork a dependency to add a go.mod file to change its `go` language version or other solutions, it seems that might be unnecessary work, especially when a more gradual option is available.

Best,

thepudds

[Robert Griesemer]

We may need to use the -lang version flag sooner than expected, depending on what we decide to do with this:  

https://golang.org/issue/28591#issuecomment-579993684

- gri

[thepud...@gmail.com]

Regarding the question of whether or not these two new vet checks should be enabled when running 'go test':

  > My vote would be yes, as those checks would be on by default. And it's trivially possible to run go test -vet=off if necessary.

One the one hand, these are somewhat smaller changes, so maybe the answer is not critical for this next release, but on the other hand, these vet changes have to some degree been labeled as precursors to possible "Go2" breaking changes, and it might be worth asking what the pattern should be in the future for other "Go2" breaking changes, and whether or not that future pattern should be followed now.

From my personal perspective, slower is usually better when it comes to the pace of breaking changes, and a more gradual introduction might help with a calmer reaction from both inside and outside the Go community.

For example, it could be that a change like this first lands in 'go vet' but not triggered by 'go test', and then starts to be later triggered by 'go test' as well, and then later the language itself is adjusted and the compiler starts failing.

The phrase "Go2" is already a bit of a trigger for some people who have seen problems with other languages incrementing their major versions. Other people might hear "They are considering making breaking changes in Go2", and not realize the planned approach means their code can continue to run in a backward-compatible way, and that they can even mix different language versions in a single build, at least as far as I understand.

Related -- at some point, it might be worth finding someone to slice Ian's "Go 2 Transition" document into something more bite-sized for a blog on golang.org to preemptively help spread the word a bit more on what "breaking changes" really mean in a Go2 context.

(Finally, before finishing posting this, I had some additional comments here that then I deleted from here and moved to https://github.com/golang/go/issues/36875#issuecomment-579967416 which is the issue Bryan just opened based on this thread).

Best,

thepudds

[haskell...@yahoo.de]

On Wednesday, 29 January 2020 19:32:16 UTC+1, gri wrote:

- Not everybody has switched to modules yet. We do know from surveys that the vast majority has (we estimate some 70-80%), but it is not the default mechanism. And even if everybody was using modules, introducing these changes would require .mod files to be adjusted to avoid breakages.

- We really don't want to break people's code

Projects without a .mod file could be compiled in 1.14 language level mode forever and projects with .mod files would compile with the language level

specified in their .mod files. So people who want the new language features (whether added or removed) have to switch to modules.

 

[Ian Lance Taylor]

I don't really disagree with anything you say. But we want to be
conservative about language removals, and it seems reasonable to me to
claim that a change to vet is more conservative than a change to the
compiler. A change to vet has an easy workaround, whereas working
around a change in the compiler can be awkward if you are using any
newer language features. So I think the conservative approach is to
use vet. If that goes well, maybe we can be more adventurous in the
future.

Ian