Redfining loop variable semantics - what's the plan?

[Amnon]

Hi Gophers,
Last year there was a discussion about removing one of the 
more common gotchas in Go.

To quote from the discussion:

the problem is that loops like this one don’t do what they look like they do:

var all []*Item

for _, item := range items { all = append(all, &item) }

That is, this code has a bug. After this loop executes, all contains len(items) identical pointers, each pointing at the same Item, holding the last value iterated over. This happens because the item variable is per-loop, not per-iteration: &item is the same on every iteration, and item is overwritten on each iteration. 

https://github.com/golang/go/discussions/56010

What was the resolution of this discussion?
Was the proposed change accepted?
Will it be released in Go 1.21 or 1.22?

It is hard to figure this out from the discussion. There are hundreds of comments,
but there is no clear marking of the resolution (apart from the discussion now being closed) either at the top of bottom of the discussion.

[Sean Liao]

https://go.dev/issue/57969

- sean

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/5d88208e-fbbf-44d5-b693-50deff176fedn%40googlegroups.com.

[Amnon]

Thanks for a very succinct response.

So if I understand the CL, there will be no change in behaviour in 1.21, unless you set GOEXPERIMENT=loopvar

- Amnon

[Eli Bendersky]

On Sat, Mar 25, 2023 at 2:33 AM Amnon <amn...@gmail.com> wrote:

Thanks for a very succinct response.

So if I understand the CL, there will be no change in behaviour in 1.21, unless you set GOEXPERIMENT=loopvar

- Amnon

That's correct. You (and everyone else) can play with this GOEXPERIMENT in 1.21 (or now, if using gotip). The most useful thing to do would be to run your tests with this experiment set and see what breaks.

Looking towards the future, this is also related to the forward compatibility proposal: https://github.com/golang/go/issues/57001

Eli

 

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/aba5e0bd-c676-45f0-a7b7-ce6e23985124n%40googlegroups.com.

[drc...@google.com]

And if there is a problem, let us know.  Probably around the time 1.21 is released we should write up "how to debug this problem if you see it" but we've been working on the tools to automate the search if/when such a bug appears.

[Marcello H]

The "scary" thing is, that if people don't have enough tests, they are probably not aware of such a bug, or can they still be aware somehow?

Op maandag 3 april 2023 om 20:19:33 UTC+2 schreef drc...@google.com:

[Brian Candler]

On Tuesday, 4 April 2023 at 07:57:58 UTC+1 Marcello H wrote:

The "scary" thing is, that if people don't have enough tests, they are probably not aware of such a bug, or can they still be aware somehow?

Do you mean bugs due to the *old* behaviour? You're quite right, the first they may be aware is when a security hole is uncovered, for example:

https://bugzilla.mozilla.org/show_bug.cgi?id=1619047

The proposed semantic change is expected to fix such hidden bugs, whilst being considered unlikely to introduce new ones - but the GOEXPERIMENT flag is a way to test this hypothesis (across codebases which *do* have good test coverage)

[drc...@google.com]

Based on studying large bodies of existing code, you should be about 25x more scared right now that there's an undetected bug in your code from the existing semantics -- especially if you haven't written many tests.  If this change does cause a failure in existing code, we have a tool to help isolate it down to the line, still working on the end-user/programmer packaging for that tool but it is based on one we used internally for compiler debugging.

[xi ool]

Little late but I have to say the rational seems foolish :

1. Currently 25% of gophers who  didn't read the language spec experienced a 'bug' and got unexpected behavior..
2. Solution is to change the language spec and behavior (despite breaking go back compat promise)
3. ...
4. Cut to the future :  25% of gophers who didn;t read the language spec experienced a 'bug' and got unexpected behaviour..

Expect to have to explain why all and all2 are different in this example :

for _, i := range []int{1, 2, 3, 4} {
all = append(all, &i)
}

var j int
for _, j = range []int{1, 2, 3, 4} {
all2 = append(all2, &j)
}

also this :

• He dreams that he is a powerful sorcerer high on top of a pinnacle commanding the stars, planets, and water. Before long, Mickey wakes up to find that the room is filled with water, and despite the cauldron overflowing, the broom is not stopping. Mickey tries to stop the broom without success; it walks right over him, bringing more and more water. Mickey even tries grabbing one of the buckets, but that too fails. Finally, when the water keeps rising, Mickey, in desperation, grabs a huge ax and chops the broom into pieces. Just when it is all over as Mickey is away, the little wooden split pieces, lying quietly on the floor, begin to come alive, stand upright, grow arms out of their sides, and turn into more brooms with buckets of water. They keep going to the vat and filling it up. Mickey tries to get the water out, but finds that there are too many brooms. Mickey goes to a book and looks for a spell to stop the brooms. Mickey finds himself in a whirlpool. Just then, Yen Sid comes in and sees this, and with a wave of his hands, the water descends and the army of brooms is decreased to one broom.

[David Finkel]

On Mon, Apr 10, 2023 at 7:23 PM xi ool <xiio...@gmail.com> wrote:

Little late but I have to say the rational seems foolish :

1. Currently 25% of gophers who  didn't read the language spec experienced a 'bug' and got unexpected behavior..

This premise is a bit suspect.

The current loop-variable behavior bites almost all gophers both those who have and haven't read the spec.

1. Solution is to change the language spec and behavior (despite breaking go back compat promise)
2. ...
3. Cut to the future :  25% of gophers who didn;t read the language spec experienced a 'bug' and got unexpected behaviour..

Expect to have to explain why all and all2 are different in this example :

for _, i := range []int{1, 2, 3, 4} {
all = append(all, &i)
}

var j int
for _, j = range []int{1, 2, 3, 4} {
all2 = append(all2, &j)
}

Given that j is outside the loop, and i is declared as part of the loop. My intuition before I re-read the language spec (after the first time the current semantics bit me) was that the loop variable should be scoped to the loop if at all possible.

The fact that all2 ends up with 4 identical pointers looks to be exactly intended in this case.

Note that it's not always so obvious where you're taking a reference to a variable, since most methods have pointer-receivers, any method-call that stores a pointer to a field on its receiver can do rather surprising things.

 

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/d5cc7e82-2928-48b1-b4ca-8b7089b2c8aen%40googlegroups.com.

[Axel Wagner]

I would add that Go strives to be a language that is easy to understand.

Any instance where you have to

    a) read the spec or be intimately familiar with the language and

    b) reason out the behavior instead of understanding it at a glance

is a case where that goal has failed. Of course, there will always be such cases (hence the occasional "Go quiz" on Twitter). But this particular case is incredibly common in actual practice. This is not about exotic code, it's about code any Go programmer writes almost every day.

So, "25% of gophers who didn't read the language spec" already is a flawed premise. You shouldn't *have* to read the language spec to understand what Go code does.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CANrC0BiizSBnq5EiadZ5t026-YSWAK8AuFVt%3DCVYp9f2oE9Mng%40mail.gmail.com.

[Jan Mercl]

Resending to the mailing list as that was my intention but I errored
again. Did the gmail UI changed again?

---------- Forwarded message ---------
From: Jan Mercl <0xj...@gmail.com>
Date: Tue, Apr 11, 2023 at 9:11 AM
Subject: Re: [go-nuts] Redfining loop variable semantics - what's the plan?
To: Axel Wagner <axel.wa...@googlemail.com>

On Tue, Apr 11, 2023 at 7:46 AM 'Axel Wagner' via golang-nuts
<golan...@googlegroups.com> wrote:

> You shouldn't *have* to read the language spec to understand what Go code does.

This is the most strange claim I have read about Go ever.

You *must* read the language spec to understand what any programming
language does. You *cannot* just guess because "other languages" do
somethíng or use similar approaches and then be surprised you have
guessed wrong. And as can be seen often, blame the language designers
they didn't meet your unfounded expectations.

What bothers me personally about the proposal and why I think it's a
bad idea: The semantics will be defined by metadata. Any Go code
snippet with such a loop, existing or future, all over the
Internet/books/blogs, ... will become ambiguous in what it does
without those metadata, which are not usually/always available
alongside. You can say "other languages have that problem as well, see
the different C, Perl, Python, you name it... versions". And you would
be right. The point is that the Go 1 compatibility promise gave us the
nice property of Go not suffering from such problems.

I oppose the idea of introducing such a problem into the language
voluntarily, 10+ years later.

Maybe it would be better to bite the bullet, introduce Go 2 and
embrace the shizma. /s

[Axel Wagner]

On Tue, Apr 11, 2023, 09:15 Jan Mercl <0xj...@gmail.com> wrote:

Resending to the mailing list as that was my intention but I errored
again. Did the gmail UI changed again?

---------- Forwarded message ---------
From: Jan Mercl <0xj...@gmail.com>
Date: Tue, Apr 11, 2023 at 9:11 AM
Subject: Re: [go-nuts] Redfining loop variable semantics - what's the plan?
To: Axel Wagner <axel.wa...@googlemail.com>


On Tue, Apr 11, 2023 at 7:46 AM 'Axel Wagner' via golang-nuts
<golan...@googlegroups.com> wrote:

> You shouldn't *have* to read the language spec to understand what Go code does.

This is the most strange claim I have read about Go ever.

You *must* read the language spec to understand what any programming
language does. You *cannot* just guess because "other languages" do
somethíng or use similar approaches and then be surprised you have
guessed wrong. And as can be seen often, blame the language designers
they didn't meet your unfounded expectations.

I disagree strongly. I've never read *any* language spec but the Go one, yet I can read most programs in many programming languages just fine. And I don't think that's at all uncommon. I know people who have been writing and maintaining widely used C software for 20 years or so, who I'm fairly certain habe never read the C spec (and I don't know many people who have). Really, Go is kind of an outlier for even *having* an approachable spec - again a sentiment I've heard a lot.

I would expect any programmer who did the Go tour to be able to productively contribute to existing Go projects. Again, this is a strength and IMO explicit design goal of Go, to be easy to pick up and use productively.

The spec exists to be able to disambiguate edge cases and answer questions about *why* a certain program behaves the way it does when you don't understand the behavior. Not to be a prerequisite to use the language.

I strongly believe none of this is controversial.

What bothers me personally about the proposal and why I think it's a
bad idea: The semantics will be defined by metadata. Any Go code
snippet with such a loop, existing or future, all over the
Internet/books/blogs, ... will become ambiguous in what it does
without those metadata, which are not usually/always available
alongside. You can say "other languages have that problem as well, see
the different C, Perl, Python, you name it... versions". And you would
be right. The point is that the Go 1 compatibility promise gave us the
nice property of Go not suffering from such problems.

I oppose the idea of introducing such a problem into the language
voluntarily, 10+ years later.

Maybe it would be better to bite the bullet, introduce Go 2 and
embrace the shizma. /s

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAA40n-UQaK1dBvo-690Z1VZDZ%2B%2BKL7gQGWi5Mq3G4x4mrxZuyg%40mail.gmail.com.

[Axel Wagner]

FTR I have been using Go for 10 years and I certainly *have* read the spec - a lot, as you're well aware. I still occasionally make this mistake. Being able to explain *why* code is buggy doesn't prevent me from introducing a specific bug.

[tapi...@gmail.com]

The plan for “for;;” is much more risky than “for-each”, because “for;;” has more variation uses in reality.

I have no doubt that many non-bug compatibility-broken or performance-degradation cases will be found in the GOEXPERIMENT phase.

[tapi...@gmail.com]

Not to mention that the change for "for;;" is so counter-intuitive and violates the try-to-be-explicit principle in Go.