evaluation of range expression in for-range loop

[Jochen Voss]

Dear all,

The spec at https://go.dev/ref/spec#For_range says

"The range expression x is evaluated once before beginning the loop, with one exception: if at most one iteration variable is present and len(x) is constant, the range expression is not evaluated." 

What does the second half of this sentence mean?

My guess was that this says that in "for i := range [3]int{1, 2, f()} ..." the function f is not called, but I quick experiment shows that this guess is wrong, see https://go.dev/play/p/MXqH_C7mllx .

All the best,

Jochen

[Jan Mercl]

I think the specs are correct: https://go.dev/play/p/yFlD98R8tNg

[Dan Kortschak]

If you have `a := [5]int{1,2,3,4,5}` and something like `for i := range
a {` then the range expression `a` is not evaluated since the length of
the array is a constant and the only information that is needed if the
the length. Similarly, if it were `for _, v := a {` the expression does
not need to be evaluated since the element values can't be mutated
during the iteration. Same for `for range a {`.

On the other hand, if you have `for i, v := range a {` then the
expression *is* evaluated before the start of the loop.

The evaluation of a function in an array is a separate issue.

[Axel Wagner]

https://go.dev/ref/spec#Length_and_capacity says:

The expression len(s) is constant if s is a string constant. The expressions len(s) and cap(s) are constants if the type of s is an array or pointer to an array and the expression s does not contain channel receives or (non-constant) function calls; in this case s is not evaluated. Otherwise, invocations of len and cap are not constant and s is evaluated.

So, in your case, the array contains a function call, so `len(…)` is not constant.

Here is a case where the clause from the spec makes an observable difference:  https://go.dev/play/p/jeCk6vHzf9u

Without that clause, the range loop would have to iterate over the element of the pointee, causing a nil-dereference panic. The clause allows this code to work.

On Fri, Mar 18, 2022 at 12:43 PM Jochen Voss <joche...@gmail.com> wrote:

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/db6d865e-233e-4120-8ce5-5feb832732b6n%40googlegroups.com.

[Jochen Voss]

Thanks Axel,

Now this makes sense to me.

All the best,

Jochen

[Marvin Renich]

* 'Axel Wagner' via golang-nuts <golan...@googlegroups.com> [220318 08:04]:

> https://go.dev/ref/spec#Length_and_capacity says:
>
> > The expression len(s) is constant if s is a string constant. The
> > expressions len(s) and cap(s) are constants if the type of s is an array or
> > pointer to an array and the expression s does not contain channel receives
> > or (non-constant) function calls; in this case s is not evaluated.
> > Otherwise, invocations of len and cap are not constant and s is evaluated.
>
> So, in your case, the array contains a function call, so `len(…)` is not
> constant.

Is the intent of that paragraph in the spec to ensure that side effects
occur? Clearly the length of [3]int{...} is constant, regardless of any
channel receives or non-constant function calls.

Why was the spec not written as:

.... The expressions len(s) and cap(s) are constants if the type of s
is an array or pointer to an array. If the expression s does not
contain channel receives or (non-constant) function calls, then s is
not evaluated. ....

And then in the documentation for the range clause, say something like:

...: if at most one iteration variable is present, the range
expression x is only evaluated if it would be evaluated by len(x).

This would allow

const x = len([3]int{1, 2, f()})

to compile with the obvious side effects, but still result in x being a
constant.

Is there some other reason to force len([3]int{«non-constant
initializer»}) to be non-constant?

...Marvin

[Axel Wagner]

One thing to keep in mind is that Go development was incremental and was done, from early on, by having multiple implementations. Incremental processes often lead to different results than designing a language from a blank slate. Ambiguities in the spec are often discovered only when someone notices that implementations disagree and must be resolved somehow. Sometimes that happens after Go 1 and the resolution must happen in a way that, as much as possible, retains stability guarantees. So, it's not unsurprising if we get non-obvious spec clauses such as this.

Specifically, I can well imagine that this particular case arose, because it was discovered after the fact that one implementation contained an optimization to not evaluate the argument to `len`, while the other didn't and the question of side-effects only arose when someone noticed.

I don't really know the answer to your question. I can't think of anything which would break. That being said, I also don't think it's something that really needs "fixing", as it seems a pretty uncommon concern.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/YjSm83cOcG1cACFP%40basil.wdw.

[Jan Mercl]

On Fri, Mar 18, 2022 at 4:53 PM 'Axel Wagner' via golang-nuts
<golan...@googlegroups.com> wrote:
>
> One thing to keep in mind is that Go development was incremental and was done, from early on, by having multiple implementations. Incremental processes often lead to different results than designing a language from a blank slate. Ambiguities in the spec are often discovered only when someone notices that implementations disagree and must be resolved somehow. Sometimes that happens after Go 1 and the resolution must happen in a way that, as much as possible, retains stability guarantees. So, it's not unsurprising if we get non-obvious spec clauses such as this.
>
> Specifically, I can well imagine that this particular case arose, because it was discovered after the fact that one implementation contained an optimization to not evaluate the argument to `len`, while the other didn't and the question of side-effects only arose when someone noticed.
>
> I don't really know the answer to your question. I can't think of anything which would break. That being said, I also don't think it's something that really needs "fixing", as it seems a pretty uncommon concern.

I don't think there's anything ambiguous here. The very first sentence
here: https://go.dev/ref/spec#Constant_expressions says

""""
Constant expressions may contain only constant operands and are
evaluated at compile time.
""""

f() does not fill that requirement for sure, it's not a constant
operand and it cannot be evaluated at compile time (in the first
approximation).

TBH, I thought that it was all clear from my first answer in this
thread. The compiler error confirms the specs.

-j

[Marvin Renich]

[Sorry, I accidentally set the Reply-To incorrectly in my previous msg;
corrected here. I don't need a CC in responses.]

* Jan Mercl <0xj...@gmail.com> [220318 12:04]:

If you had included the above quote in your original response, I would
have agreed with you. However, your other msg only contained a
playground link that demonstrated that the compiler believed the
expression to be non-constant without giving any explanation of why that
was correct according to the spec.

Axel's original response clearly explained why the spec mandated the
behavior that the OP was questioning.

On the other hand, Axel's response left me wondering why the part of the
spec he quoted was written the way it was, while your quote above at
least superficially justifies the parts of the spec Axel quoted which I
was questioning; thanks.

My question stems from the fact that

var a = [3]{1, 2, f()}
const x = len(a)

compiles with the obvious results, but

const x = len([3]{1, 2, f()})

does not compile. I.e. the value obtained from evaluating a composite
literal expression of array type has a constant length, so why is taking
the len() of that value different from assigning that value to a
variable and then taking the len() of the variable?

I was really looking for the language designers to chime in with a
justification, but I guess I can accept "we don't want constant
expressions to be allowed to have side effects" as the answer.

...Marvin

[Axel Wagner]

On Fri, Mar 18, 2022 at 5:04 PM Jan Mercl <0xj...@gmail.com> wrote:

I don't think there's anything ambiguous here.

I didn't think the spec is ambiguous. I said the spec is the result of a long process of iteration. And during that process, ambiguities where detected and fixed, which might explain why the situation today might exhibit hard to understand corner cases and phrasings.

FWIW I was curious, so before I sent my message, I looked through the git history of the spec to see how this particular section of the spec developed over time. Hoping to find some justification for the way it is currently specified. I couldn't find any good references, so I left it at that.

[Dan Kortschak]

On Fri, 2022-03-18 at 16:53 +0100, 'Axel Wagner' via golang-nuts wrote:
> One thing to keep in mind is that Go development was incremental and
> was done, from early on, by having multiple implementations.
> Incremental processes often lead to different results than designing
> a language from a blank slate. Ambiguities in the spec are often
> discovered only when someone notices that implementations disagree
> and must be resolved somehow. Sometimes that happens after Go 1 and
> the resolution must happen in a way that, as much as possible,
> retains stability guarantees. So, it's not unsurprising if we get
> non-obvious spec clauses such as this.
>
> Specifically, I can well imagine that this particular case arose,
> because it was discovered after the fact that one implementation
> contained an optimization to not evaluate the argument to `len`,
> while the other didn't and the question of side-effects only arose
> when someone noticed.
>
> I don't really know the answer to your question. I can't think of
> anything which would break. That being said, I also don't think it's
> something that really needs "fixing", as it seems a pretty uncommon
> concern.

There is some discussion on the reasons when the original narrowing of
the behaviour was introduced (https://codereview.appspot.com/4444050).

It's actually quite an interesting spec change from a language history
perspective, more because of the chan discussion than this particular
issue.

[Shivansh Rustagi]

Hi Guys, I had a question regarding:

> On the other hand, if you have `for i, v := range a {` then the

> expression *is* evaluated before the start of the loop.

I have a code (https://goplay.tools/snippet/dMwD0_rEhkB) where I am ranging over a slice, and updating (removing the zeroth element form the same slice being ranged over) here I see that the "i" values is not being re-evaluated, it just gives an increasing number even if the ranged over slice has changed . Which can also be seen when I am printing the ith element given to me by the range, and the receiving out of range panic

[Tamás Gulácsi]

Yes.

If you manipulate the meaning of the index (i) or the length of the slice, then do it explicitly:

for i := 0; i < len(a); i++ {

  if a[i] == 0 { //delete
    a[i]=a[0]
    a = a[1:]
    i--

  }

}

For other methods, see https://go.dev/wiki/SliceTricks#filtering-without-allocating