Unexpected order of global variable declaration during package initialization

[Joao Carlos]

Hi all,

I'm currently observing a behavior in the package initialization that looks incompatible with the Go language specification.

Let's consider the following two .go files which are in the same package

f1.go

package main    
   
var A int = 3    
var B int = A + 1    
var C int = A

f2.go

package main    
   
import "fmt"    
                     
var D = f()      
   
func f() int {    
  A = 1    
  return 1    
}    
   
func main() {    
  fmt.Println(A, " ", B, " ", C)    
} 

According to the Go language specification, "package-level variable initialization proceeds stepwise, with each step selecting the variable earliest in declaration order which has no dependencies on uninitialized variables".

As such, I would expect two possible orders in which the global variables can be initialized:

1. A < B < C < D - happens when you compile the project by passing f1.go first to the compiler, followed by f2.go . In this case, the output is "1 4 3"

2. A < D < B < C - happens when f2.go is passed first to the compiler. In this case, the expected output would be "1 2 1". However, the actual output is "1 2 3".

Adding to this, I observed that if we rewrite f1.go to the following, the program now has the expected behavior when we pass f2.go first to the compiler.

rewritten f1.go

package main    
   
import "fmt"    
   
var A int = initA()    
var B int = initB()    
var C int = initC()    
     
func initA() int {    
  fmt.Println("Init A")    
  return 3    
}    
     
func initB() int {    
  fmt.Println("Init B")    
  return A + 1    
}    
 
func initC() int {    
  fmt.Println("Init C")    
  return A    
}

Output

Init A
Init B
Init C
1   2   1

I observed this behavior in multiple versions of Go, including:

- go1.16.4 darwin/amd64

- go1.17.2 linux/amd64

- go1.18 linux/amd64

Is this the expected behavior? Am I overlooking any details in the Go language specification or in the Go memory model? Or is this a bug in the compiler?

Thanks in advance for any helpful comments!

[Ian Lance Taylor]

I think you're right: I think this is a bug.

Interestingly, I think the runtime package may rely on this bug. In
the runtime package I see

var maxSearchAddr = maxOffAddr
var maxOffAddr = offAddr{(((1 << heapAddrBits) - 1) + arenaBaseOffset)
& uintptrMask}

and pageAlloc.Init, which is called by mheap.init which is called by
mallocinit which is called by schedinit before package initializers
are run. So the compiler is implementing an optimization to
initialize maxSearchAddr before running package initialization
routines, which I suppose is OK provided it that it can prove that the
variable is never set by any function run during package
initialization.

Want to open a bug report at https://go.dev/issue? Thanks.

Ian

[tapi...@gmail.com]

> 2. A < D < B < C - happens when f2.go is passed first to the compiler. In this case, the expected output would be "1 2 1". However, the actual output is "1 2 3".

This is not true by my understanding of the spec.

https://go.dev/ref/spec#Package_initialization

If f2.go is passed first, then the order of uninitialized variables is D < A < B < C.

As D depends on A, so D is not initialized in the first initialization cycle.

In the first initialization cycle, A, B, and C and initialized as 3, 4, and 3.

In the second initialization cycle, D is initialized as 1 and A is changed to 1.

So the output should be 1 4 3.

The same output is for the case if f1.go is passed first.

[tapi...@gmail.com]

BTW, the rewritten version outputs

Init A
Init B
Init C

1   4   3

On my machine (go1.18 linux/amd64).

[Brian Candler]

On Thursday, 24 March 2022 at 11:30:37 UTC tapi...@gmail.com wrote:

> 2. A < D < B < C - happens when f2.go is passed first to the compiler. In this case, the expected output would be "1 2 1". However, the actual output is "1 2 3".

This is not true by my understanding of the spec.

https://go.dev/ref/spec#Package_initialization

The spec says:

"Initialization proceeds by repeatedly initializing the next package-level variable that is earliest in declaration order and ready for initialization, until there are no variables ready for initialization.

If any variables are still uninitialized when this process ends, those variables are part of one or more initialization cycles, and the program is not valid."

This hinges on how you read "next", but my reading is that you keep jumping back to the start of the list, to find the variable "earliest in declaration order" which is ready for initialization.

I think your interpretation is that "next" means you move to the next variable lexically *after* the one you've just initialized which is now ready - not the earliest one.  By that reading you have to cycle round and round until there are no more changes (which would involve keeping a flag to remember whether anything changed during a given cycle).  I don't think that's what it says.

By my reading, and given the lexical order is D < A < B < C:

1. D is not ready (because it depends on A)
2. A is ready, so it is initialized: it gets value 3.
3. Now D is the earliest variable which is ready, so D is initialized. It gets value 1, and A is set to 1.
4. B is initialized. It gets value 2
5. C is initialized. It gets value 1

We got to the end and everything is initialized, success. Result is 1 2 1. 

[Brian Candler]

On Thursday, 24 March 2022 at 12:15:16 UTC tapi...@gmail.com wrote:

BTW, the rewritten version outputs

Init A
Init B
Init C

1   4   3

On my machine (go1.18 linux/amd64).

It depends on the order, and the OP was positing what happens when f2.go is presented first to the compiler.

$ go version
go version go1.17.6 darwin/amd64

$ go run rewritten_f1.go f2.go

Init A
Init B
Init C
1   4   3

$ go run f2.go rewritten_f1.go

[Brian Candler]

Ugh, quoting got broken there.

$ go run rewritten_f1.go f2.go
Init A
Init B
Init C
1   4   3

$ go run f2.go rewritten_f1.go
Init A
Init B
Init C
1   2   1

Hopefully that will show properly.

[Joao Carlos]

Thank you all for the quick replies and helpful comments. I opened an issue in the go's issue tracker (https://github.com/golang/go/issues/51913).

[tapi...@gmail.com]

Aha, sorry, the file order really matters here.

But for this specified case, it should not.

That's not right. I remembered in an issue thread, it is mentioned that the order should be

1. D is not ready (because it depends on A)
2. A is ready, so it is initialized: it gets value 3.

4. B is initialized. It gets value 2
5. C is initialized. It gets value 1
3. Now D is the earliest variable which is ready, so D is initialized. It gets value 1, and A is set to 1.

[tapi...@gmail.com]

sorry, the quoting is missing. A groups problem.

[Brian Candler]

On Thursday, 24 March 2022 at 12:35:38 UTC tapi...@gmail.com wrote:

Aha, sorry, the file order really matters here.

But for this specified case, it should not.

That's not right. I remembered in an issue thread, it is mentioned that the order should be

1. D is not ready (because it depends on A)
2. A is ready, so it is initialized: it gets value 3.

4. B is initialized. It gets value 2
5. C is initialized. It gets value 1
3. Now D is the earliest variable which is ready, so D is initialized. It gets value 1, and A is set to 1.

Can you quote your source for that?