Working on ssa rules and runtime at the same time seems difficult

[Egon Elbre]

With the latest toolchain updates I'm having trouble regenerating ssa rules easily. Previously, I was able to use stable compiler to generate the new code.

To make things more concrete, I've been experimenting with changing `growslice` implementation in `src/runtime`, which also requires changes in `cmd/compile/internal/ssa`.

My setup uses Go tip and 1.21 as my boostrap compiler:
C:\Go.tip\src\cmd\compile\internal\ssa>where go
C:\Program Files\Go\bin\go.exe // this is my default installation

I started with trying to regenerate the code:

C:\Go.tip\src\cmd\compile\internal\ssa>go generate .
go: downloading go1.22 (windows/amd64)
go: download go1.22 for windows/amd64: toolchain not available

Of course I don't have a the go1.22 compiler available, because it's not yet released.

If I change the `C:\Go.tip\src\cmd\go.mod` to use `1.21` and do:

C:\Go.tip\src\cmd\compile\internal\ssa>go generate .
C:\Go.tip\src\cmd\compile\internal\ssa>

This doesn't seem to work, I assume it actually ended up running go generate in my default installation rather than in Go.tip.

If I run the command manually, I get:

C:\Go.tip\src\cmd\compile\internal\ssa>go run -C=_gen .
ambiguous import: found package cmd/compile/internal/ssa/_gen in multiple directories:
        C:\Program Files\Go\src\cmd\compile\internal\ssa\_gen
        C:\Go.tip\src\cmd\compile\internal\ssa\_gen

I can switch to using `C:\Go.tip` as the tool for running the ssa generation,

however, then I hit the other problem that it'll end up picking up my

`runtime` changes as well, which won't work unless the `ssa` logic has been

updated.

Which leads me to this workflow to do things:

1. modify things in ssa and runtime

2. stash changes in runtime

3. go generate cmd/compile/internal/ssa

4. unstash changes from runtime

5. run make in src to recompile the compiler

6. realize that my modifications had a bug in it

7. stash everything

8. recompile the compiler

9. go back to step 1.

I can avoid 7. - 8. by keeping a backup of a stable compiler, but there's still

a significant amount of stash/unstash to do things.

Am I missing something in my workflow to be able to regenerate

the ssa, with a separate compiler? Or should I look automating my

workflow to avoid issues?

All the best,

+ Egon

[Than McIntosh]

Hi Egon,

When doing compiler development on tip, the expected pattern is to have your path set up so that the tip version of "go" is selected as opposed to whatever you are using for bootstrapping.

In your case that would mean making sure that C:\Go.tip\bin appears in your path before C:\Program Files\Go\bin. The "go" from C:\Go.tip\bin is what you should be using to run "go generate".

Of course if you are actively hacking away on the compiler you may put yourself in a position where your <GOOROOT>/bin/compile is broken in some way, which complicates matters. To help with this most compiler developers use https://pkg.go.dev/golang.org/x/tools/cmd/toolstash to save off "known good" copies of the commands, so your workflow then becomes something like

  $ <make an edit to the compiler sources>
  $ toolstash restore
  $ go install cmd/compile
  $ go build mytestcase.go ... <- see if the edit worked

HTH.

Thanks, Than

--
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-dev/726722e1-74c5-49cf-81fa-cabf91baac11n%40googlegroups.com.

[Egon Elbre]

Thanks,

That will definitely help with the tools, but yeah, I guess I'll have to write some scripts to manage src/runtime similarly.

+ Egon

[Keith Randall]

I often regenerate the ssa rules using (run from the src directory):

cd cmd/compile/internal/ssa/_gen/; go run *.go; cd ../../../../..

Even that is broken with 1.21. It works with both 1.20 and tip, though. Not sure what 1.21's problem is. Fails both with released 1.21.1 and checking out the release branch and building.

Inside a tip source tree, I get:

khr@ultimate:~/sandbox/ro/src$ ~/go1.20.6/bin/go version
go version go1.20.6 linux/amd64
khr@ultimate:~/sandbox/ro/src$ ~/go1.21.1/bin/go version
go: downloading go1.22 (linux/amd64)
go: download go1.22 for linux/amd64: toolchain not available

khr@ultimate:~/sandbox/ro/src$ ~/sandbox/ro2/bin/go version
go version devel go1.22-53827ba49e Tue Oct 3 16:35:23 2023 +0000 linux/amd64

(ro and ro2 both reasonably close to tip. That last version string is from ro2, not ro)

If I move up from src to the repository base, then 1.21 works.

Probably caused by the go.mod saying 1.22 when the toolchain is 1.21? It would be nice if there was a way to tell the go tool to ignore that go.mod somehow. (Maybe there is?)

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-dev/91af8fca-caf4-4bb9-9ea7-26bcf771ba99n%40googlegroups.com.