Hacking gc to support methods on builtin types

[Luke Champine]

Hi all,

Recently, I managed to get this to work:

    x := []int{1, 2, 3}

    x.reverse()

    fmt.Println(x)

That is, when I run my modified compiler on this code, it prints [3, 2, 1]. Neat!

To be clear, I am not intending to get this merged upstream -- this is just a fun exercise to learn my way around the compiler.

 My approach was as follows. First, I added a reverse function to the runtime package:

    func reverse(et *_type, s slice) {

        tmp := newobject(et) // swap scratch space

        for i := 0; i < s.len/2; i++ {

            j := s.len - i - 1

            ei := add(s.array, uintptr(i)*et.size)

            ej := add(s.array, uintptr(j)*et.size)

            typedmemmove(et, tmp, ei)

            typedmemmove(et, ei, ej)

            typedmemmove(et, ej, tmp)

        }

    }

I added this function's name to gc/builtin/runtime.go and ran go generate to update gc/builtin.go.

Next, I modified the lookdot function of typecheck.go, such that when a method named "reverse" is looked for on a slice type, and no existing method is found, a sentinel method with the appropriate type is returned:

    if n.Left.Type == t || n.Left.Type.Sym == nil {

        mt := methtype(t)

        if mt != nil {

            f2 = lookdot1(n, s, mt, mt.Methods(), dostrcmp)

        }

        // new code

        if f2 == nil && t.IsSlice() {

            if s.Name == "reverse" {

                recv := types.NewField()

                recv.Type = t

                f2 = types.NewField()

                f2.Sym = &types.Sym{

                    Name: "reverse",

                    Pkg:  builtinpkg,

                }

                f2.Type = functypefield(recv, nil, nil)

            }

        }

    }

Finally, I modified gc/walk.go to replace the method call with a call to the runtime function:

    case OCALLINTER, OCALLFUNC, OCALLMETH:

        if n.Op == OCALLINTER {

            usemethod(n)

            markUsedIfaceMethod(n)

        }

        // new code

        if n.Left != nil && n.Left.Sym.Def == nil {

            methname := n.Left.Sym.Name

            if strings.HasSuffix(methname, "reverse") {

                s := n.Left.Left

                fn := syslook("reverse")

                fn = substArgTypes(fn, s.Type.Elem())

                n = mkcall1(fn, nil, init, typename(s.Type.Elem()), s)

            }

        }

This approach works, but I feel like I'm bludgeoning a very sophisticated system into doing something it was not designed to do. I'm hoping that a more experienced compiler hacker can offer some guidance as to the "Right Way" of doing this. If you were tasked with adding a builtin method to all slices, how would you do it?

In particular, how do I handle inlining? Currently I have to disable all inlining when compiling; otherwise, I get internal compiler error: no function definition. I assume this is because the inlining phase runs before the method call is replaced with a runtime call. Is it possible to hook up the method call to the runtime function definition during the typecheck phase?

Relatedly, I'm finding compiler development to be much more cumbersome than the Go programming I'm accustomed to. Specifically, I'm currently re-running make.bash after each change, which takes around 4 minutes on my machine. Is it possible to test my changes without re-running make.bash, ideally compiling in just a few seconds? Another issue is that gopls does not seem to like operating on the compiler source, at least when used with VS Code. What IDE do other compiler devs use?

Sorry for the barrage of questions -- I hope this is the right place to ask them!

[Brian Candler]

> If you were tasked with adding a builtin method to all slices, how would you do it?

Did you consider making a global function, like copy() and append() ?  Why does it have to be a method?

[Luke Champine]

Yep, in fact I have already added min and max functions that return the lesser/greater of their two arguments. Adding builtin functions is much easier than builtin methods, since other builtin functions are already supported by the compiler. You just add a new op type to gc/syntax.go, then add a switch case to typecheck1, and finally rewrite the ops to m = l; if r cmp l { m = r } in gc/walk.go. (In this case, the operation is simple enough that adding a runtime function would be overkill.) I was even able to make the expressions evaluate to a constant if their arguments are constant, which is cool.

So yes, I could add a reverse builtin function in the same way, and it would definitely be simpler than adding a method. But I wouldn't learn anything new about the compiler that way. Currently there are no builtin methods whatsoever, which means that adding one is much more involved than adding a new builtin function. I'm willing to put in the necessary effort, but I could use a few pointers so that I avoid coding myself into a corner.