Tools for developing Go compiler/ Go code from SSA

[Mohit Verma]

Hi All,

I am trying to change the ssa phase of Go compiler to do extra work during memory stores. 

For example whenever there is a memory store, I want to make a function call to one of my Go packages instead of the store. 

So, far I have some working changes and I can see the new SSA nodes when I dump SSA using GOSSAFUNC.

I have a couple of questions:

1. Is there a tool which can covert the SSA nodes back to some version of Go code?

2. How does Go team/others develop/make changes to Go compiler? So, far I have been adding prints in the Go compiler (cmd/compile/internal/syntax, gc, & ssa packages) to understand the code flow. But, this is very time consuming. I am sure a better way exists.

I'll appreciate any help!

Thanks!

Mohit

[Ian Lance Taylor]

[ +khr ]

> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAJ%2B6RRU%3D%3D33ct1%3D%2B0U10nDLnFjvBXff%2BrrmbYRvwY-TTRReLfw%40mail.gmail.com.
> For more options, visit https://groups.google.com/d/optout.

[Keith Randall]

On Wed, Jul 3, 2019 at 9:04 PM Ian Lance Taylor <ia...@golang.org> wrote:

[ +khr ]

On Wed, Jul 3, 2019 at 4:46 PM Mohit Verma <vmoh...@gmail.com> wrote:
>
> Hi All,
>
> I am trying to change the ssa phase of Go compiler to do extra work during memory stores.
> For example whenever there is a memory store, I want to make a function call to one of my Go packages instead of the store.
> So, far I have some working changes and I can see the new SSA nodes when I dump SSA using GOSSAFUNC.
>
> I have a couple of questions:
>
> 1. Is there a tool which can covert the SSA nodes back to some version of Go code?

Nope. Such a thing might be nice, actually. Handing phis, memory values, flags, etc. might be hard.

 

> 2. How does Go team/others develop/make changes to Go compiler? So, far I have been adding prints in the Go compiler (cmd/compile/internal/syntax, gc, & ssa packages) to understand the code flow. But, this is very time consuming. I am sure a better way exists.

That sounds about what I do when working on the compiler. Understanding what the compiler is doing is often 90% of the battle, and there's no easy solution. You can run the compiler under a debugger, if you want.

If you can be more specific about what you are trying to do and what you are running up against, I might be able to give you more specific recommendations.

If the compiler is generating bad code, you might be able to condition your new code on cmd/compile/internal/ssa/func.go:DebugTest which will let you binary search for a particular function whose code is bad.

[Mohit Verma]

Thanks Keith, Ian.

Specifically, I am trying to catch all the memory stores/moves during the build SSA phase, and instead call a function that can do things like record the old/new values in a journal. 

So I changed the methods like (*state).assign, (*state).storeType in gc/ssa.go and injected equivalent SSA nodes to make the function call. I can see the modified SSA code when I generate ssa.html using GOSSAFUNC. This is why I was looking for a tool that could convert SSA form back to Go code so I could see if I caught all the stores.

Right now, I am running up against something like this:

Escape analysis didn't see the function call, it saw normal assignments & marked variables in stack. Now I inject a function call during buildSSA. Now, before the function call, runtime decided that stack should migrate, and now I get a seg fault. 

I was trying to debug this, but putting up prints in the compiler/runtime seemed inefficient to me, and I thought the Go team might be doing things differently during their development/debug.

Thanks!

Mohit

[Keith Randall]

On Fri, Jul 5, 2019 at 2:05 PM Mohit Verma <vmoh...@gmail.com> wrote:

Thanks Keith, Ian.

Specifically, I am trying to catch all the memory stores/moves during the build SSA phase, and instead call a function that can do things like record the old/new values in a journal. 

So I changed the methods like (*state).assign, (*state).storeType in gc/ssa.go and injected equivalent SSA nodes to make the function call. I can see the modified SSA code when I generate ssa.html using GOSSAFUNC. This is why I was looking for a tool that could convert SSA form back to Go code so I could see if I caught all the stores.

Right now, I am running up against something like this:

Escape analysis didn't see the function call, it saw normal assignments & marked variables in stack. Now I inject a function call during buildSSA. Now, before the function call, runtime decided that stack should migrate, and now I get a seg fault. 

That shouldn't matter unless your function call would cause something to escape, than otherwise wouldn't.

Or are you passing a pointer to a stack object as a uintptr, maybe?

You can turn off escape analysis with a command-line flag, if that helps.

Adding function calls should normally be ok.  Functions marked nosplit (mostly in the runtime) would be the exception; don't add any calls to those.

[Mohit Verma]

Thanks Keith. 

Yes, the function call takes a variadic interface argument, so it would cause the variables to escape if it existed before. To take care of this, I allocate space for the interface during buildSSA & then call runtime.convT2E* functions.

1. How do I turn off escape analysis with command-line flag? I couldn't find this option anywhere.

2. Why would having calls to functions marked nosplit, be a problem? 

The function I am adding would internally call runtime.spanOfHeap() function which is marked nosplit. Will this be a problem? 

I haven't gone through the stack migration code yet, but this blog by Dave Cheney mentioned that compiler maintains a 768-byte redzone to make sure there is no memory corruption.

I had one other question. 

3. Is the stack check & runtime.morestack() code inserted by the linker? 

I see stacksplit code in cmd/internal/obj/ directory which is invoked during compileSSA() in gc/pgen.go.

Thanks a lot!

Mohit

[Keith Randall]

On Fri, Jul 5, 2019 at 9:16 PM Mohit Verma <vmoh...@gmail.com> wrote:

Thanks Keith. 

Yes, the function call takes a variadic interface argument, so it would cause the variables to escape if it existed before. To take care of this, I allocate space for the interface during buildSSA & then call runtime.convT2E* functions.

1. How do I turn off escape analysis with command-line flag? I couldn't find this option anywhere.

I thought it was part of -N, but apparently not. Nevermind.

 

2. Why would having calls to functions marked nosplit, be a problem? 

The function I am adding would internally call runtime.spanOfHeap() function which is marked nosplit. Will this be a problem? 

I haven't gone through the stack migration code yet, but this blog by Dave Cheney mentioned that compiler maintains a 768-byte redzone to make sure there is no memory corruption.

The only problem would be calling splittable functions from nosplit ones.  If you're not doing that, it shouldn't be a problem.

There is a redzone big enough for the nosplit routines.  As long as yours fit, you should be fine (the linker should check the fit for you).

 

I had one other question. 

3. Is the stack check & runtime.morestack() code inserted by the linker? 

I see stacksplit code in cmd/internal/obj/ directory which is invoked during compileSSA() in gc/pgen.go.

It's inserted by the compiler, I believe.

cmd/internal/obj.Flushplist is called from (*Progs).Flush in cmd/compile/internal/gc/gsubr.go