Programmatically test an AST?

[jacob.ho...@gmail.com]

I'm currently writing a mutation testing tool for Go, and I need to perform 1000's of mutations on a package.

I'm using the AST to modify/remove nodes and would like to be able to test that modified AST with associated the *_test.go files.

Is there anyway to just test the AST without having to write the AST to a temp directory, copy the test files in and do an os.Exec("go test") on the modified package?

Any help would be appreciated (I've only been using Go for a couple of months now coming from a C/Ruby background).

Thanks!

-Jacob

[Markus Zimmermann]

I had the same idea when I wrote https://github.com/zimmski/go-mutesting (btw can you pinpoint me to your repository so I can take a look?) but gave up since time ran out. As far as I know there is no easy solution to what you are asking. Maybe there is one now with Go 1.6 but I did not have the chance to revisit old problems. My thinking process was the following: If you run `go help test` you can read that testing something in Go simply creates and runs a binary for each tested package. Meaning, saving the binary and running it is not the challenge, it is generating it which is done using `go build` https://golang.org/src/cmd/go/build.go The goal would be to intercept the building process swap the original with the modified AST and then generate and save the new binary. Basically, you have to copy parts of `go build` and `go test` since none of them exports anything. That is where I ended my journey.

[Dave Cheney]

You should take a look at gofuzz.

[Damian Gryski]

go-fuzz writes out the new intrumented code then builds it with the normal tool chain.

Damian

[jacob.ho...@gmail.com]

I'm still in the planning stages, so no actual code yet.

This is for research, so I've been mostly focusing on how to do it the "right" way :)

My end goal is to mutation test the Go standard library packages, and chart them over time (each release).

I've looked at your go-mutesting, and also the manbearpig tool as a jumping off point.

At this point my pplan looks like this:

1. ParseDir the package directory, skipping over _*.go and *_test.go
   
1. Mutate the AST with: 
1. operator ( + to -, ++ to --)
2. comparison (< to >=)
3. logical (|| to &&)
4. node removal (emptying case/else bodies, removing default case in switches)
5. Flipping exported structs/funcs to private (uppercase to lowercase)
2. Write out AST to a tmp directory
3. Copy over *_test.go to tmp directory
4. `go test` -> record results to JSON {pkg, mutationPosition, mutationType, result, mutation, filename}
2. Aggregate the JSON results for a package

I did skim over go-fuzz, but it also looks like it use os.Command... honestly it's a huge program for a beginner like me to take apart. 

-Jacob

[Ian Lance Taylor]

On Sat, Mar 12, 2016 at 9:22 AM, <jacob.ho...@gmail.com> wrote:
> I'm currently writing a mutation testing tool for Go, and I need to perform
> 1000's of mutations on a package.
> I'm using the AST to modify/remove nodes and would like to be able to test
> that modified AST with associated the *_test.go files.
>
> Is there anyway to just test the AST without having to write the AST to a
> temp directory, copy the test files in and do an os.Exec("go test") on the
> modified package?

No, sorry.

Ian