syz-bisect: syzkaller fails to build as a subdirectory of go

[Joseph Bursey]

Hello! I'm trying to set up syz-bisect and can't seem to get it to work. First off, the syzkaller config option for syz-bisect seems like it is required to be put as a subdirectory of GOROOT (i.e. .../go/src/github.com/google/syzkaller_bisect), as it complains otherwise. However, when I put syzkaller there and try to build it (either manually or through syz-bisect), I get the following error:

============================================================

go env (err=<nil>)
warning: GOPATH set to GOROOT (/mnt/sda/jtbursey/syzkaller/go) has no effect
GO111MODULE="auto"
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/jtbursey/.cache/go-build"
GOENV="/home/jtbursey/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/mnt/sda/jtbursey/syzkaller/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/mnt/sda/jtbursey/syzkaller/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/mnt/sda/jtbursey/syzkaller/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/mnt/sda/jtbursey/syzkaller/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.17.6"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/mnt/sda/jtbursey/syzkaller/go/src/github.com/google/syzkaller_bisect/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build350306463=/tmp/go-build -gno-record-gcc-switches"

git status (err=<nil>)
On branch master
nothing to commit, working tree clean


Makefile:32: run command via tools/syz-env for best compatibility, see:
Makefile:33: https://github.com/google/syzkaller/blob/master/docs/contributing.md#using-syz-env
warning: GOPATH set to GOROOT (/mnt/sda/jtbursey/syzkaller/go) has no effect
go list -f '{{.Stale}}' ./sys/syz-sysgen | grep -q false || go install ./sys/syz-sysgen
warning: GOPATH set to GOROOT (/mnt/sda/jtbursey/syzkaller/go) has no effect
make .descriptions
warning: GOPATH set to GOROOT (/mnt/sda/jtbursey/syzkaller/go) has no effect
make[1]: *** No rule to make target 'bin/syz-sysgen', needed by '.descriptions'.  Stop.
make: *** [Makefile:156: descriptions] Error 2

===============================================================

This is the relevant section of the make file:

================================================================
.PHONY: descriptions
descriptions:
   go list -f '{{.Stale}}' ./sys/syz-sysgen | grep -q false || go install ./sys/syz-sysgen
   $(MAKE) .descriptions

.descriptions: sys/*/*.txt sys/*/*.const bin/syz-sysgen
   bin/syz-sysgen
   touch .descriptions

================================================================

I note 2 things about this error. First, it only occurs if the syzkaller directory is a subdirectory of GOROOT, which seems to be required by syz-bisect. Second, using syz-env does fix the issue, but syz-bisect does not use syz-env to build.

I can copy in syz-sysgen from another syzkaller directory to quiet the error for the first build, but after that it breaks again.

I have included my configuration below:

================================================================

{

     "compiler_type": "gcc",
     "bin_dir": "/mnt/sda/jtbursey/syzkaller/bisect_bin",
     #"ccache": "/usr/bin/ccache",
     "kernel_repo": "git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git",
     "kernel_branch": "master",
     "syzkaller_repo": "https://github.com/google/syzkaller",
     "userspace": "/mnt/sda/jtbursey/syzkaller/image/chroot",
     "kernel_config": "/mnt/sda/jtbursey/syzkaller/kernel.cfg",
     "kernel_baseline_config": "/mnt/sda/jtbursey/syzkaller/kernel.cfg",
     #"syzctl": /home/syzkaller/go/src/github.com/google/syzkaller/dashboard/config/linux/upstream.sysctl,
     #"cmdline": /home/syzkaller/go/src/github.com/google/syzkaller/dashboard/config/linux/upstream.cmdline,
     "manager":
     {
         "name" : "bisect",
         "target": "linux/amd64",
         "http": "127.0.0.1:56741",
         "workdir": "/mnt/sda/jtbursey/syzkaller/workdir",
         "kernel_obj": "/mnt/sda/jtbursey/syzkaller/kernel",
         "image": "/mnt/sda/jtbursey/syzkaller/image/stretch.img",
         "sshkey": "/mnt/sda/jtbursey/syzkaller/image/stretch.id_rsa",
         "syzkaller": "/mnt/sda/jtbursey/syzkaller/go/src/github.com/google/syzkaller_bisect",
         "procs": 8,
         "type": "qemu",
         "kernel_src": "/mnt/sda/jtbursey/syzkaller/kernel",
         "vm": {
               "count": 4,
               "kernel": "/mnt/sda/jtbursey/syzkaller/linux/arch/x86/boot/bzImage",
               "cpu": 2,
               "cmdline": "root=/dev/sda1 rw console=ttyS0 kaslr crashkernel=512M minnowboard_1:eth0::: security=none"
         }
     }
 }

===================================================================

Is there anything I am doing wrong? What can I do to fix this?

Thank you in advance!

A few other things I noticed that might not be directly related to this error:

1. syz-bisect wants syz-fuzzer to be built in the target syzkaller directory before bisecting even though it builds over it before fuzzing:

     bad config syzkaller param: can't find /mnt/sda/jtbursey/syzkaller/go/src/github.com/google/syzkaller_bisect/bin/linux_amd64/syz-fuzzer

2. I did get syz-bisect to work exactly once, though it appears to be a fluke (and I have no clue how). However, it froze before building the first kernel build. When I stopped it (ctrl-c), it gave a prompt for my sudo password as though it had been waiting for it. Does syz-bisect require sudo (maybe for docker)?

Again, thank you so much in advance!

-Joey Bursey

[Dmitry Vyukov]

Hi Joseph,

You shouldn't rename syzkaller to syzkaller_bisect. Go uses paths to
resolve package imports. It will look for github.com/google/syzkaller.

Also setting GOROOT and GOPATH to the same dir and changing contents
of GOROOT is wrong. You should set GOPATH to a separate dir and
checkout syzkaller there.

> --
> You received this message because you are subscribed to the Google Groups "syzkaller" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to syzkaller+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/syzkaller/d2c86c2c-a435-445d-a2e4-7a74258505cfn%40googlegroups.com.

[Joseph Bursey]

Thank you for the response! Though I don't think I've made much progress.

I changed the name of syzkaller_bisect back to syzkaller and set GOPATH accordingly:

     GOPATH=/mnt/sda/jtbursey/syzkaller/go/src/github.com/google/syzkaller/

But I still get the same error as before.

I also tried moving the syzkaller directory outside of GOROOT as you suggested, but bisection fails with the following error:

=====================================================================================

revisions tested: 0, total time: 3.383925008s (build: 0s, test: 0s)
error: syzkaller path "/mnt/sda/jtbursey/syzkaller/github.com/google/syzkaller" is not in GOPATH
bisection failed: syzkaller path "/mnt/sda/jtbursey/syzkaller/github.com/google/syzkaller" is not in GOPATH

=====================================================================================

GOPATH and GOROOT are as follows:

GOROOT=/mnt/sda/jtbursey/syzkaller/go

GOPATH=/mnt/sda/jtbursey/syzkaller/github.com/google/syzkaller/

This error confuses me because the syzkaller path is the same as GOPATH.

Perhaps I misunderstood your instructions?

[Joseph Bursey]

Update: I did some digging and found that the error:

=====================================================================================

revisions tested: 0, total time: 3.383925008s (build: 0s, test: 0s)
error: syzkaller path "/mnt/sda/jtbursey/syzkaller/github.com/google/syzkaller" is not in GOPATH
bisection failed: syzkaller path "/mnt/sda/jtbursey/syzkaller/github.com/google/syzkaller" is not in GOPATH

=====================================================================================

is thrown by the following section of code from pkg/instance/instance.go:

=====================================================================================
func (env *env) BuildSyzkaller(repoURL, commit string) (string, error) {
   cfg := env.cfg
   srcIndex := strings.LastIndex(cfg.Syzkaller, "/src/")
   if srcIndex == -1 {
       return "", fmt.Errorf("syzkaller path %q is not in GOPATH", cfg.Syzkaller)    }

=====================================================================================

I can get around this by setting the syzkaller path to /mnt/sda/jtbursey/syzkaller/src/github.com/google/syzkaller, but this seems like a bad idea.

Other than that, it looks like this problem is solved. Thank you!

- Joey Bursey