syz-executor compilation failure

[$rik@nth]

Hi Dmitry,

I observe syz-executor is failing with latest Android NDK version
android-ndk-r16. Did anyone also reported such issue?

/local/mnt/workspace2/android-ndk-r16/toolchains/"aarch64-linux-android-4.9"/prebuilt/linux-"x86_64"/bin/"aarch64-linux-android-g++"
-o ./bin/linux_arm64/syz-executor executor/executor_linux.cc \
-pthread -Wall -Wframe-larger-than=8192 -Wparentheses
-Werror -O1 \
-static -I
/local/mnt/workspace2/android-ndk-r16/sources/cxx-stl/llvm-libc++/include
--sysroot=/local/mnt/workspace2/android-ndk-r16/platforms/android-24/arch-"arm64"
-static -DGOOS=\"linux\"
-DGIT_REVISION=\"eb85112dc9d75d47d67e8494429f0f04dbba4b44+\"
executor/executor_linux.cc:6:19: fatal error: fcntl.h: No such file or directory
#include <fcntl.h>
^
compilation terminated.
make[1]: *** [executor] Error 1
make[1]: Leaving directory
`/local/mnt/workspace2/syzkaller-working/src/github.com/google/syzkaller'
make: *** [target] Error 2


--
Thanks & Regards,
M.Srikanth Kumar.

[Dmitry Vyukov]

Hi,

+Atul looked at compilation with r16 recently.

[Zach Riggle 🖖]

The sysroot moved, and you need to specify -B for the old sysroot, and define __ANDROID_API__.

Apparently this was "documented" as part of the unified headers, but was also a surprise to me and the rest of the internet.

$NDK/toolchains/arm-linux-androideabi-4.9/prebuilt/darwin-x86_64/bin/arm-linux-androideabi-gcc \

--sysroot $NDK/sysroot -isystem $NDK/sysroot/usr/include/arm-linux-androideabi \

-D__ANDROID_API__=27 \

-B $NDK/platforms/android-27/arch-arm/usr/lib \

hello.c

Zach Riggle |

Android Security |

rig...@google.com |

Austin, TX

[Zach Riggle 🖖]

(And specify -isystem, my line breaks didn't make that clear)

Zach Riggle |

Android Security |

rig...@google.com |

Austin, TX

[Billy Lau]

That aside, something to consider is that GCC will be dropped from future NDKs. I suggest we start looking into either steering it in the direction of using linaro-gcc or moving to clang altogether. I'm not sure of the implication of this to the overall build process for other arch. Atul and I are investigating this currently.

[Zach Riggle 🖖]

The correct solution is to use ndk-build properly with an Android.mk rather than directly invoking the compiler :P

Zach Riggle |

Android Security |

rig...@google.com |

Austin, TX

[Atul Prakash]

+1 to that.  I have tested that and it works.

The only issue with that is syzkaller relies on a compilation command with gcc for dynamically generated code during fuzzing.

- Atul

[$rik@nth]

Can someone share a working patch to unblock. I've added as suggested by Zach. But i am getting below error

In file included from executor/executor_linux.cc:24:0:

executor/executor.h:4:21: fatal error: algorithm: No such file or directory

 #include <algorithm>

                     ^

compilation terminated.

[$rik@nth]

After fixing some of the environment. Now i am getting below error

/local/mnt/workspace2/android-ndk-r16/toolchains/"aarch64-linux-android-4.9"/prebuilt/linux-"x86_64"/bin/"aarch64-linux-android-g++" -o ./bin/linux_arm64/syz-executor executor/executor_linux.cc \

                -pthread -Wall -Wframe-larger-than=8192 -Wparentheses -Werror -O1 \

                -static -I /local/mnt/workspace2/android-ndk-r16/sources/cxx-stl/llvm-libc++/include --sysroot /local/mnt/workspace2/android-ndk-r16/sysroot  -isystem /local/mnt/workspace2/android-ndk-r16/sysroot/usr/include/aarch64-linux-android  -D__ANDROID_API__=27 -B DK/platforms/android-27/arch-arm/usr/lib -static -DGOOS=\"linux\" -DGIT_REVISION=\"c9e1b85baf8369cdcded7468f617cb213ffa7bbb+\"

In file included from executor/executor_linux.cc:20:0:

executor/common_linux.h:566:18: error: 'uintptr_t syz_open_procfs(uintptr_t, uintptr_t)' defined but not used [-Werror=unused-function]

 static uintptr_t syz_open_procfs(uintptr_t a0, uintptr_t a1)

                  ^

cc1plus: all warnings being treated as errors

make[1]: *** [executor] Error 1

make[1]: Leaving directory `/local/mnt/workspace2/syzkaller-working/src/github.com/google/syzkaller'

make: *** [target] Error 2

[$rik@nth]

Hi All,

It would great, if anyone can help me in unblocking the executor compilation.

[Billy Lau]

Hi Srikanth,

Several workarounds currently:

1. Revert back to NDK 15c, which is known to work.

2. Create a standalone toolchain (https://developer.android.com/ndk/guides/standalone_toolchain.html) and reference that instead.

3. Create an Android.mk and an Application.mk file and then using ndk-build included in NDK.

Note that these are workarounds, which should get you unblocked until we find the definitive way of going forward.

Cheers,

- billy

[$rik@nth]

Hi Billy,

I am getting different issue after reverting to android-ndk-r15c and android-ndk-r15-beta2 toolchains. Which used to work for me earlier with out any issues. But after pulling the changes from master to local branch, i started seeing this issue.

/local/mnt/workspace2/android-ndk-r15c/toolchains/"aarch64-linux-android-4.9"/prebuilt/linux-"x86_64"/bin/"aarch64-linux-android-g++" -o ./bin/linux_arm64/syz-executor executor/executor_linux.cc \

                -pthread -Wall -Wframe-larger-than=8192 -Wparentheses -Werror  -O1 \

                -static -I /local/mnt/workspace2/android-ndk-r15c/sources/cxx-stl/llvm-libc++/include --sysroot=/local/mnt/workspace2/android-ndk-r15c/platforms/android-24/arch-"arm64" -static -DGOOS=\"linux\" -DGIT_REVISION=\"c9e1b85baf8369cdcded7468f617cb213ffa7bbb+\"

In file included from executor/executor_linux.cc:20:0:

executor/common_linux.h:566:18: error: 'uintptr_t syz_open_procfs(uintptr_t, uintptr_t)' defined but not used [-Werror=unused-function]

 static uintptr_t syz_open_procfs(uintptr_t a0, uintptr_t a1)

                  ^

cc1plus: all warnings being treated as errors

make[1]: *** [executor] Error 1

make[1]: Leaving directory `/local/mnt/workspace2/syzkaller-working/src/github.com/google/syzkaller'

make: *** [target] Error 2

[Dmitry Vyukov]

On Tue, Dec 5, 2017 at 11:28 AM, $rik@nth <srikan...@gmail.com> wrote:
>
> Hi Billy,
>
> I am getting different issue after reverting to android-ndk-r15c and android-ndk-r15-beta2 toolchains. Which used to work for me earlier with out any issues. But after pulling the changes from master to local branch, i started seeing this issue.
>
> /local/mnt/workspace2/android-ndk-r15c/toolchains/"aarch64-linux-android-4.9"/prebuilt/linux-"x86_64"/bin/"aarch64-linux-android-g++" -o ./bin/linux_arm64/syz-executor executor/executor_linux.cc \
> -pthread -Wall -Wframe-larger-than=8192 -Wparentheses -Werror -O1 \
> -static -I /local/mnt/workspace2/android-ndk-r15c/sources/cxx-stl/llvm-libc++/include --sysroot=/local/mnt/workspace2/android-ndk-r15c/platforms/android-24/arch-"arm64" -static -DGOOS=\"linux\" -DGIT_REVISION=\"c9e1b85baf8369cdcded7468f617cb213ffa7bbb+\"
> In file included from executor/executor_linux.cc:20:0:
> executor/common_linux.h:566:18: error: 'uintptr_t syz_open_procfs(uintptr_t, uintptr_t)' defined but not used [-Werror=unused-function]
> static uintptr_t syz_open_procfs(uintptr_t a0, uintptr_t a1)
> ^
> cc1plus: all warnings being treated as errors
> make[1]: *** [executor] Error 1
> make[1]: Leaving directory `/local/mnt/workspace2/syzkaller-working/src/github.com/google/syzkaller'
> make: *** [target] Error 2

This function is used in executor/syscalls_linux.h:

#if defined(__aarch64__) || 0
...
unsigned syscall_count = 1482;
call_t syscalls[] = {
...
{"syz_open_procfs", 1000006, (syscall_t)syz_open_procfs},

Are you sure you have clean checkout?

[$rik@nth]

Hi Dmitry,

I did make generate and still not able to see above code segments in
executor/syscalls_linux.h. Did i am doing something wrong? Rather i
need to create one more branch of master snapshot to unblock myself.

[Dmitry Vyukov]

You don't even need to run 'make generate', it's right there:
https://github.com/google/syzkaller/blob/master/executor/syscalls_linux.h#L6008

[$rik@nth]

But my local changes needs to get reflect syz-executor. Until i run
`make generate` it won't right?

[Dmitry Vyukov]

On Tue, Dec 5, 2017 at 1:32 PM, $rik@nth <srikan...@gmail.com> wrote:
> But my local changes needs to get reflect syz-executor. Until i run
> `make generate` it won't right?

Right.

I don't know what are your changes and can't help with them. You have
a choice of:
- upstreaming all your changes, then we will keep them working
- never rebasing to new syzkaller revisions, then what's working will
continue working
- rebase, debug and maintain your private changes

[$rik@nth]

Thanks Dmitry for your suggestions.

[Dmitry Vyukov]

Srikanth, if you build binaries just for linux (make TARGETOS=linux
TARGETARCH=arm64), do they work for you? They are linked as static
binaries by default and don't depend on libc, so should work on any
linux kernel including android.
Atul, uses this configuration. So if it works for you, I would rather
drop that android mess that tries to mimic what android build system
does and breaks on every ndk update.
Billy, Zach?

On Tue, Dec 5, 2017 at 11:28 AM, $rik@nth <srikan...@gmail.com> wrote:
>

[$rik@nth]

Hi Dmitry,

It is working fine with "make TARGETOS=linux TARGETARCH=arm64".

[Dmitry Vyukov]

On Wed, Dec 13, 2017 at 1:33 PM, $rik@nth <srikan...@gmail.com> wrote:
> Hi Dmitry,
>

> It is working fine with "make TARGETOS=linux TARGETARCH=arm64".

Thanks for testing.
Then I propose we drop TARGETOS=android support from Makefile (but
need to be careful to keep 'make extract TARGETOS=android` working).

[Dmitry Vyukov]

Filed https://github.com/google/syzkaller/issues/478 to keep track of this.

[Billy Lau]

Yeap, echoing the result earlier, that make TARGETOS=linux would still work, however it is when describing new syscalls that we need to be wary, as there are android specific if/else when doing make extract and make generate I think.

Additionally, it seems like it is hardcoded to aarch64-linux-gnu-gcc in targets.go even though NDK is specified - I haven't gone down the path of investigating if there's discrepancies caused by using aarch64-linux-android-gcc in building some binaries while using the *-gnu-gcc for syscall info extraction and generation w.r.t. Android.