AOSP Kernel build for Pixel 5

[Andrew Fernandes]

I'm following the instructions on source.android.com for building kernels and unable to get it working on the Pixel 5.

There are a couple of paths I've taken with differing results.

1. Check out the aosp-kernel android-msm-redbull-4.19-android11-qpr2 branch
2. Run the build_redbull.sh (or build_rebull-gki.sh) script.

3. In the aosp directory, export TARGET_PREBUILT_KERNEL to the built Image.lz4-dtb from aosp-kernel
4. flash aosp using `fastboot flashall -w`

5. Observe bootloop, the devices never reaches a point where adbd starts and I can shell in

Then I saw from other posts that when recompiling the kernel the prebuilt kernel modules that are in the aosp repository are out of date and are being packaged into the vendor_boot image and failing to load.

So... next I tried copying all of the compiled aosp-kernel modules and images into the aosp/device/google/redbull-kernel directory and rebuilding.

This gets farther and I'm now able to adb shell into the device, but the display stays on the Android boot logo and various other services/sensors don't work.

So, my question is, what's the process for a successful build/flash of the aosp kernel on the Pixel 5?

For reference I'm using the aosp android-11.0.0_r37 branch and aosp kernel android-msm-redbull-4.19-android11-qpr2 branch.

Any help/instruction would be greatly appreciated!

[Andrew Fernandes]

I've found a couple of different solutions for anyone else who stumbles on this issue.

If you follow the directions and use TARGET_PREBUILT_KERNEL, all of the kernel modules in vendor.img and vendor_boot.img are out of sync. Not only do you need to get the updated kernel in the aosp build, but all of the updated kernel modules.

After building the kernel, I would suggest ignoring the TARGET_PREBUILT_KERNEL option and instead, just copy everything from aosp-kernel/out/android-msms-pixel-4.19/dist to aosp/device/google/redbull-kernel.

A little background, the vendor_boot.img has a ramdisk with kernel modules pulled from this directory.

Next is the problem with the vendor.img, which is cumbersome to unpack/repack with the updated kernel modules.

For the vendor.img, I suggest using product-based vendor overlay files. Vendor overlay files populate the product.img with the updated kernel modules that get overlaid on the vendor image on boot.

https://source.android.com/devices/bootloader/partitions/system-as-root#using-vendor-overlay

After doing all of the above and rebuilding asop, your vendor_boot.img and product.img (with vendor_overlay) has the updated kernel modules that will work with the new kernel.

[Droid SMU]

I am also encountering the same issue and used the same setup as you did.

The furthest I could go was copy all the kernel modules *.ko files into the aosp/device/google/redbull-kernel directory to solve the flashing issue.

After a successful flash, the device will boot until the point where it shows the android logo and gets stuck there.

Would appreciate some guidance on this. Thanks!

[Droid SMU]

Hi,

I managed to solve this issue! Was able to build and boot a custom kernel on the Pixel 5!

It seems that Google has made some architectural changes to the kernel in Android 11 as shown in the following link:

https://source.android.com/setup/start/android-11-release#kernel

What I realized is that after building the kernel, it generates a boot.img and vendor_boot.img

Usually we only have boot.img but this time there is an additional vendor boot file. This led me to the idea that we need to also flash this file to the phone.

These are the steps that I generally performed:

1. repo init -u https://android.googlesource.com/kernel/manifest -b android-msm-redbull-4.19-android11-qpr2
2. repo sync
   
3. Build custom kernel with build_redbull.sh script:
   ./build_redbull.sh
   
4. After building, boot.img and vendor_boot.img gets created in kernel/out/android-msm-pixel-4.19/dist/ along with loadable modules and Image.lz4 and Image.lz4-dtb
5. Go to AOSP folder.  Copy Image.lz4, Image.lz4-dtb, kernel-uapi-headers.tar.gz and all the *.ko kernel modules from
   kernel/out/android-msm-pixel-4.19/dist/ to aosp/device/google/redbull-kernel folder.
   
   You may back-up the aosp/device/google/redbull-kernel folder at another location in case we would like to restore to the default files.
   
6. Run make to rebuild the images
   
7. Copy the vendor_boot.img from kernel to out/target/product/redfin folder
   
8. adb reboot bootloader
9. fastboot flashall -w

fastboot version: 31.0.2-7242960

This was tested with AOSP android-11.0.0_r34 but I believe it can also work for other versions.

Hope this is able to help you save two weeks of development time. At least that was the time it took me to setup a base AOSP and kernel on the Pixel 5.

On Wednesday, 16 June 2021 at 06:35:08 UTC+8 afer...@gmail.com wrote:

[Seonung Jang]

Hello I've got same issue.. I try this solution exactly same. but i can't still solve.

Today I've checked the difference between the original factory image and the aosp image.

And I saw that the size of the original product.img is bigger(x10) then my product.img.

Could you tell me that how you solved the problem through https://source.android.com/devices/bootloader/partitions/system-as-root#using-vendor-overlay ?

I can't find vendor_overlay folder.. 

Thank you.

2021년 6월 23일 수요일 오후 2시 11분 13초 UTC+9에 droi...@gmail.com님이 작성:

[Zion Pan]

Thank you for the tips!

I was building AOSP android-12.0.0_r11 and android-msm-redbull-4.19-android12 kernel. However when I fastboot flashall, it hangs on step2 (rebooting to userspace fastbootd) then throw errors and fallback to recovery mode, showing some error like 'failed to find /dev/block/.../misc'.

I think it's not about the vendor_boot.img but the vendor.img (which packs all the .ko kernel modules) since I unpack and verify that the vendor_boot.img  generate by AOSP and the kernel modules are same as those from the kernel build.

Still trying to find a solution, or try Android 11 instead :<

[Droid SMU]

Hi Serena,

An yeong ha se yo!

I checked the size of my product.img in out/target/product/redfin and it is 229MB. Maybe you can compare the size of your product.img with mine.

I am using branch refs/tags/android-11.0.0_r34 for AOSP.

I didn't use the vendor_overlay method which you shared. I just copied the vendor_boot.img file from my kernel output folder to AOSP. Then I just flash directly without rebuilding again.

May I know which AOSP branch and phone are you using?

Hi Pangao,

Ni hao!

Sorry I haven't moved to Android 12 yet and I am still on Android 11. Maybe we might move to the newer android version once I have completed my work on 11.

Are you building an unmodified image of AOSP and kernel for Android 12?

Ok if you have checked that the vendor_boot.img are the same then you may skip that step. Have you also copied the kernel modules (.ko) files to device/google/<product> folder?

Perhaps try to also include all the .img and .tar.gz files.

Thank you.

Best regards,

Joshua

[Zion Pan]

Hi Joshua,

Thanks for your reply! 

> Are you building an unmodified image of AOSP and kernel for Android 12?

Yes and no. I did add some kernel options to the default, such as CONFIG_KPROBES, CONFIG_KPROBE_EVENT, CONFIG_FTRACE_SYSCALLS and so on. But I tried with unmodified kernel and it still won't boot.

> Have you also copied the kernel modules (.ko) files to device/google/<product> folder?

Yes, I've copied all the regular files to that folder (rm device/google/redbull-kernel/*; cp -v /build/android-kernel/out/android-msm-pixel-4.19/dist/* device/google/redbull-kernel/)

FYI, I just switch my AOSP repo to android-11_r34 and build the project using the above tricks (build kernel -> copy ko/img -> build aosp -> repalce vendor_boot), it works and successfully boot into android launcher!

Then I switch the kernel branch to android-msm-redbull-4.19-android12 and rebuid, it works, too.

Then I add my KPROBE config to the BUILD_CONFIG and rebuild, and also successfully boot to android launcher. However some of the APK will randomly crash, which is really wierd.

Now I am checking about LineageOS's repo, It seems that the kernel source is included (unlike AOSP, which only includes the kernel prebuilts). 

Thanks again,

Pangao

[Droid SMU]

Hi Pangao,

That's great to hear that you managed to build and load the android OS.

Seems like it needs a combination of android-12 kernel with android-11 aosp. I wonder what additional steps are needed to make it work with android-12 aosp + android-12 kernel.

Perhaps you could try with an unmodified and clean kernel and see if those apks that are crashing will work or not. This can help to verify whether the changes you made to the kernel configurations would affect those apks.

That's a great reference to use. I built LineageOS previously as well for Android 11 just to test out my development environment.

Hope it would help with your project as well.

Thanks.

Best regards,

Joshua

[Zion Pan]

Hi Joshua,

After some debugging, I finally make my custom kernel buid work (turns out that it's not a kernel issue). Let me rephrase the builds so far:

1. android-11 aosp with android-11 kernel: works, even with modified kernel config

2. android-11 aosp with android-12 kernel: works, even with modified kernel config

3. android-12 aosp with android-12 kernel: doesn't work, even with unchanged kernel cofnig

4. android-11 LineageOS (lineage-18.1) with in-tree kernel source: works, but changing kernel config results in boot-loop

Google is enforcing GKI (Generic Kernel Image) in recent devices, and vendors build their private drivers with the GKI driver interface.

Does it means that we cannot introduct breaking changes to the custom kernel in order to be compatible with vendor prebuilt drivers?

Thanks again for all your help,

Best reguards,

Pan

[Seonung Jang]

Hi :)

Hello, I solved this problem by copying the kernel module with this. (https://groups.google.com/g/android-building/c/ou630PviyDc/m/4Jdy0bEQBwAJ)

but, It was not reliable to boot.

I also ran into a little problem. After copying the custom kernel module (.ko) and kernel image (.lz4, lz4-dtb) to the aosp/device/google/redbull-kernel folder, I flash aosp images builded with custom kernel. 

The kernel was modified normally, but the kernel module(in /vendor/lib/modules/) was maintained in its original(same hash). 

I deleted even the all of .ko from the aosp/device/google/redbull-kernel, and tried to build aosp again with custom kernel image, module. but the original .ko was always installed. (this mean vendor_boot.img was not changed probably. i don't know why...)

I had the this problem with the latest aosp-12 on pixel5 and pixel4.

Thank you.

Seonung Jang

2021년 12월 8일 수요일 오전 11시 33분 3초 UTC+9에 pangao...@gmail.com님이 작성:

[shaoyang ding]

Hi 

I am trying to build AOSP 12 and kernel and unable to get it working on the Pixel 6.

my AOSP branch is  android-12.1.0_r2   and my kernel  branch is android-gs-raviole-5.10-android12L.

Pixel 6 works well if only flash the AOSP images, however, it won't boot up if I flash the boot.img build with kernel.

Here is my step:

1. mkdir android-kernel && cd android-kernel

2. repo init -u https://android.googlesource.com/kernel/manifest -b android-gs-raviole-5.10-android12L

3. repo sync

4. build/build.sh

I could see the logs below after I build successfully.
--------------------------------------------------------------------------------------------------------------------------
Files copied to /mnt/code/shaoyang/kernel/out/android-gs-pixel-5.10/dist
vendor_ramdisk-oriole.img is LZ4 compressed boot image created at boot.img
Signing the boot.img...
vendor boot image created at /mnt/code/shaoyang/kernel/out/android-gs-pixel-5.10/dist/vendor_boot.img
--------------------------------------------------------------------------------------------------------------------------

Note that the boot.img and vendor_boot.img had already been generated, then I use “fastboot flash boot boot.img” and “fastboot flash vendor_boot vendor_boot.img”to flash the boot.img and vendor_boot.img.

Unfortunately, the Pixel6 couldn’t boot up, it always stay in the fastboot mode.

Any help/instruction would be greatly appreciated!

Best Regards

Shaoyang