Pixelbook fails to boot up after updating kernel to v5.4

[Eason]

Hi all,

Today I updated my Pixelbook with pre-built image R88-13527.0.0-eve-arc-r, when it worked.

But after I build the latest kernel on branch v5.4 in cros_sdk, and deployed the kernel remotely, the pixelbook failed to boot up, stuck after info screen 'OS verification if off', just black screen.

Several weeks ago, the kernel v5.4 could boot into desktop at least, but failed to launch Linux Beta. But it seems it turn to worse now.

Do you guys plan to update the kernel to v5.4 branch on 'eve' board ?

BTW, another question how can I roll back the kernel to 4.4, because I'm not able to boot my pixelbook, and I don't want to flash it with u-disk that can empty all my data on partition /mnt/stateful_partition.

Great thanks.

--Yi

[Evan Benn]

I dont know the status of kernels on eve, but you can get your kernel
back by booting from usb:

https://chromium.googlesource.com/chromiumos/docs/+/master/kernel_development.md#Recover-from-a-bad-kernel-update

> --
> --
> Chromium OS Developers mailing list: chromiu...@chromium.org
> View archives, change email options, or unsubscribe:
> https://groups.google.com/a/chromium.org/group/chromium-os-dev
> ---
> To unsubscribe from this group and stop receiving emails from it, send an email to chromium-os-d...@chromium.org.

[Eason]

Hi Evan,

Thank your reply. According to the guide, I recovered the kernel to the original one. But meanwhile it dropped all my data saved in /mnt/stateful_partition/.

Do you have any idea, how can I recover kernel image only, but not touch any other files.

[Kevin Chowski]

Hello Yi!

Sorry about the breakage of crostini on 5.4. I was investigating that issue (specifically for the Pixelbook), and it has been fixed by 13507.0.0. It was also cherry-picked to the 87 branch, at version R87-13505.5.0. Please let us know if you are still having trouble past those versions.

We are also planning on shipping 5.4 to eve devices in the 87 release, which is currently rolled out to dev users. I do not know the schedule for when this will happen for eve-arc-r, sorry about that. I'll reach out to someone about that now. I'll also forward the report about this broken screen issue.

As for how to recover, you could consider booting from USB and copying the chromebook's /mnt/stateful_partition out to some other device, and then load it back after a recovery which wipes that partition.

[Eason]

Hi Kevin, 

Nice to meeting you.

Glad to know kernel 5.4 will ship to eve.

I'm going to try that version of crostini.  About the bug fix, which eve built has already had it, eve or eve-arc-r? As my understanding 'eve' built with Android-p as container, while 'eve-arc-r' built with Android-r as a VM.

The crostini issue I mentioned previously happened on board 'eve-arc-r'.

Is there a web tool to track Chrome OS issue, especial for Linux/Android VM issue? Or here is good enough ?

Thanks

  --Yi

[Kevin Chowski]

Hey Yi, thanks for the reply.

For the crostini bug-fix, I should clarify that there are actually two issues that we are tracking:

1) The issue I personally was looking into (the one I mentioned in my previous mail) was related to crostini not installing correctly, and was fixed in the ChromeOS-specific 5.4 kernel directly (CL landed in the main branch here: https://crrev.com/c/2446009, cherry-picked to the 87 branch here: https://crrev.com/c/2456307). So your build should pick it up when you try to upgrade your kernel, assuming your repository has sync'd past the necessary changes. (Also, we are not 100% sure why this change fixes the crostini installation, but I validated with/without the change many times and it does seem to at least mitigate setup failures.)

2) We are currently investigating issues related to the crostini container around suspend/resume of the laptop. There is not yet a fix for this problem, unfortunately, but it is a high priority for our team.

As for how to report bugs: it is OK to ask questions on this forum, but messages email inboxes often get lost. Instead, prefer to file issues using our online tracker: https://bugs.chromium.org/p/chromium/issues/entry . You can use a predefined template (in this case, "Defect on Chrome OS" might work), but it's also important to find a good component (for example, in this case "OS>Kernel>Display" might be appropriate, and it might be useful to add in some component related to ARC as well). And finally, if you've filed a bug and no one has commented on it, it's always OK to email this group with a link to the bug you filed to try to get someone to look at it :)

-Kevin

[Kevin Chowski]

Also, to follow up: we are not quite sure how you have access to eve-arc-r, since according to some members of that team it should not be publicly available :) Are you getting access to prebuilt images because your employer is a partner with Google? If so, please use the internal bug tracker (buganizer) instead of the bugs.chromium.org link I provided in my prior email, under the "ChromeOS > Software > ARC++ > ARCVM" component.

[Eason]

Status update.

Thanks Kevin's explanation. I found the black screen issue is likely related to  my using '--ignore_verity' for script update_kernel.sh. I have to disable the verify before installing a kernel instead of ignore it.

So we can ignore the the black screen issue now, But the crostini failure happens again.

I synced all repos and rebuilt kernel, make sure I have following change that you mentioned. But unfortunately, crostini still failed to launch. I'm using 'eve' built instead of eve-arc-r this time. 

Previously I though the some  android vm failure influenced the Linux VM. But after trying eve built, it seems nothing business with 'arc'.

BTW, could you guide me a little bit, how can I gather some useful information of crostini of crosvm, I can update here, assisting you root cause.

commit b4f64ec190f4aa8e176a06f9a1f437a06731ae40                                                      

Author:     Joel Fernandes <joe...@google.com>                                                       

AuthorDate: Fri Oct 2 08:59:15 2020 -0400                                                            

Commit:     Commit Bot <commi...@chromium.org>                                                     

CommitDate: Fri Oct 2 19:25:48 2020 +0000                                                            

                                                                                                     

    FIXUP: CHROMIUM: arch/x86: Prevent buffer flushing for trusted tasks                             

                                                                                                     

    This patch causes a crostini tast test failure. Fix it.                                          

                                                                                                     

    v4.19 already has the original patch fixed.                                                      

                                                                                                     

    BUG=b:152605392                                                                                  

    TEST=crostini.basic.artifact tast test.                                                          

    Signed-off-by: Joel Fernandes <joe...@google.com>                                                

    Change-Id: Ie64b6cbac5f45caa173cffcf17d99d30b61d0774                                             

    Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/2446009  

[Eason]

Thanks! That guide helps me a lot, pretty cool! :p

On Monday, October 12, 2020 at 2:44:09 PM UTC+8 evan...@google.com wrote:

[Kevin Chowski]

We have solved all outstanding crostini problems (at least on Eve) that we know about. The fix was committed to the master branch (https://crrev.com/c/2477396, available in 13538.0.0) and also the M87 branch (https://crrev.com/c/2481611, available in 87-13505.14.0).

If you are still seeing problems after those versions, please file feedback by pressing Alt+Shift+i  right after you experience a problem and describe what has gone wrong. That's the best way for you to help us debug what's going on. See also https://support.google.com/chromebook/answer/2982029?hl=en. Filing a bug as I mentioned before is also a good idea sometimes, but filing feedback is definitely important for us to get a good view of what's gone wrong.