Blank screen when booting ChromiumOS kernel alone

[gm]

Hello there,

I have been successful at building&booting a full ChromiumOS (dev) via USB/SD - although shutting down via poweroff doesn't seem to work.

However, I get a blank screen if I try to use its kernel stand-alone with a custom command line:

console=tty1 debug verbose root=/dev/sda1 rootwait rw lsm.module_locking=0

If I use the vmlinuz (v3.10) extracted from the OEM ChromeOS clapper, the above boots fine and seems fully functional; during the very fast boot sequence I can see a crash dump about fbdev-something, but it's too fast for me to catch.

If I rebuild the kernel partition with exactly the same vbutil_kernel command but with the ChromiumOS kernel, I get instead a blank screen.

I've also tried using i915.modeset=1, but doesn't change the net effect. As a side note, I can shutdown with a single power button press when I am in such blank screen mode, and the chromebook is not rebooting - so it must think kernel is doing something (I guess).

I'm using kernel 3.10 from release-R42-6812.B and this is the net CONFIG_ diff of the working ChromeOS kernel (left) and non-working ChromiumOS one (right): http://pastebin.com/raw.php?i=7gY05PWb

To me, none of those options can interfere with framebuffers, I suspect the kernel is instantly panicking when coming alive...

Any pointers on how to troubleshoot this? Next on my list would be serial-over-USB output.

--

  gm

[Sonny Rao]

On Tue, Jun 2, 2015 at 2:53 AM, gm <g.g.ma...@gmail.com> wrote:
> Hello there,
>
> I have been successful at building&booting a full ChromiumOS (dev) via
> USB/SD - although shutting down via poweroff doesn't seem to work.
>
> However, I get a blank screen if I try to use its kernel stand-alone with a
> custom command line:
>
> console=tty1 debug verbose root=/dev/sda1 rootwait rw lsm.module_locking=0
>
>
> If I use the vmlinuz (v3.10) extracted from the OEM ChromeOS clapper, the
> above boots fine and seems fully functional; during the very fast boot
> sequence I can see a crash dump about fbdev-something, but it's too fast for
> me to catch.
>
> If I rebuild the kernel partition with exactly the same vbutil_kernel
> command but with the ChromiumOS kernel, I get instead a blank screen.

how are you building the kernel? emerge?

> I've also tried using i915.modeset=1, but doesn't change the net effect. As
> a side note, I can shutdown with a single power button press when I am in
> such blank screen mode, and the chromebook is not rebooting - so it must
> think kernel is doing something (I guess).
>
> I'm using kernel 3.10 from release-R42-6812.B and this is the net CONFIG_
> diff of the working ChromeOS kernel (left) and non-working ChromiumOS one
> (right): http://pastebin.com/raw.php?i=7gY05PWb


> To me, none of those options can interfere with framebuffers, I suspect the
> kernel is instantly panicking when coming alive...
>
> Any pointers on how to troubleshoot this? Next on my list would be
> serial-over-USB output.

One trick for debugging is to boot te kernel that panics followed by a
known working kernel so you can look at /dev/pstore/console-ramoops
and then you can see the panic.

A way to do this is to either boot the panicing kernel using a USB
stick or boot the good kernel on a USB stick after the panic.

Another (more tricky way) is to put the bad kernel on the kernel
partition which isn't used, but don't set the successful bit and have
try count of one so that it falls back to the good kernel after it
panics. For this I use the update_kernel.sh script with the
--partition flag to explicitly specify the kernel partition, the
--bootonce flag and the --rootoff flag (you might not need this if
you're hardcoding the partition to /dev/sda1)


>
> --
> gm
>
> --
> --
> Chromium OS Developers mailing list: chromiu...@chromium.org
> View archives, change email options, or unsubscribe:
> http://groups.google.com/a/chromium.org/group/chromium-os-dev?hl=en
>
> To unsubscribe from this group and stop receiving emails from it, send an
> email to chromium-os-d...@chromium.org.

[gm]

On Tuesday, 2 June 2015 21:46:20 UTC+2, Sonny Rao wrote:

On Tue, Jun 2, 2015 at 2:53 AM, gm <g.g.ma...@gmail.com> wrote:
> Hello there,
>
> I have been successful at building&booting a full ChromiumOS (dev) via
> USB/SD - although shutting down via poweroff doesn't seem to work.
>
> However, I get a blank screen if I try to use its kernel stand-alone with a
> custom command line:
>
> console=tty1 debug verbose root=/dev/sda1 rootwait rw lsm.module_locking=0
>
>
> If I use the vmlinuz (v3.10) extracted from the OEM ChromeOS clapper, the
> above boots fine and seems fully functional; during the very fast boot
> sequence I can see a crash dump about fbdev-something, but it's too fast for
> me to catch.
>
> If I rebuild the kernel partition with exactly the same vbutil_kernel
> command but with the ChromiumOS kernel, I get instead a blank screen.

how are you building the kernel? emerge?

Tried both the vmLinuz of an emerge (FEATURES=noclean to be able to retrieve byproducts) and mine, same net effect.

When building it myself, I use commands alike to emerge (I think):

make ARCH=x86_64 oldnoconfig WIFIVERSION=-3.8

make ARCH=x86_64 -k WIFIVERSION=-3.8

make ARCH=x86_64 INSTALL_PATH=$PREFIX/boot INSTALL_FW_PATH=$PREFIX/lib/firmware INSTALL_MOD_PATH=$PREFIX/ firmware_install modules_install WIFIVERSION=-3.8

make ARCH=x86_64 -s --no-print-directory kernelrelease WIFIVERSION=-3.8

 

> I've also tried using i915.modeset=1, but doesn't change the net effect. As
> a side note, I can shutdown with a single power button press when I am in
> such blank screen mode, and the chromebook is not rebooting - so it must
> think kernel is doing something (I guess).
>
> I'm using kernel 3.10 from release-R42-6812.B and this is the net CONFIG_
> diff of the working ChromeOS kernel (left) and non-working ChromiumOS one
> (right): http://pastebin.com/raw.php?i=7gY05PWb


> To me, none of those options can interfere with framebuffers, I suspect the
> kernel is instantly panicking when coming alive...
>
> Any pointers on how to troubleshoot this? Next on my list would be
> serial-over-USB output.


One trick for debugging is to boot te kernel that panics followed by a
known working kernel so you can look at /dev/pstore/console-ramoops
and then you can see the panic.

A way to do this is to either boot the panicing kernel using a USB
stick or boot the good kernel on a USB stick after the panic.

Another (more tricky way) is to put the bad kernel on the kernel
partition which isn't used, but don't set the successful bit and have
try count of one  so that it falls back to the good kernel after it
panics.  For this I use the update_kernel.sh script with the
--partition flag to explicitly specify the kernel partition, the
--bootonce flag and the --rootoff flag (you might not need this if
you're hardcoding the partition to /dev/sda1)

Wow, many thanks for telling me about /dev/pstore/console-ramoops.

I have the "bad" one on a USB stick, plus the working one both on SSD and on another USB stick, so it should be easy for me to switch.

I assume no special kernel command line is needed for /dev/pstore/console-ramoops ? Sweet, will give this a try.

--

  gm

[Alexandru Stan]

On Tue, Jun 2, 2015 at 2:53 AM, gm <g.g.ma...@gmail.com> wrote:

To me, none of those options can interfere with framebuffers, I suspect the kernel is instantly panicking when coming alive...

We disabled the vty console which his built in the kernel because it interferes with frecon and freon.

If you're just trying to use the chromeos tree with your own distro i suggest compiling with these USE flags: vtconsole fbconsole. It re-enables the console.

Here's the change that might interest you: https://chromium-review.googlesource.com/#/c/223215/

Added Dominik if you have any more questions.

Alexandru Stan (amstan)

[gm]

Thanks Alexandru, that now makes a lot of sense. I had double-checked that all VT and FB options where same as working one in the .config, but apparently it's not enough. Will give the emerge a try.

 

By the way, would it possible to find which kernel commit was used to build my ChromeOS kernel, or that comes from a non-public tree?

I have on SSD an up-to-date firmware Google_Clapper.5216.199.7, platform 6812.88.0 stable-channel, ideally I'd just always be using the corresponding commit of what is on my official ChromeOS partition.

--

  gm

[Yuly Novikov]

You could have found this info in a recent thread here instead of asking.

You can use https://chromium.googlesource.com/chromiumos/manifest-versions/+/master to find the git commit hash of 3.10 kernel corresponding to some CrOS version.

For example, for CrOS 6812.88.0 in https://chromium.googlesource.com/chromiumos/manifest-versions/+/master/LKGM-candidates/buildspecs/42/6812.88.0-rc2.xml it's:

<project name="chromiumos/third_party/kernel" path="src/third_party/kernel/v3.10" revision="4322752ed3450ac3aedeb92bef4f1d9fc25c2820" upstream="refs/heads/release-R42-6812.B-chromeos-3.10"/>

Also, if you are working on a branch, I find it convenient to have the branch checked out: http://www.chromium.org/chromium-os/how-tos-and-troubleshooting/working-on-a-branch

[Dominik Behr]

You can see what branch was used in which release using git in your
chromium tree
~/trunk/src/third_party/kernel/v3.14 $ git branch -r | grep R44
cros/release-R44-7077.B-chromeos-3.10
cros/release-R44-7077.B-chromeos-3.14
cros/release-R44-7077.B-chromeos-3.18
cros/release-R44-7077.B-chromeos-3.4
cros/release-R44-7077.B-chromeos-3.8

VT and FB console eat memory and they are incompatible with freon
project. When using freon we dont have X server and virtual terminals.
Chrome talks directly to KMS/DRM and console (frecon is implemented
all in user space on top of KMS/DRM as well).

You may consider using kmscon
(https://wiki.freedesktop.org/www/Software/kmscon/) or something
similar for your project instead of relying of VT/FB console.

[Mike Frysinger]

none of our devices use private kernels.  that wouldn't be very nice ;).

-mike

--

[gm]

On Tuesday, 2 June 2015 23:30:00 UTC+2, Dominik Behr wrote:

You can see what branch was used in which release using git in your
chromium tree
~/trunk/src/third_party/kernel/v3.14 $ git branch -r | grep R44
  cros/release-R44-7077.B-chromeos-3.10
  cros/release-R44-7077.B-chromeos-3.14
  cros/release-R44-7077.B-chromeos-3.18
  cros/release-R44-7077.B-chromeos-3.4
  cros/release-R44-7077.B-chromeos-3.8

Thanks Dominik for your kind reply, I was looking for a confirmation :)

 

VT and FB console eat memory and they are incompatible with freon
project. When using freon we dont have X server and virtual terminals.
Chrome talks directly to KMS/DRM and console (frecon is implemented
all in user space on top of KMS/DRM as well).

That sounds efficient :)

 

You may consider using kmscon
(https://wiki.freedesktop.org/www/Software/kmscon/) or something
similar for your project instead of relying of VT/FB console.

Thing is, the kernel taken from my SSD partition (not built by me) worked at first try when booting a Debian/Ubuntu rootfs (VTs etc), hence I was wondering if I were building the right version. Thanks for these pointers.

--

  gm

[gm]

On Wednesday, 3 June 2015 05:42:08 UTC+2, Mike Frysinger wrote:

none of our devices use private kernels.  that wouldn't be very nice ;).

-mike

Yeah, I understand :) I thought more along the lines: maybe stays private for a while (like a "cool-down" period), then it's published. It turns out it was a newbie mistake: I was not getting exactly same kernel because my ~/chromiumos repo had been initialized with R44, and then I had manually switched only the kernel to R42. By using the right SDK tools instead, I got the expected results.

Good to know that in R44 VTs/FBs will be gone; I don't have a particular requirement for them, was just looking for a positive boot test with custom-built kernel (and I have it now).

--

  gm

[Mike Frysinger]

during very early bring up we might have a private vendor kernel, but that gets moved to the open source side pretty quickly, and certainly long before the device is actually released
-mike

[Marc Herbert]

On Tuesday, 2 June 2015 02:53:14 UTC-7, gm wrote:

Any pointers on how to troubleshoot this? Next on my list would be serial-over-USB output.

Dunno about serial-over-USB, however if by any chance you're in some lab somewhere with a servo board or an early platform then you should really like "real" serial ports:

http://www.chromium.org/chromium-os/how-tos-and-troubleshooting/serial-debugging-howto

This page looks complete; I think it may only miss some examples like:

USE="pcserial tty_console_ttyS0" cros_workon_make --install sys-kernel/...
.build_image --boot_args="console=ttyS0,115200 loglevel=7" # this one just works for me; ./build_image is too time-consuming to go and try the other, documented one.

BTW, any news on the "Gerritification" of this documentation? I mean this: "Improved collaboration for Chromium development documentation" https://groups.google.com/a/chromium.org/d/msg/chromium-os-dev/k40tGJhnzC4/X9Kza9-I1JwJ