Running CROS inside a VM for fast developmnet.

[Adam Kearn]

Good evening,

I am not sure how to get CrOS to run inside a QEMU environment.

I have read the documentation (https://chromium.googlesource.com/chromiumos/docs/+/master/developer_guide.md#building-an-image-to-run-in-a-virtual-machine)

But it doesn't show you how to start the VM etc..

I would like to use this for faster development on my project, as at the moment I am having to: Build the image, flash to a USB and then finally boot it off my laptop.

It would be much easier to be able to just run it inside my dev machine.

Please, could you guide me in the steps in order to get CrOS to boot inside QEMU.

Thanks in advance.

Adam.

[Achuith Bhandarkar]

Please take a look at:

https://chromium.googlesource.com/chromiumos/docs/+/master/cros_vm.md

--
--
Chromium OS Developers mailing list: chromiu...@chromium.org
View archives, change email options, or unsubscribe:
https://groups.google.com/a/chromium.org/group/chromium-os-dev

[Mike Frysinger]

that doc is woefully specific to the simple chrome workflow. there's a lot of irrelevant steps in there for non browser work.

-mike

[Lepton Wu]

you can follow go/virgl which is for virgl bring up. But it give you
instructions for how to build a vm board for chrome os and how to
build
virgl enabled qemu (so you can have better graphics for your chrome os vm)

On Wed, Jul 24, 2019 at 12:39 PM Adam Kearn <adamk...@gmail.com> wrote:
>

[Achuith Bhandarkar]

We have a section here on how to build a VM image and launch it from within the chroot:

https://chromium.googlesource.com/chromiumos/docs/+/master/cros_vm.md#launch-a-locally-built-vm-from-within-the-chroot

Mike: we can expand this section as needed. Let me know if you have suggestions on what to add.

[Adam Kearn]

Thanks Achuith, Lepton and Mike,
Some point this week I’ll recompile and give that a go.
I’ll report back if I have any problems.

Thanks
Adam

[Adam Kearn]

Lepton,
I don’t have access to the internal documentation
So I can’t view your link (go/virgl)

[Lepton Wu]

OK. Google can bring up some instructions: (I haven't tried it yet)

https://snapcraft.io/qemu-virgil

[Achuith Bhandarkar]

If you discover holes in the documentation or have suggestions on how to make it better, let us know.

[Denny Lockhart]

This reminds me of a very early (2013) endeavor with QEMU & Chromium OS by 'the drinking cat' (drinkcat@). ;-)

~Denny

[Mike Frysinger]

it isn't about expanding the section, it's about repointing the doc.  "cros_vm" shouldn't be "you're using simple chrome", it should be "here's how you run CrOS in a VM, and here's how simple chrome users do it".

-mike

[Adam Kearn]

Hi all,

I am experiencing these problems when starting the VM.

What do you think?

.... Running ./image_to_vm.sh --test_image --board=$BOARD ....
Creating final image

Created image at /mnt/host/source/src/build/images/amd64-generic/R77-12368.0.2019_07_25_1236-a1

You can start the image with:

cros_vm --start --image-path /mnt/host/source/src/build/images/amd64-generic/R77-12368.0.2019_07_25_1236-a1/chromiumos_qemu_image.bin

(cr) ((e83ebd6...)) orbital-os@ubuntu ~/trunk/src/scripts $ cros_vm --start --image-path /mnt/host/source/src/build/images/amd64-generic/R77-12368.0.2019_07_25_1236-a1/chromiumos_qemu_image.bin

13:35:50: INFO: RunCommand: sudo -- kill -9 7271

kill: (7271): No such process

13:35:50: INFO: RunCommand: sudo -- /mnt/host/source/chroot/usr/bin/qemu-system-x86_64 --version

13:35:50: INFO: QEMU version 3.0.0

13:35:50: INFO: Pid file: /tmp/cros_vm_9222/kvm.pid

13:35:50: INFO: RunCommand: sudo -- /mnt/host/source/chroot/usr/bin/qemu-system-x86_64 -m 8G -smp 8 -vga virtio -daemonize -usbdevice tablet -pidfile /tmp/cros_vm_9222/kvm.pid -chardev 'pipe,id=control_pipe,path=/tmp/cros_vm_9222/kvm.monitor' -serial file:/tmp/cros_vm_9222/kvm.monitor.serial -mon 'chardev=control_pipe' -cpu SandyBridge,-invpcid,-tsc-deadline,check -device 'virtio-net,netdev=eth0' -device 'virtio-scsi-pci,id=scsi' -device 'scsi-hd,drive=hd' -drive 'if=none,id=hd,file=/mnt/host/source/src/build/images/amd64-generic/R77-12368.0.2019_07_25_1236-a1/chromiumos_qemu_image.bin,cache=unsafe,format=raw' -netdev 'user,id=eth0,net=10.0.2.0/27,hostfwd=tcp:127.0.0.1:9222-:22' -enable-kvm

qemu-system-x86_64: -usbdevice tablet: '-usbdevice' is deprecated, please use '-device usb-...' instead

Could not access KVM kernel module: No such file or directory

qemu-system-x86_64: failed to initialize KVM: No such file or directory

cros_vm: Unhandled exception:

Traceback (most recent call last):

  File "/mnt/host/source/chromite/bin/cros_vm", line 170, in <module>

    DoMain()

  File "/mnt/host/source/chromite/bin/cros_vm", line 166, in DoMain

    commandline.ScriptWrapperMain(FindTarget)

  File "/mnt/host/source/chromite/lib/commandline.py", line 961, in ScriptWrapperMain

    ret = target(argv[1:])

  File "/mnt/host/source/chromite/scripts/cros_vm.py", line 16, in main

    vm.VM(opts).Run()

  File "/mnt/host/source/chromite/lib/vm.py", line 364, in Run

    self.Start()

  File "/mnt/host/source/chromite/lib/vm.py", line 428, in Start

    self.RunCommand(qemu_args)

  File "/mnt/host/source/chromite/lib/device.py", line 92, in RunCommand

    return cros_build_lib.SudoRunCommand(*args, **kwargs)

  File "/mnt/host/source/chromite/lib/cros_build_lib.py", line 331, in SudoRunCommand

    return RunCommand(sudo_cmd, **kwargs)

  File "/mnt/host/source/chromite/lib/cros_build_lib.py", line 660, in RunCommand

    raise RunCommandError(msg, cmd_result)

chromite.lib.cros_build_lib.RunCommandError: return code: 1; command: sudo -- /mnt/host/source/chroot/usr/bin/qemu-system-x86_64 -m 8G -smp 8 -vga virtio -daemonize -usbdevice tablet -pidfile /tmp/cros_vm_9222/kvm.pid -chardev 'pipe,id=control_pipe,path=/tmp/cros_vm_9222/kvm.monitor' -serial file:/tmp/cros_vm_9222/kvm.monitor.serial -mon 'chardev=control_pipe' -cpu SandyBridge,-invpcid,-tsc-deadline,check -device 'virtio-net,netdev=eth0' -device 'virtio-scsi-pci,id=scsi' -device 'scsi-hd,drive=hd' -drive 'if=none,id=hd,file=/mnt/host/source/src/build/images/amd64-generic/R77-12368.0.2019_07_25_1236-a1/chromiumos_qemu_image.bin,cache=unsafe,format=raw' -netdev 'user,id=eth0,net=10.0.2.0/27,hostfwd=tcp:127.0.0.1:9222-:22' -enable-kvm

cmd=['sudo', '--', '/mnt/host/source/chroot/usr/bin/qemu-system-x86_64', '-m', '8G', '-smp', '8', '-vga', 'virtio', '-daemonize', '-usbdevice', 'tablet', '-pidfile', '/tmp/cros_vm_9222/kvm.pid', '-chardev', 'pipe,id=control_pipe,path=/tmp/cros_vm_9222/kvm.monitor', '-serial', 'file:/tmp/cros_vm_9222/kvm.monitor.serial', '-mon', 'chardev=control_pipe', '-cpu', 'SandyBridge,-invpcid,-tsc-deadline,check', '-device', 'virtio-net,netdev=eth0', '-device', 'virtio-scsi-pci,id=scsi', '-device', 'scsi-hd,drive=hd', '-drive', 'if=none,id=hd,file=/mnt/host/source/src/build/images/amd64-generic/R77-12368.0.2019_07_25_1236-a1/chromiumos_qemu_image.bin,cache=unsafe,format=raw', '-netdev', 'user,id=eth0,net=10.0.2.0/27,hostfwd=tcp:127.0.0.1:9222-:22', '-enable-kvm']

(cr) ((e83ebd6...)) orbital-os@ubuntu ~/trunk/src/scripts $ 

[Lepton Wu]

Make sure you've installed kvm kernel model. You can run "modporbe
kvm" before entering chroot.
Also check if /dev/kvm is there after you installed it.

[Mike Frysinger]

> Could not access KVM kernel module: No such file or directory
> qemu-system-x86_64: failed to initialize KVM: No such file or directory

you need to enable KVM support in your development system

-mike

--

[Adam Kearn]

Just checked my hardware,  it doesn't support KVM.  It's probably because I'm running it inside VMWare.

I'll have a look if I can enable it somewhere.

adam@ubuntu:~$ egrep -c '(vmx|svm)' /proc/cpuinfo

0

It needs to be >0 for it to work apparently.  According to this (https://sharadchhetri.com/2014/10/09/install-kvm-kernel-based-virtual-machine-ubuntu-14-04-lts-desktop/)

Thanks for the speedy reply.

-Adam

[Lepton Wu]

If you really can't have a physical machine for dev, here is the doc
for enabling nested kvm for vmware, but I guess performance will be
very poor.

https://communities.vmware.com/docs/DOC-8970

[Adam Kearn]

I am considering building a system just for development.

I was thinking of getting an AMD Ryzen Threadripper 1950X as they are 1/2 price at the moment.

But I am not sure how it would handle compiling chromium/chromiumos.  (16cores/32threads)

What CPU do you recommend?  I'm not sure if I should go AMD or Intel.

[Mike Frysinger]

both work fine for CrOS development

-mike

[Adam Kearn]

Ok thanks Mike.

This is where I am at right now.

When I run the following command nothing happens.  The VM doesn't open.

-Adam

adam@ubuntu:~/chromiumos/tot$ egrep -c '(vmx|svm)' /proc/cpuinfo

4

adam@ubuntu:~/chromiumos/tot$ modprobe kvm

adam@ubuntu:~/chromiumos/tot$ cros_sdk

(cr) ((e83ebd6...)) adam@ubuntu ~/trunk/src/scripts $ cros_vm --start --image-path /mnt/host/source/src/build/images/amd64-generic/R77-12368.0.2019_07_25_1236-a1/chromiumos_qemu_image.bin

14:25:25: INFO: RunCommand: sudo -- kill -9 299

kill: (299): No such process

14:25:25: INFO: RunCommand: sudo -- /mnt/host/source/chroot/usr/bin/qemu-system-x86_64 --version

14:25:25: INFO: QEMU version 3.0.0

14:25:25: INFO: Pid file: /tmp/cros_vm_9222/kvm.pid

14:25:25: INFO: RunCommand: sudo -- /mnt/host/source/chroot/usr/bin/qemu-system-x86_64 -m 8G -smp 8 -vga virtio -daemonize -usbdevice tablet -pidfile /tmp/cros_vm_9222/kvm.pid -chardev 'pipe,id=control_pipe,path=/tmp/cros_vm_9222/kvm.monitor' -serial file:/tmp/cros_vm_9222/kvm.monitor.serial -mon 'chardev=control_pipe' -cpu SandyBridge,-invpcid,-tsc-deadline,check -device 'virtio-net,netdev=eth0' -device 'virtio-scsi-pci,id=scsi' -device 'scsi-hd,drive=hd' -drive 'if=none,id=hd,file=/mnt/host/source/src/build/images/amd64-generic/R77-12368.0.2019_07_25_1236-a1/chromiumos_qemu_image.bin,cache=unsafe,format=raw' -netdev 'user,id=eth0,net=10.0.2.0/27,hostfwd=tcp:127.0.0.1:9222-:22' -enable-kvm

qemu-system-x86_64: -usbdevice tablet: '-usbdevice' is deprecated, please use '-device usb-...' instead

VNC server running on 127.0.0.1:5900

14:25:25: INFO: RunCommand: ssh -p 9222 '-oConnectionAttempts=4' '-oUserKnownHostsFile=/dev/null' '-oProtocol=2' '-oConnectTimeout=30' '-oServerAliveCountMax=8' '-oStrictHostKeyChecking=no' '-oServerAliveInterval=15' '-oNumberOfPasswordPrompts=0' '-oIdentitiesOnly=yes' -i /tmp/ssh-tmp1g4QGu/testing_rsa root@localhost -- true

14:25:58: INFO: (stdout):

Warning: Permanently added '[localhost]:9222' (ED25519) to the list of known hosts.

(cr) ((e83ebd6...)) adam@ubuntu ~/trunk/src/scripts $ 

[Achuith Bhandarkar]

You need to install vncviewer and run vncviewer 127.0.0.1:5900 to see the UI

[Lepton Wu]

See this line: VNC server running on 127.0.0.1:5900

Try to use vncviewer to connect to it

[Mike Frysinger]

everything looks correct to me.  what exactly are you expecting to happen next ?

ssh into the VM or run whatever tests you want.

-mike

[Adam Kearn]

Ahhh I was expecting a window to open with the VM running inside that. (like on the simple chrome setup)

It's all working now.

Thank you, everyone!!!!  I wouldn't have done it without you.

Regards,

Adam.

[Alpha Tang]

Sorry to steal an old thread as I found the subject most relevant.

For those who are interested in running CrOS under VM for fast Chromium development, there is another option to build and run it under VMWare clients, including Workstation and Fusion (on macOS). Overlays open-sourced with build instructions and a click-to-play prebuilt image is also provided:

https://github.com/FydeOS/overlay-variant-amd64-generic-vmware/

HTH.

-Alpha

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-os-d...@chromium.org.

[Adam Kearn]

Thank you soo much, this is really helpful.
Nice work!!!

I’ll have a go at compiling it tonight.

Thanks again.
-Adam