fdisk in a VM in a shell script

[Barret Rhoden]

Hi -

This started as a dirt-simple VM app that uses virtio-block and
virtio-cons, and only outputs: fdisk -l (not the full fdisk).
It's a pretty easy example of making a VM app - and it is less trivial
than "date" or "echo". And with some minor changes, we can support a
fully interactive fdisk. Read on!

Here's the "fdisk that is really fdisk -l". It's a little simpler and
smaller than the full fdisk:

---------------------
(akaros) / $ fdisk /dev/sdE0/data
Found valid GPT with protective MBR; using GPT

Disk /dev/vda: 234441648 sectors, 3881M
Logical sector size: 512
Disk identifier (GUID): a95f9f66-3f08-49d9-bbba-e33cedefb9ec
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 234441614

Number Start (sector) End (sector) Size Code Name
1 6144 33554432 15.9G 0700 Linux reserved
3 33554496 67108864 15.9G 0700 Linux reserved
5 64 2047 992K 0700 BIOS boot partition
6 2048 6143 2048K 0700 Linux filesystem

---------------------

It's a small shell script (~30 LOC) with an embedded initramfs.

(workstation) $ file obj/fdisk
obj/fdisk: Bourne-Again shell script executable (binary data)

The script is roughly this (minus the details of the embedded
initramfs and a little error handling):

---------------------
cd -P '#tmpfs'

cat > guest_cmdline <<EOF
console=none
lapic=notscdeadline
nortc
nozswap
EOF

# extracts the initramfs from this script, putting it in 'initramfs' in
# our tmpfs, which gets cleaned up when this script exits

extract_initramfs

vmrunkernel -k guest_cmdline -m 0x4000000 -f $DISK -s /lib/vmlinux -i initramfs

exit 0
---------------------

That's it on Akaros's side.

The initramfs is tinycore, with the rcS script (easiest to override)
replaced with the following: (Note the "-l")

---------------------
#!/bin/sh
# fdisk TC override

mount -t devtmpfs devtmpfs /dev
fdisk -l /dev/vda > /dev/hvc0
tinyreboot
---------------------

tinyreboot just calls sync() and reboot(), and Linux is paravirtualized
to use an AKAROS_VMCALL to shutdown.


Ah, but I'd like to run the *real* fdisk. Hit 'p' and 'q' and all
that. And pass "-l" if I want! Have no fear!

Actually, have some fear, since screwing with arguments and spaces in
sh (not bash) is a pain. I'll spare the details; suffice to say you
can get the shell script args from Akaros, put them on the guest's
cmdline, and pass them along to fdisk in the VM.

So we add this on the Akaros side:

cat > guest_cmdline <<EOF
...
fdisk_args=$FDISK_ARGS
EOF

And change the fdisk line in the guest rcS init script:

fdisk $FDISK_ARGS /dev/vda < /dev/hvc0 &> /dev/hvc0

That's right - we can *read* from /dev/hvc0 too! How does it feel
using this? Just like fdisk on Linux. I hit those 'p's and 'q's below
on Akaros:

---------------------
(akaros) / $ fdisk /dev/sdE0/data
Found valid GPT with protective MBR; using GPT


Command (m for help): p
Disk /dev/vda: 234441648 sectors, 3881M
Logical sector size: 512
Disk identifier (GUID): a95f9f66-3f08-49d9-bbba-e33cedefb9ec
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 234441614

Number Start (sector) End (sector) Size Code Name
1 6144 33554432 15.9G 0700 Linux reserved
3 33554496 67108864 15.9G 0700 Linux reserved
5 64 2047 992K 0700 BIOS boot partition
6 2048 6143 2048K 0700 Linux filesystem

Command (m for help): q

(akaros) / $
---------------------

How big was the initrd? Not too bad: 2.9 MB (gz). Here are the
offenders.

(workstation) $ ./cat_cpio_gz.sh initramfs.cpio.gz | tail
115160 sbin/blkid
118384 sbin/udevd
124424 lib/ld-2.22.so
153152 lib/libblkid.so.1.1.0
154544 usr/bin/sudo
220664 sbin/e2fsck
241968 lib/libext2fs.so.2.4
541731 bin/busybox
981256 lib/libm-2.22.so
1485904 lib/libc-2.22.so

I yanked out libstdc++, and can yank more if needs be. One could even
automate that process too (which is what LightVM/TinyX did).

Since I'm only calling three programs, we can check which libraries they
need. Mount and fdisk are both busybox, and tinyreboot just needs
libc. All together, it's just:

/lib64/ld-linux-x86-64.so.2
/lib/x86_64-linux-gnu/libcrypt.so.1
/lib/x86_64-linux-gnu/libm.so.6
/lib/x86_64-linux-gnu/libc.so.6

It probably took me two minutes to remove all files >= 20KB that were
not one of those, down to about 1.9 MB (compressed). To be fair,
vmlinux isn't small; it's around 13 MB (uncompressed). I like have a
single vmlinux to use, and didn't feel like messing around with
config=tinyconfig. Actually, the same Linux kernel and tinycore
initramfs that works in Akaros works in Qemu too.

Making these customizations is relatively easy with the new versions of
my linux-guest/akaros/ scripts.

full_guest_build.sh has a spot where you can tell it which programs you
want added to tinycore, e.g. my tinyreboot or ufs for the 9p filesystem
mounter. You can also tell it what files you want removed, e.g.
libstdc++.so.

You can also adjust the PACKAGES variable in setup_tinycore.sh, which
will specify which tinycore tcz packages to pull in. For fdisk, I
pulled in no packages. For a typical Linux VM, I'll usually have
openssh and maybe strace. setup_tinycore.sh will figure out the
tinycore dependencies and install the right packages. (The old version
required *you* to figure out the deps).

If you want to override rcS to run a VM app, setup_tinycore.sh can
handle that too. Just change the rcS-link to point to whatever file you
want (or nothing, if you want to keep the TC default).

I can quickly rebuild 'fdisk' after a change too - either to the shell
template on akaros (fdisk.sh) or the rcS-fdisk, and have a new fdisk
read to go in seconds:

---------------------
(workstation) $ time SKIP_KERNEL=1 ./full_guest_build.sh ~/akaros/linux-guest/
Downloading TC distro
Extracting TC distro
24280 blocks
Downloading packages
Extracting TCZs
Rebuilding CPIO
7349 blocks
Compressed initramfs at /home/brho/akaros/linux-guest//akaros/initramfs.cpio.gz
Copying to devbox
fdisk 100% 1890KB 1.8MB/s 00:01

real 0m4.211s
user 0m1.050s
sys 0m1.374s
---------------------

SKIP_KERNEL=1 is pretty sweet. No need to rebuild the entire vmlinux
when we have a standalone initramfs (which VMRK has been able to handle
for a while now).

Anyway, check out the linux-guest repo and pull the latest Akaros
master branch if you want to play around. (And reinstall your kernel
headers, make xcc-install-headers).

Barret