Importing a disk image

[Jonathan Bayer]

I have a situation where I have several VMs on an older system, stored
as disk images. These were created using virt-manager.

I'd like to be able to shut them down and import them (somehow) into
Ganeti.

Are there any pointers or how-tos on this? For various reasons, these
cannot be rebuilt.


Thanks in advance.


JBB

[Nicolas Sebrecht]

Here is how I did it manually for servers in production. Be aware that
while it should work, there is no guaranty and you must make you own
tests.

I have in mind to write something for the official docs but I still have
to get support from my managers. I assume DRBD target disk in Ganeti but
you can do something somewhat similar for LVM.


Conversion from raw disk (libvirt or physical disk) to DRBD in Ganeti
=====================================================================

This method is POTENTIALLY VERY HARMFUL. Make sure to understand the
whole documentation before proceeding.

First, install whatever driver in the VM you might need due to the
conversion (refer to the Ganeti doc to know what kvm/xen drivers are
necessary).

Then, convert your libvirt (created with virt-manager) disk images to
raw format:

# qemu-img convert disk-libvirt.img -O raw disk-libvirt.raw

Then, get the virtual size of the disk in bytes:

# qemu-img info disk-libvirt.raw
image: disk.img
file format: qcow2
virtual size: 29G (31457280000 bytes)
disk size: 5.9G
cluster_size: 65536
#

Here is a disk of 31457280000 bytes. You can check the value with fdisk

# fdisk- l disk.raw
Disk disk.raw: 31.5 GB, 31457280000 bytes
255 heads, 63 sectors/track, 3824 cylinders, total 61440000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0007053b

Device Boot Start End Blocks Id System
disk.raw1 * 2048 61437951 30717952 83 Linux
#

reporting the same 31457280000 bytes.

Next, you create the Ganeti VM. The disk size must exactly match.
Convert the bytes into mebibytes. In this case:

31457280000 bytes / 1024 / 1024 = 30000 mebibyte (which is NOT 30 Giga Bytes)

So, when creating the Ganeti instance, give this calculated value to the
disk option size:

--disk 0:size=30000m

Wait until the VM is fully created (DRBD is synchronized). Get the DRBD
file device name with:

# gnt-instance info my.inst
<...>
Disk template: drbd
Disks:
- disk/0: drbd8, size 29.3G
access mode: rw
nodeA: radium.piing.lan, minor=6
nodeB: hafnium.piing.lan, minor=6
port: 11011
auth key: 37228a75a8579acaefee5586f4ca10eba95a13ef
on primary: /dev/drbd6 (147:6) in sync, status ok
on secondary: /dev/drbd6 (147:6) in sync, status ok
<...>

Here, we want /dev/drbd6. BE CARE this is the correct instance and
correct file device to not erase other data.

Active the disks:

# gnt-instance activate-disks my.inst

Check the target has the exact same size as the source:

# fdisk -l /dev/drbd6

Disk /dev/drbd6: 31.5 GB, 31457280000 bytes
255 heads, 63 sectors/track, 3824 cylinders, total 61440000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/drbd6 doesn't contain a valid partition table
#

In this sample, the 31457280000 bytes is what we expect.

Copy the disk with dd

# dd if=disk-libvirt.raw of=/dev/drbd6

Finally, deactivate the disks of the ganeti instance

# gnt-instance deactivate-disks my.inst

You should be able to start and use the Ganeti VM. If not, please report
it here to give me a chance to improve this "mini how-to".

--
Nicolas Sebrecht

[Jean-François Maeyhieux]

If it's linux systems, you could use ganeti-intance-image with a specific variant tarball based.

We deploy gentoo linux here this way but that could be used for any linux systems that have been tarballed.

So the way is:

- install ganeti-instance-image

- mount your image

- make a tar file with the image content

- create a variant image+"your-variant" adapted (filesystem type, swap size, arch) based on the default one with IMAGE_TYPE="tarball" and IMAGE_DIR set to a directory where ganeti could find the tarball.

- activate grub hook if you wish ganeti installed it for you

- activate others hooks if needed like network if you need to reconfigure instance IP/subnet at creation time

- install your instance based on the precedent variant using -o image+"your-variant"

You could re-use the variant definition with another tarball or use a new one for the next system.

May be there is another solution using "gnt-backup import" based on dump of your disk images with a manual config.ini creation.

Jean-François

[Jonathan Bayer]

This won't work, since the images do not conform to the layout which ganeti-instance-image requires.

JBB

[Lance Albertson]

Are these Linux VMs?

The method I used before when converting my old system to ganeti should still work in this case. Your mileage may vary:

1. Manually create a "no-install" VM on the ganeti cluster, manually format/mkfs, mount the volumes on the node
2. Do an initial rsync of all files using "rsync -avH --numeric-ids" while the system is live
3. When ready to switch, shutdown old VM, mount the raw file disk somehow, then do a final rsync to node OR boot VM to single user and rsync manually.
4. Fix fstab/grub/etc, install the MBR if needed, on new VM
5. Boot and success (hopefully)

When I did it I was going from an iSCSI block device instead of a file based device. I believe you can mount file based systems using qemu-img but I might be wrong.

Its not nearly as automated but it gets the job done and worked for me in the past.

--
Lance Albertson

Associate Director of Operations

Oregon State University | Open Source Lab 

[Jean-François Maeyhieux]

Could you please argument on this point ?

Which layout is require by ganeti-instance-image ?

I could migrate any kind of linux/unix by a tarball with ganeti-instance-image.

By the way you have a way to mount your disk image using qemu-nbd.

Jean-Francois

[Jonathan Bayer]

Ganeti-instance-image requires a specific disk layout (https://code.osuosl.org/projects/ganeti-image/wiki/Creating_OS_Images):

vda1    /boot
vda2    swap
vda3    /

These VMs are most definitely NOT setup that way.

I know of the tarball install, but the partition layout is my sticking point.

Regarding qemu-nbd, there aren't any packages yet for nbd for RHEL 5/6.  I can easily make a tarball by starting the instance and mounting an NFS share, and then tar'ing the entire system into a tarball on the NFS share.  But beyond that, I'm a bit lost.

I'll try this with a test VM and see how it goes.


JBB

[Jonathan Bayer]

Followup on my previous message, specifically about mounting a qcow image.

since the nbd kernel module isn't in the RHEL5/6 systems, the instructions in the following link show how to mount the partitions:

http://dailypackage.fedorabook.com/index.php?/categories/11-System-Recovery-Week


An abbreviated version is:

to mount:

losetup -f -v disk1.img
kpartx -a -v /dev/loop0
vgscan
vgchange -ay

Find and mount the root filesystem on a temporary directory
Cd into that directory, then find and mount all other partitions which contain filesystems on their appropriate directories.

to unmount

vgchange -an (vgname)
kpartx -d /dev/loop0
losetup -d /dev/loop0

On 11/29/2012 6:49 AM, Jean-François Maeyhieux wrote:

[Jean-François Maeyhieux]

Allright, tarball based install doesn't care about the layout:

Ganeti should use /boot content in the tarball to populate the the /boot partition.

That is a way to create the initial tarball:

So you have to make a specific tarball that include all system partitions but exclude specific mounting point like /dev, /sys, /proc

If you don't have separate partitions for /usr /var, etc... it's easy using cp -x , rsync -x or tar --one-file-system.

But there is a better option:  use a new mounting point with the bind option of mount:

#mkdir /mnt/backup

#mount -o bind / /mnt/backup/

#mount /boot && mount -o bind /boot /mnt/backup/boot (if separate boot partition)

#mount -o bind /var /mnt/backup/var                             (if separate var partition)

#mount -o bind /usr /mnt/backup/usr                              (if separate usr partition)

....

* /dev could be tricky:

  - if you don't use udev, you have to include /dev: 

    # mount -o bind /dev /mnt/backup/dev

  - If you use udev, you have to exclude it but you need to be sure to have some specific devices in /dev in order to boot the system.

    If you don't have them, you have to create them with mknod

    # DIR_BACKUP=/mnt/backup

    # test -c ${DIR_BACKUP}/dev/null    || mknod -m 660 ${DIR_BACKUP}/dev/null c 1 3

    # test -p ${DIR_BACKUP}/dev/initctl   || mknod -m 600 ${DIR_BACKUP}/dev/initctl p

    # test -c ${DIR_BACKUP}/dev/tty1     || mknod -m 660 ${DIR_BACKUP}/dev/tty1 c 4 1

So you're ready to make the tarball but you have to exclude specific distribution runtime folder too like /var/run, /var/lock or other directory that your init system use to save /etc/init.d status, lock, pid , etc...

Else your first boot couldn't start properly init services. 

# tar cvzf --exclude '/var/run/*' --exclude '/var/run/lock/*' mysystem.tar.gz -C /mnt/backup .

Obviously It's really more easy to do the tarball if you could stop the system since you only have to mount all system partitions and make a tarball of the whole since you don't have to care about /proc, /dev, runtime directory and so on.

Jean-François

[Jonathan Bayer]

I've written a script to mount all filesystems from a disk image and then make a complete tarball.

Once I've finished testing it, it will be posted on my blog as well as being sent the Lance along with a few other scripts I have for possible inclusion into instance-image


Thanks


JBB

[Lucas Yamanishi]

I can attest to this method. I am in the process of migrating from
qcow2 files and it works great. It's much easier than it sounds, and
would be scriptable to a point.

I've also been extending my images as I do this. All my old images
are 16GB in size, but with new hardware I can afford to increase my
base size to 32GB-- thank you, hspace! To do this, I just create new
instances with 32GB disks (no need to worry about exact size), then
proceeded with the above method. Once they are in place, I extended
the partitions with fdisk, reboot, resize my LVM PV and voilà! VGs
automatically detect the change and extend.

More info on LVM partition resizing is available here:
http://theducks.org/2009/11/expanding-lvm-partitions-in-vmware-on-the-fly/

--
*question everything*learn something*answer nothing*
------------
Lucas Yamanishi
------------------
Systems Administrator, ADNET Systems, Inc.
NASA Space and Earth Science Data Analysis (606.9)
gpg: 0xE23F3D7A

[Thomas Ott]

Hi JBB,

if I want to migrate a phyisical machine to ganeti, i use this Howto of my
University.
https://wiki.fbihome.de/Netzwerk:VM#Bestehendes_System_in_VM_migrieren
I think you can use it the same way.

Thomas Ott

-----Ursprüngliche Nachricht-----
Von: gan...@googlegroups.com [mailto:gan...@googlegroups.com] Im Auftrag von
Jonathan Bayer
Gesendet: Dienstag, 27. November 2012 15:06
An: gan...@googlegroups.com
Betreff: Importing a disk image