Experimental: ZFS backend support with live migration.

0 views
Skip to first unread message

Sean Reifschneider

unread,
12:16 AM (4 hours ago) 12:16 AM
to ganeti-devel
Noting up front: this was done using AI.

For the last couple years I've been toying with the idea of a better ZFS backend for Ganeti, in particular one that supports live migration by using zfs send and snapshots.  This branch implements that: https://github.com/linsomniac/ganeti/blob/master/README-ZFS.md

I had previously experimented with the normal zfs external storage module, but decided against using it at that time and went back to LVM (which has been a real workhorse for me for over a decade).

I have a 2 node ZFS cluster at home that is what I've been running this on.  It seems to work fine for gnt-instance move and migrate.

It's basically like the DRBD backend, but instead of keeping a live copy, it makes an on-demand copy of the disk devices, using snapshots so that the VM doesn't have to be paused for the entire duration of the disk block file copy.

At the moment I'm labeling this entirely experimental.  I have a 2 node cluster running and have done 50-100 move and migration operations between them, successfully.  My plan from here is to start setting it up on my dev/stg cluster.  I might end up waiting until Ubuntu 26.04 is released, as I'm planning on reinstalling dev/stg then.  I *MIGHT* start testing before then, if I can come up with the hardware.

Here's an example "gnt-instance migrate" run:

Sun Feb 22 03:06:49 2026 Migrating instance test.example
Sun Feb 22 03:06:50 2026 * checking disk consistency between source and target
Sun Feb 22 03:06:50 2026 * preparing ZFS replication for 1 disks
Sun Feb 22 03:06:50 2026 * replicating ZFS dataset ganeti/b5248300-fca0-4e2e-beca-1871a1bf9e6e.zfs.disk0
Sun Feb 22 03:06:50 2026   - creating and sending initial snapshot ganeti-migration-1771729610-38bc9ce5-initial
Sun Feb 22 03:07:41 2026   - creating and sending incremental snapshot ganeti-migration-1771729661-38bc9ce5-incremental
Sun Feb 22 03:07:43 2026 * ZFS replication preparation complete
Sun Feb 22 03:07:43 2026 * opening instance disks on node gnt1.example in shared mode
Sun Feb 22 03:07:43 2026 * opening instance disks on node gnt2.example in shared mode
Sun Feb 22 03:07:43 2026 * preparing gnt2.example to accept the instance
Sun Feb 22 03:07:43 2026 * migrating instance to gnt2.example
Sun Feb 22 03:07:43 2026 * starting memory transfer
Sun Feb 22 03:07:54 2026 * memory transfer progress: 8.59 %
Sun Feb 22 03:08:05 2026 * memory transfer progress: 17.11 %
Sun Feb 22 03:08:06 2026 * memory transfer complete
Sun Feb 22 03:08:06 2026 * performing final ZFS synchronization
Sun Feb 22 03:08:06 2026   - final sync for ganeti/b5248300-fca0-4e2e-beca-1871a1bf9e6e.zfs.disk0
Sun Feb 22 03:08:08 2026 * closing instance disks on node gnt1.example
Sun Feb 22 03:08:08 2026 * opening instance disks on node gnt2.example in exclusive mode
Sun Feb 22 03:08:08 2026 * removing instance's disks None from gnt1.example
Sun Feb 22 03:08:08 2026 * cleaning up ZFS migration snapshots
Sun Feb 22 03:08:09 2026 * done
Reply all
Reply to author
Forward
0 new messages