Using current node name in kvm_extra

[Ronny Adsetts]

Hi all,

Is there a way, does anyone know, to use the current (or target) node name in instance kvm_extra data?

Is this something I should be changing as a hook?

Specifically I want to specify a state dir on shared storage for swtpm to provide TPM devices to instances. The TPM state is migrated as part of the QEMU migration so the state dir needs to be node specific hence wanting to have the kvm_extra data be different on the migration target.

Thanks.

Ronny

--
Ronny Adsetts
Technical Director
Amazing Internet Ltd, London
t: +44 20 8977 8943
w: www.amazinginternet.com

Registered office: 85 Waldegrave Park, Twickenham, TW1 4TJ
Registered in England. Company No. 4042957

[Sascha Lucas]

Hi Ronny,

On Mon, 23 Jan 2026, Ronny Adsetts wrote:

> Is there a way, does anyone know, to use the current (or target) node
> name in instance kvm_extra data?

I think it's not possible ATM.

> Is this something I should be changing as a hook?

Sounds difficult to use a hook therefore. Maybe in pre migration you can
switch a static symlink to the target node. But I'm unsure if the source
instance has resolved the real path from original symlink.

> Specifically I want to specify a state dir on shared storage for swtpm
> to provide TPM devices to instances. The TPM state is migrated as part
> of the QEMU migration so the state dir needs to be node specific hence
> wanting to have the kvm_extra data be different on the migration target.

Thanks for looking into TPM and live migration. If the state is migrated,
one might think of using a path on the node instead of shared storage?
But loosing the node will also loose the TPM state. Probably a bad idea?

Maybe an A/B path or device would be the proper solution for Ganetis disk
templates?

Also EVI NVRAM would need additional disks, which sounds slightly odd
to implement inside Ganeti (mainly the DRBD case with many small disks).

Any ideas on how to solve UEFI/secure-boot/TPM are welcome.

Thanks, Sascha.

[Ronny Adsetts]

Hi Sascha,

Wow, this reply dragged out - I started writing it on Monday, it's mid-afternoon on Wednesday now. Let's see if I can grab an hour to finish it... :-).

'Sascha Lucas' via ganeti wrote on 26/01/2026 06:54:
> On Mon, 23 Jan 2026, Ronny Adsetts wrote:
>
>> Is there a way, does anyone know, to use the current (or target)
>> node name in instance kvm_extra data?
>
> I think it's not possible ATM.

Thought so.

>> Is this something I should be changing as a hook?
>
> Sounds difficult to use a hook therefore. Maybe in pre migration you
> can switch a static symlink to the target node. But I'm unsure if the
> source instance has resolved the real path from original symlink.

Having got a bit further in my thinking about this, we don't actually need to specify the TPM state dir in kvm_extra. It's only used in starting up the swtpm device. We only specify the device socket which is local to the node. So that bit of the problem goes away. No need to change kvm_extra when an instance is moved.

>> Specifically I want to specify a state dir on shared storage for
>> swtpm to provide TPM devices to instances. The TPM state is
>> migrated as part of the QEMU migration so the state dir needs to be
>> node specific hence wanting to have the kvm_extra data be different
>> on the migration target.
>
> Thanks for looking into TPM and live migration. If the state is
> migrated, one might think of using a path on the node instead of
> shared storage? But loosing the node will also loose the TPM state.
> Probably a bad idea?

Yes, I'd prefer the TPM state not to be on the node for increased reliability. If a node crashes, at least you can then (probably) recover the TPM state. No idea how often it changes or what the implications are of losing the state.

> Maybe an A/B path or device would be the proper solution for Ganetis
> disk templates?

Yes, perhaps. Disk templates are disk-specific now, right? Rather than per-instance.

The disk images for EFI are quite small - 3.5MB for the firmware and 528KB for the vars. More details later.

I don't understand enough of the internal workings of Ganeti to know how best to fit EFI and TPM within the available framework. I'm more than happy to help figure it out though.

> Also EVI NVRAM would need additional disks, which sounds slightly odd
> to implement inside Ganeti (mainly the DRBD case with many small
> disks).
>
> Any ideas on how to solve UEFI/secure-boot/TPM are welcome.

This is written from a QEMU TPM/EFI beginners point of view so just skip over any bits that are too basic. :-).

EFI seems best done using two images: the first, a read-only firmware; the second is for EFI vars.

Both the firmware and template vars files are provided in Debian in the ovmf package. The idea is to make a copy of the vars template as the instance will change it. It will need to be stored on shared storage to support migration. The firmware image remains the packaged one so it can pick up updates as needed.

/Untested/ kvm_extra string on how I'm working on solving this for EFI vars for now:

-drive if=pflash,format=raw,readonly=on,file=/usr/share/OVMF/OVMF_CODE_4M.ms.fd
-drive if=pflash,format=raw,file=/srv/vmuefi/$INSTANCE_NAME/uefi/OVMF_VARS_4M.ms.fd

The path /srv/vmuefi is shared and mounted on all nodes.

For the TPM, I'm looking at using something like this in kvm_extra:

-chardev socket,id=chrtpm,path=/run/swtpm/$INSTANCE_NAME.sock
-tpmdev emulator,id=tpm0,chardev=chrtpm
-device tpm-tis,tpmdev=tpm0

and then the following for the swtpm device itself:

swtpm socket --tpmstate dir=/srv/vmuefi/$INSTANCE_NAME/tpm/$NODE_NAME/swtpm-state --tpm2 --ctrl type=unixio,path=/run/swtpm/$INSTANCE_NAME.sock --daemon --pid /run/swtpm/$INSTANCE_NAME.pid

Note that if you want the PID file as above, /run/swtpm is not created by default on Debian and should probably be owned by root and mode 1777.

You obviously need to start the TMP device before the instance. Where would you even start to build in that integration?

It's just occurred to me that there are complications with the TPM state. If you do a FAILOVER of an instance, you need to make sure the state is copied after the instance shutdown but before it's started on the new node. If you SHUTDOWN an instance, then MOVE or FAILOVER, you again need to update the state on the target node. Maybe you have to make sure the state is sync'd on SHUTDOWN. Lots of ways to end up with either no state or out of date state I think. :-). Does it matter? Probably quite a lot.

Anyway that's as far I've got so far.

I'm happy to help looking in to this further and can contribute as much as my time and skills will allow. If that's useful then any guidance would be appreciated.