[Confidential Computing WG] TDX Attestation Beta Design Questions
18 views
Skip to first unread message
Aseef Imran
Jul 14, 2026, 2:38:51 PM (7 days ago) Jul 14
to kubevirt-dev
Hi everyone,
TDX Attestation merged in https://github.com/kubevirt/kubevirt/pull/15958 but created several open design questions as listed in https://github.com/kubevirt/enhancements/blob/main/veps/sig-compute/attestation-proposal.md#beta.
This threads aim to resolve these questions, and collect feedback about my proposed resolutions. Please let me know if you have comments on any of the below.
1. Decide whether to keep the `confidentialCompute.tdx.attestation.qgsSocketPath` configuration option to custom QGS path locations or to remove it in favor of a default location (i.e. `/var/run/tdx-qgs/qgs.socket`).
Proposed resolution: Remove this field.
Reasoning: Intel's QGS implementation in https://github.com/intel/confidential-computing.tee.dcap/blob/main/QuoteGeneration/quote_wrapper/qgs/server_main.cpp hardcodes the QGS socket path. This implementation is what most users of TDX deploys. Since API decisions are permanent, it is better to keep this out of the API until we have a concrete use case showing its value.
2. Decide whether to enforce the existence of QGS on the KubeVirt side (i.e use of the `confidentialCompute.tdx.attestation.enforced` field) and if so whether it makes sense to extend these enforcements to support per-VMI enforcements (as opposed to only cluster-wide).
Proposed resolution: Keep field, rename to `confidentialCompute.tdx.attestation.requireQGS`, change default to requireQGS=true.
Alternative resolution: Remove this field.
Reasoning:
Cryptographic attestation (quote → verify → unlock secrets) remains the guest/workload’s job. KubeVirt can only gate on infra (QGS socket present), not “enforce attestation.” The name enforced is a misnomer; requireQGS matches the actual behavior (TDX device plugin unhealthy / TDX not schedulable unless the socket exists).
Intentional mixed fleets (QGS on only some TDX nodes) are unlikely—QGS is cheap and node-local—so there is no strong case for per-VMI require-QGS.
With requireQGS: true (proposed default), missing or misdeployed QGS fails at scheduling with a clear infra cause. TDX without QGS still provides stronger isolation than a non-confidential VM, but it is not remotely attestable. Allowing that mode as the default is a poor UX: VMs start successfully, then guest attestation fails later. That mode should be explicit opt-in (requireQGS: false) for lab/dev.
Removing the field remains an option if SIG wants a smaller API, but it reintroduces that late, confusing failure mode. If we keep a knob, document it as “require QGS for TDX scheduling,” not as attestation security.
(Alpha→Beta: rename + default flip is an acceptable break while the feature is still Alpha.)
3. Decide whether TDX detection should be removed from node-labeller (since the device plugin already achieves this purpose) or whether it should be kept for either consistency with SEV-SNP or for user-friendliness.
Proposed resolution: Keep TDX on node labeller
Reasoning: The label and the device plugin answer different questions. kubevirt.io/tdx=true is capability discovery (same pattern as SEV/SNP). devices.kubevirt.io/tdx is scheduling capacity plus QGS mount/health. Ops and tooling already use node labels; forcing them to infer TDX support from a device-plugin resource couples discovery to whichever capacity mechanism we use today. If we later move capacity to DRA or an external NFD resource, the label should still mean “this node can run TDX.” Keep the label; keep scheduling authority on the resource request.
4. For the purpose of the TDX device counting mechanism, can we assume that KubeVirt will have exclusive access to the TDX hardware? Or should we support the use case of external stacks creating TDX VMs.
Context: This is relevant because a single host can have finite number of TDX VMs. Source of truth for max allowed confidential VMs is `/sys/fs/cgroup/misc.capacity`. And the source of truth for current usage is in `/sys/fs/cgroup/misc.current`.
Proposed resolution: Kick the problem down the road to when someone complains.
Alternate resolution 1: Move to DRA and drive availability from misc.capacity / misc.current.
Alternate resolution 2: Add support for requesting custom resources when TDX VMs are created as an escape hatch to allow compatibility with operators like CoCo/Kata. In API it might look something like this:
```
```
Reasoning: Alt 1 is a large platform change and is not how neighboring stacks (e.g. OpenShift CoCo) account for TDX today (NFD extended resources such as tdx.intel.com/keys, not DRA). Even perfect KubeVirt accounting cannot stop other runtimes from oversubscribing unless everyone shares one counter. Alt 2 is the right shape for co-existence with NFD/CoCo, but it is a permanent API surface and should wait on people who need it and not speculative API surface.
For Beta, oversubscribe with another TDX stack on the same node may cause VM start failure; that is an acceptable, documented limitation until co-existence is a real requirement.
5. Permission configuration for the QGS socket is currently the cluster-admin's responsibility. We should revisit the permission model for the QGS socket and decide whose responsibility permission configuration should be and whether KubeVirt can assume that it has exclusive access to the socket.
Proposed resolution: Cluster admin (or QGS operator) deploys and owns the host QGS socket. KubeVirt does not assume exclusive access and does not chown/relabel the host socket. When `/var/run/tdx-qgs/qgs.socket` is present, KubeVirt creates a relay socket it owns (qemu-accessible), proxies to the host QGS socket, and points libvirt/QEMU at the relay—same idea as PR-helper (qemu → KV-owned socket → privileged helper → host service).
Alternative resolution: If technical cost is too high for Beta, keep mount-only and leave host QGS permissions to the deployer.
Reasoning:
Documenting “make this usable by QEMU (107)” leaks runtime internals and couples QGS deployers to KubeVirt’s uid/SELinux choices. Other clients may also use the same QGS socket. A transparent unix relay keeps guest-facing perms under KubeVirt (like PR-helper) while the QGS deployer keeps ownership of the real socket. KubeVirt only needs the node agent/helper to connect to host QGS.
Reasoning:
Documenting “make this usable by QEMU (107)” leaks runtime internals and couples QGS deployers to KubeVirt’s uid/SELinux choices. Other clients may also use the same QGS socket. A transparent unix relay keeps guest-facing perms under KubeVirt (like PR-helper) while the QGS deployer keeps ownership of the real socket. KubeVirt only needs the node agent/helper to connect to host QGS.
Roman Mohr
Jul 15, 2026, 2:26:34 AM (7 days ago) Jul 15
to Aseef Imran, kubevirt-dev
Hi Aseef,
Those are great questions.
On Tue, Jul 14, 2026 at 8:39 PM 'Aseef Imran' via kubevirt-dev <kubevi...@googlegroups.com> wrote:
Hi everyone,TDX Attestation merged in https://github.com/kubevirt/kubevirt/pull/15958 but created several open design questions as listed in https://github.com/kubevirt/enhancements/blob/main/veps/sig-compute/attestation-proposal.md#beta.This threads aim to resolve these questions, and collect feedback about my proposed resolutions. Please let me know if you have comments on any of the below.1. Decide whether to keep the `confidentialCompute.tdx.attestation.qgsSocketPath` configuration option to custom QGS path locations or to remove it in favor of a default location (i.e. `/var/run/tdx-qgs/qgs.socket`).Proposed resolution: Remove this field.Reasoning: Intel's QGS implementation in https://github.com/intel/confidential-computing.tee.dcap/blob/main/QuoteGeneration/quote_wrapper/qgs/server_main.cpp hardcodes the QGS socket path. This implementation is what most users of TDX deploys. Since API decisions are permanent, it is better to keep this out of the API until we have a concrete use case showing its value.
I would propose to keep it for two reasons:
1. We run QGS for instance inside a container and have a hostPath mounted where we expose that hostPath has little to do with a hardcoded path in a binary inside a container. We have it for instance in `/var/run/<projectprefix>/tdx/<socket_and_other_stuff>.
2. The QGS binary is just a reference implementation. One can take the core signed library from intel and build their own quote generation service around it (while not used by us and at this point probably not kubevirt users, that's done by people as well).
1. We run QGS for instance inside a container and have a hostPath mounted where we expose that hostPath has little to do with a hardcoded path in a binary inside a container. We have it for instance in `/var/run/<projectprefix>/tdx/<socket_and_other_stuff>.
2. The QGS binary is just a reference implementation. One can take the core signed library from intel and build their own quote generation service around it (while not used by us and at this point probably not kubevirt users, that's done by people as well).
2. Decide whether to enforce the existence of QGS on the KubeVirt side (i.e use of the `confidentialCompute.tdx.attestation.enforced` field) and if so whether it makes sense to extend these enforcements to support per-VMI enforcements (as opposed to only cluster-wide).Proposed resolution: Keep field, rename to `confidentialCompute.tdx.attestation.requireQGS`, change default to requireQGS=true.Alternative resolution: Remove this field.Reasoning:Cryptographic attestation (quote → verify → unlock secrets) remains the guest/workload’s job. KubeVirt can only gate on infra (QGS socket present), not “enforce attestation.” The name enforced is a misnomer; requireQGS matches the actual behavior (TDX device plugin unhealthy / TDX not schedulable unless the socket exists).Intentional mixed fleets (QGS on only some TDX nodes) are unlikely—QGS is cheap and node-local—so there is no strong case for per-VMI require-QGS.
With requireQGS: true (proposed default), missing or misdeployed QGS fails at scheduling with a clear infra cause. TDX without QGS still provides stronger isolation than a non-confidential VM, but it is not remotely attestable. Allowing that mode as the default is a poor UX: VMs start successfully, then guest attestation fails later. That mode should be explicit opt-in (requireQGS: false) for lab/dev.
Removing the field remains an option if SIG wants a smaller API, but it reintroduces that late, confusing failure mode. If we keep a knob, document it as “require QGS for TDX scheduling,” not as attestation security.
(Alpha→Beta: rename + default flip is an acceptable break while the feature is still Alpha.)
I don't mind if it is set to true or false by default, but QGS is an external project dependency. I would be against the alternative proposed, so keeping it sounds good.
3. Decide whether TDX detection should be removed from node-labeller (since the device plugin already achieves this purpose) or whether it should be kept for either consistency with SEV-SNP or for user-friendliness.Proposed resolution: Keep TDX on node labellerReasoning: The label and the device plugin answer different questions. kubevirt.io/tdx=true is capability discovery (same pattern as SEV/SNP). devices.kubevirt.io/tdx is scheduling capacity plus QGS mount/health. Ops and tooling already use node labels; forcing them to infer TDX support from a device-plugin resource couples discovery to whichever capacity mechanism we use today. If we later move capacity to DRA or an external NFD resource, the label should still mean “this node can run TDX.” Keep the label; keep scheduling authority on the resource request.
+1 on keeping the label. One concrete example where the TDX label is needed, the main purpose is to schedule appropriate infrastructure to the nodes. Especially for the following:
1. Scheduling QGS
2. In the same pod or separate pod scheduling infrastructure which handled node registration with intel
4. For the purpose of the TDX device counting mechanism, can we assume that KubeVirt will have exclusive access to the TDX hardware? Or should we support the use case of external stacks creating TDX VMs.Context: This is relevant because a single host can have finite number of TDX VMs. Source of truth for max allowed confidential VMs is `/sys/fs/cgroup/misc.capacity`. And the source of truth for current usage is in `/sys/fs/cgroup/misc.current`.Proposed resolution: Kick the problem down the road to when someone complains.Alternate resolution 1: Move to DRA and drive availability from misc.capacity / misc.current.Alternate resolution 2: Add support for requesting custom resources when TDX VMs are created as an escape hatch to allow compatibility with operators like CoCo/Kata. In API it might look something like this:``````Reasoning: Alt 1 is a large platform change and is not how neighboring stacks (e.g. OpenShift CoCo) account for TDX today (NFD extended resources such as tdx.intel.com/keys, not DRA). Even perfect KubeVirt accounting cannot stop other runtimes from oversubscribing unless everyone shares one counter. Alt 2 is the right shape for co-existence with NFD/CoCo, but it is a permanent API surface and should wait on people who need it and not speculative API surface.For Beta, oversubscribe with another TDX stack on the same node may cause VM start failure; that is an acceptable, documented limitation until co-existence is a real requirement.
Sounds good to me.
5. Permission configuration for the QGS socket is currently the cluster-admin's responsibility. We should revisit the permission model for the QGS socket and decide whose responsibility permission configuration should be and whether KubeVirt can assume that it has exclusive access to the socket.Proposed resolution: Cluster admin (or QGS operator) deploys and owns the host QGS socket. KubeVirt does not assume exclusive access and does not chown/relabel the host socket. When `/var/run/tdx-qgs/qgs.socket` is present, KubeVirt creates a relay socket it owns (qemu-accessible), proxies to the host QGS socket, and points libvirt/QEMU at the relay—same idea as PR-helper (qemu → KV-owned socket → privileged helper → host service).
That's a possibility. I just want to point out that attestation is often a periodic task (we have projects where it happens every few minutes), so it would be great if it is designed so that it does not depend on the availability of virt-handler after the initial VM start.
Best regards,
Roman
Alternative resolution: If technical cost is too high for Beta, keep mount-only and leave host QGS permissions to the deployer.
Reasoning:
Documenting “make this usable by QEMU (107)” leaks runtime internals and couples QGS deployers to KubeVirt’s uid/SELinux choices. Other clients may also use the same QGS socket. A transparent unix relay keeps guest-facing perms under KubeVirt (like PR-helper) while the QGS deployer keeps ownership of the real socket. KubeVirt only needs the node agent/helper to connect to host QGS.
--
You received this message because you are subscribed to the Google Groups "kubevirt-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubevirt-dev...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/kubevirt-dev/2fe0e4f3-946a-4916-b446-911145c98f63n%40googlegroups.com.
Matias Ezequiel Vara Larsen
Jul 15, 2026, 10:07:47 AM (6 days ago) Jul 15
to Roman Mohr, Aseef Imran, kubevirt-dev
Hello Aseef, thanks for sharing these questions. I added some
comments. I do not have an opinion on some of them. I guess I'll wait
to see how people use it before deciding.
On Wed, Jul 15, 2026 at 8:26 AM 'Roman Mohr' via kubevirt-dev
>>
>>
>> 2. Decide whether to enforce the existence of QGS on the KubeVirt side (i.e use of the `confidentialCompute.tdx.attestation.enforced` field) and if so whether it makes sense to extend these enforcements to support per-VMI enforcements (as opposed to only cluster-wide).
>>
>> Proposed resolution: Keep field, rename to `confidentialCompute.tdx.attestation.requireQGS`, change default to requireQGS=true.
>> Alternative resolution: Remove this field.
>> Reasoning:
>> Cryptographic attestation (quote → verify → unlock secrets) remains the guest/workload’s job. KubeVirt can only gate on infra (QGS socket present), not “enforce attestation.” The name enforced is a misnomer; requireQGS matches the actual behavior (TDX device plugin unhealthy / TDX not schedulable unless the socket exists).
>>
>> Intentional mixed fleets (QGS on only some TDX nodes) are unlikely—QGS is cheap and node-local—so there is no strong case for per-VMI require-QGS.
>>
>> With requireQGS: true (proposed default), missing or misdeployed QGS fails at scheduling with a clear infra cause. TDX without QGS still provides stronger isolation than a non-confidential VM, but it is not remotely attestable. Allowing that mode as the default is a poor UX: VMs start successfully, then guest attestation fails later. That mode should be explicit opt-in (requireQGS: false) for lab/dev.
>>
>> Removing the field remains an option if SIG wants a smaller API, but it reintroduces that late, confusing failure mode. If we keep a knob, document it as “require QGS for TDX scheduling,” not as attestation security.
>>
>> (Alpha→Beta: rename + default flip is an acceptable break while the feature is still Alpha.)
>
> I don't mind if it is set to true or false by default, but QGS is an external project dependency. I would be against the alternative proposed, so keeping it sounds good.
>
+1 to keep it. Probably, `requireQGS` sounds cleaner.
>>
>>
>> 3. Decide whether TDX detection should be removed from node-labeller (since the device plugin already achieves this purpose) or whether it should be kept for either consistency with SEV-SNP or for user-friendliness.
>>
>> Proposed resolution: Keep TDX on node labeller
>> Reasoning: The label and the device plugin answer different questions. kubevirt.io/tdx=true is capability discovery (same pattern as SEV/SNP). devices.kubevirt.io/tdx is scheduling capacity plus QGS mount/health. Ops and tooling already use node labels; forcing them to infer TDX support from a device-plugin resource couples discovery to whichever capacity mechanism we use today. If we later move capacity to DRA or an external NFD resource, the label should still mean “this node can run TDX.” Keep the label; keep scheduling authority on the resource request.
>
>
> +1 on keeping the label. One concrete example where the TDX label is needed, the main purpose is to schedule appropriate infrastructure to the nodes. Especially for the following:
> 1. Scheduling QGS
> 2. In the same pod or separate pod scheduling infrastructure which handled node registration with intel
>
+1 to keep it.
>>
>>
>> 4. For the purpose of the TDX device counting mechanism, can we assume that KubeVirt will have exclusive access to the TDX hardware? Or should we support the use case of external stacks creating TDX VMs.
>>
>> Context: This is relevant because a single host can have finite number of TDX VMs. Source of truth for max allowed confidential VMs is `/sys/fs/cgroup/misc.capacity`. And the source of truth for current usage is in `/sys/fs/cgroup/misc.current`.
>> Proposed resolution: Kick the problem down the road to when someone complains.
>> Alternate resolution 1: Move to DRA and drive availability from misc.capacity / misc.current.
>> Alternate resolution 2: Add support for requesting custom resources when TDX VMs are created as an escape hatch to allow compatibility with operators like CoCo/Kata. In API it might look something like this:
>> ```
>> confidentialCompute:
>> tdx:
>> sharedCapacityResourceName: tdx.intel.com/keys
>> ```
>> Reasoning: Alt 1 is a large platform change and is not how neighboring stacks (e.g. OpenShift CoCo) account for TDX today (NFD extended resources such as tdx.intel.com/keys, not DRA). Even perfect KubeVirt accounting cannot stop other runtimes from oversubscribing unless everyone shares one counter. Alt 2 is the right shape for co-existence with NFD/CoCo, but it is a permanent API surface and should wait on people who need it and not speculative API surface.
>>
>> For Beta, oversubscribe with another TDX stack on the same node may cause VM start failure; that is an acceptable, documented limitation until co-existence is a real requirement.
>
>
> Sounds good to me.
>
+1 to ignore until we have the use-case.
>>
>>
>> 5. Permission configuration for the QGS socket is currently the cluster-admin's responsibility. We should revisit the permission model for the QGS socket and decide whose responsibility permission configuration should be and whether KubeVirt can assume that it has exclusive access to the socket.
>>
>> Proposed resolution: Cluster admin (or QGS operator) deploys and owns the host QGS socket. KubeVirt does not assume exclusive access and does not chown/relabel the host socket. When `/var/run/tdx-qgs/qgs.socket` is present, KubeVirt creates a relay socket it owns (qemu-accessible), proxies to the host QGS socket, and points libvirt/QEMU at the relay—same idea as PR-helper (qemu → KV-owned socket → privileged helper → host service).
>
>
> That's a possibility. I just want to point out that attestation is often a periodic task (we have projects where it happens every few minutes), so it would be great if it is designed so that it does not depend on the availability of virt-handler after the initial VM start.
>
I do not follow here. What do you mean with `it does not depend on the
availability of virt-handler after the initial VM start`? Do you mean
that virt-handle does _something_ after the initialization of the VM
that will prevent attestation after VM initialization?
Matias
> Best regards,
> Roman
>
>>
>>
>> Alternative resolution: If technical cost is too high for Beta, keep mount-only and leave host QGS permissions to the deployer.
>>
>> Reasoning:
>> Documenting “make this usable by QEMU (107)” leaks runtime internals and couples QGS deployers to KubeVirt’s uid/SELinux choices. Other clients may also use the same QGS socket. A transparent unix relay keeps guest-facing perms under KubeVirt (like PR-helper) while the QGS deployer keeps ownership of the real socket. KubeVirt only needs the node agent/helper to connect to host QGS.
>>
>> --
>> You received this message because you are subscribed to the Google Groups "kubevirt-dev" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to kubevirt-dev...@googlegroups.com.
>> To view this discussion visit https://groups.google.com/d/msgid/kubevirt-dev/2fe0e4f3-946a-4916-b446-911145c98f63n%40googlegroups.com.
>
> --
> You received this message because you are subscribed to the Google Groups "kubevirt-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to kubevirt-dev...@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/kubevirt-dev/CA%2BNAZzwg2dOQpStiUffBmXd-%2BrD%2BCojqCrHeB0Gr%2BBMZYfrc4A%40mail.gmail.com.
comments. I do not have an opinion on some of them. I guess I'll wait
to see how people use it before deciding.
On Wed, Jul 15, 2026 at 8:26 AM 'Roman Mohr' via kubevirt-dev
<kubevi...@googlegroups.com> wrote:
>
> Hi Aseef,
>
> Those are great questions.
>
> On Tue, Jul 14, 2026 at 8:39 PM 'Aseef Imran' via kubevirt-dev <kubevi...@googlegroups.com> wrote:
>>
>> Hi everyone,
>>
>> TDX Attestation merged in https://github.com/kubevirt/kubevirt/pull/15958 but created several open design questions as listed in https://github.com/kubevirt/enhancements/blob/main/veps/sig-compute/attestation-proposal.md#beta.
>>
>> This threads aim to resolve these questions, and collect feedback about my proposed resolutions. Please let me know if you have comments on any of the below.
>>
>>
>> 1. Decide whether to keep the `confidentialCompute.tdx.attestation.qgsSocketPath` configuration option to custom QGS path locations or to remove it in favor of a default location (i.e. `/var/run/tdx-qgs/qgs.socket`).
>>
>> Proposed resolution: Remove this field.
>> Reasoning: Intel's QGS implementation in https://github.com/intel/confidential-computing.tee.dcap/blob/main/QuoteGeneration/quote_wrapper/qgs/server_main.cpp hardcodes the QGS socket path. This implementation is what most users of TDX deploys. Since API decisions are permanent, it is better to keep this out of the API until we have a concrete use case showing its value.
>
>
> I would propose to keep it for two reasons:
>
> 1. We run QGS for instance inside a container and have a hostPath mounted where we expose that hostPath has little to do with a hardcoded path in a binary inside a container. We have it for instance in `/var/run/<projectprefix>/tdx/<socket_and_other_stuff>.
> 2. The QGS binary is just a reference implementation. One can take the core signed library from intel and build their own quote generation service around it (while not used by us and at this point probably not kubevirt users, that's done by people as well).
>
+1 to keep it.
>
> Hi Aseef,
>
> Those are great questions.
>
> On Tue, Jul 14, 2026 at 8:39 PM 'Aseef Imran' via kubevirt-dev <kubevi...@googlegroups.com> wrote:
>>
>> Hi everyone,
>>
>> TDX Attestation merged in https://github.com/kubevirt/kubevirt/pull/15958 but created several open design questions as listed in https://github.com/kubevirt/enhancements/blob/main/veps/sig-compute/attestation-proposal.md#beta.
>>
>> This threads aim to resolve these questions, and collect feedback about my proposed resolutions. Please let me know if you have comments on any of the below.
>>
>>
>> 1. Decide whether to keep the `confidentialCompute.tdx.attestation.qgsSocketPath` configuration option to custom QGS path locations or to remove it in favor of a default location (i.e. `/var/run/tdx-qgs/qgs.socket`).
>>
>> Proposed resolution: Remove this field.
>> Reasoning: Intel's QGS implementation in https://github.com/intel/confidential-computing.tee.dcap/blob/main/QuoteGeneration/quote_wrapper/qgs/server_main.cpp hardcodes the QGS socket path. This implementation is what most users of TDX deploys. Since API decisions are permanent, it is better to keep this out of the API until we have a concrete use case showing its value.
>
>
> I would propose to keep it for two reasons:
>
> 1. We run QGS for instance inside a container and have a hostPath mounted where we expose that hostPath has little to do with a hardcoded path in a binary inside a container. We have it for instance in `/var/run/<projectprefix>/tdx/<socket_and_other_stuff>.
> 2. The QGS binary is just a reference implementation. One can take the core signed library from intel and build their own quote generation service around it (while not used by us and at this point probably not kubevirt users, that's done by people as well).
>
>>
>>
>> 2. Decide whether to enforce the existence of QGS on the KubeVirt side (i.e use of the `confidentialCompute.tdx.attestation.enforced` field) and if so whether it makes sense to extend these enforcements to support per-VMI enforcements (as opposed to only cluster-wide).
>>
>> Proposed resolution: Keep field, rename to `confidentialCompute.tdx.attestation.requireQGS`, change default to requireQGS=true.
>> Alternative resolution: Remove this field.
>> Reasoning:
>> Cryptographic attestation (quote → verify → unlock secrets) remains the guest/workload’s job. KubeVirt can only gate on infra (QGS socket present), not “enforce attestation.” The name enforced is a misnomer; requireQGS matches the actual behavior (TDX device plugin unhealthy / TDX not schedulable unless the socket exists).
>>
>> Intentional mixed fleets (QGS on only some TDX nodes) are unlikely—QGS is cheap and node-local—so there is no strong case for per-VMI require-QGS.
>>
>> With requireQGS: true (proposed default), missing or misdeployed QGS fails at scheduling with a clear infra cause. TDX without QGS still provides stronger isolation than a non-confidential VM, but it is not remotely attestable. Allowing that mode as the default is a poor UX: VMs start successfully, then guest attestation fails later. That mode should be explicit opt-in (requireQGS: false) for lab/dev.
>>
>> Removing the field remains an option if SIG wants a smaller API, but it reintroduces that late, confusing failure mode. If we keep a knob, document it as “require QGS for TDX scheduling,” not as attestation security.
>>
>> (Alpha→Beta: rename + default flip is an acceptable break while the feature is still Alpha.)
>
> I don't mind if it is set to true or false by default, but QGS is an external project dependency. I would be against the alternative proposed, so keeping it sounds good.
>
>>
>>
>> 3. Decide whether TDX detection should be removed from node-labeller (since the device plugin already achieves this purpose) or whether it should be kept for either consistency with SEV-SNP or for user-friendliness.
>>
>> Proposed resolution: Keep TDX on node labeller
>> Reasoning: The label and the device plugin answer different questions. kubevirt.io/tdx=true is capability discovery (same pattern as SEV/SNP). devices.kubevirt.io/tdx is scheduling capacity plus QGS mount/health. Ops and tooling already use node labels; forcing them to infer TDX support from a device-plugin resource couples discovery to whichever capacity mechanism we use today. If we later move capacity to DRA or an external NFD resource, the label should still mean “this node can run TDX.” Keep the label; keep scheduling authority on the resource request.
>
>
> +1 on keeping the label. One concrete example where the TDX label is needed, the main purpose is to schedule appropriate infrastructure to the nodes. Especially for the following:
> 1. Scheduling QGS
> 2. In the same pod or separate pod scheduling infrastructure which handled node registration with intel
>
>>
>>
>> 4. For the purpose of the TDX device counting mechanism, can we assume that KubeVirt will have exclusive access to the TDX hardware? Or should we support the use case of external stacks creating TDX VMs.
>>
>> Context: This is relevant because a single host can have finite number of TDX VMs. Source of truth for max allowed confidential VMs is `/sys/fs/cgroup/misc.capacity`. And the source of truth for current usage is in `/sys/fs/cgroup/misc.current`.
>> Proposed resolution: Kick the problem down the road to when someone complains.
>> Alternate resolution 1: Move to DRA and drive availability from misc.capacity / misc.current.
>> Alternate resolution 2: Add support for requesting custom resources when TDX VMs are created as an escape hatch to allow compatibility with operators like CoCo/Kata. In API it might look something like this:
>> ```
>> confidentialCompute:
>> tdx:
>> sharedCapacityResourceName: tdx.intel.com/keys
>> ```
>> Reasoning: Alt 1 is a large platform change and is not how neighboring stacks (e.g. OpenShift CoCo) account for TDX today (NFD extended resources such as tdx.intel.com/keys, not DRA). Even perfect KubeVirt accounting cannot stop other runtimes from oversubscribing unless everyone shares one counter. Alt 2 is the right shape for co-existence with NFD/CoCo, but it is a permanent API surface and should wait on people who need it and not speculative API surface.
>>
>> For Beta, oversubscribe with another TDX stack on the same node may cause VM start failure; that is an acceptable, documented limitation until co-existence is a real requirement.
>
>
> Sounds good to me.
>
>>
>>
>> 5. Permission configuration for the QGS socket is currently the cluster-admin's responsibility. We should revisit the permission model for the QGS socket and decide whose responsibility permission configuration should be and whether KubeVirt can assume that it has exclusive access to the socket.
>>
>> Proposed resolution: Cluster admin (or QGS operator) deploys and owns the host QGS socket. KubeVirt does not assume exclusive access and does not chown/relabel the host socket. When `/var/run/tdx-qgs/qgs.socket` is present, KubeVirt creates a relay socket it owns (qemu-accessible), proxies to the host QGS socket, and points libvirt/QEMU at the relay—same idea as PR-helper (qemu → KV-owned socket → privileged helper → host service).
>
>
> That's a possibility. I just want to point out that attestation is often a periodic task (we have projects where it happens every few minutes), so it would be great if it is designed so that it does not depend on the availability of virt-handler after the initial VM start.
>
availability of virt-handler after the initial VM start`? Do you mean
that virt-handle does _something_ after the initialization of the VM
that will prevent attestation after VM initialization?
Matias
> Best regards,
> Roman
>
>>
>>
>> Alternative resolution: If technical cost is too high for Beta, keep mount-only and leave host QGS permissions to the deployer.
>>
>> Reasoning:
>> Documenting “make this usable by QEMU (107)” leaks runtime internals and couples QGS deployers to KubeVirt’s uid/SELinux choices. Other clients may also use the same QGS socket. A transparent unix relay keeps guest-facing perms under KubeVirt (like PR-helper) while the QGS deployer keeps ownership of the real socket. KubeVirt only needs the node agent/helper to connect to host QGS.
>>
>> --
>> You received this message because you are subscribed to the Google Groups "kubevirt-dev" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to kubevirt-dev...@googlegroups.com.
>> To view this discussion visit https://groups.google.com/d/msgid/kubevirt-dev/2fe0e4f3-946a-4916-b446-911145c98f63n%40googlegroups.com.
>
> --
> You received this message because you are subscribed to the Google Groups "kubevirt-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to kubevirt-dev...@googlegroups.com.
Aseef Imran
Jul 16, 2026, 11:14:25 AM (5 days ago) Jul 16
to kubevirt-dev
>> I would propose to keep it for two reasons:
>>
>> 1. We run QGS for instance inside a container and have a hostPath mounted where we expose that hostPath has little to do with a hardcoded path in a binary inside a container. We have it for instance in `/var/run/<projectprefix>/tdx/<socket_and_other_stuff>.
>> 2. The QGS binary is just a reference implementation. One can take the core signed library from intel and build their own quote generation service around it (while not used by us and at this point probably not kubevirt users, that's done by people as well).
OK fair enough. If you have already found a use for custom paths then we can keep it. I believe it is also possible (in theory anyway) to run multiple QGS services on a single host too.
>> I don't mind if it is set to true or false by default, but QGS is an external project dependency. I would be against the alternative proposed, so keeping it sounds good.
Just to be clear, the alternatives I proposed aren't the only ones I'm open to. Just some I had in mind that I thought would be reasonable as well. I think there is also an argument to be made for keeping the default to false (though I do think true might make marginally more sense). I would love to hear more thoughts on this!
>> That's a possibility. I just want to point out that attestation is often a periodic task (we have projects where it happens every few minutes), so it would be great if it is designed so that it does not depend on the availability of virt-handler after the initial VM start.
Like Matias, I'm also a bit confused on this point.
Thanks for your valuable feedback!
Aseef
Alan Caldelas
Jul 20, 2026, 6:34:30 PM (yesterday) Jul 20
to kubevirt-dev
Hey, just added my votes to this.
On Wednesday, July 15, 2026 at 9:07:47 AM UTC-5 Matias Ezequiel Vara Larsen wrote:
Hello Aseef, thanks for sharing these questions. I added some
comments. I do not have an opinion on some of them. I guess I'll wait
to see how people use it before deciding.
On Wed, Jul 15, 2026 at 8:26 AM 'Roman Mohr' via kubevirt-dev
<kubevi...@googlegroups.com> wrote:
>
> Hi Aseef,
>
> Those are great questions.
>
> On Tue, Jul 14, 2026 at 8:39 PM 'Aseef Imran' via kubevirt-dev <kubevi...@googlegroups.com> wrote:
>>
>> Hi everyone,
>>
>> TDX Attestation merged in https://github.com/kubevirt/kubevirt/pull/15958 but created several open design questions as listed in https://github.com/kubevirt/enhancements/blob/main/veps/sig-compute/attestation-proposal.md#beta.
>>
>> This threads aim to resolve these questions, and collect feedback about my proposed resolutions. Please let me know if you have comments on any of the below.
>>
>>
>> 1. Decide whether to keep the `confidentialCompute.tdx.attestation.qgsSocketPath` configuration option to custom QGS path locations or to remove it in favor of a default location (i.e. `/var/run/tdx-qgs/qgs.socket`).
>>
>> Proposed resolution: Remove this field.
>> Reasoning: Intel's QGS implementation in https://github.com/intel/confidential-computing.tee.dcap/blob/main/QuoteGeneration/quote_wrapper/qgs/server_main.cpp hardcodes the QGS socket path. This implementation is what most users of TDX deploys. Since API decisions are permanent, it is better to keep this out of the API until we have a concrete use case showing its value.
>
>
> I would propose to keep it for two reasons:
>
> 1. We run QGS for instance inside a container and have a hostPath mounted where we expose that hostPath has little to do with a hardcoded path in a binary inside a container. We have it for instance in `/var/run/<projectprefix>/tdx/<socket_and_other_stuff>.
> 2. The QGS binary is just a reference implementation. One can take the core signed library from intel and build their own quote generation service around it (while not used by us and at this point probably not kubevirt users, that's done by people as well).
>
+1 to keep it.
+1 keep it.
>>
>>
>> 2. Decide whether to enforce the existence of QGS on the KubeVirt side (i.e use of the `confidentialCompute.tdx.attestation.enforced` field) and if so whether it makes sense to extend these enforcements to support per-VMI enforcements (as opposed to only cluster-wide).
>>
>> Proposed resolution: Keep field, rename to `confidentialCompute.tdx.attestation.requireQGS`, change default to requireQGS=true.
>> Alternative resolution: Remove this field.
>> Reasoning:
>> Cryptographic attestation (quote → verify → unlock secrets) remains the guest/workload’s job. KubeVirt can only gate on infra (QGS socket present), not “enforce attestation.” The name enforced is a misnomer; requireQGS matches the actual behavior (TDX device plugin unhealthy / TDX not schedulable unless the socket exists).
>>
>> Intentional mixed fleets (QGS on only some TDX nodes) are unlikely—QGS is cheap and node-local—so there is no strong case for per-VMI require-QGS.
>>
>> With requireQGS: true (proposed default), missing or misdeployed QGS fails at scheduling with a clear infra cause. TDX without QGS still provides stronger isolation than a non-confidential VM, but it is not remotely attestable. Allowing that mode as the default is a poor UX: VMs start successfully, then guest attestation fails later. That mode should be explicit opt-in (requireQGS: false) for lab/dev.
>>
>> Removing the field remains an option if SIG wants a smaller API, but it reintroduces that late, confusing failure mode. If we keep a knob, document it as “require QGS for TDX scheduling,” not as attestation security.
>>
>> (Alpha→Beta: rename + default flip is an acceptable break while the feature is still Alpha.)
>
> I don't mind if it is set to true or false by default, but QGS is an external project dependency. I would be against the alternative proposed, so keeping it sounds good.
>
+1 to keep it. Probably, `requireQGS` sounds cleaner.
>>
>>
>> 3. Decide whether TDX detection should be removed from node-labeller (since the device plugin already achieves this purpose) or whether it should be kept for either consistency with SEV-SNP or for user-friendliness.
>>
>> Proposed resolution: Keep TDX on node labeller
>> Reasoning: The label and the device plugin answer different questions. kubevirt.io/tdx=true is capability discovery (same pattern as SEV/SNP). devices.kubevirt.io/tdx is scheduling capacity plus QGS mount/health. Ops and tooling already use node labels; forcing them to infer TDX support from a device-plugin resource couples discovery to whichever capacity mechanism we use today. If we later move capacity to DRA or an external NFD resource, the label should still mean “this node can run TDX.” Keep the label; keep scheduling authority on the resource request.
>
>
> +1 on keeping the label. One concrete example where the TDX label is needed, the main purpose is to schedule appropriate infrastructure to the nodes. Especially for the following:
> 1. Scheduling QGS
> 2. In the same pod or separate pod scheduling infrastructure which handled node registration with intel
>
+1 to keep it.
+1 to keep it as well, I can see the argument that eventually we will want to move to the DRA implementation but for now I think this should suffice.
>>
>>
>> 4. For the purpose of the TDX device counting mechanism, can we assume that KubeVirt will have exclusive access to the TDX hardware? Or should we support the use case of external stacks creating TDX VMs.
>>
>> Context: This is relevant because a single host can have finite number of TDX VMs. Source of truth for max allowed confidential VMs is `/sys/fs/cgroup/misc.capacity`. And the source of truth for current usage is in `/sys/fs/cgroup/misc.current`.
>> Proposed resolution: Kick the problem down the road to when someone complains.
>> Alternate resolution 1: Move to DRA and drive availability from misc.capacity / misc.current.
>> Alternate resolution 2: Add support for requesting custom resources when TDX VMs are created as an escape hatch to allow compatibility with operators like CoCo/Kata. In API it might look something like this:
>> ```
>> confidentialCompute:
>> tdx:
>> sharedCapacityResourceName: tdx.intel.com/keys
>> ```
>> Reasoning: Alt 1 is a large platform change and is not how neighboring stacks (e.g. OpenShift CoCo) account for TDX today (NFD extended resources such as tdx.intel.com/keys, not DRA). Even perfect KubeVirt accounting cannot stop other runtimes from oversubscribing unless everyone shares one counter. Alt 2 is the right shape for co-existence with NFD/CoCo, but it is a permanent API surface and should wait on people who need it and not speculative API surface.
>>
>> For Beta, oversubscribe with another TDX stack on the same node may cause VM start failure; that is an acceptable, documented limitation until co-existence is a real requirement.
>
>
> Sounds good to me.
>
+1 to ignore until we have the use-case.
+1 I can see this eventually also being an issue for SEV-SNP
Reply all
Reply to author
Forward
0 new messages