service "kubevirt-operator-webhook" not found

[ALAN FERGUS]

# kubectl replace -f /tmp/kubectl-edit-r9bfc.yaml
Error from server (InternalError): error when replacing "/tmp/kubectl-edit-r9bfc.yaml": Internal error occurred: failed calling webhook "kubevirt-update-validator.kubevirt.io": Post https://kubevirt-operator-webhook.kubevirt.svc:443/kubevirt-validate-update?timeout=30s: service "kubevirt-operator-webhook" not found

Hello,
I upgrade to the latest version of KubeVirt and was having issues. Decided to try do clean install as such i followed the documentation to delete. Once it was complete I did a reinstall but facing an issue where the "service "kubevirt-operator-webhook" not found".Was hoping someone might be able to help and i aplogise if this is the wrong channel, if so please point me to where i should post.
Thanks in advance,
Alan (edited) 

[Ashley Schuett]

What is the version of the operator you are running? can you point me to the documentation you used and steps you took for re-installation so I can recreate the issue ?

Thanks!

[ALAN FERGUS]

Hello Ashley,

I went to V0.38.1 following https://kubevirt.io/user-guide/operations/updating_and_deletion/#updating-kubevirt and used https://kubevirt.io/user-guide/operations/updating_and_deletion/#deleting-kubevirt to delete.

Even with everything seeming to be gone, when i reinstall I get timeout on kubectl -n kubevirt wait kv kubevirt --for condition=Available.

Any suggestions would be greatly appreciated.

Regards,

Alan

[dvo...@redhat.com]

On Thursday, February 11, 2021 at 11:04:20 AM UTC-5 alanf...@gmail.com wrote:

Hello Ashley,

I went to V0.38.1 following https://kubevirt.io/user-guide/operations/updating_and_deletion/#updating-kubevirt and used https://kubevirt.io/user-guide/operations/updating_and_deletion/#deleting-kubevirt to delete.

Even with everything seeming to be gone, when i reinstall I get timeout on kubectl -n kubevirt wait kv kubevirt --for condition=Available.

Any suggestions would be greatly appreciated.

I think this a bug in kubevirt. It appears the operator webhook leaked into the logic that uninstalls KubeVirt. We need to address this. 

I can see this in "pkg/virt-operator/resource/generate/install/strategy.go". i don't think we should be adding the operator webhook there to the list. 

        strategy.validatingWebhookConfigurations = append(strategy.validatingWebhookConfigurations, components.NewOpertorValidatingWebhookConfiguration(operatorNamespace))

        strategy.validatingWebhookConfigurations = append(strategy.validatingWebhookConfigurations, components.NewVirtAPIValidatingWebhookConfiguration(config.GetNamespace()))

        strategy.mutatingWebhookConfigurations = append(strategy.mutatingWebhookConfigurations, components.NewVirtAPIMutatingWebhookConfiguration(config.GetNamespace()))

[dvo...@redhat.com]

On Friday, February 12, 2021 at 8:58:59 AM UTC-5 dvo...@redhat.com wrote:

On Thursday, February 11, 2021 at 11:04:20 AM UTC-5 alanf...@gmail.com wrote:

Hello Ashley,

I went to V0.38.1 following https://kubevirt.io/user-guide/operations/updating_and_deletion/#updating-kubevirt and used https://kubevirt.io/user-guide/operations/updating_and_deletion/#deleting-kubevirt to delete.

Even with everything seeming to be gone, when i reinstall I get timeout on kubectl -n kubevirt wait kv kubevirt --for condition=Available.

Any suggestions would be greatly appreciated.

I think this a bug in kubevirt. It appears the operator webhook leaked into the logic that uninstalls KubeVirt. We need to address this. 

I can see this in "pkg/virt-operator/resource/generate/install/strategy.go". i don't think we should be adding the operator webhook there to the list. 

        strategy.validatingWebhookConfigurations = append(strategy.validatingWebhookConfigurations, components.NewOpertorValidatingWebhookConfiguration(operatorNamespace))

        strategy.validatingWebhookConfigurations = append(strategy.validatingWebhookConfigurations, components.NewVirtAPIValidatingWebhookConfiguration(config.GetNamespace()))

        strategy.mutatingWebhookConfigurations = append(strategy.mutatingWebhookConfigurations, components.NewVirtAPIMutatingWebhookConfiguration(config.GetNamespace()))

I was incorrect here, we do need to register the operator's validation webhook during the installation process, at least for now. This logic is nearly correct, but I did find one inconsistency that i've addressed here [1]. I don't think this is what you are experiencing though. 

Running the command below will likely resolve the issue you're encountering with re-install. 

kubectl delete validatingwebhookconfigurations virt-operator-validator

1. https://github.com/kubevirt/kubevirt/pull/5022