Question about "the object has been modified; please apply your changes to the latest version and try again"

[Jean-Marc Andre]

Hello,

After an update on my CRD object, the Reconcile method is called and I read my object again using the reconciler struct client.

I do what I have to do to reconcile and then I update my object again and from time to time I have the following error:

"the object has been modified; please apply your changes to the latest version and try again"

I have the feeling that the client read happens a bit too quickly after the first update and that it somehow reads an "old" value that has not yet been updated by the apiserver.

Could something like that happen?

I workaround this by "sleeping" a bit at the very beginning of the Reconcile method before the client reads my object again.

I wonder if there is something more appropriate to do than sleeping every single time I enter the reconcile method.

Thanks for your help

Jim

[Anwar C]

The best approach overcome this is to use server-side apply if your k8s version supports it and if you can refactor your code. I had this problem where I was reading the object , make modifications and then sending the update for update, which would fail since the versions mismatch. I had to manually copy over the version value to the new updated object and the code became too much repetitive and maintenance nightmare.

[Robert Metzger]

I'm not sure if using server-side apply is something to recommend when implementing an operator / controller, due to this bug: https://github.com/kubernetes/kubernetes/issues/95460.

I found this topic, because I'm running into an "the object has been modified" error while using SSA in a K8s operator (another argument against using SSA at the moment)

[Camila Macedo]

Hi Jean-Marc Andre,

See that when are developing operators, it is essential for the controller's reconciliation loop to be idempotent. By following the Operator pattern[1], we will create Controllers[2] who provide a reconcile function responsible for synchronizing resources until the desired state is reached on the cluster. That means that the reconciliation is like a loop that will still be running until it can ensure the desired state on the cluster. 

The error `" the object has been modified; please apply your changes to the latest version and try again"` usually happens because we fetched the resource and stored it in a variable in our reconcile. Then after a while, we try to update this resource using the data stored in the variable/controller. However, the state of the resource on the cluster changed from when we fetched the data using the client until when we try to update it. 

An alternative solution for the proposed idea, to use the server-side, can be to ensure that you re-fetch the resource (client.GET) before any update (client.Update). 

I hope that helps you out. 

[1] - https://kubernetes.io/docs/concepts/extend-kubernetes/operator/

[2] - https://kubernetes.io/docs/concepts/architecture/controller/

CAMILA MACEDO

SR. SOFTWARE ENGINEER 

RED HAT Operator framework

Red Hat UK

She / Her / Hers

IM: cmacedo

I respect your work-life balance. Therefore there is no need to answer this email out of your office hours.

--
You received this message because you are subscribed to the Google Groups "kubebuilder" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubebuilder...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubebuilder/20f5f8eb-ea0e-459f-a860-7f8c710be967n%40googlegroups.com.

[Phillip Wittrock]

+1 to what Camila said.

If you are reconciling reading from the client cache then object you are writing may be stale.  If this is an issue for you, then you could write your reconcile in 2 phases: phase 1) are any changes required based on the object read from the client cache?  phase 2) if yes, read a fresh copy of the object and update using that one