Self Node Remediation Controller on OKD - Error with Minor Kubernetes version number

49 views
Skip to first unread message

Rainer Molitor

unread,
Jun 7, 2023, 4:52:13 AM6/7/23
to medik8s
Hi guys,

I was running the Operator over a year without problems, but during the update (or install) of version v0.6.0 the self-node-remediation-controller-manager did not start with following error:

-----------------------------------------------------------------------------------------------------------------------------------------------

I0607 08:43:08.437365       1 request.go:601] Waited for 1.038655836s due to client-side throttling, not priority and fairness, request: GET:https://10.0.192.1:443/apis/operators.coreos.com/v1?timeout=32s

1.6861273904918637e+09           INFO     controller-runtime.metrics         Metrics server is starting to listen                {"addr": "127.0.0.1:8080"}

1.6861273904922063e+09           INFO     setup    Starting as a manager that installs the daemonset

1.6861273904934714e+09           ERROR  utils-taints          couldn't parse k8s minor version             {"minor version": "25+", "error": "strconv.Atoi: parsing \"25+\": invalid syntax"}

github.com/medik8s/self-node-remediation/pkg/utils.InitOutOfServiceTaintSupportedFlag

                /workspace/pkg/utils/taints.go:70

main.initSelfNodeRemediationManager

                /workspace/main.go:135

main.main

                /workspace/main.go:109

runtime.main

                /root/sdk/go1.19/src/runtime/proc.go:250

1.68612739049354e+09                ERROR  setup    unable to verify out of service taint support       {"error": "strconv.Atoi: parsing \"25+\": invalid syntax"}

main.initSelfNodeRemediationManager

                /workspace/main.go:136

main.main

                /workspace/main.go:109

runtime.main

                /root/sdk/go1.19/src/runtime/proc.go:250

---------------------------------------------------------------------------------------------------------------------------------------------

The problem may be the minor k8s version show by OKD (25+) in contrast with OpenShift (25).

Is there any help or solution?

Kind regards
Rainer Molitor

Michael Shitrit

unread,
Jun 8, 2023, 3:14:11 AM6/8/23
to Rainer Molitor, medik8s
Hi Rainer,

Thanks for a great catch and the root cause analysis  :-)
We will definitely fix that.
At the moment we are assessing the priority of this issue, if you have any feedback please let us know.

Michael

--
You received this message because you are subscribed to the Google Groups "medik8s" group.
To unsubscribe from this group and stop receiving emails from it, send an email to medik8s+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/medik8s/f45773b9-31e3-4cbd-8e3e-49b8cd87c073n%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Carlo Lobrano

unread,
Jun 8, 2023, 5:39:32 AM6/8/23
to medik8s
Hi Rainer,

what was the full OKD version string? I'd like to understand why the minor version is a string and not a digit

Thank you,
Carlo

Marc Sluiter

unread,
Jun 8, 2023, 6:08:24 AM6/8/23
to Carlo Lobrano, medik8s
On Thu, Jun 8, 2023 at 11:39 AM Carlo Lobrano <clob...@redhat.com> wrote:
Hi Rainer,

what was the full OKD version string? I'd like to understand why the minor version is a string and not a digit

Just to be clear, we need the kubernetes version :) `oc version` will report both OKD and K8s versions.

Regards, Marc

Message has been deleted
Message has been deleted

Marc Sluiter

unread,
Jun 9, 2023, 6:14:55 AM6/9/23
to Carlo Lobrano, medik8s
On Thu, Jun 8, 2023 at 12:08 PM Marc Sluiter <mslu...@redhat.com> wrote:


On Thu, Jun 8, 2023 at 11:39 AM Carlo Lobrano <clob...@redhat.com> wrote:
Hi Rainer,

what was the full OKD version string? I'd like to understand why the minor version is a string and not a digit

Just to be clear, we need the kubernetes version :) `oc version` will report both OKD and K8s versions.

Regards, Marc

Quick update: we were able to reproduce the issue on OKD 4.12.

BR, Marc
Message has been deleted
Message has been deleted

Rainer

unread,
Jun 11, 2023, 3:07:37 AM6/11/23
to Marc Sluiter, Carlo Lobrano, medik8s

Hi guys,

I tried to answer your question yesterday and today via the guy but I see only message deleted?!

So I try it this way:

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

$ oc version

Server Version: 4.12.0-0.okd-2023-04-16-041331

Kubernetes Version: v1.25.0-2824+27e744f55d2e99-dirty

$ kubectl version

Server Version: version.Info{Major:"1", Minor:"25+", GitVersion:"v1.25.0-2824+27e744f55d2e99-dirty", GitCommit:"27e744f55d2e99eb8265c837e2571e47559079f6", GitTreeState:"dirty", BuildDate:"2023-04-06T20:32:34Z", GoVersion:"go1.19.7", Compiler:"gc", Platform:"linux/amd64"}

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

It took me some time to find out where the "25+" came from. Someone must use the kubectl variant instead of the oc one.

The interesting thing is that Node Healthcheck Operator did not have the problem.

Thank you for your support.

Regards
Rainer
You received this message because you are subscribed to a topic in the Google Groups "medik8s" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/medik8s/kEA7jRlOsKw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to medik8s+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/medik8s/CAH5Q-kVLyduHq%3D3Xvz5yrzjFAVDk4ztr%2Bh4HxCuhFMgrP4eB%3DQ%40mail.gmail.com.

Rainer

unread,
Jun 11, 2023, 3:07:37 AM6/11/23
to Marc Sluiter, Carlo Lobrano, medik8s

should read GUI instead of guy!

And here a screenshot about the missing messages:


Regards
Rainer

Michael Shitrit

unread,
Jun 11, 2023, 3:12:02 AM6/11/23
to Rainer, Marc Sluiter, Carlo Lobrano, medik8s
Hi Rainer,

Thanks for your reply, since we consider this issue a blocker we are working on a fix that will be released before the next scheduled release.
Will update further once we have more concrete dates.

Michael

Rainer

unread,
Jun 12, 2023, 2:16:21 AM6/12/23
to Michael Shitrit, Marc Sluiter, Carlo Lobrano, medik8s

Hi Michael,

thank you very much.

Kind Regards
Rainer

Michael Shitrit

unread,
Jun 21, 2023, 4:30:34 PM6/21/23
to Rainer, Marc Sluiter, Carlo Lobrano, medik8s
Hi Rainer,

Fix is currently released as part of SNR 0.6.1 version.

Best regards,
Michael
Reply all
Reply to author
Forward
0 new messages