Unrecognized configuration key "quarkus.kubernetes-config.config-maps[1]"

[KimJohn Quinn]

Hello everyone,

Hoping this is a quick answer and something on my end configuration-wise that needs to change (or has changed with recent updates).

I am getting this error when building the native image and I believe it is the root of my issue when deploying to K8S:

Unrecognized configuration key "quarkus.kubernetes-config.config-maps[1]"

From the application.yml, the relevant section looks like:

    kubernetes-config:

      enabled: true

      fail-on-missing-config: false

      config-maps:

        - my-config

        - my-config-override

Previously, this deployed...

[Georgios Andrianakis]

This seems to be a mismatch in out YAML configuration handling...

@Roberto Cortez would know more

--
You received this message because you are subscribed to the Google Groups "Quarkus Development mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to quarkus-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/quarkus-dev/fe6ffb42-9eed-4f83-a567-967a947ddccbn%40googlegroups.com.

[KimJohn Quinn]

The larger stack trace causing me to ask above is this but the more I think about it I dont know if the above is my problem (it looks to be trying to get the configmap just using the wrong user):

io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: GET at: https://10.100.0.1/api/v1/namespaces/platform-445-staging/configmaps/ my-config. Message: configmaps "my-config" is forbidden: User "system:anonymous" cannot get resource "configmaps" in API group "" in the namespace "platform-445-staging". Received status: Status(apiVersion=v1, code=403, details=StatusDetails(causes=[], group=null, kind=configmaps, name= my  -config, retryAfterSeconds=null, uid=null, additionalProperties={}), kind=Status, message=configmaps " my-config" is forbidden: User "system:anonymous" cannot get resource "configmaps" in API group "" in the namespace "platform-445-staging", metadata=ListMeta(_continue=null, remainingItemCount=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=Forbidden, status=Failure, additionalProperties={}). at io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:681) at io.fabric8.kubernetes.client.dsl.base.OperationSupport.assertResponseCode(OperationSupport.java:620) at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:560) at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:521) at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleGet(OperationSupport.java:488) at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleGet(OperationSupport.java:470) at io.fabric8.kubernetes.client.dsl.base.BaseOperation.handleGet(BaseOperation.java:827) at io.fabric8.kubernetes.client.dsl.base.BaseOperation.getMandatory(BaseOperation.java:197) at io.fabric8.kubernetes.client.dsl.base.BaseOperation.get(BaseOperation.java:164) at io.fabric8.kubernetes.client.dsl.base.BaseOperation.get(BaseOperation.java:88) at io.quarkus.kubernetes.client.runtime.KubernetesConfigSourceProvider.getConfigMapConfigSources(KubernetesConfigSourceProvider.java:69) at io.quarkus.kubernetes.client.runtime.KubernetesConfigSourceProvider.getConfigSources(KubernetesConfigSourceProvider.java:45) at io.quarkus.runtime.configuration.ConfigUtils.addSourceProvider(ConfigUtils.java:172) at io.quarkus.runtime.configuration.ConfigUtils.addSourceProviders(ConfigUtils.java:186) at io.quarkus.runtime.generated.Config.readConfig(Config.zig:2358) at io.quarkus.deployment.steps.RuntimeConfigSetup.deploy(RuntimeConfigSetup.zig:60) at io.quarkus.runner.ApplicationImpl.doStart(ApplicationImpl.zig:524) at io.quarkus.runtime.Application.start(Application.java:101) at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:101) at io.quarkus.runtime.Quarkus.run(Quarkus.java:66) at io.quarkus.runtime.Quarkus.run(Quarkus.java:42) at io.quarkus.runtime.Quarkus.run(Quarkus.java:119) at io.quarkus.runner.GeneratedMain.main(GeneratedMain.zig:29)

[KimJohn Quinn]

Sorry, not trying to send anyone on a wild goose chase...just summarizing my issue:

- Deployment used to work (a month or so back)

- I dont recall seeing that config-map error so, unless i did through the pipelines, I dont know if that is the cause.

- It does look to be trying to read the config map from the right namespace but with the wrong user

Thanks.

[KimJohn Quinn]

I will have to dig through the pipelines to find the release that did work but as you point out it still appears to be an issue with reading the configmaps in K8S and not the config.  Right now we are running Quarkus 2.1.1.Final.

We are trying to investigate further as we just recently updated our EKS cluster (and Gitlab which manages the EKS cluster) to the latest version as well.

Of course this would happen right before a deployment ;)

>kubectl logs fusion-platform-client-c85ff7478-m7876 -n platform-445-staging
Aug 06, 2021 4:08:37 PM io.fabric8.kubernetes.client.Config
WARN: Error reading service account token from: [/var/run/secrets/kubernetes.io/serviceaccount/token]. Ignoring.
Aug 06, 2021 4:08:37 PM io.fabric8.kubernetes.client.Config
WARN: Error reading service account token from: [/var/run/secrets/kubernetes.io/serviceaccount/token]. Ignoring.
Aug 06, 2021 4:08:37 PM io.fabric8.kubernetes.client.Config
WARN: Error reading service account token from: [/var/run/secrets/kubernetes.io/serviceaccount/token]. Ignoring.
Aug 06, 2021 4:08:37 PM io.quarkus.runtime.ApplicationLifecycleManager run
ERROR: Failed to start application (with profile my-release)
io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: GET at: https://10.100.0.1/api/v1/namespaces/platform-445-staging/configmaps/my-config. Message: configmaps "my-config" is forbidden: User "system:anonymous" cannot get resource "configmaps" in API group "" in the namespace "platform-445-staging". Received status: Status(apiVersion=v1, code=403, details=StatusDetails(causes=[], group=null, kind=configmaps, name=my-config, retryAfterSeconds=null, uid=null, additionalProperties={}), kind=Status, message=configmaps "my-config" is forbidden: User "system:anonymous" cannot get resource "configmaps" in API group "" in the namespace "platform-445-staging", metadata=ListMeta(_continue=null, remainingItemCount=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=Forbidden, status=Failure, additionalProperties={}).

On Fri, Aug 6, 2021 at 3:26 PM Roberto Cortez <radc...@yahoo.com> wrote:

Hi,

Which version was working and which version is not working?

I believe this is related with https://github.com/quarkusio/quarkus/issues/19262, but it shouldn’t affect the configuration behaviour.

Cheers,

Roberto

-- 
You received this message because you are subscribed to the Google Groups "Quarkus Development mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to quarkus-dev...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/quarkus-dev/5452b177-27da-4460-8e8d-a5d767a3460cn%40googlegroups.com.

[KimJohn Quinn]

Further update,

We are starting to think this is an issue between our K8S upgrade to 1.21 from 1.19.

- 1.19 - it was working here (no code/config changes)

- 1.20 - dont know if it works or not we only upgraded here to get to latest

- 1.21 - does not work

We know it worked before and as a quick test we deployed it into an older cluster (1.17) and it looks to work.

Our plan, because we are in need, is to try and downgrade (recreate the cluster) to 1.20 and if that does not work go to 1.19.

We will investigate further too and provide any updates...

[KimJohn Quinn]

Maybe it is this according to the upgrade log for K8S?

Service account tokens bound to a pod is now a stable feature. The feature gates will be removed in 1.21 release. For more information, refer to notes below on the changelogs.

[Roberto Cortez]

Hi,

Which version was working and which version is not working?

I believe this is related with https://github.com/quarkusio/quarkus/issues/19262, but it shouldn’t affect the configuration behaviour.

Cheers,

Roberto

On 6 Aug 2021, at 20:22, KimJohn Quinn <k...@logicdrop.com> wrote:

[KimJohn Quinn]

Hello,

Depends on which question you are referring to...

- The Kubernetes/ConfigMap extension is working up to 1.20 on K8S.  1.21 (latest on EKS) gives an error regarding the token.  We verified this by downgrading to 1.19, then back up to 1.20 and both worked.

- The configuration key message seemed to be benign but I will have to dig through our pipelines to figure out where it occurred originally.  Using the latest Quarkus I do not see that issue.  If it helps any, it would probably be 1-2 versions back from the date of the original post.  I'll try to double-check and see where it was occurring.

You received this message because you are subscribed to a topic in the Google Groups "Quarkus Development mailing list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/quarkus-dev/LjrPX-xXMSk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to quarkus-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/quarkus-dev/613B7CBF-AD9A-41BE-980A-82C02613B512%40yahoo.com.