Need help with KIALI OpenId with RBAC on AKS clusters

[Kulbhushan Azad]

hi

i am facing issue while trying to setup KIALI with RBAC enabled using OpenID on AKS clusters.

Looking for any help or suggestion to get the RBAC working with openID

below is  my openId Configuration

spec:

  auth:

    openid:

      authorization_endpoint: https://login.microsoftonline.com/****/oauth2/v2.0/authorize

      client_id: ****

      disable_rbac: false

      issuer_uri: https://login.microsoftonline.com/****/v2.0

      scopes:

      - openid

      - profile

      - email

      username_claim: preferred_username

    password: secret:kiali:oidc-secret

    strategy: openid

kiali pod logs:

2021-03-30T18:28:13Z INF Starting Metrics Server on [:9090]

2021-03-30T18:32:46Z INF Not handling OpenId code flow authentication: No nonce code present. Login window timed out.

2021-03-30T18:33:29Z DBG Rest perf config QPS: 175.000000 Burst: 200

2021-03-30T18:33:29Z TRC [Kiali Cache] cacheIstioTypes map[AuthorizationPolicy:true DestinationRule:true Gateway:true PeerAuthentication:true RequestAuthentication:true ServiceEntry:true Sidecar:true VirtualService:true]

2021-03-30T18:33:29Z INF Kiali Cache is active for namespaces [.*]

2021-03-30T18:33:29Z DBG Rest perf config QPS: 175.000000 Burst: 200

2021-03-30T18:33:29Z INF [Prom Cache] Enabled

on web page: after succesful authentication with IDP , page gets redirected to kiali 

{"error":"Token is not valid or is expired","detail":"Unauthorized"}

[Marc Boorshtein]

i am facing issue while trying to setup KIALI with RBAC enabled using OpenID on AKS clusters.

Looking for any help or suggestion to get the RBAC working with openID

I don't think this will work because you can't generate an OIDC token that works with your cluster.  I think you'll need to use impersonation instead.

 

[Kulbhushan Azad]

ok so you are saying i can't use RBAC with AKS using openID approach ?

fyi i am able do get the auhtentication working fine and get the user claims etc without RBAC (disable_rbac: true )

But i have a strict need to have RBAC.

also , how can i  configure impersonation approach that you are suggesting ? any documentation/reference would really help

[Edgar Hernandez Garcia]

Hi, Kulbhushan!

It's a known issue that RBAC is not working with Kiali and AKS, even if AzureAD integration is turned on. It's mentioned in a note at the end of the "Requirements" section on Kiali's OpenID docs: https://kiali.io/documentation/latest/configuration/authentication/openid/#_requirements. As mentioned in this page, the current workaround is to use kube-oidc-proxy (or something similar) to make mappings of AzureAD users/groups with regular Kubernetes ServiceAccounts.

However, on this comment: https://github.com/kiali/kiali/issues/3294#issuecomment-761676735

it was mentioned that there is a chance that authentication with AzureAD may succeed by using the "accees_token" (Kiali is currently using the "id_token"). This goes in line with what is mentioned in step 4 of this Azure docs: https://docs.microsoft.com/en-us/azure/aks/concepts-identity#webhook-and-api-server. However, since I don't have an Azure account, I cannot try this.

I saw your messages on Slack -- sorry for not replying, I was doing some errands.

If you have some time to try the "access_token" approach, I can prepare a modified Kiali with some hard coded changes and see if authentication succeeds on Azure with RBAC turned on.

We may need several trials to find how to do the right communication with AzureAD/AKS... or... you know.... also iterate because of coding bugs.

If we can confirm that the "access_token" works, I will do the permanent changes that can be made available in the next Kiali release. I will need you to try the permanent changes.

Does this work for you? If so, I'll do the changes and let you know when you can try.

The "Impersonation" that Marc is suggesting is another alternative. In this case, you use Kiali's Header authentication to handle authentication externally by using a reverse proxy (see https://kiali.io/documentation/latest/configuration/authentication/header/) and inject impersonation headers. This probably can be achieved with oauth2_proxy, although I'm not sure how.

--
You received this message because you are subscribed to the Google Groups "kiali-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kiali-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kiali-dev/5e2290de-74d9-4bb4-a373-0e4004d00980n%40googlegroups.com.

[Kulbhushan Azad]

Thanks Edgar for the response

Yes i can try the "access_token" approach.

Pls let me know how and where to pick the changes.

--

Thanks
Kulbhusan

[Edgar Hernandez Garcia]

Hi, again

Check if you can use this image: quay.io/edgarhz/kiali:azure-poc-20210331-1223

(it's published on https://quay.io/repository/edgarhz/kiali?tab=tags)

You can see the code changes here: https://github.com/kiali/kiali/pull/3838

Notice there are some log messages echoing the tokens (which are sensitive).

If login doesn't work, I'll appreciate if you provide the log messages (of course, with the sensitive data removed).

If you haven't done so already, I think you should follow this guide to enable AKS+AAD integration: https://docs.microsoft.com/en-us/azure/aks/managed-aad

There are some holidays this week. If I don't reply soon, I surely should reply on monday.

I wait for your feedback!

- Edgar

[Kulbhushan Azad]

ok Thanks

i will try and let you know the result

--

Thanks
Kulbhusan

[Kulbhushan Azad]

Below is

auth:
  openid:
    authentication_timeout: 300
    client_id: xxx
    client_secret: xxx
    issuer_uri: https://login.microsoftonline.com/xxx/v2.0

    scopes:
    - openid
    - profile
    - email
    username_claim: preferred_username

2021-03-31T19:25:55Z INF Not handling OpenId code flow authentication: No nonce code present. Login window timed out.
2021-03-31T19:26:08Z ERR This is the raw token response: {"token_type":"Bearer","scope":"openid profile email Directory.Read.All Group.Read.All","expires_in":3599,"ext_expires_in":3599,"access_token":"xxx"}
2021-03-31T19:26:08Z ERR Parsed response: IdToken = xxx
2021-03-31T19:26:08Z ERR Parsed response: AccessToken = xxx
2021-03-31T19:26:08Z DBG Rest perf config QPS: 175.000000 Burst: 200
2021-03-31T19:26:08Z TRC [Kiali Cache] cacheIstioTypes map[AuthorizationPolicy:true DestinationRule:true Gateway:true PeerAuthentication:true RequestAuthentication:true ServiceEntry:true Sidecar:true VirtualService:true]
2021-03-31T19:26:08Z INF Kiali Cache is active for namespaces [.*]
2021-03-31T19:26:08Z DBG Rest perf config QPS: 175.000000 Burst: 200
2021-03-31T19:26:08Z INF [Prom Cache] Enabled
2021-03-31T19:26:09Z ERR The cluster is rejecting the IdToken. Trying the AccessToken...
2021-03-31T19:26:09Z ERR Original httpStatus = 401
2021-03-31T19:26:09Z ERR Original errMsg = Token is not valid or is expired
2021-03-31T19:26:09Z ERR Original detailedError = Unauthorized
2021-03-31T19:26:09Z DBG Rest perf config QPS: 175.000000 Burst: 200
2021-03-31T19:26:09Z ERR The cluster is rejecting the AccessToken. Nothing else to do...

the AKS cluster is AD enabled

and i do have the cluster admin role to the cluster

So what could be the next thing to try or look ?

--

Thanks
Kulbhusan

[Edgar Hernandez Garcia]

What's the message you are getting in the browser?

Is it the same bare "Token is not valid or is expired" / "Unauthorized"? or does it have more details about why it's not working?

[Kulbhushan Azad]

same as before

{"error":"Token is not valid or is expired","detail":"Unauthorized"}

--

Thanks
Kulbhusan

[Edgar Hernandez Garcia]

Hi again!

Can you, please, check your "App Registration"?

I have read this small guide for App Registrations in AzureAD: https://docs.pivotal.io/p-identity/1-11/azure-oidc/config-azure.html#register-an-app

Also, I'm aware that when you enable AzureAD integration in AKS, an "App" is created automatically. This is what I understand from this comment of another user using AKS: https://github.com/kiali/kiali/issues/3294#issuecomment-704500897

A common mistake is that people tend to create a new "App" to use for Kiali. The "Token is not valid or is expired" error is caused by this very common misconfiguration.

However, the right configuration is to reuse the existing App created for the cluster. In the case of AKS, you should reuse the App that is automatically created when enabling AzureAD integration.

So, please, make sure you are reusing this automatically created App, rather than creating a new one. I want to make sure that the config is right before going further.

[Kulbhushan Azad]

Yes I do have a new app created to be used for Kiali.

I am not sure if Microsoft would share the access and details to the app they implicitly created with the AKS.

Also the issuer uri might be different and/or the callback url they might have different(something standard) unlike what kiali needs as callback

I would consult Microsoft and seek their opinion and guidance too on this.

--

Thanks
Kulbhusan

[Edgar Hernandez Garcia]

I thought that the automatically created app was available in the Azure console. Isn't this the case?

If you cannot see that app in the console and make changes to it, that's definitely a blocker. That would make it impossible to get a token that is valid for the cluster.

[Kulbhushan Azad]

right , i don't have access to those client/server app , i believe they share among multiple tenants 

--

Thanks
Kulbhusan

[Kulbhushan Azad]

hi Edgar, 

some good news, after putting several hours on this i was able to get the right access token which the backend AKS cluster would accept.

so below is the url which i used in local to get the access token which works with the AKS 

https://login.windows.net/<tenant-id>/oauth2/authorize?resource=<resource-id>&client_id=<client-id>&response_type=token&redirect_uri=http://localhost:8080/dumpToken&response_mode=form_post&scope=openid&state=12345&nonce=678910

*Note the resource id is in public domain and shared across multiple tenant 

so if you could help incorporate that above url in kiali while calling the IDP for access token and send a patch then i can test the same.

i guess we can hardcode the resourceid for now and if it works then we can have it configurable in the kiali cr.

Let me know if you need the resourceid

--

Thanks
Kulbhusan

[Edgar Hernandez Garcia]

Hi!

It's very good news.

OK, pass the resource-id and I'll hard code it for testing. It should be a quick change.

You could either reply to the mail-list or reply privately. Since the resource-id is public, I think it doesn't matter what you choose.

Regards!