Dynamic/custom scopes PR https://github.com/keycloak/keycloak/pull/7721

[Schuster Sebastian (IOC/PAU1)]

Hi everybody,

 

as Stian suggested, I would like to kick off a discussion on the topic of dynamic/custom scopes.

 

Background:

Before KC10, Keycloak accepted any scope parameter coming from the client. With Keycloak 10, it started to validate the scopes sent from the client and rejected unknown scopes (https://github.com/keycloak/keycloak/pull/6943) because the spec says the authorization server should respond with “invalid_scope” if “The requested scope is invalid, unknown, or malformed.”

 

This change caused some problem for people relying on the old behaviour.  In https://issues.redhat.com/browse/KEYCLOAK-14143, there was a discussion to solve this problem by

1. adding a switch to just reenable the old behaviour,
2. to make scope validation an SPI or
3. to add a standard mechanism to validate scopes via regular expressions.

 

For the approach of turning scope validation into an SPI, a PR was already submitted:

https://github.com/keycloak/keycloak/pull/7721

 

In our case, we do have quite a lot of scopes that are essentially validated elsewhere. So all three of the above-mentioned approaches would work for us.
I assume there are definitely use-cases for more dynamic scopes than the built-in Keycloak scopes, because they could be short-lived / bound to specific transactions / time-limited resources etc.

The SPI approach allows to capture these scenarios so I do like it.

 

If I get Stians concerns correctly, he is a little worried that people will just implement an SPI accepting all scopes making Keycloak break the spec and potentially creating an environment with security issues.

 

It would be really great to get other peoples’ opinions on this topic.

 

Thanks and best regards,

Sebastian

 

 

Mit freundlichen Grüßen / Best regards

Dr.-Ing. Sebastian Schuster

Project Delivery Berlin 22 (IOC/PDL22)
Bosch.IO GmbH | Ullsteinstr. 128 | 12109 Berlin | GERMANY | www.bosch.io
Tel. +49 30 726112-485 | Mobil +49 152 02177668 | Telefax +49 30 726112-100 | Threema / Threema Work: MF9VMEAE | Sebastian...@bosch.io

Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B
Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten Lücke; Geschäftsführung: Dr. Stefan Ferber, Dr. Aleksandar Mitrovic, Yvonne Reckling

[Leandro de Bortoli]

We have a similar requirement for dynamic scopes,  we are implementing a consent API based on the fapi lodging intent, where the intent id it's passed as a dynamic scope, so It is essential to have an SPI to validate the scopes dynamically or the ability to disable scope validation altogether.

As for breaking the specification, I'm not sure how this would do it, the RFC 6749 states "The authorization server MAY fully or partially ignore the scope requested by the client, based on the authorization server policy". My understanding it's that the policy It's not specified and can be tweaked by the authorization server.

Also, the Keycloak ClientModel interface has a getDinamicScope for this scenario, but unfortunately, it's not implemented.

[Schuster Sebastian (IOC/PAU1)]

Reading that part of the spec, I think it leaves a little room for interpretation (surprise, surprise):

  The authorization server MAY fully or partially ignore the scope

   requested by the client, based on the authorization server policy or

   the resource owner's instructions.  If the issued access token scope

   is different from the one requested by the client, the authorization

   server MUST include the "scope" response parameter to inform the

   client of the actual scope granted.

The way I would understand this section is that it would actually be fine to ignore unknown scopes
as long as the authorization server responds with the set of known/granted scopes specifically.

 

In that light the section https://datatracker.ietf.org/doc/html/rfc6749#section-4.1.2.1

invalid_scope

               The requested scope is invalid, unknown, or malformed.

just states what error is returned when the authorization server choses to not ignore a scope.

I would say the above argumentation would at least allow to override the now-standard behaviour
of Keycloak to always return invalid_scope on unknown scopes using an SPI as implemented in https://github.com/keycloak/keycloak/pull/7721

@stian WDYT?

 

Best regards,

Sebastian

 

Mit freundlichen Grüßen / Best regards

Dr.-Ing. Sebastian Schuster

Project Delivery Berlin 22 (IOC/PDL22)
Bosch.IO GmbH | Ullsteinstr. 128 | 12109 Berlin | GERMANY | www.bosch.io
Tel. +49 30 726112-485 | Mobil +49 152 02177668 | Telefax +49 30 726112-100 | Threema / Threema Work: MF9VMEAE | Sebastian...@bosch.io

Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B
Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten Lücke; Geschäftsführung: Dr. Stefan Ferber, Dr. Aleksandar Mitrovic, Yvonne Reckling

 

--
You received this message because you are subscribed to the Google Groups "Keycloak Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to keycloak-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-dev/faf36e2f-96c7-4f78-82ad-4d7709be446bn%40googlegroups.com.

[Pedro Igor Craveiro e Silva]

Did you check the `ClientStorageProviderSpi`? 

In the client model, there is a `getDynamicClientScope` that can be used to resolve additional scopes. I think the only implementation we have for this method is related to the Openshift integration work we did a long time ago. There we had to resolve additional scopes to make things work.

In your case, perhaps the client storage could use delegation to delegate calls to the JPA storage while overriding the behavior of that method.

If that works for you, I would use it. Otherwise, I agree about having an SPI.

W.r.t to people breaking spec and creating an environment with security issues, I would say people can do that anyway through other SPIs too. Not sure if it is a problem.

To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-dev/141601B1-547F-46FD-A93B-2857F9EFF9F6%40bosch.com.

[Aline Borges]

In regards of the following section from RFC 6749:

"The authorization server MAY fully or partially ignore the scope requested by the client, based on the authorization server policy or the resource owner's instructions. If the issued access token scope is different from the one requested by the client, the authorization server MUST include the "scope" response parameter to inform the client of the actual scope granted."

My interpretation is in line of yours, Sebastian. Therefore, invalid_scopes should be processed according to a custom validation configuration specifically to Authorization server in question. In addition, in the light of this RFC, my vision is that the more appropriate manner is to provide a SPI, Pigor. In this way, the implementation of the FAPI Lodging Intent would be possible, as suggested by you, Leandro.

[Pedro Igor Craveiro e Silva]

On Thu, Jun 17, 2021 at 10:24 AM Aline Borges <afsoare...@gmail.com> wrote:

In regards of the following section from RFC 6749:

"The authorization server MAY fully or partially ignore the scope requested by the client, based on the authorization server policy or the resource owner's instructions. If the issued access token scope is different from the one requested by the client, the authorization server MUST include the "scope" response parameter to inform the client of the actual scope granted."

My interpretation is in line of yours, Sebastian. Therefore, invalid_scopes should be processed according to a custom validation configuration specifically to Authorization server in question. In addition, in the light of this RFC, my vision is that the more appropriate manner is to provide a SPI, Pigor. In this way, the implementation of the FAPI Lodging Intent would be possible, as suggested by you, Leandro.

There are other ways to support Lodging Intent other than using scopes, right? Is using the scope the most common solution to this problem?

 

To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-dev/8d631977-214b-48c4-b8fa-0f2c9d133becn%40googlegroups.com.

[Leandro de Bortoli]

There are other ways, like and custom parameter or custom claims, but we have to follow a defined security profile, the section 7.1.2 defines the dynamic scope.

[Marek Posolda]

Few points to this:

- As pointed, the specification allows various interpretations around scope validations. IMO it will be good to stick to current behaviour and reject unknown scopes by default, but at the same time, have an easy way to allow "dynamic" scopes. I think it is needed that people can relax on scopes validations without need to implement their own provider of any SPI.

- Regarding scope validation, I am thinking that instead of introducing separate SPI, we can possibly re-use the existing client policies SPI, which was recently added, and possibly have the executor dedicated to validate scopes? This would mean that we may need to add the default validation to the separate executor, which will be added to some default policy. This will first require to have auto-created default client policy added to each realm. Auto-created client policy is not yet available, but I think that from the long term perspective, it may be way to go https://issues.redhat.com/browse/KEYCLOAK-18463. Administrators might be able to remove auto-created policy or link to the client profile, which won't have the default executor. This will allow effectively disable scope validations without any changes. And I hope it will work fine with the use-cases like lodging intent (See below).

- Related note: It will be nice if Keycloak has built-in support for the Brasil OpenBanking FAPI Lodging intent and dynamic scopes. IMO it will be nice that whatever we do regarding dynamic scopes validations, it would be also aligned with the lodging intent needs. Leandro, do you have a chance to start a discussion for this and eventually send the PR for this feature? Or at least share your implementation if you already have something? Just a note, that there is FAPI WG meeting every two weeks where are discussions about FAPI related specifications and features. See https://groups.google.com/g/keycloak-dev/c/0K9XI0t5zLI. This is just a note for the case you missed this, but I don't want to force you to join :-)

Thanks,
Marek

To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-dev/fddbcaaf-d075-4ded-bd45-7c07e7c21633n%40googlegroups.com.

[Pedro Igor Craveiro e Silva]

On Tue, Jun 22, 2021 at 10:54 AM Marek Posolda <mpos...@redhat.com> wrote:

Few points to this:

- As pointed, the specification allows various interpretations around scope validations. IMO it will be good to stick to current behaviour and reject unknown scopes by default, but at the same time, have an easy way to allow "dynamic" scopes. I think it is needed that people can relax on scopes validations without need to implement their own provider of any SPI.

- Regarding scope validation, I am thinking that instead of introducing separate SPI, we can possibly re-use the existing client policies SPI, which was recently added, and possibly have the executor dedicated to validate scopes? This would mean that we may need to add the default validation to the separate executor, which will be added to some default policy. This will first require to have auto-created default client policy added to each realm. Auto-created client policy is not yet available, but I think that from the long term perspective, it may be way to go https://issues.redhat.com/browse/KEYCLOAK-18463. Administrators might be able to remove auto-created policy or link to the client profile, which won't have the default executor. This will allow effectively disable scope validations without any changes. And I hope it will work fine with the use-cases like lodging intent (See below).

IMO, this is not only about validation but also how to handle these dynamic scopes, including how they are rendered in the consent screen. I would also expect some logic to query additional information from external systems?

A new SPI sounds more reasonable to me. Or even a review of the client storage SPI to make this customization easier to implement. I would prefer the former though.

 

To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-dev/ffb23cfa-d3e7-81e3-4b15-ba49f7fa0f71%40redhat.com.

[Schuster Sebastian (IOC/PAU1)]

Hi everybody,

 

thanks for the hint with the getDynamicClientScopes() @Pedro. I took a look at it and although I think it could do the job, I don’t think it would be a great solution since
a) ClientStorageProviderSPI is pretty heavyweight and
b) from the name it should be more concerned with how clients are stored technically and not about the business logic of how scopes are validated for an individual client – this sounds like a different concern to me.

 

I also like Marek’s idea make it easy to turn off scope validation, maybe as a simple switch in the default implementation of scope validation.

 

I also agree with Pedro that for proper dynamic scopes, especially when it comes to examples like lodging intent, this should be visible on the consent screen.
I would also prefer to have a separate SPI for this. But then https://github.com/keycloak/keycloak/pull/7721 would probably not be sufficient as things like the calculation of scopes to show on the consent screen
(https://github.com/keycloak/keycloak/blob/458c841c390589ec95e173e2d0955be54cdce887/services/src/main/java/org/keycloak/services/managers/AuthenticationManager.java#L1143) and more would have to be adapted.

 

Tbh. I find it hard to judge whether having a separate SPI for this is the best way to go or whether this should be part of client policies… It touches client, client scope and user (consent), so its somewhere in the middle…

 

Best regards,

Sebastian

 

 

Mit freundlichen Grüßen / Best regards

Dr.-Ing. Sebastian Schuster

Project Delivery Berlin 22 (IOC/PDL22)
Bosch.IO GmbH | Ullsteinstr. 128 | 12109 Berlin | GERMANY | www.bosch.io
Tel. +49 30 726112-485 | Mobil +49 152 02177668 | Telefax +49 30 726112-100 | Threema / Threema Work: MF9VMEAE | Sebastian...@bosch.io

Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B
Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten Lücke; Geschäftsführung: Dr. Stefan Ferber, Dr. Aleksandar Mitrovic, Yvonne Reckling

 

To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-dev/CA%2B3s2iR5KVE2AyMJOr89Xys75-%3Dsu550%3DpHT%3D%2BN0ZiD1rSK51A%40mail.gmail.com.

[Marek Posolda]

On 22. 06. 21 18:24, Schuster Sebastian (IOC/PAU1) wrote:

Hi everybody,

 

thanks for the hint with the getDynamicClientScopes() @Pedro. I took a look at it and although I think it could do the job, I don’t think it would be a great solution since
a) ClientStorageProviderSPI is pretty heavyweight and
b) from the name it should be more concerned with how clients are stored technically and not about the business logic of how scopes are validated for an individual client – this sounds like a different concern to me.

+1 to these concerns. Using Client Storage SPI seems more like a workaround to me.

 

I also like Marek’s idea make it easy to turn off scope validation, maybe as a simple switch in the default implementation of scope validation.

 

I also agree with Pedro that for proper dynamic scopes, especially when it comes to examples like lodging intent, this should be visible on the consent screen.
I would also prefer to have a separate SPI for this. But then https://github.com/keycloak/keycloak/pull/7721 would probably not be sufficient as things like the calculation of scopes to show on the consent screen
(https://github.com/keycloak/keycloak/blob/458c841c390589ec95e173e2d0955be54cdce887/services/src/main/java/org/keycloak/services/managers/AuthenticationManager.java#L1143) and more would have to be adapted.

 

Tbh. I find it hard to judge whether having a separate SPI for this is the best way to go or whether this should be part of client policies… It touches client, client scope and user (consent), so its somewhere in the middle…

+1 for Pedro and your point here. I proposed client policies just for the use-case of disabling strict validation of scope parameterc. But regarding full support of dynamic client scopes including consent screen etc, it seems that separate SPI would have more sense.

Maybe the support for lodging intent can be added as prototype to see what would be all the capabilities required for the SPI? Maybe some design/prototype of it can help. Hopefully it will help us to see what would be best approach to easily disable scope validations for those users, who are interested just in that use-case of disable strict scopes validation.

Marek

[Leandro de Bortoli]

We are using an external IDP for user identification and acquisition of consent, so regarding lodging intent on keycloak the only requirement we have is to disable strict validation of scopes. Because of this, I'm not familiar with how the consent screen is rendered on keycloak, but to add full lodging intent support to it, I think it would be necessary to query the intent on an external resource server and render the consent screen accordingly, this would need to be customizable. IMO, the support for lodging intent on the consent screen and support for dynamic scopes can be treated as separated things, especially since it's possible to use lodging intent without dynamic scopes.

It crossed my mind to make a contribution adding the attribute "dynamicScope" to the ClientScopeModel and add the validation accordingly(by prefix), the issue that I found is that there is not a specified way to validate custom scopes, so I think that an SPI it's more appropriate.

[Stian Thorgersen]

On Thu, 17 Jun 2021 at 09:06, Schuster Sebastian (IOC/PAU1) <Sebastian...@bosch.io> wrote:

Reading that part of the spec, I think it leaves a little room for interpretation (surprise, surprise):

  The authorization server MAY fully or partially ignore the scope

   requested by the client, based on the authorization server policy or

   the resource owner's instructions.  If the issued access token scope

   is different from the one requested by the client, the authorization

   server MUST include the "scope" response parameter to inform the

   client of the actual scope granted.

The way I would understand this section is that it would actually be fine to ignore unknown scopes
as long as the authorization server responds with the set of known/granted scopes specifically.

 

In that light the section https://datatracker.ietf.org/doc/html/rfc6749#section-4.1.2.1

invalid_scope

               The requested scope is invalid, unknown, or malformed.

just states what error is returned when the authorization server choses to not ignore a scope.

I would say the above argumentation would at least allow to override the now-standard behaviour
of Keycloak to always return invalid_scope on unknown scopes using an SPI as implemented in https://github.com/keycloak/keycloak/pull/7721

@stian WDYT?

I'd say that's even an argument that Keycloak shouldn't return an error with invalid scopes, but rather just return the subset of known scopes.

I can see the need to make scopes themselves more dynamic, but not sure I see the argument to make the requested scopes validation pluggable.

[Schuster Sebastian (IOC/PAU1)]

But if you want to support things like lodging intent, Keycloak would have to return a dynamically requested scope for a specific transaction if it was granted by the user (or no scope at all if all requested scopes were granted).

 

Mit freundlichen Grüßen / Best regards

Dr.-Ing. Sebastian Schuster

Project Delivery Berlin 22 (IOC/PDL22)
Bosch.IO GmbH | Ullsteinstr. 128 | 12109 Berlin | GERMANY | www.bosch.io
Tel. +49 30 726112-485 | Mobil +49 152 02177668 | Telefax +49 30 726112-100 | Threema / Threema Work: MF9VMEAE | Sebastian...@bosch.io

Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B
Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten Lücke; Geschäftsführung: Dr. Stefan Ferber, Dr. Aleksandar Mitrovic, Yvonne Reckling

 

From: Stian Thorgersen <stho...@redhat.com>

Reply to: "st...@redhat.com" <st...@redhat.com>
Date: Wednesday, 23. June 2021 at 12:30
To: "Schuster Sebastian (IOC/PAU1)" <Sebastian...@bosch.io>
Cc: Leandro de Bortoli <leandrob...@gmail.com>, Keycloak Dev <keyclo...@googlegroups.com>

[Schuster Sebastian (IOC/PAU1)]

So how do we proceed here? I think https://github.com/keycloak/keycloak/pull/7721 would be the KISS solution of the problem.

Maybe it could be beneficial if the SPI instead of only returning true/false on the whole list returns a (sub-)-list of valid scopes, empty

list if invalid_scpope should be returned. Then you could also tell the client he got less scopes from the SPI for other reasons than

the user not granting a certain scope (future Keycloak feature, hopefully). But not sure if that’s super useful…

 

Best regards,

Sebastian

 

Mit freundlichen Grüßen / Best regards

Dr.-Ing. Sebastian Schuster

Project Delivery Berlin 22 (IOC/PDL22)
Bosch.IO GmbH | Ullsteinstr. 128 | 12109 Berlin | GERMANY | www.bosch.io
Tel. +49 30 726112-485 | Mobil +49 152 02177668 | Telefax +49 30 726112-100 | Threema / Threema Work: MF9VMEAE | Sebastian...@bosch.io

Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B
Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten Lücke; Geschäftsführung: Dr. Stefan Ferber, Dr. Aleksandar Mitrovic, Yvonne Reckling

 

To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-dev/DCC56653-45A8-4C14-9DCA-0D2D7DD0CD5D%40bosch.com.

[Pedro Igor Craveiro e Silva]

From an implementation perspective, I'm wondering that instead of a simple method returning a boolean we could allow returning a client scope instance.

The reason is that it should help to support future requirements such as customizations to consent. 

Similarly to the getDynamicClientScope from the ClientModel. As I mentioned before.

[Everton Godoi]

Hello everybody! Someone managed to do this? Because I'm working in a project to implement Open Banking Brasil, and I need to use a dynamic scope or by parameters, but I couldn't in keycloak configuration, if someone may help me I would appreciate.

Best regards,

Everton Godoi

[Daniel Gozalo Barquilla]

Hi Everton,

This is a very timely message! 

I have a working POC making use of Dynamic / Parameterized Scopes to control the information that we include in the generated JWTs with great results. 

I'm in the middle of creating a design proposal to have this implemented in Keycloak, so I'll let you know when I open the PR. Feedback will be greatly appreciated.

Regards,

Daniel.

Daniel Gozalo

Principal Software Engineer

dgoz...@redhat.com

To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-dev/36fd78f2-2291-4c5a-a082-3da355cbe06dn%40googlegroups.com.

[Everton Godoi]

Hi Daniel, 

Firstly thank you for to sent this good news! =) 

Let's will wait the version with this implementation, to start the tests, and I'll send a feedback! 

Regards,

Everton Godoi.