Recommended setup for REST-like web service

523 views
Skip to first unread message

Jesse Bickel

unread,
Nov 16, 2022, 5:19:43 PM11/16/22
to Keycloak User
I would like to protect some web API endpoints with keycloak. These will be accessed by machine users across the internet. From what I can see in OpenID Connect and OAuth 2.0 definitions, a "Client" would be the protected entity, e.g. our web service, while a user would be the machine across the internet accessing the service. With keycloak, I hoped to create a single realm with a single client and many users to access that client. The examples and guides I found show setting up a single client and then logging in with that client but the guidance goes no further. It doesn't make sense to provide users with client credentials, yet I cannot seem to get a token for a user without providing client credentials. Am I really supposed to create multiple clients for a single web application? I am probably missing something obvious here, but what is it?

I am trying keycloak 20.0.1.

Best,

Jesse

David Cook

unread,
Nov 16, 2022, 5:41:23 PM11/16/22
to Jesse Bickel, Keycloak User

I’m sure someone else will have a better answer for machine to machine interactions but I’ll share what I know generally.

 

When securing a web app with a Keycloak OIDC provider, you set up a Keycloak realm and a Keycloak client using the Authorization Grant. You configure your web app’s OIDC client using the Keycloak client credentials. When a human user visits your web app using a web browser, your web app redirects the human user’s browser to the Keycloak realm. The human user fills in their username and password, Keycloak verifies it, and then redirects the web browser back to your web app’s OIDC client which then retrieves an access token and an ID token.

 

Now for your case there is no human and no web browser so you can’t use the Authorization Grant, but I think you might be able to use Direct Access Grants. In that case, your web service will have the OIDC client (including the client id and client secret), and it’ll need to collect and on-send the username and user password from the machine using the web service to the OIDC provider. How you collect that user name and user password is up to you.

 

If you’re just using Keycloak for authentication, then that should be it. If you’re using it for authorization too, then you’ll need to get more information from someone else.

 

David Cook

Senior Software Engineer

Prosentient Systems

Suite 7.03

6a Glen St

Milsons Point NSW 2061

Australia

 

Office: 02 9212 0899

Online: 02 8005 0595

--
You received this message because you are subscribed to the Google Groups "Keycloak User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to keycloak-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-user/cfb0db28-d90b-4d85-a704-a032aa01db11n%40googlegroups.com.

Jesse Bickel

unread,
Nov 16, 2022, 5:49:16 PM11/16/22
to Keycloak User
On Wednesday, November 16, 2022 at 4:41:23 PM UTC-6 David Cook wrote:

When securing a web app with a Keycloak OIDC provider, you set up a Keycloak realm and a Keycloak client using the Authorization Grant. You configure your web app’s OIDC client using the Keycloak client credentials. When a human user visits your web app using a web browser, your web app redirects the human user’s browser to the Keycloak realm. The human user fills in their username and password, Keycloak verifies it, and then redirects the web browser back to your web app’s OIDC client which then retrieves an access token and an ID token.

This is helpful to know the typical flow, yes, thank you.

Now for your case there is no human and no web browser so you can’t use the Authorization Grant, but I think you might be able to use Direct Access Grants. In that case, your web service will have the OIDC client (including the client id and client secret), and it’ll need to collect and on-send the username and user password from the machine using the web service to the OIDC provider. How you collect that user name and user password is up to you.

Doesn't it also seem incorrect to reveal the username and password to the client? Perhaps the first flow you mentioned is OK for machine users too?

If you’re just using Keycloak for authentication, then that should be it. If you’re using it for authorization too, then you’ll need to get more information from someone else.

Yes, we will also want at least two groups (e.g. users and administrators) and to allow for authorization to particular routes (and probably more) within the service.

Thanks for the quick reply and explanation!

C R

unread,
Nov 17, 2022, 7:54:28 AM11/17/22
to Jesse Bickel, Keycloak User
Hi,

The Direct Access grant is almost never what you want or need, and if
a was a user, I would not use your service because of fear of you
stealing my credentials. Like David writes, for regular users you just
need the "standard" flow, known as the authorization code grant.

For machine-to-machine connections, there are different ways to
implement it. The simplest is just **locally** validate a JWT that the
remote machine retrieves itself from somewhere. Just make sure the
signature is valid and the token has not expired. For long lived
tokens, if you want, you can contact the token endpoint of Keycloak.
If there is no user context, the remote service can receive a service
account token which it can receive after authenticating with its own
client id and secret. You can restrict which token are valid by
requiring specific scopes or roles.

Regards,

CR

Le mer. 16 nov. 2022 à 23:49, Jesse Bickel
<jbi...@opentechstrategies.com> a écrit :
> --
> You received this message because you are subscribed to the Google Groups "Keycloak User" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to keycloak-use...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-user/f1103079-2a3f-4842-a9f4-c52c22031ee3n%40googlegroups.com.

Jesse Bickel

unread,
Nov 17, 2022, 12:48:46 PM11/17/22
to Keycloak User
Thank you all for your help and patience. As you can tell, I am new to OAuth, OpenID Connect, and keycloak. I think I am missing some more basic assumptions about what is happening between the User and the service/RP/Client. An as-yet unstated goal when using keycloak is to be able to use existing libraries both in our service/RP/Client and also in the Users software. The thought is that authentication and authorization could be streamlined without much customization and with common tool support. The service/RP/Client would like to distinguish Users by group membership for some requests. See more questions inline below.

On Thursday, November 17, 2022 at 6:54:28 AM UTC-6 CR wrote:
The Direct Access grant is almost never what you want or need, and if
a was a user, I would not use your service because of fear of you
stealing my credentials. Like David writes, for regular users you just
need the "standard" flow, known as the authorization code grant.

Thank you for confirming my intuition regarding sharing User credentials with the service/RP/Client. If the "standard" authorization code grant flow is truly standard, would many tools support this workflow for machine Users as well as human Users? In this case the service/RP/Client would get a token for a User from the OpenID Connect Provider (OP), but then what? Does the whole flow happen for every request or is there assumed to be some session between the User and the service/RP/Client bound to this token in service/RP/Client state? Or does the service/RP/Client then publish the token to the User who includes it on subsequent requests?
 
For machine-to-machine connections, there are different ways to
implement it.

Darn. I was hoping there is a streamlined way, a de facto standard way that is not too cumbersome to machine Users (either by a simple nature or by availability of libraries) and decently secures the service/RP/Client against unauthenticated/unauthorized use.
 
The simplest is just **locally** validate a JWT that the
remote machine retrieves itself from somewhere.

Does "locally" mean the service/RP/Client? The remote machine is the machine User? The User retrieves this JWT from the OP? Using which flow? Implicit flow?
 
Just make sure the
signature is valid and the token has not expired. For long lived
tokens, if you want, you can contact the token endpoint of Keycloak.

Does "you" here mean the service/RP/Client or the User again? Or the service/RP/Client asks keycloak whether the token is still valid?

If there is no user context, the remote service can receive a service
account token which it can receive after authenticating with its own
client id and secret. You can restrict which token are valid by
requiring specific scopes or roles.

Does "user context" mean context between the User and the service/RP/Client such as session state? If so, then it would be good to avoid such state if possible. I hoped or assumed the state would all be in whatever token the User passes to the service/RP/Client on subsequent requests. That kind of sounds like what you are suggesting with the JWT. Is the flow something like this for two requests?

[User] --login--> [OP]
[User] <--JWT-- [OP]
[User] --request 1 w/JWT --> [service] (service internally validates the consistency of the JWT as well as some values are as expected)
[User] <--response-- [service]
[User] --request 2 w/JWT --> [service] (service internally validates the consistency of the JWT as well as some values are as expected)
[User] <--response-- [service]

Thanks again for all your help.

Jesse

David Cook

unread,
Nov 17, 2022, 5:44:54 PM11/17/22
to Jesse Bickel, Keycloak User

You’ve got way too many questions for me to try to answer this line by line, but I had a thought…

 

Originally, you said “I would like to protect some web API endpoints with keycloak. These will be accessed by machine users across the internet.”

 

What machine users are these? Is this an IoT scenario? And you’re hoping to have each device be a “User”?

 

If so, you might want the “OAuth 2.0 Device Authorization Grant”. I don’t really know anything about it, but I just learned about Keycloak’s support of it after a bit of Googling. Check this link out: https://keycloak.discourse.group/t/oauth-2-0-device-authorization-grant/8890/5

 

--

 

I’d encourage you to keep exploring Keycloak, as I think your idea that “authentication and authorization could be streamlined without much customization” is a worthwhile one. I mostly support existing apps with their own custom AuthN/AuthZ, but if I were to write something new from scratch, I’d be trying to use IDAM software like Keycloak to take care of AuthN/AuthZ.

 

As I said before, I haven’t used Keycloak’s more advanced AuthZ capabilities, so I can’t help there, but I’d encourage you to read and experiment. (That said, I have created custom claim mappers to surface Groups and User Attributes, so that downstream apps can make authorization decisions based off Keycloak data, but that’s still custom AuthZ based on custom logic – it’s not using Keycloak’s fine-grained authorization policies.)

 

David Cook

Senior Software Engineer

Prosentient Systems

Suite 7.03

6a Glen St

Milsons Point NSW 2061

Australia

 

Office: 02 9212 0899

Online: 02 8005 0595

 

From: keyclo...@googlegroups.com <keyclo...@googlegroups.com> On Behalf Of Jesse Bickel


Sent: Friday, 18 November 2022 4:49 AM
To: Keycloak User <keyclo...@googlegroups.com>

--

You received this message because you are subscribed to the Google Groups "Keycloak User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to keycloak-use...@googlegroups.com.

Jesse Bickel

unread,
Nov 18, 2022, 10:28:17 AM11/18/22
to Keycloak User
Thanks for the encouragement, David, see inline replies below.

On Thursday, November 17, 2022 at 4:44:54 PM UTC-6 David Cook wrote:

Originally, you said “I would like to protect some web API endpoints with keycloak. These will be accessed by machine users across the internet.”

What machine users are these? Is this an IoT scenario? And you’re hoping to have each device be a “User”?

These would be distinct organizations' software users. It could be anything from manual curl calls to fully integrated and automated software callers in whatever languages those organizations use. So it's not a hardware user or device in particular but by "machine" I meant primarily "automated" or "software" or "non-browser" users.

I’d encourage you to keep exploring Keycloak, as I think your idea that “authentication and authorization could be streamlined without much customization” is a worthwhile one. I mostly support existing apps with their own custom AuthN/AuthZ, but if I were to write something new from scratch, I’d be trying to use IDAM software like Keycloak to take care of AuthN/AuthZ.

Yes, and during this exploration I am confident keycloak can support our needs, but I need to get past this initial hump.

My key question at this point (pardon the pun): will these software users be modeled as Users or as Clients or what? It seems strange to model them as Clients because I do not know what to put for all the redirect URIs, etc., because this is not our software it is someone else's software, and these really are users from our perspective. If I can create exactly one Client and then multiple Users (one for each organization or user at that organization), this seems most congruent to me, but then I am trying to discover which flow to use, because the "Implicit Flow" sounds like it is discouraged and on the way out. A follow-up question (if still unclear) is what is the expected request-response workflow for two requests from a User, including the User initial request? The specs and so many tutorials start with a Client, but a whole flow actually starts with a User.

I will keep experimenting. One thing I think I need to try is to avoid the `token` endpoint when trying to get a token for these users, and try using the `auth` endpoint for a user attempting to access a client set up with zero boxes checked as described in https://groups.google.com/g/keycloak-user/c/75iuKmiYvBU/m/qapTtBASAAA.

C R

unread,
Nov 18, 2022, 5:32:27 PM11/18/22
to Jesse Bickel, Keycloak User
Le jeu. 17 nov. 2022 à 18:48, Jesse Bickel
<jbi...@opentechstrategies.com> a écrit :
> Thank you for confirming my intuition regarding sharing User credentials with the service/RP/Client. If the "standard" authorization code grant flow is truly standard, would many tools support this workflow for machine Users as well as human Users? In this case the service/RP/Client would get a token for a User from the OpenID Connect Provider (OP), but then what? Does the whole flow happen for every request or is there assumed to be some session between the User and the service/RP/Client bound to this token in service/RP/Client state? Or does the service/RP/Client then publish the token to the User who includes it on subsequent requests?

Standard oidc/oauth2 libraries for your language will do the job. You
may need to unpack the claims to verify a scope or role is there, but
it's trivial and it should be documented by the library.

>> For machine-to-machine connections, there are different ways to
>> implement it.
>
>
> Darn. I was hoping there is a streamlined way, a de facto standard way that is not too cumbersome to machine Users (either by a simple nature or by availability of libraries) and decently secures the service/RP/Client against unauthenticated/unauthorized use.

You can use the same code as for users. Zero extra coding. See below,

>>
>> The simplest is just **locally** validate a JWT that the
>> remote machine retrieves itself from somewhere.
>
>
> Does "locally" mean the service/RP/Client? The remote machine is the machine User? The User retrieves this JWT from the OP? Using which flow? Implicit flow?
[...]

My 2c.
- ignore all the other flows besides the authorization code, except if
you **really** know what you're doing.
- If there is a user context (==user logs in interactively), use the
authorization code.
- If there is no user context (== a background job in de client
application, not one run by the user), use the authorization code for
the service account of the application.

C.

Jesse Bickel

unread,
Nov 21, 2022, 10:50:37 AM11/21/22
to Keycloak User
C, thank you again for your help. I am going to try to clarify the problem a little further below.

On Friday, November 18, 2022 at 4:32:27 PM UTC-6 C wrote:
Standard oidc/oauth2 libraries for your language will do the job. You
may need to unpack the claims to verify a scope or role is there, but
it's trivial and it should be documented by the library.
 
This is encouraging, thank you.
 
>> For machine-to-machine connections, there are different ways to
>> implement it.
>
>
> Darn. I was hoping there is a streamlined way, a de facto standard way that is not too cumbersome to machine Users (either by a simple nature or by availability of libraries) and decently secures the service/RP/Client against unauthenticated/unauthorized use.

You can use the same code as for users. Zero extra coding. See below,

>>
>> The simplest is just **locally** validate a JWT that the
>> remote machine retrieves itself from somewhere.
>
>
> Does "locally" mean the service/RP/Client? The remote machine is the machine User? The User retrieves this JWT from the OP? Using which flow? Implicit flow?
[...]

My 2c.
- ignore all the other flows besides the authorization code, except if
you **really** know what you're doing.
- If there is a user context (==user logs in interactively), use the
authorization code.
- If there is no user context (== a background job in de client
application, not one run by the user), use the authorization code for
the service account of the application.

Our product is a public-facing web service to be called by our users. Everything I see regarding calling services (e.g. with a bearer-only token) assumes that the caller is software under our control (one of our services/RPs/Clients) that makes a request to another bit of software under our control (another service/RP/Client), for example the diagram: https://invidious.weblibre.org/watch?v=FyVHNJNriUQ&t=2200.

In your first case, "user logs in interactively", there is an html login page to process with cookies, sessions, etc., not great for a software caller but maybe technically possible.
In your second case, "background job in de client application", this again falls under the assumption of a second service/RP/Client under our control.

The use case I am presenting is exactly one service/RP/Client under our control. I don't want to share that Client's secrets with our users or their software because that seems inappropriate as you confirmed. I also don't want an html form involved in our users' workflows because that is going to be brittle. I also don't want to share our users secrets with our one Client.  I also do not want to create and maintain multiple Clients and associated service accounts for each user because that seems inappropriate and overkill. Does this clarify the dilemma? Am I being obtuse?

C R

unread,
Nov 21, 2022, 12:37:10 PM11/21/22
to Jesse Bickel, Keycloak User
Is it a command line application we're talking about? In that case,
David already mentioned the Device Authorization Grant. It works like
how to link a Chromecast in case you've seen it before. Make sure the
token is kept long enough in the client or that it can be refreshed.

C.

Le lun. 21 nov. 2022 à 16:50, Jesse Bickel
<jbi...@opentechstrategies.com> a écrit :
>
> --
> You received this message because you are subscribed to the Google Groups "Keycloak User" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to keycloak-use...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-user/ce0d8802-09ee-463a-a8a5-d35848a34dc1n%40googlegroups.com.

Jesse Bickel

unread,
Dec 2, 2022, 6:45:24 PM12/2/22
to Keycloak User
See below.

On Monday, November 21, 2022 at 11:37:10 AM UTC-6 C wrote:
Is it a command line application we're talking about? In that case,
David already mentioned the Device Authorization Grant. It works like
how to link a Chromecast in case you've seen it before. Make sure the
token is kept long enough in the client or that it can be refreshed.

Yes, it could be a command-line application running from cron, for example. The Device Authorization Grant looks like it would work well enough for this scenario, thank you both!

After much discussion, there is still the open question of how folks typically do this. Do folks typically model these entities (scripts or software calling a bearer-only back-end service via internet) as Clients or as Users? I think I was stuck on modeling them as Users, but it sounds like folks use Clients, e.g. https://stackoverflow.com/questions/52230634/issuing-api-keys-using-keycloak
Reply all
Reply to author
Forward
0 new messages