Authenticating with Oauth2/OpenID Connect

[Mike Orr]

I'm trying to get my Pyramid application to use a Keycloak
authentication server. Keycloak supports OAuth2 with OpenID Connect. I
don't need authorization: just authenticating the user and retrieving
their roles and metadata and maybe the refresh feature. I've looked
through several Oauth2, OpenID, and OpenID Connect libraries and there
seem to be a range of issues with all of them. So which one is most
supported and people here are using, which class should I use, and how
do I plug in my client ID, auth server URL, and secret? Some of them
use strange terminology and I'm not sure if I have what they need
under a different name.

pyramid_oauth2_client, oauth2demo, and yasso (a Python OAuth2
provider) The easiest to understand, but it hasn't been updated since
2012. I had to make it compatible with Python 3 and Pyramid 1.9.2. I
got the demo to sometimes log in to yasso but when it came back to the
site it got an invalid state error (akin to a CSRF token mismatch) --
it predates Pyramid's CSRF token support too.

pyramid_oauthlib, oauthlib, and requests-oauthlib -- This looks the
best supported and most actively developed, but I can't tell which
parts I need as a client; much of it is for building an Oauth2 server.
The OAuth2 terminology is new to me so that makes it harder.
'requests-oauthlib' is a wrapper for the 'requests' library to send
Oauth2 requests; I can't tell from the example whether I can use it
for authentication and if so how.

authomatic -- This is easier to understand than oauthlib and recently
updated (2017) but I'm not sure how to configure my provider. The
built-in classes are for Facebook et al. For other providers it steers
you to the OpenID class. I may be able to use the base class or make a
subclass like the provided Yahoo and Google ones, but its arguments
don't match my server parameters so I'm not sure how to configure it.
It delegates to the 'openid' package, which below.

openid -- There are several openid packages on PyPI: python-openid
2.25 (2010 [last download file]), python-openid2 (2018),
python-openid3 (2013), python3-openid (2017). All of these have
similarly-worded descriptions as if they're by the same team.
'python-openid2' has the most detailed description and is recent
(2018), but authomatic says it depends on 'openid' which I assume is
'python-openid'. Still, the authomatic doc may be old and
'python-openid2' may be compatible, but is it?

Now, can I just use OpenID or OpenID Connect and forget about the
Oauth2 packages since I don't need authorization? Going back to
'python-openid2', it says it's easy to use as a consumer, but the
example has its own BaseHTTPServer and strange method names and how
would I use it from Pyramid?

For OpenID Connect, the Keycloak docs suggest 'oic' (2019). Ah,
recent. But the docs say, "Unfortunately, the documentation has been
largely left unmaintained and there are various issues.... the current
examples included in this repository are unmaintained and there are
many issues." Great, recent code but out-of-date examples. And again
hard to figure out how to use.

Other OpenID Connect packages on PyPI are 'openid-connect' (2019):
"This is a low-level Python library for authentication against OpenID
Providers (e.g. Google). For high-level libraries see the Aiakos
project." Aiakos is a "Passwordless authentication gateway." And
'oidc' (2014). I think 'oidc' was suggested in one of the previous
ones.

There's also a repoze.who plugin and something about Venusian but
those sound old.

So what are people using now, and is there an example of configuring
it for a non-Big-Name provider or specifically Keycloak?




--
Mike Orr <slugg...@gmail.com>

[Bert JW Regeer]

https://github.com/requests/requests-oauthlib

> --
> You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com.
> To post to this group, send email to pylons-...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/CAH9f%3DuqvnNdEEowGt%3DWhqi4Y3TXdiTqoF0r%2Bo9JeewaLUxJ5iA%40mail.gmail.com.
> For more options, visit https://groups.google.com/d/optout.

[Jonathan Vanasco]

I have a project that I've been meaning to Open Source.  I'll try to make time tomorrow to repackage it... it's called pyramid_oauthlib_lowlevel and is a VERY LOWLEVEL integration tool for oauthlib against Pyramid, it was somewhat inspired by flask-oauthlib and there are example oauth1 and oauth2 system in the test suite.  

Something that may be confusing you, is that you essentially do need an oAuth server.  There are really three main ways to integrate oAuth :

* Server: Identity Authority

* Server: Client Application
* Developer Client

The "Client Application" and "Developer Client" are very similar in that they both authenticate against the Identity Authority, but they typically use very different methods of handling a credential exchange.  You can implement both with the requests-oauthlib library.

[Jonathan Vanasco]

mike-

if you haven't figured it out yet, hopefully my examples can help:  https://github.com/jvanasco/pyramid_oauthlib_lowlevel 

i've been meaning to release this for a while now, thanks for giving me a reason to.

in the tests, you can see a full flow of interacting with clients and servers:

https://github.com/jvanasco/pyramid_oauthlib_lowlevel/blob/master/pyramid_oauthlib_lowlevel/tests/oauth2.py

the client and server views are implemented in this file:

https://github.com/jvanasco/pyramid_oauthlib_lowlevel/blob/master/pyramid_oauthlib_lowlevel/tests/oauth2_app/views.py

[Mike Orr]

Thanks. This presents the same difficulty as 'pyramid_ouathlib',
figuring out which parts I need. It's like looking through a forest
for the right fern plants. My current authentication code and views is
4 pages long, the linked modules are 32 pages, and 'pyramid-oauthlib',
'ouathlib', and 'requests-ouathlib' are each around thirty pages of
code and docs. Since these modules are only part of your library, I'm
in awe of how much you know and wrote; it must have taken you weeks to
figure it all out and write the code.

Let describe my current auth system, what I've accomplished on OAuth
so far, and my assumptions so far of what a solution would look like.
Then you can tell me whether my assumptions are correct.

Current system:
A. The Unauthorized view mimics the login form (username/password(,
POSTing to the login view).
B. The login view checks the username syntax, and either invokes LDAP
(python-ldap3) to authenticate, or authenticates locally using a User
database record. The LDAP result may cause the User record to be
created or updated.
C. The Pyramid group principles are calculated from the User record
for authorization.
D. It's using SessionAuthenticationPolicy with 'pyramid_redis_sessions'.
E. We're moving to Keycloak because the LDAP server will be retired
soon and Keycloak manages passwords better than the existing code.

Oauth attempts:
- I implemented the 'requests-oauth' console script and made a simple
Pyramid application modeled after 'oauth2demo' from
'pyramid_oauth2_client'. Both of these instantiate a
'requests_ouathlib.OAuth2Session' and query the server for an
authorization URL.
- When I go to the URL, via redirect in the app or manually in the
script, I get a kind of home page instead of an authentication dialog.
This may be a problem with the server; I'm asking the server admins
about that. Have you seen this before?
- I need to get it to the point of successfully authenticating so I
can see what user data it returns and whether it's sufficient.

Assumptions about a solution:
1. The Unauthorized view will fetch an authorization URL and redirect to it.
2. Keycloak will authenticate the user, verify they're allowed to use
this application, and return user metadata, including the user's
roles. The roles are a list of strings and can be converted to Pyramid
group principals for authorization.
3. The login view will receive this information and store the relevant
parts in the session.
4. It can do authorization and property lookup from the session, so
the User record may go away or be converted into a log of past logins
and metadata.
5. If Keycloak accepts a nonce, I can pass it in step 1 and receive it
in step 3 as a kind of CSRF token.
6. Do I need to do the third step and fetch an access token? Or is
that just ito access a third-party resource (which I don't have any
of). Where do I fetch the access token and what do I do with it? Will
I want to refresh it? If so, where do I do that?

Also, one of the docs somewhere recommended using an Authorizion:
header, saying it was OAuth's preferred method. Is that compatible
with a Pyramid workflow? Who sends the header to whom? Or is that
again only for accessing third-party resources? How would I generate
it and what should I do if I receive it?

> --
> You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com.
> To post to this group, send email to pylons-...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/75af89f8-fb92-40e7-a014-aca324943e17%40googlegroups.com.

> For more options, visit https://groups.google.com/d/optout.

--
Mike Orr <slugg...@gmail.com>

[Jonathan Vanasco]

Thanks for the kind words.  This was actually pretty fast to do.  I think it took 2-3 days to build oAuth into our apps and almost everything was repackaged into this within that same week. This has been sitting in a private repo for a year or so, because it required some updates to oauthlib and requests-oauthlib that I wrote but hadn't been merged/released yet.  The tough bit was getting the testing harness set up right. 

I had written a lot of stuff explaining some things for you, and then realized I may be all wrong in terms of what you're trying to accomplish.  It reads like you're trying to do a lot of different authorization things, but then you specifically said no authorization but authentication.  So I'm getting tripped up a bit, especially by this line:  "I don't need authorization: just authenticating the user and retrieving their roles and metadata and maybe the refresh feature"

If the roles and metadata for the user are in Keycloak, then Keycloak is handling the Authentication (e.g. the user authenticates to keycloak by logging in) and you actually want Authorization to happen (which is what you described above).

So let's pull back for minute and get a better idea of what you're trying to do in terms of use-cases.

What is the intended interplay of your Keycloak server and your Pyramid application?  

oAuth can be very annoying because one spec/project/initiative/framework covers a handful of ways to implement it.  The different implementation methods are called "oAuth Flows" or "oAuth Grants".  The first think you need to do, is to decide exactly which grant you want do implement.  Depending on how you look at it, there are  between 4 and 8 flows covered by a handful of RFCs, all under the umbrella of oAuth. 

[Mike Orr]

On Tue, May 21, 2019 at 4:14 PM Jonathan Vanasco <jona...@findmeon.com> wrote:
>
> Thanks for the kind words. This was actually pretty fast to do. I think it took 2-3 days to build oAuth into our apps and almost everything was repackaged into this within that same week. This has been sitting in a private repo for a year or so, because it required some updates to oauthlib and requests-oauthlib that I wrote but hadn't been merged/released yet. The tough bit was getting the testing harness set up right.
>
>
> I had written a lot of stuff explaining some things for you, and then realized I may be all wrong in terms of what you're trying to accomplish. It reads like you're trying to do a lot of different authorization things, but then you specifically said no authorization but authentication. So I'm getting tripped up a bit, especially by this line: "I don't need authorization: just authenticating the user and retrieving their roles and metadata and maybe the refresh feature"
>
> If the roles and metadata for the user are in Keycloak, then Keycloak is handling the Authentication (e.g. the user authenticates to keycloak by logging in) and you actually want Authorization to happen (which is what you described above).
>
> So let's pull back for minute and get a better idea of what you're trying to do in terms of use-cases.
>
> What is the intended interplay of your Keycloak server and your Pyramid application?

I want it to do username/password checking and give me the user
metadata including the list of roles. Then I can do authorization from
the roles. By "no authorization" I mean the role/view ACLs would
remain in Pyramid; I don't want to put them into Keycloak and ask it
every page view whether this user has permission to this view. We
decided to migrate our applications this way because authentication is
the same across all web applications and Keycloak does it better, but
authorization is application-specific and every Pyramid, Django, or
other application has different needs and already has a working
system. To tie the authentication and authorization together I need
more user metadata than just the user ID. With LDAP, I fetch the
user's OU property, and that indirectly tells what their group
principals should be. I'm looking for something similar in Keycloak.

> oAuth can be very annoying because one spec/project/initiative/framework covers a handful of ways to implement it. The different implementation methods are called "oAuth Flows" or "oAuth Grants". The first think you need to do, is to decide exactly which grant you want do implement. Depending on how you look at it, there are between 4 and 8 flows covered by a handful of RFCs, all under the umbrella of oAuth.

Yes, that's what's so complicated, trying to figure out which classes
I need. I'm trying to implement the "Web Application Flow" in the
'requests-oauthlib' docs. So I would redirect to Keycloak, and it
would authenticate the user and redirect back. I'm not sure what
exactly to do after that. Keycloak would manage the user's account,
password, whether the user can access this application, and a list of
roles related to the application.

[Jonathan Vanasco]

On Tuesday, May 21, 2019 at 8:13:11 PM UTC-4, Mike Orr wrote:

I want it to do username/password checking and give me the user
metadata including the list of roles. Then I can do authorization from
the roles. By "no authorization" I mean the role/view ACLs would
remain in Pyramid; I don't want to put them into Keycloak and ask it
every page view whether this user has permission to this view.

....

 

Yes, that's what's so complicated, trying to figure out which classes
I need. I'm trying to implement the "Web Application Flow" in the
'requests-oauthlib' docs. So I would redirect to Keycloak, and it
would authenticate the user and redirect back. I'm not sure what
exactly to do after that. Keycloak would manage the user's account,
password, whether the user can access this application, and a list of
roles related to the application.

Okay. Within the realm of oAuth and Identity Management, Keycloak is handling Authentication of the user and your Pyramid App will be delegating Authorization to it.  Everything else you're talking about in regards to "Authorization" and interpreting the ACLs is within Pyramid and a slightly different concept of Authorization.  In terms of oAuth... Pyramid is a client and using Keycloak for Authorization, even though it seems like you're using it for Authentication.

Let me talk about a few things you brought up of order, and then try to answer some questions while I leave you with more...

Authorization Headers

This is fully compatible with Pyramid, don't worry. The oAuth specs prefer/suggest/request that certain oAuth information is exchanged within request headers and not within query strings or post data.  This data exchange generally happens between different servers and is implemented by the oauth client (likely requests-oauthlib) or the upstream authority server.  It's an implementation detail that is really unrelated to your usage.  IIRC, the two places this happens are in the post-authorization redirect from the Authority to your Callback URL and then again in a backend request from your server to the Authority.

oAuth Flows and Grants

Forget about classes in libraries and just focus on the Spec/Use-Case. I'm pretty sure you want the standard Authorization Code grant, but just to be sure here If you search "oauth flows", you'll see a bunch of blogs and resources describing the different grants and flows.  Some of the things you wrote about sound like the legacy Resource Owner Password Credentials flow; if that's what you're trying to do... i suggest you take another look at the Authorization Code flow

* https://nordicapis.com/8-types-of-oauth-flows-and-powers/

* https://oauth.net/2/

* https://auth0.com/docs/api-auth/which-oauth-flow-to-use

* https://medium.com/@darutk/diagrams-and-movies-of-all-the-oauth-2-0-flows-194f3c3ade85
* https://requests-oauthlib.readthedocs.io/en/latest/oauth2_workflow.html

Let me explain a bit of the Authorization Code Grant using my library, because that sounds like what you should be doing (that flow, not my library)...

1. The user visits your Application, and is redirected to the Authority with a payload which contains the application id, a random id, a set of credentials to authorize, what they want, and callback URL that you already configured as valid with the authority.  At the Authority, they will either login or be logged in, and Authorize the grant.

     This Pyramid view is a user being redirected from your application: https://github.com/jvanasco/pyramid_oauthlib_lowlevel/blob/master/pyramid_oauthlib_lowlevel/tests/oauth2_app/views.py#L654-L671

2. At the Authority, they generate a temporary oAuth Grant Token to track the authorization and redirect the user to a predetermined callback page on your application.  There is some oAuth information in the headers of the redirect. 

     This is the user at the Authority view- https://github.com/jvanasco/pyramid_oauthlib_lowlevel/blob/master/pyramid_oauthlib_lowlevel/tests/oauth2_app/views.py#L178-L241

     The GET params on the request to the authority will have a clientId (your app), a redirect url (which must match the one in their database) and a 'state' nonce/session

     The authority allows the Authenticated user to Authorize the credentials request request, create a session on their side, then redirect to your page.

3. Your application processes the oAuth information contained in the headers on the callback page, and then (4) redirects to a success page or the page originally requested.  A library will handle processing all this stuff for you, you should just be pulling the information from validated request via the library's API.

    This is where that happens https://github.com/jvanasco/pyramid_oauthlib_lowlevel/blob/master/pyramid_oauthlib_lowlevel/tests/oauth2_app/views.py#L673-L724

While this is going on, there are some behind the scenes communications between your server and the Authority.  

Within Step3, your application will make a request behind-the-scenes to the authority with the information from the callback. The authority will generate a "Server Bearer Token" on their side, and send you a payload with the "Client Bearer Token", which you save on your side and associate with the user in the session.  At this point, the "Grant Token" should cease to exists, and both the client (your app) and the authority server will now have a stored "BearerToken".

Generally speaking, once someone makes an oAuth authorization for your application to the Identity authority, the authority should pick up a subsequent request and just redirect to the callback and use the existing BearerToken -- instead of asking them to authorize again.  Some authorities work differently though.

The authorized BearerTokens are generally time-limited and provided with two components - an "Access Token" and a "Refresh Token".  The Access Token is a secret string you make all your various requests with and has the expiry time limit; the Refresh Token is a different secret string you use to obtain a new token.  If you control both servers, you may not want to time-limit them.

After you complete the oAuth grant, you can use the stored token to make automated queries against the upstream server's API to update the user's profile information, ensure they are still registered, etc.  The openid connect stuff basically bootstraps a bunch of profile information into the callback payloads.  While that is useful, it's generally a one-time data transfer (unless there is some way to resync data), and having the oAuth token is preferred.

I hope this quick overview makes sense.  

---- 

In terms of the issue you experienced 

There are MANY things that can be happening there.  two possible ones are:

1. if you are already logged into that service and have already authenticated once on their end, they may be redirecting you to their homepage

2. you may be missing some arguments or you supplied some invalid values in your request, and there is no proper error messaging on the server.

I would drop in some `pdb.set_trace()` lines in this console script, and inspect the request objects to see if you have any redirects going on.  

[Mike Orr]

OK, I got it working enough to authenticate the user and get a
userinfo dict, but it doesn't have the all-important roles
information. I may need to add some scopes; I'm asking the server
admin about that. The original problem was I received an incomplete
authorization URL, '/auth' instead of
'auth/realms/REALM/protocol/openid-connect/auth'. Here's my code
(paraphrased) and a bunch more questions:

======
import pprint
import secrets
import requests_oauthlib

# Utilities
def get_oauth2_session(request, state):
redirect_uri = request.route_url("login")
client = request.registry.settings["oauth2.client"]
scope = request.registry.settings["oauth2.scope"] # scope == None.
oauth = requests_oauthlin.OauthSession(
client, redirect_uri=redirect_uri, scope=scope, state=state)
return oauth

# View callables
def home(request):
"""Display 'Login with Keycloak' link."""
auth_url = request.registry.settings["oauth2.url.auth"]
state = secrets.token_urlsafe()
request.sesion["oauth2_state"] = state
oauth = get_oauth2_session(request, state)
authorization_url, state2 = oauth.authorization_url(auth_url)
if state2 != state:
log.error("STATE MISMATCH: %r != %r", state2, state)
return {"authorization_url": authorization_url}

def login(request):
"""Receive redirect from Keycloak server, display userinfo dict."""
secret = request.registry.settings["oauth2.secret"]
token_url = request.registry.settings["oauth2.url.token"]
userinfo_url = request.registry.settings["oauth2.url.userinfo"]
state = request.session{"oauth2_state"]
oauth = get_ouath2_session(request, state)

token = oauth.fetch_token(
token_url, client_secret=secret, authorization_response=request.url)
# Token dict:
# access_token: string
# expires_in: 300
# refresh_expires_in: 600
# refresh_token: string
# token_type: "bearer"
# not-before-policy: 0
# session_state: string
# scope: ["profile", "email"]
# expires_at: float

data = oauth.get(userinfo_url).json()
# Userinfo dict:
# sub: string (a short token)
# email_verified: False
# name: string (full name)
# preferred_username: string (user ID)
# given_name: string (first name)
# family_name: string (last name)
# email: string (email address)

return {"formatted_data": pprint.pformat(data)}
======

Questions from top to bottom:

- Am I matching the state correctly? The 'authorization_url' method
returns the state but the 'fetch_token' and 'get' methods don't so I
don't have anything to compare it against.
- Should I create a new state every time it displays the login link?
- Would this be amenable to Pyramid's CSRF token checking? But the
login request is GET and Pyramid's checking seems to be only for POST
requests.
- Should I delete the state from the session after matching it?
- What should I do if there's a state mismatch? I don't want to give
the user an Internal Server Error, or tell them they're bad when
there's nothing they can do about it.
- What should I do with the token?
- Should I save the token in the session?
- What's the difference between the token keys 'expires_in' and
'refresh_expires_in'?
- What does the token's 'session_state' key mean? Should I care about it?
- What does the userinfo's 'sub' key mean? Should I care about it?
- Should I refresh the token? Where would I do that? In every view
callable? My existing LDAP implementation doesn't have this concept;
it leaves the user logged in until the Pyramid session expires and is
deleted in Redis.
- I'm instantiating the OAuth2Session in each view. Should I save it
somewhere? I can't put it in the Pyramid session because it's not
JSONable. Is it expensive to reinstantiate it like this?


On Tue, May 21, 2019 at 6:45 PM Jonathan Vanasco <jona...@findmeon.com> wrote:
> oAuth Flows and Grants

[Jonathan Vanasco]

Many of the things you mentioned are specific to Keycloak.  I can't answer those.

I can take a deeper look at the oauth details tomorrow.  the following are things that just popped at me.

On Thursday, May 30, 2019 at 8:57:28 PM UTC-4, Mike Orr wrote:

- What should I do if there's a state mismatch? I don't want to give 

the user an Internal Server Error, or tell them they're bad when
there's nothing they can do about it.

 

If there is a state mismatch, something very bad happened and you should not let them login.

- What should I do with the token?
- Should I save the token in the session?

Are they using oAuth to login to your app every time?  If so, you should save it in the database.  If they will authorize via oAuth every time they use the app, you could store it in the session or trash it.

 

- What's the difference between the token keys 'expires_in' and
'refresh_expires_in'?

expires_in the expiry of the access_token.  the auth server will not recognize the access_token after that point.

refresh_expires_in the expiry of the refresh_token.  the auth server will not issue a new access_token in exchange for the refresh_token after that point.

- Should I refresh the token? Where would I do that? In every view
callable? My existing LDAP implementation doesn't have this concept;
it leaves the user logged in until the Pyramid session expires and is
deleted in Redis.

You only refresh the token when it is about to expire.

When a user approves your app, the server gives you two secret tokens.
You use the access token to make all your requests to the server.  That is likely saved in the database and the user's session.  Think of it as a "session_id" for the upstream api.
You keep the refresh token tucked away in the database. You only use that token to ask for new access tokens.  Think of it as an autologin cookie to the upstream api.

- I'm instantiating the OAuth2Session in each view. Should I save it
somewhere? I can't put it in the Pyramid session because it's not
JSONable. Is it expensive to reinstantiate it like this?

 

You should only require/create it on the login and callback.  At the callback, if everything is successful, you can log the user in as a user of your application.

[Mike Orr]

On Thu, May 30, 2019 at 6:25 PM Jonathan Vanasco <jona...@findmeon.com> wrote:
>
> Many of the things you mentioned are specific to Keycloak. I can't answer those.
>
> I can take a deeper look at the oauth details tomorrow. the following are things that just popped at me.
>
> On Thursday, May 30, 2019 at 8:57:28 PM UTC-4, Mike Orr wrote:
>>
>> - What should I do if there's a state mismatch? I don't want to give
>>
>> the user an Internal Server Error, or tell them they're bad when
>> there's nothing they can do about it.
>
>
> If there is a state mismatch, something very bad happened and you should not let them login.

But what should I say? If it's not a real human then I can just return
an Internal Server Error, but if it's a person suffering a
malfunctioning server or network then i want to give a polite message.
and what would be the right thing to say?

>> - What should I do with the token?
>> - Should I save the token in the session?
>
>
> Are they using oAuth to login to your app every time? If so, you should save it in the database. If they will authorize via oAuth every time they use the app, you could store it in the session or trash it.

They'll authorize at the beginning of the Pyramid session, or if the
session expires, or if they quit the browser and restart it. It sounds
like I can store the userinfo data in the session and then I don't
need the token.

> When a user approves your app, the server gives you two secret tokens.
> You use the access token to make all your requests to the server. That is likely saved in the database and the user's session. Think of it as a "session_id" for the upstream api.

I don't need a session ID for the upstream API. There are no remote
resources to access.

[Theron Luhn]

> But what should I say? If it's not a real human then I can just return
an Internal Server Error, but if it's a person suffering a
malfunctioning server or network then i want to give a polite message.
and what would be the right thing to say?

This is a good question.

State mismatch happens more often than I intuitively expected, so you should handle it gracefully.

I give an error message along the lines of “Something went wrong when signing in, please try again” and then a link which starts the whole OAuth flow over again.

— Theron

--
You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com.
To post to this group, send email to pylons-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/CAH9f%3DurU8AmtiHsYE_boOGkk2oeQ__nHEJQS88CKBhNuUYGNbQ%40mail.gmail.com.

[Mike Orr]

On Fri, May 31, 2019 at 9:12 AM Theron Luhn <the...@luhn.com> wrote:
>
> > But what should I say? If it's not a real human then I can just return
> an Internal Server Error, but if it's a person suffering a
> malfunctioning server or network then i want to give a polite message.
> and what would be the right thing to say?
>
> This is a good question.
>
> State mismatch happens more often than I intuitively expected, so you should handle it gracefully.

That's what I'm concerned about because I don't know enough about what
might cause it and who's to blame (hackers, malfunctioning server,
user mistake, sunspots).

When I implemented a CSRF token on the existing login page
(authenticating via LDAP or a local db), users started complaining
about getting Pyramid's default CSRF error and not understanding it
and thinking the site was broken. The problem turned out to be that
the session timeout was 30 minutes (redis.sessions.timeout = 1800) but
users were leaving it at the login page longer than that. Apparently
they open it in the morning or in a tab and then come back sometime
later to log in. So I changed the implementation to treat that as an
invalid form submission, so it reloads the page with the form and an
error message, "Login attempt timed out. Please try again."

This OAuth situation is more confusing because the user didn't cause
the error and has no control over it: it's an internal problem between
my server and the oauth server before the user is even involved.

There are other similar situation such as the LDAP server timing out
or giving an unrecognized response, so I have error messages for
those, and also a novel one. Our LDAP servers sometimes do a socket
reset (hangup) immediately on connection; it happens in bursts for
five minutes or so and then fixes itself. So I catch that and say,
"Authentication server error. Please wait a few minutes and try
again."

> I give an error message along the lines of “Something went wrong when signing in, please try again” and then a link which starts the whole OAuth flow over again.

I'll probably treat it as an unexpected response like above.

[Mike Orr]

To follow up, I got OAuth/Keycloak authentication working with the
following code pattern. The userinfo contains only name/email-related
attributes, not the role attributes and origin directory info I need
to calculate Pyramid pincipals. (Origin directory = where the user is
defined; e.g., an enterprise LDAP directory.)

Our Keycloak admin thinks these are not in the userinfo but in the
access token itself, which is not a random string as I thought but
JWT-encoded JSON. I was able to Base64-decode the token and get what
looks like JSON with a "JWT" key. I'm evaluating 'jwt' and a few other
libraries and seeing if there's a public key I need to decrypt it. The
Base64-decoded file is not fully JSON: the JWT value is binary, and
the file ends without a closing quote and brace. Maybe that's a JWT
format that predates JSON.

My original intention was to calculate the group principals based on
the Keycloak roles and origin directory info, and get rid of my User
record in the database that contains these, or rather convert the User
record to an archive of the user's latest login date and Keycloak
attributes.

But without the Keycloak roles I can't do that, so I'm falling back to
the existing User records for that information. This means I can
authorize users who already have a User record, but if a new Keycloak
user comes in (somebody who's configured in Keycloak to access the
application but doesn't have a User record), I'll either have to not
support them or create a User record with default roles because I
don't know what their roles should be. The project team is deciding
whether to do this and what the default roles should be. The admins
can modify the User records online, but somebody will have to do it
before the user first logs in or soon afterward, because otherwise
they'll have fewer permissions than they should.

Here's the code again from my prototype app:

===
import pprint
import secrets
import requests_oauthlib

# Utilities
def get_oauth2_session(request, state):
redirect_uri = request.route_url("login")
client = request.registry.settings["oauth2.client"]
scope = request.registry.settings["oauth2.scope"] # scope == None.
oauth = requests_oauthlin.OauthSession(
client, redirect_uri=redirect_uri, scope=scope, state=state)
return oauth

# View callables
def home(request):
"""Display 'Login with Keycloak' link."""
auth_url = request.registry.settings["oauth2.url.auth"]
state = secrets.token_urlsafe()
request.sesion["oauth2_state"] = state
oauth = get_oauth2_session(request, state)
authorization_url, state2 = oauth.authorization_url(auth_url)
if state2 != state:
log.error("STATE MISMATCH: %r != %r", state2, state)

requests.session["oauth2_state"] = state

return {"authorization_url": authorization_url}

def login(request):

"""Callback page; receive authn from Keycloak server."""

# TODO: Should delete state in session?
# TODO: Should convert state to Pyramid CSRF token?

### Configuration settings:
# oauth2.client: Client ID registered in Keycloak. String, required.
# oauth2.secret: Client secret registered in Keycloak. String, required.
# oauth2,scope: Oauth2 scopes. Space-delimited string, optional,
default None.
# oauth2.url;auth: Server's authorization URL. String, required.
# oauth2.url.token: Server's fetch token URL. String, required.
# oauth2.url.userinfo: Server's userinfo URL. String, required.
======

In my real application, the "home" page is protected, and the
Unauthorized View displays the "login" page which contains a button to
log in. The callback URL is "/oauth2/login", route "login_oauth2",
view name "login_oauth2". It currently displays the userinfo and has
links to continue to the originally-requested page or the home page.
Ultimately it will redirect to these automatically.

The callback page can only be used once after logging in. If the user
refreshes the page I get an InvalidGrantError saying 'code' is wrong.
I'm catching the error and displaying a page "Error from authorization
server. Please log in again, [login link]".

> - What does the userinfo's 'sub' key mean? Should I care about it?

It's the "subject" of the userinfo, a UUID-like string. This is
supposed to match a "sub" somewhere else in the protocol.

[Mike Orr]

So I can extract the data from my OAuth2 tokens now. My remaining
questions are how to integrate refreshing into a Pyramid application.

First, do I need to? I don't care if the token is refreshed; I'll keep
using the claims stored in the Pyramid session until it expires.But if
I want to contribute to the enterprise's Single Sign-In, do I need to
tell the server the user is still logged into my application so it
doesn't expire the SSO account? Do I do this by refreshing the token?

If I do want to refresh the token, do I do it in a NewRequest subscriber?
The refresh_expire is 60 minutes, so how close to the end should I do
the refresh? If a request comes in 10 minutes before the end, I don't
know whether the next request will be in 1 second or 20 minutes. What
if the request contains POST data? Would I have to save the data in
the session, generate an authorization URL, redirect to the server,
come back through the callback view, redirect back to the original
URL, and extract the POST data in the session. (Where it's no longer
in request.POST.) That sounds like a lot of code overhead.

I may also have to put a session lock in the site; i.e., Javascript
that waits for an idle timeout and puts up a modal dialog, "Do you
want to extend your session?" or "Your session is expired. Click here
to log in again." How would that interact with tokens and refreshing
tokens, since my token processing is in the backend? How could the
Javascript extend the session or have the user log in again without
throwing away a partially-filled-in form? I've gotten user complaints
about that, that the user leaves the page on a form and then come back
later and submits the form and blammo! the server-side session is
deleted so they have to log in again and reenter the form input from
scratch. So I want to avoid that.

--
Mike Orr <slugg...@gmail.com>

[Jonathan Vanasco]

On Friday, June 21, 2019 at 8:43:32 PM UTC-4, Mike Orr wrote:

But if  I want to contribute to the enterprise's Single Sign-In, do I need to
tell the server the user is still logged into my application so it
doesn't expire the SSO account?

That is up to your upstream identity provider.

Your token should be assumed valid until the expiry timestamp. The upstream provider may have it's own protocol where you are supposed to check validity at periodic intervals.

 

Do I do this by refreshing the token?

No.

 

If I do want to refresh the token, do I do it in a NewRequest subscriber?

Generally this is done by a background process.  If you want to do it within a Pyramid request, you could do it in a tween or subscriber.

The refresh_expire is 60 minutes, so how close to the end should I do
the refresh? If a request comes in 10 minutes before the end, I don't
know whether the next request will be in 1 second or 20 minutes.

This is largely an organizational issue on how long you can honor the tokens.  TBH, expiring a refresh in 60 minutes sounds like a bad configuration.  Typically an access token would expire in 60 minutes, but the refresh token would expire in 30 days.  That would mean you would need to automatically refresh the access token behind-the-scenes every 60 minutes.

 

What  if the request contains POST data? Would I have to save the data in
the session, generate an authorization URL, redirect to the server,
come back through the callback view, redirect back to the original
URL, and extract the POST data in the session. (Where it's no longer
in request.POST.) That sounds like a lot of code overhead.

If you need the user to re-authorize because your refresh token expired... you would need to stash the POST data, so that is one way of doing it.  That is a lot of overhead, but should not happen because the refresh token should live longer.

I may also have to put a session lock in the site; i.e., Javascript  

that waits for an idle timeout and puts up a modal dialog, "Do you
want to extend your session?" or "Your session is expired. Click here
to log in again." How would that interact with tokens and refreshing
tokens, since my token processing is in the backend?

If you have csrf field on the form, that csrf token might have changed. You could update it with javascript.
 

How could the  Javascript extend the session or have the user log in again without
throwing away a partially-filled-in form?

Do the reauth in an ajax request or a browser popup.
 

[Bert JW Regeer]

On Jun 24, 2019, at 15:44, Jonathan Vanasco <jona...@findmeon.com> wrote:

On Friday, June 21, 2019 at 8:43:32 PM UTC-4, Mike Orr wrote:

But if  I want to contribute to the enterprise's Single Sign-In, do I need to
tell the server the user is still logged into my application so it
doesn't expire the SSO account?

No. This is not something that the refresh token/access token is concerned with.

That is up to your upstream identity provider.

Your token should be assumed valid until the expiry timestamp. The upstream provider may have it's own protocol where you are supposed to check validity at periodic intervals.

 

Do I do this by refreshing the token?

No.

 

If I do want to refresh the token, do I do it in a NewRequest subscriber?

Generally this is done by a background process.  If you want to do it within a Pyramid request, you could do it in a tween or subscriber.

See my question below... you likely don't need to do any refreshing

The refresh_expire is 60 minutes, so how close to the end should I do
the refresh? If a request comes in 10 minutes before the end, I don't
know whether the next request will be in 1 second or 20 minutes.

This is largely an organizational issue on how long you can honor the tokens.  TBH, expiring a refresh in 60 minutes sounds like a bad configuration.  Typically an access token would expire in 60 minutes, but the refresh token would expire in 30 days.  That would mean you would need to automatically refresh the access token behind-the-scenes every 60 minutes.

Mike, before you try to go through the whole rigamarole of using the refresh token to update the access token, let me ask you an important question:

Are you using the access token after you fetch the initial information about the user from Keycloak? If you NEVER ever make another request to Keycloak as long as the user is logged in to your Pyramid application, then you don't need to worry about refreshing the access token. In which case you can just discard it and the refresh token.

If you ever need another access token you'd basically make the user do the login dance again. At that point keycloak will go "Ah, but the user is already logged in, and already authorized this application" and fast forward back to your application, you get a new access token, you contact Keycloak and then forget about it ever again.

 

What  if the request contains POST data? Would I have to save the data in
the session, generate an authorization URL, redirect to the server,
come back through the callback view, redirect back to the original
URL, and extract the POST data in the session. (Where it's no longer
in request.POST.) That sounds like a lot of code overhead.

If you need the user to re-authorize because your refresh token expired... you would need to stash the POST data, so that is one way of doing it.  That is a lot of overhead, but should not happen because the refresh token should live longer.

This is the case even if the refresh token hasn't expired. If the user's session has expired (for any reason what so ever). It's up to you to decide whether to save the users input or if it is lost while they do the login/auth dance again.

I may also have to put a session lock in the site; i.e., Javascript  

that waits for an idle timeout and puts up a modal dialog, "Do you
want to extend your session?" or "Your session is expired. Click here
to log in again." How would that interact with tokens and refreshing
tokens, since my token processing is in the backend?

If you have csrf field on the form, that csrf token might have changed. You could update it with javascript.
 

How could the  Javascript extend the session or have the user log in again without
throwing away a partially-filled-in form?

Do the reauth in an ajax request or a browser popup.
 

--
You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com.
To post to this group, send email to pylons-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/6eb8c654-073d-428d-b404-2fb22d751092%40googlegroups.com.