Json-rpc (or REST) with JWT token auth in py4web - how?

[Alexei Vinidiktov]

Hello Massimo,

Since web2py is very slow (according to a benchmark I've seen), and py4web is much faster, I'm interested in moving to py4web but I'm not sure how to handle the transition.

My web2py app is used as a service layer communicating with the JavaScript frontend via json-rpc.

It uses web2py JWT tokens for auth (sent as Authorization: Bearer <token> header).

How can I handle the same architecture in py4web?

I think the same question should be valid for REST services requiring authentication. 

Basically, something like this:

myjwt = AuthJWT(auth, secret_key='SecretKey', expiration=3000000)

@cors_allow
@catch303
@myjwt.allows_jwt()
@auth.requires_login()
def call():
    session.forget()
    return service()

@service.jsonrpc2
def create(lesson_id, question, answer, reading, context, weight, examples):
    card_id = db.card.insert(
        lesson_id=lesson_id,
        user_id=auth.user.id,
        question=question,
        answer=answer,
        reading=reading,
        context=context,
        weight=weight,
        examples=examples
    )
    return int(card_id)

How can I get the token, refresh the token, revoke it, how can I use it to authenticate json-rpc requests (or REST requests for that matter)?

[Massimo]

I am going to have to write this example. Will post it within one week.

[Alexei Vinidiktov]

I'm looking forward to it, Massimo.

[Massimo]

I create two distinct mechanism to do this. Please help me test before I make it official. They are in master anyway.

I create a general mechanism in auth to add arbitrary auth.token_plugins and two distinct plugins SimpleTokenPlugin and JwtTokenPlugin.

# SimpleTokenPlugin

This is the recommended plugin for API authentication. 

In common.py enable it:

simple_token_plugin = SimpleTokenPlugin(auth)

auth.token_plugins.append(simple_token_plugin)

In controllers.py create a page to manage your tokens

from py4web import action

from py4web.utils.grid import Grid

from .common import auth, db

@action("tokens/<path:path>")

@action.uses("generic.html", auth.user)

def _(path):

db.auth_simple_token.user_id.default = auth.user_id

grid = Grid(path, db.auth_simple_token.user_id==auth.user_id, create=True, deletable=True)

return dict(grid=grid)

Your API do not need to do anything special, for example,

@action("test_api")

@action.uses(auth.user)

def test_api():

return {"hello": "world"}

You are requiring an auth.user so you can specify the user via the token

curl http://127.0.0.1:8000/test1/test_api -H "Authorization: Bearer {token}"

When the token is provided actions decorated with auth or auth.user will try authenticate the user from the token if provided, and if the user is not in the session.

The token is just a UUID. It can be created, viewed, expired, and deleted via the above "tokens" management interface.

In order to verify the user, if the token is provided, it requires two trivial DB queries.

SimpleTokenPlugin is the recommended API token.

# JwtTokenPlugin

This token is different from the above because it is not a UUID. It actually stores information such as user_id and other info. You can put what you want in the token, but normally you would only put the user.

To enable it, in common.py:

jwt_token_plugin = JwtTokenPlugin(auth)

auth.token_plugins.append(jwt_token_plugin)

In controllers.py create an action that makes a token for you:

@action("make_token")

@action.uses("generic.html", auth.user)

def make_token():

return dict(token=jwt_token_plugin.make(auth.current_user, expiration=utcnow()+datetime.timedelta(days=10)))

From the API prospective it works as the one above or any other authentication method,

@action("test_api")

@action.uses(auth.user)

def test_api():

return {"hello": "world"}

Call the API with:

curl http://127.0.0.1:8000/test1/test_api -H "Authorization: Bearer {token}"

Since this token has its own expiration and has its own expiration (must be a UTC datetime) it cannot be deleted or revoked. On the plus side the server does not require any database access to validate the token and retrieve the user.

# Make you own token validator

You can make your own token token validator, it only neet a get_user() method to retrieve the user from the HTTP request:

from py4web import request

class ImpersonateTokenPlugin:

def get_user(self):

authorization = request.headers.get("Authentication")

if authorization and authorization.startswith("auth_as "):

username = authorization.split()[1].strip()

user = db.auth_user(username==username)

if user:

return user.as_list()

return None

auth.token_plugins.append(ImpersonateTokenPlugin())

and use it with

curl http://127.0.0.1:8000/test1/test_api -H "Authentication: auth_as myusername"

(this is not safe but can be useful for testing)

Notice you can support multiple token plugins at the same time auth.token_plugins is a list and py4web will try them all in order.

Sorry this took a while but it required some thought.

I could use some feedback. Is this clear and easy enough? Is this general enough?

Massimo

[Alexei Vinidiktov]

Thanks a lot, Massimo!

I'm already experimenting with it.

What isn't clear so far, is what should the standard sign up, log in, log out, recover lost password flows look like for a REST or json-rpc based app that doesn't rely on sessions.

The docs say that The auth.enable() step creates and exposes the following RESTful APIs:

• {appname}/auth/api/register (POST)

• {appname}/auth/api/login (POST)

• {appname}/auth/api/request_reset_password (POST)

• {appname}/auth/api/reset_password (POST)

• {appname}/auth/api/verify_email (GET, POST)

• {appname}/auth/api/logout (GET, POST) (+)

• {appname}/auth/api/profile (GET, POST) (+)

• {appname}/auth/api/change_password (POST) (+)

• {appname}/auth/api/change_email (POST) (+)

I don't fully understand, how these APIs are supposed to work.

I've tried the login endpoint (haven't found any examples, had to guess how to create a proper request) and I can see that the response returns set-cookie headers. One of the cookies is the session cookie. So I deduce that the login endpoint sets a session in the browser.

But how can this be used from a mobile app, for example, where there are no cookies? The response doesn't contain an auth token.

Here's a response from the login endpoint:

{
  "code": 200,
  "status": "success",
  "user": {
    "email": "alexei@***.ru",
    "first_name": "Alexei",
    "id": 2,
    "last_name": "Vinidiktov",
    "username": "alexei"
  }
}

How do the rest of the APIs work? Are there examples of requests, of integrating the APIs into an app?

And most importantly, how can they be used with auth tokens?
If they can't, do we have to create a separate set of endpoints for user and session management for token based flows?

[Massimo DiPierro]

You auth api are best explained with the examples in the tests  https://github.com/web2py/py4web/blob/master/tests/test_auth.py You would only use the API to login if you want to get a session cookie and authenticate using that. There is no api to use the username:password to obtain an authentication token. Is it assumed a user logins and uses that UI to get the token. Are you trying to automate this step? Don't you want a human to manually register and obtain the token? What  is the desired workflow?

--
You received this message because you are subscribed to the Google Groups "py4web" group.
To unsubscribe from this group and stop receiving emails from it, send an email to py4web+un...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/py4web/5b049afc-bb54-4a6e-b36b-a76dc6d29863n%40googlegroups.com.

[Alexei Vinidiktov]

Let's say my frontend is hosted on a different domain than the backend. It is written in Vuejs or React and it communicates with the backend via json-rpc (or REST).

I need the user to be able to register, login, etc. via this frontend and then send requests that require the user to be logged in (authenticated).

Would the session cookie work in this scenario? 

If it would, is it possible to obtain a token via a json-rpc or REST request (not via the tokens UI)? 

Would I even need to do that (use tokens)? Maybe session cookies will be enough for all my subsequent REST (json-rpc) requests that require auth?

Or consider another scenario.

I have a desktop or mobile app that communicates with the backend via json-rpc (or REST). 

I need my app to be able to register, log the user in, etc. - the same as in the first scenario.

Would session based auth work?
If it would, what's the point of having tokens? Maybe I can just use cookies?

In my web2py based current app, I use jwt tokens provided by the framework. 

I'm beginning to realize that maybe I asked the wrong question. What if I don't need tokens at all?

It's just that all the RESTful APIs that I've seen only support token based flows.

I thought the token-based approach is more suitable for APIs since it's stateless and works better with mobile/frontend clients.

[Massimo DiPierro]

In my experience if you need api for a frontend, like vue or react, then you do not need tokens because you have sessions.

If you need api access for third party service access you need a token. This token should be requested and managed from the ui.

One can make things more complicated and definitively I have seen it but I have not found a good reason for it.

In py4web with auth.uses(auth.user) you automatically support both, session and token, so you do not need two create two sets of apis.

To view this discussion visit https://groups.google.com/d/msgid/py4web/f43117c5-03fc-49a1-bc19-ead41d4ad517n%40googlegroups.com.

[Alexei Vinidiktov]

I will try to rethink the whole approach to authentication in my app.

The question remains: does the proposed tokens API expose methods for programmatically managing (creating, expiring, deleting) tokens (as opposed to using a Grid UI)? 

[Dave S]

On Tuesday, January 14, 2025 at 12:43:22 AM UTC-8 Alexei Vinidiktov wrote:

I will try to rethink the whole approach to authentication in my app.

The question remains: does the proposed tokens API expose methods for programmatically managing (creating, expiring, deleting) tokens (as opposed to using a Grid UI)? 

As I read it, Massimo's JWT Token example does not use a Grid UI.  I suspect that the .make() function is also in the Simple Token API.  And if that is there, then a destructor should also be there.  What else do you think is needed?

Also, I think that Massimo is suggesting that when your user logs in to the mobile app, the mobile app then has given to your front end server what is needed to request a token from your backend server.  It is similar to how OAuth tokens work, ISTM, but with the authorization server not directly involved in the backend.  

If you are using OAuth, then your front end server could choose to use the OAuth token instead of username as the key to the backend server.  I don't know if that has an advantage;  I've picked a bit about OAuth, mainly from IRL presentations, but I haven't actually used it or and Directory Services type login.

Oh, and you could also choose to have the front end get a token based on its identity, as long as you only need the user to log in to front end.  Seems like that's vaguely similar to how Google's "authorized application" tokens work.

I admit I'm just commenting from the peanut gallery, but it looks to me like Massimo put a lot of thought into making this simple for the application writer in a way that can make it simple for the user.

/dps

[untrimmed thread below]

[Massimo]

OAuth and API tokens are two different things for different purposes. Auth tockes exist for a transient short login period and they are fully specified by the OAuth protocol for SSO. py4web supports those for Facebook, Google, GitHub etc. They simply establish who the user his using a third party SSO.

An API token is given to an authenticated user (already known to the py4web app) so that it can use it to access an API that does not understand sessions, in an automated manner, at some later time. They can be simple UUID or JWT. With the proposed code we support both. None of them is for use with a web UI. Web UIs do not need them because they already know the user from the session cookies.

UUID tokens can be managed (change expiration, deleted). We accomplish this with a Grid as an example but you can make your own UI. A token is just a record in the corresponding database table. The table stores the corresponding info: uuid, expiration, owner, creator, modification date, etc.

JWT tokens, by definition, can only be created. We accomplish this with the provided make() function. They cannot be deleted. You cannot change the expiration. This is because, by definition, they store this information in the token itself which is given to the user. The only way to expire a JWT token is by changing the secret used to digitally sign it. py4web uses a always associates a secret with every app automatically and uses this one but, if you look at the code, you can use your own secret. But if you change it you invalidate all the tokens for all the users. I do not recommend using JWT tokens every.

Massimo

[Massimo]

I added this to the docs and in version 1.20250125.1

[Alexei Vinidiktov]

Hello, Massimo, Dave.

I haven't left the discussion.
In the coming couple of weeks I'll start porting my app from web2py to py4web.

I'll first try to use sessions for authenticating my json-rpc APIs. 

If sessions work fine for my use case I'll probably keep using them instead of auth tokens.

But I think I will still need auth tokens down the road so I'll be experimenting with them as well.

As soon as I have something useful to say on the topic of auth tokens, I'll get back to the discussion.

At least I'll let you know how porting goes and if sessions turn out to be a good fit for my use case.

Thanks for your input, Dave, and for your work, Massimo.