Migration from Flask - options?

[p...@thirdfloor.com.au]

Hi. 

We have a non-trivial Flask app that I'm in the process of moving chunks of to Pyramid and would like to get some input on options for how we run two systems until the whole lots is migrated.

The data layer is agnostic with regards web framework (using ZODB) so that there are no problems having two systems concurrently access it. However I'm struggling with the best way to go with auth across both. Conceptually the simplest would be to have a auth cookie that is valid in both, it could be set to only be created in one and honoured in the other. A call back from one system to the other to verify a cookie is another way but that just increases moving parts.

Has anyone done this before or have any other ideas?

Thanks for any input.

Peter W.

[Michael Merickel]

Are you trying to host the apps in the same process? Where do you want certain shared parts to live as you migrate? In Pyramid code? In Flask code? Agnostic?

Option 1: If you want Pyramid to be able to use any Flask code then you'll have to likely setup Flask's threadlocal variables / request object inside the Pyramid request so that you can invoke Flask APIs.

Option 2: If you want Flask to be able to use Pyramid code as you move it into Pyramid then you'll have to setup Pyramid's threadlocal variables / request object so that you can invoke Pyramid APIs.

Option 3: If you want it agnostic then both Flask and Pyramid code need to pass their relevant data (path, headers, etc) into a function that doesn't care where it came from.

You can do any of the things I mentioned above - I'm not sure what's the best for your situation. If you move your auth into Pyramid then you're basically talking about giving Flask a way to invoke `pyramid_request.has_permission(...)` which falls into Option 2 and would likely be best going forward if you're trying to move code into Pyramid.

If you just want the cookie to be parseable in both frameworks, and write most of the security code separately in each, that should be quite doable as well because in Pyramid you can just define your own AuthenticationPolicy which could probably reuse a lot of code to parse/validate the cookie. Worst case scenario you can always write an AuthenticationPolicy that just hits a trusted REST API or something in the Flask code to parse/validate it, and then you just have Flask do the work there.

If you're running the apps in the same process then they can potentially call each other pretty easily. For example with WebOb it has `request.get_response(app)` in which you construct a WebOb request object, and it can talk directly to another WSGI app (Flask, etc) and get back a WebOb response. This would automatically handle all the threadlocal stuff I mentioned above at least between Flask/Pyramid - no promises for whatever you're doing with other globals / threadlocals from other frameworks you're using for ZODB etc.

Anyway without more details I can only focus on these high level options but I hope it helps. There are certain advantages of them both being Python WSGI apps in that you can actually run them together in the same process, and invoke code from each other without hitting a network. You can setup something like rutter (path-based dispatch between multiple WSGI apps) or pyramid.wsgi.wsgiapp2 (Pyramid can wrap other WSGI apps) to handle what you want in Pyramid and dispatch the rest to Flask.

- Michael

--
You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/124cda96-4016-4049-b591-bb1fa25da663n%40googlegroups.com.

[Peter Wilkinson]

Thanks Michael, lots to think about there!

I’m hoping to have the code run in separate processes, the data layer has been extracted and any code that touches that moved into services code. Having separate processes turns out to be much cleaner as a migration since any code that does “the wrong thing” in touching data with any lingering Flask-isms blows up straight away. 

It sounds like a custom AuthenticationPolicy might be the way to go. I could bring in the Flask cookie handing code to parse it’s cookie on the Pyramid side, leaving login and the setting of the cookie as the last part of the migration. 

As an aside, would you recommend starting with 1.10 or the current dev state? It looks like it’s very close to being ready as the next version. 

Peter W.

On 30 Aug 2020, at 7:34 am, Peter Wilkinson <p...@thirdfloor.com.au> wrote:



To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/2A72C370-0A31-4086-B70D-A36441D83E4F%40gmail.com.

[Mike Orr]

On Fri, Aug 28, 2020 at 7:27 PM p...@thirdfloor.com.au
<p...@thirdfloor.com.au> wrote:
>
> Hi.
>
> We have a non-trivial Flask app that I'm in the process of moving chunks of to Pyramid and would like to get some input on options for how we run two systems until the whole lots is migrated.
>
> The data layer is agnostic with regards web framework (using ZODB) so that there are no problems having two systems concurrently access it. However I'm struggling with the best way to go with auth across both. Conceptually the simplest would be to have a auth cookie that is valid in both, it could be set to only be created in one and honoured in the other. A call back from one system to the other to verify a cookie is another way but that just increases moving parts.

I had a pair of Pyramid applications that shared a session cookie. One
was at e.g., 'foo.example.com' and the other at 'm.foo.example.com'. I
set the cookie domain attribute to span both. (I don't have the code
at hand but maybe '*.foo.example.com'.) Both applications had the same
codebase, just different configuration options to dispay a desktop or
mobile layout. They both shared the same Redis session backend. I
wasn't doing authentication, just saved search criteria.

I had to change the configuration a bit for the staging servers
because I couldn't replicate the subdomain relationship; the domains
were like 'foo.staging.example.com' and 'm-foo.staging.example.com'. I
didn't want to wildcard at the 'staging' level because that would
bring in lots of enterprise sites I didn't know what they are. But I
couldn't find a configuration that would span just 'foo' and 'm-foo'.
So instead I put them on different ports in the same domain,
'staging.foo.example.com:444' and 'staging.foo.example.com:7446'. The
default cookie configuration successfully spanned between the ports.

Another application has OAuth2 authentication. It doesn't have a
companion but its structure may be useful. I may have used
'pyramid_oauthlib'. When a successful login comes back from the auth
server it returns a dict of claims (email, name, server groups, etc).
I parse the claims into principles and store that in the session along
with some metadata. Subsequent requests use the session data for
authorization/metadata, so it's the same pattern as the one above and
could share the cookie with a companion application if I had one. A
disadvantage of this approach is that after login, the "logged-in"
state in the application is independent of the logged-in state in the
auth server so they will expire at different times, and if the account
is modified you won't see the changes until they log in again. If this
is unacceptable you can save the server's auth token and revalidate it
on every request. Then both applications would have to do this.

[Jonathan Vanasco]

I've done a handful of side-by-side migrations or deployments.

I think you already identified the generally best approach IMHO:

    >  Conceptually the simplest would be to have a auth cookie that is valid in both, it could be set to only be created in one and honoured in the other.

However I recommend it with one caveat -- I would approach this functionality as if it were it's own authentication micro-service.  It can live in one of the two processes, it doesn't need a third, but I would structure the UX, tests and development as if it were totally independent -- so the cookie value would just contain login state the core account info needed for auth and permissions.  If either process needs to store other data in a cookie, use another cookie.

Reading Pyramid cookies in Flask is pretty simple, Pyramid sessions just automate loading webob cookies with defaults

* https://github.com/Pylons/pyramid/blob/master/src/pyramid/session.py
* https://github.com/Pylons/webob/blob/master/src/webob/cookies.py


In terms of Pyramid versions, if you need Python2 support - 1.10 is your only option.  Otherwise, just pay attention to the deprecations on Pyramid2 and you should be able to transition from 1.10 to 2 very easily if you don't want to run the pre-release.  Michael Merickel is the expert on this, but I think he and his team have done a great job in planning the 2.0 changes by offering a lot of "forward compatibility' options in 1.10 to prepare us for switching to 2.0.  I could be wrong, but I think almost every one of the 2.0 changes can be elected in 1.10.  The only difference I know of is the drastic change to authentication/authorization (which may be worth using the pre-release). 

[Michael Merickel]

The 2.0 security system is pretty backward compatible so I'd recommend just starting with 1.x but I might suggest using the master version of pyramid-cookiecutter-starter as it has significant improvements to the default test fixtures for new projects.

With respect to 2.0, sorry everyone but I've been dragging my feet on an issue in the new security api so it isn't out yet! Mailing list activity is a great motivator though!

- Michael

--
You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/b0d9e559-d070-4522-82ef-217af2c63de2o%40googlegroups.com.

[Peter Wilkinson]

Thanks everyone for the great responses!

I’m planning on having a go at making an authentication policy along the lines of SessionAuthenticationPolicy and have the Flask cookie signing bits imported just to handle that.

The apps are definitely going to dealt with as seperate, the migration is a good chance to sanitise the structure from years of accumulated cruft.

This is a 3.8+ only app so I could start with 2.0 - are there any areas that are likely to change in it? Everything that I’ve read in the changes looks nice and simple to deal with.

Another thing… I’ve jumped onto the IRC channel a few times and not seen any traffic, potentially timezone differences being on the other side of the world but is there much activity there?

Peter W.

To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/7146D524-E61C-4B70-A78A-505F42059524%40gmail.com.

[Theron Luhn]

With respect to 2.0, sorry everyone but I've been dragging my feet on an issue in the new security api so it isn't out yet!

Hey Michael, anything I can do to help out with the security API?

—Theron

To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/7146D524-E61C-4B70-A78A-505F42059524%40gmail.com.