TG2 Identity ("Authority")

[Kevin H]

OK, so as some of you know I've started work on a new package to
replace identity for TG2. I'm calling this package "authority",
though if you have a different suggestion for the name, please feel
free to let me know.

I managed to do some work on this over the weekend (I probably should
have spent the time on other things...but coding for TG is so much
more fun! :)

Here's what I've got so far:
- a basic middleware piece that intercepts 401s from the main app and
performs authentication related tasks
- a "scheme" object that represents the way users are authenticated/
authorized (i.e. HTTPBasic, HTTPDigest, FormBased). Users should be
able to build a custom scheme without too much trouble.
- a "require" decorator that will eventually behave similarly to the
TG1 require decorator. It checks the beaker session for auth info
stored by the middleware, and either passes the request through to the
main app or raises a 401 or 403. Right now this just checks that the
user is logged in, but will eventually be able to check against a
"condition" like TG1 identity does.
- storage objects (not yet done) which will represent how the user
credentials, group/permission info are stored
- a user object which gets stored in a beaker session that stores auth-
related info. custom user object should be possible in future.

Once all this infrastructure is in place it should be relatively
simple to add new user objects, scheme objects, and storage objects
which can be mixed and matched to create whatever kind of auth system
is needed. My intention is to have a "TG2" scheme which will closely
match the identity API, and I'm hoping that you'll be able to do
something like:

import authority as identity

in your Tg2 apps to make porting TG1 apps simple

As of this moment, authority is dependent on webob, beaker (which will
probably not change) and pylons (which might be changeable in the
future). My goal is to eventually make it framework agnostic, but
that may require a small adapter piece for each framework.

Right now basic HTTP auth sort of works (doesn't actually check a back
end for the password info, just checks that username==password), but I
haven't moved forward much on anything else at the moment.

I'll try to get the code up someplace in an SVN repos over the next
couple of days, so that others can take a look.

Please send your thoughts and opinions!

Kevin Horn

[iain duncan]

> I'll try to get the code up someplace in an SVN repos over the next
> couple of days, so that others can take a look.
>
> Please send your thoughts and opinions!

This is much needed, go Kevin!

My immediate thought is that when making the login methods for identity,
we could make that code a lot easier to follow. I have to admit, I found
figuring out how to get login/logout doing what I want in tg1 identity
much more troublesome than I expected.

That's a bit that could have a clearer api or maybe just way more
comments in the code, or something!

Thanks
Iain

[Daniel Fetchinson]

> My intention is to have a "TG2" scheme which will closely
> match the identity API, and I'm hoping that you'll be able to do
> something like:
>
> import authority as identity
>
> in your Tg2 apps to make porting TG1 apps simple

This would be really great, I mean if the API would not change too
much relative to tg1.

> My goal is to eventually make it framework agnostic,

Any reason why you would want to do that? (apart from "let's be as
general as possible")
Being general is great but if all pieces of a software package are
uber-general with duct tape holding everything together then sometimes
the amount of duct tape is more than the actual stuff that needs to be
held together. Or you have a well defined target audience apart from
tg users?

Cheers,
Daniel

[Kevin Horn]

Well, obviously my main priority is to support TG2. That will take precedence over everything else.  I would also like to support pylons, which shouldn't be too hard to do, since TG2 is built on top of Pylons (and since right now, nothing depends on TG code, only on Pylons code)

After that, it's mostly the "let's be as general as possible" reason, though I think that's appropriate.  I don't intend to go outside the WSGI realm with this, though.  From what I have so far, it looks like there's only one place in the code that there is a hard dependency on pylons, and I think that can be generalized pretty simply.

We'll see how it turns out.  It may not be feasible to try and generalize this to all WSGI applications, in which case, too bad, we won't do it.  But I don't think that it will add too much in this particular case.

Kevin Horn

[Mark Ramm]

> We'll see how it turns out. It may not be feasible to try and generalize
> this to all WSGI applications, in which case, too bad, we won't do it. But
> I don't think that it will add too much in this particular case.

Sounds like a good approach to me. Ideally we want to attract
contributors outside of the TG2 world, and we want to make this
something that people want to contribute too (so that we can get
OpenID support, ect). This is more likely to happen if we make it
work with the tools they allready use. But, TurboGears defines one
of the most popular toolsets in the python community, so even if it's
TG specific it should attract people.

Here's my basic thoughts on the subject off framework independent
components: The trade-off between generality and framework
specificness should be handled on a case by case basis, depending on
who might end up wanting to use it, how much work it is to make it
framework independent, and how willing the authors of the component
are to work to support non TG2 users.

So, we definitely don't want our own ORM, or template system, and we
probably don't need our own authentication middleware, but I'd doubt
that a user-registration module would be worth making framework
neutral, since it will have to have templates, depend directly on the
ORM, and it would be very hard indeed to abstract all of that away.
And I doubt that the component would really grown non TG2 users. So,
it's all something that we have to deal with for each component we
create.

--Mark Ramm

[percious]

This is really awesome work Kevin, and very timely. Are you
separating this work out as a googlecode project or is it part of the
tg svn?

Also, is it possible to run this if TG1 was running on WSGI? In terms
of support, it would be great if it was backwards compatible.

I am looking forward to using this on Grok.

-chris

[percious]

> So, we definitely don't want our own ORM, or template system, and we
> probably don't need our own authentication middleware, but I'd doubt
> that a user-registration module would be worth making framework
> neutral, since it will have to have templates, depend directly on the
> ORM, and it would be very hard indeed to abstract all of that away.
> And I doubt that the component would really grown non TG2 users. So,
> it's all something that we have to deal with for each component we
> create.

Hmm, this sounds very similar to a problem I had to solve in
DBSprockets to make it work the way I wanted. Maybe I should release
a toolset which makes it possible to abstract concepts like Provider
(ORM), Widgets, Validation, etc.

I'd love to see Authority running on ZODB.

-chris

[Kevin Horn]

On Thu, Feb 14, 2008 at 4:40 PM, percious <ch...@percious.com> wrote:

This is really awesome work Kevin, and very timely.  Are you
separating this work out as a googlecode project or is it part of the
tg svn?

Also, is it possible to run this if TG1 was running on WSGI?  In terms
of support, it would be great if it was backwards compatible.

I am looking forward to using this on Grok.

-chris

Yeah, I was trying to work on Docudo (the next generation) as a sort of testbed app for TG2, and I just couldn't get past the lack of identity in TG2.  I looked at some of the other WSGI auth toolkits, but they just didn't satisfy.  So I waited for someone to build TG2 identity...and they didn't, so eventually I just picked up the ticket and ran. (though it feels like I'm running through molasses the last couple of weeks..I can't wait to get this code out of my head!)

Hmmm...I hadn't considered using it with TG1. And I haven't yet run a TG1 app under WSGI.
So I guess the answer to that one is...maybe?  The plan is to make it depend only on beaker and WebOb, but early versions might depend on pylons (mine does right now, but only in one tiny line of code)


Big things to do:
- hook into the paste config system...probably will shamelessly swipe ideas from beaker (beaker's source is beautiful)
- figure out all the setuptools magic I want to happen, though I guess it's not strictly necessary
- fill out the infrastructure a bit to support what I want to do for the "TG2 Scheme"
- write the "TG2 scheme"
- write other schemes (httpbasic kinda works now, need httpdigest, OpenID, etc.)
- write tests...not entirely sure how to go about testing this thing
- write docs


The code right now is pretty much on my desktop PC...getting it up on an SVN repos is one of the tasks I hope to accomplish this weekend.  Not currently part of TG svn.  I think it probably shouldn't be, but if others have opinions, let's hear em.

Kevin Horn

[Mark Ramm]

> The code right now is pretty much on my desktop PC...getting it up on an SVN
> repos is one of the tasks I hope to accomplish this weekend. Not currently
> part of TG svn. I think it probably shouldn't be, but if others have
> opinions, let's hear em.

Authority it self is going to need to have multiple providers
(obviously) and I think it will likely be usable outside of TG itself
(perhaps in Grok as Chris has suggested), so it should probably get
it's own repo. Now that apidya does google code wiki style output
for docs, I think google code is an pretty reasonable choice these
days, and it's certainly easy to get up and running.

--Mark

[iain duncan]

> Big things to do:
> - hook into the paste config system...probably will shamelessly swipe
> ideas from beaker (beaker's source is beautiful)
> - figure out all the setuptools magic I want to happen, though I guess
> it's not strictly necessary
> - fill out the infrastructure a bit to support what I want to do for
> the "TG2 Scheme"
> - write the "TG2 scheme"
> - write other schemes (httpbasic kinda works now, need httpdigest,
> OpenID, etc.)
> - write tests...not entirely sure how to go about testing this thing

OMG, a question I can answer! ;-)

Use Twill and import the twill commands into a nose test suite. Running
functional tests with twill through nose is really easy and it does all
your cookie stashing for you and let's you pretend to be different
agents and stuff. Plus Titus and Gregio (sp?) are really helpful on the
twill and testing python mailing lists. And there would be no reason you
couldn't combine it with checks on what is in the raw db at the same
time ( though I haven't quite figured out the best way to do this yet,
baby steps for me).

It is really easy to run long progressions of login and outs. I could
actually help there, I've been writing twill tests of member sign up and
status changing and editing ad nauseum for the last four days.

Iain

[Kevin Horn]

A quick update...

I'm afraid both of the projects that I hoped would be finished on Friday weren't so i didn't get anything really accomplished this past weekend.

With any luck, I'll have some time later in the week to get an SVN set up and get some more actual code done.

If nothing else, I'll try to have the SVN up and available before the sprint next weekend.

Kevin Horn

[Mark Ramm]

I got permission from work, so I'll throw some authorization code up
on a SVN repo tonight that provides several of the necessary features.

It sounds like Kevin's plan is the way forward, but perhaps this code
(which was written for a specific project) will provide some useful
stuff that we can hack out.

BTW, the code I'm posting used Beaker middleware and provided a WSGI
authentication wrapper around a TG1 application. It's been used
against TG2 as well, and I'll try to get a bit of related
authorization code for TG2 up as well.

Hopefully this will be of some use to the Authority project, if only
as a reference.

BTW, if you want me to Kevin, I can set up a google code project for
Authority tonight as the same time I post this stuff.

--Mark

--
Mark Ramm-Christensen
email: mark at compoundthinking dot com
blog: www.compoundthinking.com/blog

[Kevin Horn]

OK, I've created a Google Code project and Google Group for Authority.

here:
http://code.google.com/p/authority/

and here:
http://groups.google.com/group/authority-wsgi

I've uploaded my code (what there is) to the SVN repository.

Not much there yet, whole chunks missing, but at least it's finally up someplace.

BTW, Mark, did you have a chance to put your auth code up someplace?

Kevin Horn

[Mark Ramm]

> BTW, Mark, did you have a chance to put your auth code up someplace?

Not yet, I had more trouble with the Routes stuff than expected, and
then I found a few places which needed to be cleaned up on wsgiauth
before I could post it. I do expect to have it up somewhere this
afternoon, I'll let you know as soon as it's there.

BTW, do you think you could add me as a project member on authority?

--Mark

[Kevin Horn]

Will do.

Kevin Horn

[Devin Torres]

I can't believe we're having this discussion on this mailing list. Use
AuthKit. If you're having problems understanding it contact me and
I'll try to help you, but we don't need to be reimplementing the wheel
here. Pylons has already chosen the best of breed packages for you, if
you have a problem with one of them then improve it.You said "it needs
a bit of continuing love to make it into all it could be." Well? Give
it that bit of continuing love!

I just don't understand why we're trying to replace Pylons. We chose
it, now build on top of it and improve it.

-Devin

[percious]

Devin,

And how would you go about telling users of TG1.x that they have to
switch over to a completely different way of doing identity?

I think Kevin has some great ideas which will help the upgrade path
and further TG2.

If we all want to "just use what already exists" why don't we all use
Zope?

-chris

On Feb 22, 10:10 am, "Devin Torres" <devin.tor...@gmail.com> wrote:
> I can't believe we're having this discussion on this mailing list. Use
> AuthKit. If you're having problems understanding it contact me and
> I'll try to help you, but we don't need to be reimplementing the wheel
> here. Pylons has already chosen the best of breed packages for you, if
> you have a problem with one of them then improve it.You said "it needs
> a bit of continuing love to make it into all it could be." Well? Give
> it that bit of continuing love!
>
> I just don't understand why we're trying to replace Pylons. We chose
> it, now build on top of it and improve it.
>
> -Devin
>

> On Thu, Feb 21, 2008 at 4:00 PM, Kevin Horn <kevin.h...@gmail.com> wrote:
>
> > On Thu, Feb 21, 2008 at 3:13 PM, Mark Ramm <mark.mchristen...@gmail.com>

[Mark Ramm]

I'm pretty sure that a largeish number of people on the pylons list
have expressed some dissatisfaction with AuthKit as a solution. I've
seen this on the mailing list, and the IRC channel, and I've
personally had a couple of people I respect in the Python and Pylons
community suggest that we not use AuthKit.

Even with all that, I suggested that we try to make it work, and after
a couple of people did that, we ended up deciding that while we could
make it work, it would require a lot of work to get it into the kind
of shape we need it to be.

We really want an Auth solution that could be embraced by the Pylons
and TG communities, and could be accessable to the greater WSGI world,
and I have my doubts that AuthKit will become that solution. There's
already too much frustration and dissatisfaction, and very little
activity on the development side to answer any of those issues.

If however, we've been misinformed and AuthKit has a healthy and
vibrant community which is willing to help us, and which is willing to
take on some new developers who will fix things and turn it into a
viable solution for us, I'm sure the TG dev community is wiling to
re-open the issue, and I for one know I could be convinced to change
my mind. But, I don't think it's likely unless the AuthKit project
starts becoming more open, and more active.

--Mark Ramm

On Fri, Feb 22, 2008 at 10:10 AM, Devin Torres <devin....@gmail.com> wrote:
>
> I can't believe we're having this discussion on this mailing list. Use
> AuthKit. If you're having problems understanding it contact me and
> I'll try to help you, but we don't need to be reimplementing the wheel

> here. Pylons has already chosen the best of breedI packages for you, if

> you have a problem with one of them then improve it.You said "it needs
> a bit of continuing love to make it into all it could be." Well? Give
> it that bit of continuing love!
>
> I just don't understand why we're trying to replace Pylons. We chose
> it, now build on top of it and improve it.
>
> -Devin
>
>
>
> On Thu, Feb 21, 2008 at 4:00 PM, Kevin Horn <kevin...@gmail.com> wrote:
> >
> >
> > On Thu, Feb 21, 2008 at 3:13 PM, Mark Ramm <mark.mch...@gmail.com>
> > wrote:
> > >
> > >
> > > > BTW, Mark, did you have a chance to put your auth code up someplace?
> > >
> > > Not yet, I had more trouble with the Routes stuff than expected, and
> > > then I found a few places which needed to be cleaned up on wsgiauth
> > > before I could post it. I do expect to have it up somewhere this
> > > afternoon, I'll let you know as soon as it's there.
> > >
> > > BTW, do you think you could add me as a project member on authority?
> > >
> > > --Mark
> > >
> > >
> > >
> >
> > Will do.
> >
> > Kevin Horn
> >
> >
> >
> > >
> >
>
> >
>

--

[Kevin Horn]

On Fri, Feb 22, 2008 at 11:10 AM, Devin Torres <devin....@gmail.com> wrote:

I can't believe we're having this discussion on this mailing list. Use
AuthKit. If you're having problems understanding it contact me and
I'll try to help you, but we don't need to be reimplementing the wheel
here. Pylons has already chosen the best of breed packages for you, if
you have a problem with one of them then improve it.You said "it needs
a bit of continuing love to make it into all it could be." Well? Give
it that bit of continuing love!

I just don't understand why we're trying to replace Pylons. We chose
it, now build on top of it and improve it.

-Devin

Personally, my problem with AuthKit was not that it was hard to understand, but that:
1) It couldn't do some things I wanted to do
2) It would have been more work to retrofit those features than to "roll my own"...and the result would not have looked anything like AuthKit.

So I decided to create a library that would handle what I wanted.  When the topic came up on the TG-trunk mailing list, I mentioned my thoughts, people seemed interested, and so now I'm working on something that _may_ become the default for TG2.

AuthKit may be the "best of breed", but for me, it isn't good enough.  I don't mean it's bad, just that it isn't what I want in an auth framework.

Of course, even if it does, nothing stops people from using AuthKit if it meets their needs.  It should be as simple as some config file changes and changing a line or two in middleware.py

Kevin Horn

[Devin Torres]

Well let me know what these features are that everyone wants.

I'm also not sure how TG1.x accomplished authentication (wasn't it just
forward authentication?), so if anyone could enlighten me on how TG1.x
authentication was performed I could probably provide a counter-example.

-Devin

[percious]

In short, identity is accomplished with a decorator and a provider to
the database. Here is some documentaion:

http://docs.turbogears.org/1.0/UsingIdentity?action=show&redirect=1.0%2FIdentityDecorator

So, how would you like to contribute at the sprints tomorrow? I think
Auth is going to be taken care of by Kevin.

-chris

On Feb 22, 3:33 pm, Devin Torres <devin.tor...@gmail.com> wrote:
> On Friday 22 February 2008 15:58:04 Kevin Horn wrote:
>

> > On Fri, Feb 22, 2008 at 11:10 AM, Devin Torres <devin.tor...@gmail.com>

[Kevin Horn]

The particular itch I wanted to scratch was having users with (in TG1 terminology) multiple groups and multiple permissions (or in authkit terminology, multiple groups and multiple roles).  AuthKit specifically disallows a user from having more than one group.  I suppose I could have hacked around this (maybe used roles for both groups _and_ permissions), but it seemed more trouble than it was worth.

I also like to assign permissions to my groups, and then have the users belong to them.  This might be possible with AuthKit, but I couldn't find any info on it (that was a while ago though).

Also, AuthKit does not (AFAIK) have an analogue to the TG1 Identity "Any" condition, which is not often strictly necessary, but is really handy to have.

There are a number of other "features" that have been mentioned on the TG and Pylons mailing lists that people would like to have, many of which AuthKit can do (as can Identity), but some of which cannot be done (or cannot easily be done) by either.

It would also be pretty difficult (IMO) to give TG1 users an easy upgrade path when/if they decide to port their applications to TG2 if we used AuthKit.

AuthKit is very flexible, but I think I/We can create something even _more_ flexible, and still make it easier to use for the average TG/Pylons user (many of which are new to Python web development).

As for how TG1 handled authentication, it was more like AuthKit's "form and cookie" than its "forward", though the line is kinda blurry since Identity really was part of the application (sort of), rather than a wrapper.

Kevin Horn

[Felix Schwarz]

Devin Torres wrote:
> I can't believe we're having this discussion on this mailing list. Use
> AuthKit.

While I did not look into TG2 (yet), I need a more flexible auth* solution
(compared to identity) for a mission-critical TG1-based project.

I tried AuthKit but I was not able to get AuthKit running with
TG1/CherryPy: Just sending a request and get a response (no auth* related
restrictions) but AuthKit sitting in between as a middleware. Maybe this is
a problem with CherryPy 2.x but I can't exchange that one.

Therefore I'm interested in another auth solution, independently if it is
used in TG2 (although that would be nice). I will make sure that a new
auth* solution will be usable in TG1 which will smooth the upgrade path for
some users.

fs

PS: I will do some coding on this during the sprint but my time is somewhat
limited today.

[Paul Johnston]

Hi,

> While I did not look into TG2 (yet), I need a more flexible auth*
> solution (compared to identity) for a mission-critical TG1-based project.

Can everyone on this thread please document their requirements on this page:
http://docs.turbogears.org/2.0/AuthReqs

If we get a good set of use cases here, that will really help designing
a future auth* solution.

Paul

[Kevin Horn]

On Sat, Feb 23, 2008 at 11:00 AM, Felix Schwarz <felix....@oss.schwarz.eu> wrote:

Hi,

I'll finish working on authority for today. I got my dev environment set up
and patched authority so that beaker is not required any more (this is
useful for auth schemes like HTTP Basic).

I don't have any commit rights to the repo, so I uploaded my patch to:
http://www.felix-schwarz.name/files/misc/2008/2008-02-23_authorize.patch

Furthermore I wrote a small unit test for HTTP basic auth (please have a
look at the disabled test which tests the handling of malformed headers):
http://www.felix-schwarz.name/files/misc/2008/test_basic_auth.py

Have fun,

--
Felix Schwarz
Dipl.-Informatiker
software development and consulting

www.schwarz.eu

I've applied your patches, Felix.  Thanks for your work!

Kevin Horn

[percious]

Just an FYI, the tests are not passing at this time, not sure why yet.

Should we start up a separate google groups for authority, thus
uncoupling from Turbogears?

-chris

[Lee McFadden]

On Mon, Feb 25, 2008 at 10:35 PM, percious <ch...@percious.com> wrote:
>
> Just an FYI, the tests are not passing at this time, not sure why yet.
>
> Should we start up a separate google groups for authority, thus
> uncoupling from Turbogears?
>

The group is already up and running :)

http://groups.google.com/group/authority-wsgi/

--
Lee McFadden

blog: http://www.splee.co.uk
work: http://fireflisystems.com
skype: fireflisystems

[Kevin Horn]

As Lee said, the group is already set up.  But we should continue our discussions there, rather than here.

Kevin Horn