Thoughts on library architecture
Will Norris
in the DiSo Project, and have been quite happy overall with how it
works. For the long term, I've been talking with Marc a little bit
about some patches I'd like to make for the library. I figured it
would make more sense to discuss this openly on the list to see if
there is other interest in this, and if this effort can help to create
a better OAuth library for all of us to use.
So to preface this a little bit, I'm not looking to write a whole new
OAuth library from scratch... like I said, I've been quite happy with
what I'm using. What you see below are my initials thoughts on how
the library *could* be architected. This is actually far more than
what I originally intended to provide to Marc in the way of patches,
but it turned out it was easier to formulate my thoughts on a blank
canvas.
For now, almost everything just has an interface, for two reasons.
Primarily, I find it far more useful to start by thinking exclusively
about the library API and how the different pieces fit together. It
doesn't matter how good the actual implementation is, if the API is
not flexible enough for what you need. Second, most of the code is
already written between the two libraries we have. The variable names
are a little different and the some of the logic has been moved to
different places in the code, but by and large that work has already
been done.
So, my early and ever-changing thoughts on an OAuth library
architecture... Drill down into "library/Auth/OAuth" in the github
repository below:
http://github.com/willnorris/oauth-php/tree/master
So the key things to notice in this code (as compared to the existing
Mediamatic library):
- I've broken the class hierarchy, where many of the core classes
were extending from OAuthRequest. This just seemed really weird to
me, because an OAuthServer is not actually a more specific kind of
OAuthRequest, it's something different entirely. Instead,
OAuth_Server, OAuth_Consumer, and OAuth_Signer are all standalone
classes that simply act on OAuth_Request objects.
- OAuth_Server and OAuth_Consumer are sort of serving double duty
right now. They are both used to represent the very light
representation of servers and consumers, returned by OAuth_Store. But
they also contain the actual logic (in static methods) for making and
handling requests (previously in OAuthServer and OAuthRequester). I
don't actually like having them overloaded in this manner, and will
probably split those apart later.
- With the exception noted above of OAuth_Server and OAuth_Consumer
serving double duty, these two classes along with OAuth_Request,
OAuth_Token, and (to some degree) OAuth_Store are designed to be very
lightweight. They should certainly have formal interfaces so that the
local platform can provide custom implementations of them if need be.
- OAuth_Store has been simplified to basic CRUD operations. It is
responsible for persistent storage and retrieval of consumer and
server entries, as well as consumer and server tokens. It has no
concept of "admin users", or "secrets for verify/signature". When
exchanging request tokens for access tokens, the store is not
responsible for checking that the request token has been authorized,
or for removing the old request token after it has been exchanged.
All of this logic should be done outside of the OAuth_Store
implementation, so that it can be shared. The simpler we make the
OAuth_Store, the easier it will be to create custom implementations.
- OAuth_Signer combines OAuthRequestSigner and
OAuthRequestVerifier, as well as some of the signing specific stuff
that was in OAuthRequest. It has all of the message signing logic.
Again, this was done in order to make OAuth_Request as lightweight as
possible.
- All of the utility functions from OAuthRequest have been moved
into OAuth_Util. Again, in order to make OAuth_Request lightweight.
- There is a little actual implementation code in OAuth_ServerImpl,
specifically the accessToken() function. That was primarily so I
could make sure the OAuth_Store interface would work how I was
thinking. It gives a small indication of how I picture this all
working.
So I welcome any thoughts or comments on this architecture. Does this
seem like a reasonable approach? Do you think this will provide the
level of flexibility we need? There are a lot of things that need to
be considered later, but first I want to make sure the API is pretty
solid.
-will
Morten Fangel
Okay - I'm a bit tired and have only just scanned the code, so
if this feels like a rant, that's my excuse.. ;)
I really really like the idea of splitting everything up, and having
interfaces that just states which function an object has to
perform in order to be considered correct in some specific
region of the code..
But a few things struck me as odd
1) Consumer & Server
There is both a Consumer that has a key+secret, and a server
which has a key+secret? When are what used, and ehh.. I just
really didn't get what the heck it was all about..
They both have required methods for creating new tokens etc..
2) Data storage is overly complex
Seriously, it has 17 required methods. The current OAuth storage
requires 5. That's less than a third, and that still functions really
well.
I guess the layout makes sense if you want the data-storage
and the server to require all sorts of methods because it forces
developers to build the required functionality.
But how many will actually need a method in their datastorage
that deletes consumers etc. How come this has to go through the
datastorage and can't just be implemented somewhere in some
other application-specific code?
All in all, I favour a much more pragmatic approach. Only make
the library require what it actually needs to function.
The current data-storage has these methods:
* Lookup consumer by key (String -> OAuth_Consumer)
* Create new request-token (OAuth_Consumer -> OAuth_Token)
* Create new access-token, but only if the request-token is authed.
Also delete the request-token
(OAuth_Consumer * OAuth_Token -> OAuth_Token)
* Lookup a token (OAuth_Consumer * String -> OAuth_Token)
* Check Nonce (OAuth_Consumer * String -> boolean)
And I can't really see why there need to be more. It might be
misguiding to call it a data storage, because it is merely a sort of
bridge between the OAuth library and your Application.
Can anyone tell my what the awesome reason to include more than
three times as many function? I can't see it, but it might be there..
And this is why I kind of like the current library. There's a lot of
connecting the invisible dots. But when you figure out how to tie it
all together, it really doesn't take a lot to make it play nice with
your
existing application..
-Morten
Marc Worrell
I had an earlier reply on another thread in oauth-php, though google
seems to have eaten that one. Wise lesson: never use the web interface.
I will try to add some parts of my earlier reply into this one.
First a small introduction: I am the original author of the Mediamatic
OAuth-PHP library.
In an earlier mail Will was right when he said that the library was
not meant to be rigid. It just evolved into what it was from our
requirements. One big requirement is that the library should be able
to support multiple users, and then as a consumer and as a provider.
Besides that it should be easy to plug into an existing system.
With this in mind, I looked at the structure Will proposed and the
remarks he made. It makes a lot of sense to me. I will look into it
in more detail tomorrow, together with Arjan, the other maintainer of
the oauth-php library.
What I missed was a split in the OAuth_Store between a consumer and a
provider part. When the store is split then it will be easier to make
either a consumer or a provider. It might also remove some confusion
about the consumer registering servers and the server registering
consumers :-)
I think the store might be implemented using four different interfaces:
- Consumer side
- Provider side
- Consumer side management (by the user - "which providers can I
access?")
- Provider side management (by the user - "which consumers access my
account?")
And then there is the request logging, which is more or less the same
for the consumer and provider side.
On 27 jan 2009, at 23:42, Morten Fangel wrote:
> 1) Consumer & Server
> There is both a Consumer that has a key+secret, and a server
> which has a key+secret? When are what used, and ehh.. I just
> really didn't get what the heck it was all about..
> They both have required methods for creating new tokens etc..
Quite simple, the library implements both consumer and server side
tokens. They are two different registries. As a test case, try to
connect to yourself using the library, that should be possible.
Besides that the consumer and the server have quite different
administration requirements around their tokens.
> 2) Data storage is overly complex
> Seriously, it has 17 required methods. The current OAuth storage
> requires 5. That's less than a third, and that still functions really
> well.
The oauth-php datastore is quite complex because it supports a lot of
use cases. I wouldn't be surprised if the store Will proposes is in
fact simpler than the current one. The store in the oauth library is
a bit too lightweight. As an example: try to connect to yourself
using that store. You really shouldn't mix the consumer and server
side registration in the same table.
> I guess the layout makes sense if you want the data-storage
> and the server to require all sorts of methods because it forces
> developers to build the required functionality.
> But how many will actually need a method in their datastorage
> that deletes consumers etc. How come this has to go through the
> datastorage and can't just be implemented somewhere in some
> other application-specific code?
It needs to be done in the data store because the logic in the library
might want to perform, for example, just such a deletion. Think of
time-to-live tokens and other extensions.
> All in all, I favour a much more pragmatic approach. Only make
> the library require what it actually needs to function.
Which I do not totally agree to. As you also would like the library to
be able to function as a kind of black box. In which case you also
need some interfaces for management of your data store. Users do like
to manage the access tokens to their account and to other servers.
Of course you can also define some data store that on one side
implement the interfaces as needed by either the consumer and/or the
provider library code, and on the other side implement interfaces to
manage the store.
Providing a functional datastore, and not just interfaces, makes it
possible to give (almost) turnkey solutions to consumer and provider
builders. Preferably the datastore should be provided for mysql,
postgresql and sqlite.
> The current data-storage has these methods:
> * Lookup consumer by key (String -> OAuth_Consumer)
> * Create new request-token (OAuth_Consumer -> OAuth_Token)
> * Create new access-token, but only if the request-token is authed.
> Also delete the request-token
> (OAuth_Consumer * OAuth_Token -> OAuth_Token)
> * Lookup a token (OAuth_Consumer * String -> OAuth_Token)
> * Check Nonce (OAuth_Consumer * String -> boolean)
>
> And I can't really see why there need to be more. It might be
> misguiding to call it a data storage, because it is merely a sort of
> bridge between the OAuth library and your Application.
>
> Can anyone tell my what the awesome reason to include more than
> three times as many function? I can't see it, but it might be there..
When you separate server and consumer side tokens, then it really adds
up quickly.
In this case: provider + consumer + management.
- Marc Worrell
Will Norris
On Jan 27, 2009, at 2:42 PM, Morten Fangel wrote:
>
> Hi Will
>
> Okay - I'm a bit tired and have only just scanned the code, so
> if this feels like a rant, that's my excuse.. ;)
no problem, it wasn't to rant-y. :)
> 1) Consumer & Server
> There is both a Consumer that has a key+secret, and a server
> which has a key+secret? When are what used, and ehh.. I just
> really didn't get what the heck it was all about..
> They both have required methods for creating new tokens etc..
So that is basically the double-duty those two classes serve. There
are four different logic parts here:
- OAuth_Consumer - contains the logic for requesting tokens from an
OAuth server somewhere across the web. This would be the main class
your application would touch if you were implementing an OAuth consumer
- OAuth_Server - contains the logic for handling token requests and
verifying the signature of OAuth-signed requests to protected
resources. This would be the main class your application would touch
if you were implementing an OAuth server.
- OAuth_Consumer_Entry - very lightweight entry that is returned by
OAuth_Store::getConsumer(). At it's simplest, this is just a consumer
key and secret, so maybe we don't even need an object. The Mediamatic
library also stores things like the requester's name, email, and
application name, I think.
- OAuth_Server_Entry - very lightweight entry that is returned by
OAuth_Store::getServer(). At it's simplest, this would have the
consumer key and secret that should be used when communicating with
this server, along with the three endpoint URLs. The Mediamatic
library also stores the URL of the server, so that you can just give
the library a protected resource you want to access, and it will
figure out which endpoints, keys, and tokens to use.
These should absolutely be split apart, I just hadn't come up with a
better name for OAuth_Consumer_Entry, that's all.
Well to begin with, you're only taking into account the methods that
are necessary if you are an OAuth server. If you are a consumer, then
you need the functions:
* Lookup server by key (String -> OAuth_Server)
* store request/access token (OAuth_Token)
* retrieve request/access token (String -> OAuth_Token)
How about marking request tokens as having been authorized by the
user? How do you currently store that state?
Now it is certainly possible to handle the other CRUD operations for
these things in separate application-specific code. However, since
your OAuthStore already has the logic for creating and retrieve these
entries, why not go ahead and add the additional logic to update and
delete? You're almost certainly going to need to perform these
operations (creating new Consumers, deleting or "deactivating" access
tokens that have been revoked by the user, etc), so it makes a lot of
sense to keep all of this logic in the same place. If your data store
changes later, you only have one class that needs to be updated.
For these extra functions that are not directly called by the library,
you could just stub them out in your implementation. Or we could have
an abstract class that provides stubbed functions, and you only
override them if you care to have them. Or we leave them out of the
interface all together. While I understand that these add unnecessary
weight to the OAuth_Store interface, I think the vast majority of
users of the library actually will make use of them.
> And this is why I kind of like the current library. There's a lot of
> connecting the invisible dots. But when you figure out how to tie it
> all together, it really doesn't take a lot to make it play nice with
> your
> existing application..
But you assume that the only users of this library are going to be
those that are integrating it into existing applications that can
provide 90% of what they need. While this is certainly one case, it
is by no means the only one. I think we can develop something that
can be integrated into existing applications with minimal effort, but
still provide a little more utility for those users who don't want to
write a bunch of auxiliary code to fill holes in the library.
-will
Morten Fangel
Okay - having slept and read through your emails, it makes a bit
more sense..
It's because what is always referred to as the "Consumer Secret",
is called different things, depending on whether you're using the
library as a Consumer-implementation, or as a Server
-implementation.
It was mainly all the aspects relating to using the library in a
Consumer-implementation that threw me off. I only looked at the
library as one consumer consumes one service provider, in which
case the whole "let's store service-provider-profiles" thing is pretty
overkill..
I might still think it is, because none of the APIs accessible by
OAuth is the same, so you are going to have service-provider
specific code anyway, so why not also a hard coded profile of your
consumer key for that specific service-provider..
Could your split up the data storage into two parts: one for each of
the use cases. Because assume that you only need to function as
a SP, why worry about all the storage methods that only consumer
implementations need..
It would also make implementation documentation easier (I reckon).
You could say "If you are going to be a SP, you need to create a
class that implements the OAuth_Storage_SP interface, and attach
this to your OAuth_Server instance so it knows where to look up the
different consumers calling the server"
-Morten
Will Norris
On Jan 28, 2009, at 4:25 AM, Morten Fangel wrote:
> <snip>
>
> Could your split up the data storage into two parts: one for each of
> the use cases. Because assume that you only need to function as
> a SP, why worry about all the storage methods that only consumer
> implementations need..
> It would also make implementation documentation easier (I reckon).
> You could say "If you are going to be a SP, you need to create a
> class that implements the OAuth_Storage_SP interface, and attach
> this to your OAuth_Server instance so it knows where to look up the
> different consumers calling the server"
interfaces. Of course, if you are implementing both, you can still
have a single OAuth_Store implementation that simply implements both
interfaces.
-will
Jon Crosby
like everyone has come together around a possible approach.
Having said that, I just wanted to lend some experience from CloudKit
and say that splitting the OAuth store out into its own thing was very
beneficial for the reasons that Will stated. Having a store interface
means other implementations can easily be created without having to
deal with the logic around the meaning of the items that are contained
in the store. CloudKit's OAuthStore knows that it needs to create
tokens and things like this but it leaves the storage specifics up to
an adapter. The OAuthFilter (which is only a provider in the case of
CloudKit) simply accesses the store like a generic database that
happens to have the correct types in place and manipulates properties
(such as an "authorized" value on a request token) to perform its
work.
Jon Crosby
http://joncrosby.me