Apart from my serious doubt that this can be done at all I'd like to
point out that the very notion of *predicting* URLs is not RESTful. URLs
are generated by server (by whatever logic it chooses) and client treats
them as opaque identifiers. The more you document or specify rules for
constructing URLs for clients the less RESTful your service is.
6. Proper Links / Foreign Key Resources
http://github.com/ingenieroariel/dapi/tree/master is an intersting project working on the same thing, it's probably a good idea to look at it(and any other attempts) for reference.
I actually question that very premise. Django needs to support generic
REST precisely by *not* trying to wrap it up as a "module". And I say
that as somebody who write REST-based applications almost all the time,
because it's simply a nice maintainable architecture style.
> 1. Predictable URI Naming
> A logical, clean URI structure is at the core of any REST API.
> Individual items and collections should have standard naming schemes
> across all resources. Additionally, URIs should be static to the
> greatest extent possible.
However, URI's vary infinitely across the complete set of all
applications. They need to be consistent within an application, but what
you do for a particular application is domain specific.
What's the existing problem in Django here? Clean, predictable URI's are
already possible. Lots of people already use them.
> 2. Graceful Handling of Complex Resources
> Jacob’s example is a GET method to “/heros/superman”. Such a resource
> needs to return a complete set of data on Superman, and this means
> being able to aggregate data from multiple resources.
Again, already possible with views. What isn't possible here?
> 3. Generic / Pluggable Authentication
> Resource data can be sensitive in nature and needs to be distributed
> only to those that have a right to receive it. This permissions
> system should be generic in the sense that a resource does not
> personally allow / deny access, but instead delegates permission
> requests to a generic permissions handler.
which permission system is used is up to the application. Django already
has the contrib.auth module, which can be used even without cookies
(although it requires some different middleware). There's a small gap
here in providing documentation and probably some HTTP authentication
header handling middleware.
> 4. Flexible Serialization
> Django API consumers will have disparate data format needs, and these
> need to be addressed. Formats such as XML, YAML, JSON need to be
> considered during implementation.
So we already have serialisers that handle all of those for querysets.
If you're serialising other things, there are a number of approaches
discussed in the various threads here already about GSOC for
serialisation. Subclassing the existing serialisers isn't particularly
difficult. "Solving" serialisation in general, whatever that means, is a
project all on its own (as witnessed by the broad scopes of the
proposals already discussed herre).
> 5. Clean Request Handling / Mapping
> Jacob notes that browser limitations make one-to-one mapping of HTTP
> methods (GET/POST/PUT/DELETE) to CRUD operations only a piece of the
> puzzle. Flexibility is need here to accommodate various consumer
That's not too hard to do already. Again, you're describing a bullet
point here, rather than identifying some particular problem in Django or
proposing a solution (which is a consistent problem throughout this
document), so I'm not quite sure where you're coming from.
I have done a fair bit of thinking in this area, trying to work out the
*minimum* changes to Django to allow as much flexibility as possible.
Particularly based on ideas such as JAX-RS (JSR-311 -- the Java REST API
framework) and similar things. I think most of these problems --
dispatching to different handlers for different methods can be solved
fairly easily with two small modifications to how URL resolvers (and
reversers) are picked up and a couple of decorators. If nothing else
happens in the interim, that's something I'll do early in the 1.2
timeframe, as I ran out of time for 1.1.
> 6. Proper Links / Foreign Key Resources
> All resources which contain foreign key fields should be serialized to
> contain links to said resources in the form of a URI. One potential
> expansion on this concept is adding “deep” or “full” serialization
> support, whereby a consumer can request that foreign key references be
> “folded” in to the core object. This is open for debate.
Indeed it is. General REST design (and, remember, you're claiming to be
discussing a generic REST API, not something that provides a way to
display models) completely separates the API from the persistent
storage. They are orthogonal areas. *Sometimes* there will be a nice
mapping, but that's not the general case.
> 7. Scalable Deployment / Low Coupling.
> Jacob makes the good point in his “Worst Practices” document that the
> resource usage profile of a heavily used API is nothing like that of
> the web app component of a service. Therefore, the API module must be
> capable of being run on its own server(s).
I can't think of anything in Django prevents this now. Jacob's document
points out that the developers shouldn't be stupid and should understand
sensible computer software engineering practices, which will remain true
pretty much forever.
I realise the above is going to read very negatively, but I'm highly
skeptical of this proposal, particularly at such a late date when you've
already (hopefully!) read all the feedback on other people's proposals
and seen what we are after. Primarily, there's no proposal in this
proposal. What problems are you trying to fix? How are you proposing to
Secondly, I'm not sure there's a huge problem that is solved by
implementing a lot of code in Django here. Most of the bullet points are
already possible and fairly easy. There's certainly some ground for
extra documentation, probably quite a lot of it, around this area.
Something I've been personally addressing in an, as yet incomplete,
series of blog posts, for example. However, documentation isn't a Summer
of Code project. The areas I think are missing aren't Summer of Code
length, either (they're more like Two Weekends of Code stuff.).
Don't get me wrong; there are definitely some small pieces of code
support needed so that developers can be as generic as they want to be
when designing their APIs. However, the whole space isn't conducive to
saying "this is the best practice" and forcing something on the users,
as REST is an architectural *style*, not an architecture.
So, if you're really enthused about this, please provide a lot more
details, starting with problem description. You've hopefully carefully
studied existing code such as the django-restapi project to understand
where the difficulties lie in the generic approach. I genuinely doubt
there is a generic set of helpers is possible, since "generic" covers
such a broad area. We need to loosen up a few things, such as allowing
custom URL resolving and reversing so that it can be done a bit more
easily for collections of views. I have an idea of the precise amount of
difficulty needed there, since it's something I've looked at a fair bit.
It's not a summer's worth.
I'd want to see a lot more specifics before I could evaluate a proposal
along these lines.
It's entirely up to the developer as to which pattern to follow. It's
not something the software can help with (although it's already
encouraged, since we use reg-exps to parse the URIs).
For example, there are very valid use-cases, when including a locale
specifier in a URL for putting it either at the front or the end of the
URL (e.g. /de/foo/blah vs /foo/blah/de). Both are valid and good
practice in different cases. Even within the same website, both styles
might be appropriate (it's not going to be common, but I can think of
some cases). There's no way for a set of libraries like Django to
provide more than a way to parse out useful bits of URLs for situations
like that (and locale parsing, particularly in the leading-component
case, is one of the use-cases for some of the URL resolver modifications
we'll eventually need).
Well, you said you had created a problem statement and I assumed that
mail was it. Apparently not. It's a bit confusing as this whole proposal
to do something for Summer of Code hinges on there actually being a
problem to solve and a solution that solves it, not the fact that REST
> The reason that I refer to it as a module and think there is more to
> this than just an architectural style is that by providing an API
> "service" that models can be registered with (think django-rest-
> interface from 2007) a lot of the details can be ignored by someone
> who is not interested in writing a full API from scratch.
Then please stop calling it a generic REST API. Because it's not. It's
an interface to expose models. A generic REST API requires the full
power of Python because, as I've already noted, it's an orthogonal
concept to what is stored persistently. We provide a way to retrieve
arbitrary data -- view functions and the ORM. You can't hope to make it
any simpler than that for the general case.
> objective would be to let the developer define which data can be
> accessed through the API, who can access it and have most of the
> internal details handled for him.
So is this another pass at the 2007 project again? Why not contribute to
django-restapi instead and keep that going along? I know why that
project got as far as it did and what the limitations and strengths of
it are (since I was the mentor for that student). For a
model-presentation API, it's going in the right direction and people are
The devil is in the details here and the worthiness of a proposal along
these lines very much on what you are thinking requires changes in
Django's core. I think the answer to that is "very little". Which then
leads me to wonder about what you're proposing to do.
> Does that help clarify what I wrote a bit?
Not really, because I still don't know what you're proposing to do and
the specifics are important for a Summer of Code project.
A whole 12 hours after the previous one. I had a day away from mail.
That happens sometimes.
> I have gone over the django-
> restapi library and would appreciate more guidance on the intended
> direction of the project. As you suggested, following up on the
> existing work there is a great way to get this feature to the public
> this summer.
It's already "to the public" -- it's been publicly available and
maintained (brought up to date to run with Django 1.0) on Google code
for a few years. There's even been a maintainer hand-over last year,
indicating ongoing life and maybe one day, the people maintaining it
will propose it for addition to django.contrib.
This is one of my big problems with your proposal so far. I'm not seeing
what you're bringing to the table here. I don't think working further on
django-restapi is a Summer of Code project. I don't think it's a
candidate to be merged into django.contrib (yet) either. I also don't
have particularly strong feelings about things that are needed, since I
don't really use it. I follow the code changes, but don't use it in my
applications. My REST apis don't map directly to models and there's lots
of intermediate processing going on, so I use normal Django views to do
the work. There's an issues list at Google code and a mailing list with
archives (django-rest on Google Groups) that is low enough volume you
could look through there for ideas.
I like the broad structure used in django-restapi -- Andreas did a good
job of trying out a bunch of possibilities to come up with something
that is actually an aid, rather than an obstruction, for models.
However, this is really a third-party project, not something that is
improving Django's core, so it's out of scope for Summer of Code.
Working on duplicated work just so that it's a Summer of Code project
isn't a valid technical reason to do something, either. This keeps
coming back to my questions about what modifications are needed to solve
actual problems in core that makes this a worthwhile project?
I haven't heard anything from Jacob about what he thought he discussed
with you on this issue. Your version of it is so far a bit shy on
specifics and it looks like you're still hunting around for "do
something cool with REST" project. Unfortunately, asking me for ideas
isn't going to help you, since I've already indicated that apart from a
couple of small resolver changes and short utility functions to mark up
which resolver to use for a function (the former being appropriate for
core in 1.2, the latter maybe or maybe not), I don't see much extra that
is needed. We already have a fairly viable third-party application that
people can contribute to if they want model -> front-end RESTful-style
API (django-restapi). It's not something I've found myself needing,
since writing an appropriate API for the code I have in front of me
isn't that hard.
So, once again, I'm forced to ask -- and this is something you really
need to answer if you're going to come up with a proposal: What problems
are you trying to solve that aren't already possible? Why do these
problems need core changes to Django?