Why Django Apps Suck
klaasvan...@gmail.com
rants so I figured I'd try my luck here. Django has been my favorite
framework for developing web applications for about a year and a half,
so it comes from a position of love when I say that the Django app
really sucks as a level of abstraction. And I'm really looking for a
way out so any suggestions are much appreciated. A tl;dr is provided
below.
Others have complained before me. Two particular ones worth mentioning
are:
http://www.scribd.com/doc/37113340/Why-Django-Sucks-and-How-we-Can-Fix-it
which claims that "Apps which provide models are inflexible" and
http://www.mutualinformation.org/2010/03/why-i-switched-to-pylons-after-using-django-for-six-months/
Both claim that the Django app provides the wrong level of
abstraction, neither goes in too much detail exactly why that is the
case. That's the question I'll attempt to answer.
Before we can answer the question we must first set some reasonable
expectations for an "app". I'd say an app is a small package of
closely related features as they are presented to the user (in the
context of a web project) that can be easily modified/extended for any
given project.
Examples of apps (in my own particular context):
* Dealing with customer information (a mini CRM). Adding, editing,
useful overviews.
* A small news app (Add, edit, overviews)
* Our own particular UserProfile app (editing, viewing others,
birthday calendars etc.)
* A small budgetting tool (creating budgets, see how much money has
been used)
* Logging time lines (for worked hours), searching them.
I choose this definition for "app" because it is a very useful one in
the context of web projects. At least in my context it works like
this: we have a lot of customers with similar intranets and extranets.
Each one of them might want a particular subset of the various apps we
have "on the shelf".
By the way: there's other things that can already be reused quite
well. Anything pure python for example. Or extensions to Django (extra
fields, extra context managers, middleware etc etc.) This is not my
concern, mainly because it already works well.
One thing that deserves extra stress is that apss should be easily
extendable. Exactly the fact that every customer want something that
is slightly different from the next is how we're able to provide
value. (This is also the reason I like Django in the first place:
making stuff "your own way" is very easy). So: if no one want exactly
the same, extensibility is a major prerequisite for reusability.
What are some typical things we want to do when extending (modifying)
an app for the context of a particular project?
* Deal with authorization in the context of the particular django
project
* Styling for the project
* Turn a subset of the functionality on or off for a given project (no
docs, no 'advanced features')
* Add one or more fields to one or more models and possibly related
edit pages, view pages forms etc
Given this control flow let's try to do some of the typical
modifications mentioned above:
Authorization:
Django has a beautiful mechanism for authorization, which is using
decorators (@may_do_a_b_or_c) directly in the views.py.
Tempting, but will not be modifyable in any way once you lift the app.
(because the decorator will become tied to the view function). Other
mechanisms are not presented (in a standardized way). If anyone has
ideas about this I'd love to discuss them.
Styling:
This actually works well, by providing sufficient amount of hooks in
the templates and extending.
Turning functionality on and off:
This could be made to work by working with a 'url_parts.py' in the
reusable app, that provides different url_patterns for the various
subsets. So one for the CRUD, one for the docs, one for the simple
features and one for the advanced features. Our project's urls.py
would then simply include the right subset from this url_parts.py. I'm
not sure it's very pretty though.
Extending models
This is the big one, for two reasons.
1. You almost always want to do this (for any change to the app that
is reflected in the underlying data)
2. It's pretty much impossible.
Why is it impossible?
Firstly, /app/models.py is always discovered, so unless we make all
Models abstract, or we do not include "app" in settings.py, models are
created for syncdb and any related commands.
Secondly, let's take a look at Django's control flow:
* urls.py matches and calls,
* possibly via a decorator
* a view in views.py, that may use any number of
* models,
* forms and
* templates
The "official" way to modify a reusable app for your particular app is
by making it available as a kwarg with proper default in the view, and
then customizing it from urls.py.
However, if you simply want to add one field to a much used object,
this solution implies passing around the object class (factory)
through your whole app. That's not very elegant.
The same goes for any other changes at a lower level. Say I want to
swap in a particular Form (not uncommon). Or reuse 90% of the views
from an app, but replace 10% of the alltogether. If those 10% are
referenced from {% url %} tags (or reverse in a view) we're screwed.
By the way: where do modifications of apps go anyway? The reusable app
already goes into /appname. So the modification (presuming it warrants
a full directory) goes into /my_appname? If reusable apps are common,
this means my projects look like this:
/app_a
/app_b
/app_c
/my_app_a
/my_app_b
/my_app_c
Solutions:
http://www.scribd.com/doc/37113340/Why-Django-Sucks-and-How-we-Can-Fix-it
mentions using factories "everwhere". Too bad the actual speech cannot
be heard at scribd because the slides are just a bit thin on details.
settings.py
In my humble opinion a good app adds no settings to settings.py. The
problems with settings.py are mentioned in many places, suffice to say
it's basically a big global. Also: we're programmers, not configurars.
For the models problem specifically I came up with this about 7 months
ago:
https://bitbucket.org/vanschelven/superglue/ (very much proof of
concept)
Superglue works like this:
For any reusable app that uses it, models are to be defined abstract.
Then a magic method (glue) is to be called (from the apps models.py).
This magic method looks at settings.py (meh) and possibly extends the
model. At that point the extended, non-abstract model is made
available for the rest of the app.
I don't really like it yet, but it could be a start.
Another possible way forward could be object oriented views, like
here:
http://djangosnippets.org/snippets/1009/
tl;dr
* Django Apps Suck. Your reply explaining why they don't is expected.
* Reuse of a lot of things doesn't suck. But the specific unit app,
meaning a bunch of models that can be modified and presented with
their model-specific features, is extremely hard to reuse.
* Apps are mostly hard to reuse because they are hard to reuse with
minor modifications (extensions).
* Using decorators on views to define authorization levels ties in
said authorization levels with the app (and makes it non-reusable)
* The hardest things to change are at the bottom of Django's call
chain. Deep inside the views, we find particular Models, Forms and
references to other views.
* Models are extra hard to change, because they are expected to be non-
abstract in the app.
* There is no logical separation (dir structure) between "a reusable
app" and "my modifications to a reusable app".
* Settings.py is not a solution (it is a problem), though I don't
bother to explain why.
* Some ideas for solutions exist, but this seems to be an open problem
in the Django world.
Correct me if I'm wrong.
hope to hear from you,
Klaas van Schelven
Shawn Milochik
Ramdas S
Wow! I guess your definition of an app and expectation of re-usuability from an app written by someone else
is fairly high!
We have built serveral news sites based on Django, and a lot of code gets rewritten everytime, because customer requirements are very different. We reuse 90% of code in every project, but 10% gets rewritten or gets writtenf fresh.
I know there's a number of apps out there including ones written by people like me, which are grossly unusable. I guess that's primarily because its written by developers for a specific work, and when they threw it back into the wild, they did not pay enought attention tomake it really re-usuable. However there are quite a few exceptions,
I completely agree with your views that with the way you write code with {% url in templates, and decorators, reusing the code is a pain. I had to scrap a project based on Pinax for the same reason, and then rewrite a mini scoial framework, because pinax is not customizable beyond a point.
However my personal experience is that it's true for most platforms. There are always trade-offs and trade-ins when you use a platform.
--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django...@googlegroups.com.
To unsubscribe from this group, send email to django-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
--
Ramdas S
+91 9342 583 065
Klaas van Schelven
Just started watching the video and I'm noticing my complaints are not
unique.
> Wow! I guess your definition of an app and expectation of re-usuability
> from an app written by someone else
> is fairly high!
since all of them provide models, and every use case either needs
changes to those.
>
> We have built serveral news sites based on Django, and a lot of code gets
> rewritten everytime, because customer requirements are very different. We
> reuse 90% of code in every project, but 10% gets rewritten or gets writtenf
> fresh.
>
Do you adapt models across your apps?
Do you deal with (some kind of advanced) authorization that differs
over projects?
I'd like to know how.
> I know there's a number of apps out there including ones written by people
> like me, which are grossly unusable. I guess that's primarily because its
> written by developers for a specific work, and when they threw it back into
> the wild, they did not pay enought attention tomake it really re-usuable.
> However there are quite a few exceptions,
>
> I completely agree with your views that with the way you write code with {%
> url in templates, and decorators, reusing the code is a pain. I had to scrap
> a project based on Pinax for the same reason, and then rewrite a mini scoial
> framework, because pinax is not customizable beyond a point.
app") but I was a bit disappointed. (not so much in the app, as in how
it is an example of reusability).
Problem is "{% url %}" is presented as a Good Thing (DRY) while it
makes reuse in some ways harder
What's your alternative, i.e. how do you provide links that are
project independent?
Mike Dewhirst
Ramdas S
Shawn: thanks
Just started watching the video and I'm noticing my complaints are not
unique.
Actually, a big part of the problem is reusing my own apps. Mostly
> Wow! I guess your definition of an app and expectation of re-usuability
> from an app written by someone else
> is fairly high!
since all of them provide models, and every use case either needs
changes to those.
That's interesting.
>
> We have built serveral news sites based on Django, and a lot of code gets
> rewritten everytime, because customer requirements are very different. We
> reuse 90% of code in every project, but 10% gets rewritten or gets writtenf
> fresh.
>
Do you adapt models across your apps?
Do you deal with (some kind of advanced) authorization that differs
over projects?
We don't do anything that's extra-ordinary. Tried it and it failed. While I have not quite experimented in depth with Pylons, the problem what you are describing about reusability exists in some way or other with every platform. We've been through almost all major frameworks from Java based ones to CakePHP to Rails.
I'm sorry to disappoint, but all I do is move code between projects and then customize models, views and urls according to the needs.
Even an app which is pretty well written like django-registration, I had to customize it number of times to suite specific needs of some of my customers, that my versions are impossible to reuse.
I'd like to know how.
Yeah I looked at Pinax too (because it is presented as a "reusable
> I know there's a number of apps out there including ones written by people
> like me, which are grossly unusable. I guess that's primarily because its
> written by developers for a specific work, and when they threw it back into
> the wild, they did not pay enought attention tomake it really re-usuable.
> However there are quite a few exceptions,
>
> I completely agree with your views that with the way you write code with {%
> url in templates, and decorators, reusing the code is a pain. I had to scrap
> a project based on Pinax for the same reason, and then rewrite a mini scoial
> framework, because pinax is not customizable beyond a point.
app") but I was a bit disappointed. (not so much in the app, as in how
it is an example of reusability).
Problem is "{% url %}" is presented as a Good Thing (DRY) while it
makes reuse in some ways harder
What's your alternative, i.e. how do you provide links that are
project independent?
Pinax was a bad experience for me. I am going to give glamkit a try now...
> However my personal experience is that it's true for most platforms. There
> are always trade-offs and trade-ins when you use a platform.
--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django...@googlegroups.com.
To unsubscribe from this group, send email to django-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Russell Keith-Magee
<klaasvan...@gmail.com> wrote:
> tl;dr
> * Django Apps Suck. Your reply explaining why they don't is expected.
> * Reuse of a lot of things doesn't suck. But the specific unit app,
> meaning a bunch of models that can be modified and presented with
> their model-specific features, is extremely hard to reuse.
> * Apps are mostly hard to reuse because they are hard to reuse with
> minor modifications (extensions).
> * Using decorators on views to define authorization levels ties in
> said authorization levels with the app (and makes it non-reusable)
> * The hardest things to change are at the bottom of Django's call
> chain. Deep inside the views, we find particular Models, Forms and
> references to other views.
> * Models are extra hard to change, because they are expected to be non-
> abstract in the app.
> * There is no logical separation (dir structure) between "a reusable
> app" and "my modifications to a reusable app".
> * Settings.py is not a solution (it is a problem), though I don't
> bother to explain why.
> * Some ideas for solutions exist, but this seems to be an open problem
> in the Django world.
> Correct me if I'm wrong.
I'm no sure I'd go so far as to say your 'wrong' -- you've certainly
got some valid points in here -- but you're drawing a different set of
conclusions than I do.
You're looking at the world of Django apps that has evolved over 5
years, and is still evolving, and has an evolving world of best
practice, and appear to have concluded that reusable apps can't/don't
work.
I look a the same landscape and see that reusability demonstrably
*does* work -- but it could work a whole lot better. More importantly,
I've seen reusability get steadily better over time as the community
gets more experience building larger projects, and needing to reuse
more code.
Reusability has always been *possible* -- providing you build your
apps in the "right way". The catch is that not every app is built the
"right way"... and the understanding of the what constitutes the
"right way" changes with time. Even Django's contrib apps have been
victim of this. contrib.comments was built using the best
understanding of it's time, and landed in Django 1.0. By Django 1.1,
we had significantly improved our understanding of the importance of
having multiple instances of an application, and provided tools to
allow URLs to be namespaced and handled at a class level.
Does this mean that contrib.comments isn't reusable? Well, yes an no.
It *can* be reused. It has many desirable reusable features. It can
and has been used in many projects. But it isn't 'state of the art'.
And there is a of aspects of reusability that could be improved in
that app.
The secret is that someone needs to pitch in and make those
improvements. You think contrib.comments isn't reusable enough?
Propose a change! Make your case, write a patch, shepherd it through
the release process.
The same goes for any other reusable app in the community. If you find
an app that isn't as reusable as it should be, contribute a patch back
to the authors that adds flexibility in the direction you think it
needs it.
And if there's something that needs to be acknowledged at the core
level to improve support for reusability, we're all ears. Namespaced
URLs was a big step. Class based views will be another -- they've been
on the roadmap since 1.0, and based on progress so far, there is a
good possibility we might actually get them for 1.3.
At the end of the day, I can only see four possible reasons why an app
can't be made reusable:
a) fundamentally can't
b) can't justify the engineering cost to do it
c) can't work out how to do it, or
d) can't be bothered
I'm yet to see a genuine case of (a) -- every time I've seen (a), it's
really masked version of (b), (c) or (d).
(b) is a legitimate engineering management problem that can't be
ignored, so anything Django as a project can do to lower the cost
barrier is certainly a welcome suggestion.
(c) is an education problem; Admittedly, we haven't been as good at
this as we have in other areas. There are various unofficial resources
and books that discuss reusability, but there's nothing in Django's
formal documentation corpus on this significant problem. Pinax started
as an attempt to identify and codify some of the conventions that
would lead to better reusability in apps. However, that hasn't really
resulted in a concrete set of guidelines, more a set of best practices
best observed by inspection. I completely agree that this is something
we need to address; volunteers are welcome.
(d) is a motivation problem. Maybe it's driven by (b) and (c), but
ultimately, we can lead a horse to water, but we can't make it drink.
Yours,
Russ Magee %-)
Russell Keith-Magee
I'm intrigued to know what these changes might be that they couldn't
be integrated in a reusable fashion.
I doubt even James Bennett would claim django-registration is perfect.
Every application can be improved, or made more flexible. I'm sure
there are points of customization that he either hasn't considered, or
hasn't got around to implementing. This is where you, as a third
party, can contribute back -- to engage with the builders of existing
django apps to make them more reusable.
There will also be occasions where django-registration simply isn't
the right solution. That doesn't mean reusable apps are a failure. It
just means that django-registration isn't the right reusable app for
your problem. The promise of reusability isn't that every part can
perform every task. Reusable apps only promise that if you built an
app carefully, you'll be able to reuse it in similar circumstances in
the future.
Lastly, I can't deny that reusability has it's price. Engineering an
application to be reusable takes more effort and planning than
building an application as a once-off. The aim is that if you take the
effort once to make your application reusable, the engineering payoff
will happen quickly -- hopefully on your second build. However, if you
have no intention of using a component a second time, then there isn't
a whole lot of reason to spend the effort making it reusable.
Yours,
Russ Magee %-)
indymike
On Sep 21, 4:26 pm, "klaasvanschel...@gmail.com"
> One thing that deserves extra stress is that apss should be easily
> extendable. Exactly the fact that every customer want something that
> is slightly different from the next is how we're able to provide
> value. (This is also the reason I like Django in the first place:
> making stuff "your own way" is very easy). So: if no one want exactly
> the same, extensibility is a major prerequisite for reusability.
require the developer to have a special kind of discipline if you are
going to write apps (modules, classes, beans whatever the framework
calls them :) that are 100% reusable. The challenges are *always* the
same:
* Easily extending an app without breaking upgradeability.
* Dealing with interdependencies between apps, particularly where
upgrades and patches are concerned.
* For web apps, Dealing with graphic design issues where we want to be
able to customize, but we don't want to lose compatibility with future
versions.
* Presenting back end screens.
For me, Django has been a remarkable improvement from php and Python
based tools I've used in the past primarily because Django isn't a
content manager that is being devolved from its original form to
create a new application and Django isn't a framework that forces a
strict way of doing things on you (see cake and Smarty from the PHP
world).
If anything loose coupling is what makes all of this so difficult -
and so easy. Django begs you to write lots of simple apps and tie
them together - the question is how to tie them? In my case I've
found that django.contrib.auth.user is a great tool for simplifying
things, and if you d things right, you can leverage Django's automatic
two way relationships to avoid using lots and lots of decorators to
control access. I'm sure there are other ways to do things, and
probably some that might be better (most of what I've been working on
is very transaction centric, so who has access to what transaction
data is the question, not who has access to what view).
Another item that is particularly challenging is that Django doesn't
do hooks the way that some php based systems do. Want to extend
something? Hook it. It's super easy for the developer, and can
protect upgradability if the programmer maintaining the function you
are hooking cares about backwards compatibility and doesn't go
renaming everything. On second thought, I hate hooks :)
-- Mike
Klaas van Schelven
> really masked version of (b), (c) or (d).
been adressed by any of the responses above.
Let's focus on one for now: how to extend models in any given app.
As stated in the original mail, there are at least two problems with
this in Django:
1. How to provide the models in the first place? Everything that's not
abstract is hard to extend; providing abstract models feels really
awkward.
2. How to tie in the models into the views? Long view parameter lists
that are used from urls are ugly and repetitive. Moreover: it may not
be possible to solve the entire problem this way (think admins, think
forms).
If we forget about Django's status quo for a minute and take
extendability of related webfeatures as a goal, an obvious solution
would be using a class. Classes are surely the poster child for
extendability. The class would contain everything that's relevant for
an app: all models, views, admin stuff etc, in addition to the
relevant internal API ('hooks'). Like so:
class AbstractPage(models.Model):
body = models.TextField()
class Meta:
abstract = True
class MyApp(object):
def get_models(self):
# returns something that Django understands, i.e. a list of
Model classes
# or simply lists the models, so that the cache picks up on it
def get_page_class(self):
class Page(AbstractPage):
pass
return Page
def page_detail(self, request, pk=None):
page = self.get_page(pk)
# or:
page = self.get_page_class.objects.get(pk=pk)
return render_to_response('myapp/page_detail.html', locals())
def get_urls(self):
# something that can be included by django
Obviously we can choose anything that python permits to create
organization of the above once it starts growing. Standardization
would be a good idea as well.
However, the main point is that a structure like the above would allow
for very simple extension on the app level, like so:
class CustomizedMyApp(object):
def get_page_class(self):
class Page(AbstractPage):
title = models.CharField(max_length=255)
return Page
AFAIK this is currently not possible. But I may be misinformed.
The main stumbling block is that an app is somehow special to Django,
specifically in that it is a directory, containing a file 'models.py'.
The name of the app directory is included in INSTALLED_APPS in your
settings.py. I guess other stuff might break as well (do urlpatterns
and reverse views work for any method, rather than functions in the
top level of a module?).
Django's idea of apps is great, because it gives us such an easy way
to start. However, I think it breaks down in the above example. Would
it be possible to rewrite Django so that the INSTALLED_APPS way of
working would remain available, but would be a shortcut/facade to a
manual registration of the various parts that entail an app? I think
so.
Hope to hear from you all, I will play a bit with some code in the
meantime.
p.s.
Thinking about it some more: it's not weird at all that apps are hard
to extend, since apps are limited to being modules. Modules are not a
unit for extension.
Klaas van Schelven
minimal amount of weird tricks (an empty models.py, plugging in some
app_labels in the right places).
Of course: without the tricks it would be even better.
http://bitbucket.org/vanschelven/extendible_app_experiment
A few notes:
1. app_a vs. my_app_a is still ugly
2. my approach breaks 'automatic' overriding of templates 'the django
way'; extendible_app/templates/ must be added to the templates path
'manually' (since "extendible_app" is not in settings.py, only
"my_extendible_app" is). Also ugly: my_extendible_app has a template
dir containing 'extendible_app' (not symmetric).
3. some verbosity is required in the example. I believe this could be
easily factored out. For example:
* the lazyness of get_xxx_class could be factored out (as a decorator,
as a lazy property, etc)
I'll try it somewhat more seriously on some real code in the coming
days. Will keep you posted.
Elim Qiu
Yours,
Russ Magee %-)
Russell Keith-Magee
<klaasvan...@gmail.com> wrote:
>> I'm yet to see a genuine case of (a) -- every time I've seen (a), it's
>> really masked version of (b), (c) or (d).
>
> I disagree. I think there are very specific problems, that has not
> been adressed by any of the responses above.
> Let's focus on one for now: how to extend models in any given app.
>
> As stated in the original mail, there are at least two problems with
> this in Django:
> 1. How to provide the models in the first place? Everything that's not
> abstract is hard to extend; providing abstract models feels really
> awkward.
If a model isn't abstract but needs to be, then that speaks to
applications not being built to be as extensible as possible. Django
hasn't always had abstract models -- they were added just before 1.0
landed (IIRC).
Unfortunately, it's very difficult to migrate from a concrete model
back to an abstract model, so it's not easy to introduce an abstract
model into an existing app without breaking backwards compatibility.
There are also complications around handling foreign keys to abstract
models. This is an area where there is room for improvement; there
have been some very recent discussions on this very topic. Alex Gaynor
discussed some of the issues and approaches in his recent DjangoCon
presentation [1].
[1] http://djangocon.blip.tv/file/4108781/
> 2. How to tie in the models into the views? Long view parameter lists
> that are used from urls are ugly and repetitive. Moreover: it may not
> be possible to solve the entire problem this way (think admins, think
> forms).
I'm afraid I have no idea what you're talking about here. HTTP is a
stateless protocol. The only reliable way you have to pass around
state is the arguments in a URL (technically, the request, so you can
use cookies too). This is one of the core *features* of the HTTP
protocol. Django provides a way (technically, several ways) of mapping
URLs to specific callables, and provides a way to hierarchically
organize those callables based on argument structure.
I fail to see either the 'ugly', or the set of problems that can't be
solved. And it's somewhat spurious to say that "admin and forms" are
two things that Django can't do when Django ships with a forms
library, and Django's automated admin is one of the major initial
selling points of the framework.
I think you'll find you are. Why is what you are describing not
possible right now? Django is, at the end of the day, a machine for
turning a HTTP request into a function call, and turning the response
for that function call into a HTTP response. If you want to use a
particular class structure to compose that logic, go right ahead.
There's nothing in Django stopping you.
This may mean that you discover some magnificent class structure for
composing full websites from parts. What will convince me isn't a
dozen lines in an email -- it's a fully fledged project, with the
complexity of something like a CMS.
And it isn't me you need to convince anyway -- it's the wider
community. Trust me -- if you find the magic formula for perfect
reusability, my opinion won't matter. You'll be beating away
contributors with a stick. :-)
> The main stumbling block is that an app is somehow special to Django,
> specifically in that it is a directory, containing a file 'models.py'.
> The name of the app directory is included in INSTALLED_APPS in your
> settings.py. I guess other stuff might break as well (do urlpatterns
> and reverse views work for any method, rather than functions in the
> top level of a module?).
*This* is the stumbling block? Honestly? Can you please describe an
occasion where the fact that you need to predeclare the application
modules (and it is the *module*, not the directory) that you intend to
use has been an impediment to reusability? I fail to see how this is
any different to complaining that Python requires you to put "import
os.path" before you can do directory path manipulations.
> Django's idea of apps is great, because it gives us such an easy way
> to start. However, I think it breaks down in the above example. Would
> it be possible to rewrite Django so that the INSTALLED_APPS way of
> working would remain available, but would be a shortcut/facade to a
> manual registration of the various parts that entail an app? I think
> so.
You seem to be implying that your example is fundamentally constrained
by the existence of INSTALLED_APPS. Either I'm missing something, or
you haven't explained some subtle detail, because I simply don't see
the limitation.
> Hope to hear from you all, I will play a bit with some code in the
> meantime.
>
> p.s.
> Thinking about it some more: it's not weird at all that apps are hard
> to extend, since apps are limited to being modules. Modules are not a
> unit for extension.
Why not?
If you want to write a custom Email class, you don't put the code in
the Python standard library. You write a "my.custom.email" module, and
put an extension class in it. Why can't a module be used to organize
extensions?
Yours,
Russ Magee %-)
Steve Holden
> Lastly, I can't deny that reusability has it's price. Engineering an
> application to be reusable takes more effort and planning than
> building an application as a once-off.
Well, Unix certainly came a long way with this philosophy
> The aim is that if you take the effort once to make your application
> reusable, the engineering payoff will happen quickly -- hopefully on
> your second build. However, if you have no intention of using a
> component a second time, then there isn't a whole lot of reason to
> spend the effort making it reusable.
True to a certain extend, but it takes a certain amount of experience to
recognize a true YAGNI.
regards
Steve
--
DjangoCon US 2010 September 7-9 http://djangocon.us/
Klaas van Schelven
wrote:
>
> >> I'm yet to see a genuine case of (a) -- every time I've seen (a), it's
> >> really masked version of (b), (c) or (d).
>
> > I disagree. I think there are very specific problems, that has not
> > been adressed by any of the responses above.
> > Let's focus on one for now: how to extend models in any given app.
>
> > As stated in the original mail, there are at least two problems with
> > this in Django:
> > 1. How to provide the models in the first place? Everything that's not
> > abstract is hard to extend; providing abstract models feels really
> > awkward.
>
> If a model isn't abstract but needs to be, then that speaks to
> applications not being built to be as extensible as possible. Django
> hasn't always had abstract models -- they were added just before 1.0
> landed (IIRC).
>
> Unfortunately, it's very difficult to migrate from a concrete model
> back to an abstract model, so it's not easy to introduce an abstract
> model into an existing app without breaking backwards compatibility.
> There are also complications around handling foreign keys to abstract
> models. This is an area where there is room for improvement; there
> have been some very recent discussions on this very topic. Alex Gaynor
> discussed some of the issues and approaches in his recent DjangoCon
> presentation [1].
>
> [1]http://djangocon.blip.tv/file/4108781/
I'll try to make my case again with the insight I've gained over the
last couple of mails.
Let me reiterate that a large part of the reusable app problem has
been solved. However, the following has not:
Reusing a small set of functionality around some typical, extensible
data structures.
Again, examples:
* A reusable address book
* A reusable "who's at the office" calendar
* A reusable small library tool (which books are in our library)
* A reusable time tracking tool
In any of the above, the data structure is typical and revolves around
a few related model classes. The views revolve around common CRUD for
the objects, and around "whatever makes the app special". Special
stuff is, in the case of time tracking: various ways of inputting time
and searching for it. In the case of an address book: displaying
relevant lists, detail views.
However, pretty much every actual use of the reusable app will involve
some additions to the models. Contacts in the USA? We'll add a State
field to your address model. Your particular use case requires us to
categorize time-lines in a specific manner? We'll slap a foreign key
on it to some "TimeLineCategory".
The key is, that this new extended address, time-line or whatever
should become the idea of an address, timeline or whatever from the
perspective of the app.
So, we run into a few problems:
* How do we extend the models from the original extendible
application?
For this to work, we need to provide hooks in the application for
extendiblity. In the case of models, such a hook would be twofold:
1. an abstract model to extend to provide a baseline (a default with
the minimum amount of fields)
2. something overridable that is referred to by the rest of our app,
providing the concrete instance of [1] in the plain vanilla app, and
the concrete instance plus extensions in our extended app. (the actual
hook)
* how do we refer to ourselves from the various parts of the
extendible application?
Self-referral with overrides is a solved problem in the object
oriented world ("inheritance"). However, it's impossible in Django's
standard way of organizing apps, because we all use modules (not
classes or instances) for the various parts of the app ("models.py",
"views.py" etc).
It is not possible to pass context into a module (other than monkey
patching it)
Every views.py begins with a bunch of imports from the associated
models.py, tying the views into those particular app's models. My
alternative would be like so:
class View(...):
def page_detail(self, request, pk=None):
page = self.models.Page.objects.get(pk=pk)
return render_to_response(.... page ...)
For models:
class Models(...):
def get_Page(self):
class Page(models.Model):
#abstract
category = models.ForeignKey(self.models.Category)
return Page
Page = property(get_Page)
(The above is a bit less elegant than the views example because models
are classes and the following therefor does not work:)
class Models(...)
class Page(self, model.Model):
...
category = models.ForeignKey(self.models.Category)
If anyone has an example of how to do this without a property I'd be
much obliged.
As said, I'll try to come up with something in the evening hours that
allows for this and still feels Django-esque. (Which should make
conversion of my own apps easier as well).
>
> > 2. How to tie in the models into the views? Long view parameter lists
> > that are used from urls are ugly and repetitive. Moreover: it may not
> > be possible to solve the entire problem this way (think admins, think
> > forms).
>
> I'm afraid I have no idea what you're talking about here. HTTP is a
> stateless protocol. The only reliable way you have to pass around
> state is the arguments in a URL (technically, the request, so you can
> use cookies too). This is one of the core *features* of the HTTP
> protocol. Django provides a way (technically, several ways) of mapping
> URLs to specific callables, and provides a way to hierarchically
> organize those callables based on argument structure.
>
> I fail to see either the 'ugly', or the set of problems that can't be
> solved. And it's somewhat spurious to say that "admin and forms" are
> two things that Django can't do when Django ships with a forms
> library, and Django's automated admin is one of the major initial
> selling points of the framework.
>
1. Those that are passed in via urls.py from the request path. Love
'em, couldn't live without 'em. This is what you're talking about.
2. Those we add (with usefull defaults) to make our app "reusable".
What I was saying, is that I consider these in particular as unusable
in the general sense. If I subclass the idea of address in an address
book app (to add some field), would it make sense to refer to the
subclassed address class in every call to every single view function?
I'd say no. I'd rather say "give me my views where each view
understands that it should use the improved address class).
back.
>
> This may mean that you discover some magnificent class structure for
> composing full websites from parts. What will convince me isn't a
> dozen lines in an email -- it's a fully fledged project, with the
> complexity of something like a CMS.
>
> And it isn't me you need to convince anyway -- it's the wider
> community. Trust me -- if you find the magic formula for perfect
> reusability, my opinion won't matter. You'll be beating away
> contributors with a stick. :-)
Bear"/"Rubber Duck".
>
> > The main stumbling block is that an app is somehow special to Django,
> > specifically in that it is a directory, containing a file 'models.py'.
> > The name of the app directory is included in INSTALLED_APPS in your
> > settings.py. I guess other stuff might break as well (do urlpatterns
> > and reverse views work for any method, rather than functions in the
> > top level of a module?).
>
> *This* is the stumbling block? Honestly? Can you please describe an
> occasion where the fact that you need to predeclare the application
> modules (and it is the *module*, not the directory) that you intend to
> use has been an impediment to reusability? I fail to see how this is
> any different to complaining that Python requires you to put "import
> os.path" before you can do directory path manipulations.
various levels. You're right.
> > Django's idea of apps is great, because it gives us such an easy way
> > to start. However, I think it breaks down in the above example. Would
> > it be possible to rewrite Django so that the INSTALLED_APPS way of
> > working would remain available, but would be a shortcut/facade to a
> > manual registration of the various parts that entail an app? I think
> > so.
>
> You seem to be implying that your example is fundamentally constrained
> by the existence of INSTALLED_APPS. Either I'm missing something, or
> you haven't explained some subtle detail, because I simply don't see
> the limitation.
>
> > Hope to hear from you all, I will play a bit with some code in the
> > meantime.
>
> > p.s.
> > Thinking about it some more: it's not weird at all that apps are hard
> > to extend, since apps are limited to being modules. Modules are not a
> > unit for extension.
>
> Why not?
>
> If you want to write a custom Email class, you don't put the code in
> the Python standard library. You write a "my.custom.email" module, and
> put an extension class in it. Why can't a module be used to organize
> extensions?
Modules cannot be extended because they cannot refer to parts of their
extended self from the original. See above.
Klaas
>
> Yours,
> Russ Magee %-)
Russell Keith-Magee
<klaasvan...@gmail.com> wrote:
> On Sep 23, 2:01 am, Russell Keith-Magee <russ...@keith-magee.com>
> wrote:
>
> So, we run into a few problems:
> * How do we extend the models from the original extendible
> application?
> For this to work, we need to provide hooks in the application for
> extendiblity. In the case of models, such a hook would be twofold:
> 1. an abstract model to extend to provide a baseline (a default with
> the minimum amount of fields)
> 2. something overridable that is referred to by the rest of our app,
> providing the concrete instance of [1] in the plain vanilla app, and
> the concrete instance plus extensions in our extended app. (the actual
> hook)
Sure. Abstract models form the first part of this; the 'reference to
the concrete instance' is a known problem -- the most obvious example
of this surfacing is defining a custom User class to use in
contrib.auth. Alex provides one possible solution (join models) in his
talk; another (LazyForeignKey) was raised by Eric Florenzano in his
keynote [1]. However, this is still very much an open area of debate.
Any suggestions are welcome.
[1] http://djangocon.blip.tv/file/4112452/
Sure. There's at least two approaches. The first -- a method based
approach -- is used by contrib.auth (amongst others):
def my_view(request, arg1, arg2, klass=Page):
objects = klass.objects.filter(arg1=arg1, arg2=arg2)
...
The second is the class based approach, used by contrib.admin (and
others), which is to define a class-based view, and provide methods
like get_objects() that you can subclass to perform whatever behaviour
you need.
> Views.py methods generally have two types of parameters:
> 1. Those that are passed in via urls.py from the request path. Love
> 'em, couldn't live without 'em. This is what you're talking about.
> 2. Those we add (with usefull defaults) to make our app "reusable".
> What I was saying, is that I consider these in particular as unusable
> in the general sense. If I subclass the idea of address in an address
> book app (to add some field), would it make sense to refer to the
> subclassed address class in every call to every single view function?
> I'd say no. I'd rather say "give me my views where each view
> understands that it should use the improved address class).
Sure. And that's why you use class-based views. That's why Django's
admin is a class.
That's also why Django is in the process of getting a class-based view
framework - again, from DjangoCon, see Ben Firshman's talk [2]
[2] http://blip.tv/file/4109272
Yours,
Russ Magee %-)
Yo-Yo Ma
Do you think a round table discussion in a real person context with
whiteboards and the best of the bunch (ie, at Django con or similar
event) would be a good time/place to re architecture of the
abstraction layers, perhaps to address some of the concerns brought up
in the slide show by Eric? By abstraction layers I don't just refer to
the "app" (which I don't see to be a problem as much as others do,
although I don't like how so many apps have models in them). I'm also
referring to things like; putting global functionality into urls.py,
tight coupling of built-in apps, etc.
Michael
On Sep 23, 5:40 am, Russell Keith-Magee <russ...@keith-magee.com>
wrote:
>
Russell Keith-Magee
> Hey Russell,
>
> Do you think a round table discussion in a real person context with
> whiteboards and the best of the bunch (ie, at Django con or similar
> event) would be a good time/place to re architecture of the
> abstraction layers, perhaps to address some of the concerns brought up
> in the slide show by Eric? By abstraction layers I don't just refer to
> the "app" (which I don't see to be a problem as much as others do,
> although I don't like how so many apps have models in them). I'm also
> referring to things like; putting global functionality into urls.py,
> tight coupling of built-in apps, etc.
Sure. However, Djangocon 2010 finished a week ago, so you're going to
have to wait until next year before we have the next serious
opportunity to do this.
It also helps to have a concrete proposal on the table, rather than
just a bunch of abstract "hey this sucks" bullet points.
Yours,
Russ Magee %-)
Klaas van Schelven
> Sure. However, Djangocon 2010 finished a week ago, so you're going to
> have to wait until next year before we have the next serious
> opportunity to do this.
>
how many of the "best of the bunch" are going to make it to
Amsterdam :-)
Venkatraman S
http://djangocon.eu/ is going to be in the spring of 2011. Let's see
how many of the "best of the bunch" are going to make it to
Amsterdam :-)
How was it?
-V-
http://twitter.com/venkasub
Carles Barrobés
> modifications mentioned above:
> Authorization:
> Django has a beautiful mechanism for authorization, which is using
> decorators (@may_do_a_b_or_c) directly in the views.py.
> Tempting, but will not be modifyable in any way once you lift the app.
> (because the decorator will become tied to the view function). Other
> mechanisms are not presented (in a standardized way). If anyone has
> ideas about this I'd love to discuss them.
people forget the fact that it is only syntactic sugar. You do not
have to use it, and specifically for the purpose you describe it is a
really bad idea. When dealing with authorization, using this syntax
ties you up and doesn't give the convenience of having authorisation
declared in a more centralised location.
An alternative approach I use is to centralise authorization is to not
use the @ decorator syntax, and use the decorator directly in urls.py.
E.g. from one of my apps:
-----------
from django.conf.urls.defaults import *
from babystats import views
from django.contrib.auth.decorators import login_required
urlpatterns = patterns('',
url(r'^$',
login_required(views.user_babies),
name='babystats_user_babies'),
url(r'^(?P<id>\d*)/$',
login_required(views.show_baby),
name = 'babystats_baby_detail'),
#etc...
)
-----------
This is a start. You can always replace the app's urls with your own
and apply different decorators/access controls. It just requires you
to include your views as callables instead of strings.
If you want to do it even more globally, I don't think it should be
too hard to write a function to replace
django.conf.urls.defaults.include that takes a decorator as a second
parameter and applies it to all the included views, e.g. it could be
used:
(r'^path/', include_decorated('appxyz.urls', my_decorator)),
Carles
Delusional Insanity
It's just a matter of making the app reusable and flexible enough to adapt modifications.