admin code is really really "not best practice"

[Robert Wittams]

And that wasn't my initial subject line...
I have been attempting to work out what is going on with the admin code
and manipulators.

These are just a few of the features of the admin code:

1) 154 column lines, many of them unreadable. Huge functions, ditto.

2) Writing out templates as massive strings that then get parsed and
applied. This smacks of document.write()-ism, and I can only assume it
arose before the template system was in its current very useful state.

3) Work arounds strewn about the code to compensate for the deficiencies
of manipulators/formfields when applied to particular field types.

My proposed solutions are:

1) add in some newlines!
2) change the whole output philosophy of the admin code to one of
templates + extensive tag libraries. Minimal fruity output logic in the
view code.
3) fix the manipulators/formfields so no work arounds are required.
Virtual functions exist for a reason.

The problem is that this amounts to quite the rewrite. But I don't think
that django is going to be able to rapidly evolve if people have to fix
the admin in its current state. I hope that this is something all
djangonistas would be happy to see/ work on too.

3) is what started me on this odyssey when I realised that workarounds
were needed for all but the simplest fields, to poke around in data
dictionaries etc. At the moment, the admin views serve almost as a
"false advertisement", because they hack around the framework so much to
achieve their impressive aesthetic. We need to eat our own dogfood.

Hopefully the end result of this will be
*) admin view with understandable, maintainable code + same or better
user experience, which also provides an effective example of how to use
the django framework without screwing yourself
*) formfields/ manipulators that work without hacks in view functions
*) more reusable stuff out of the admin views, so people don't get so
much of a "Oh, now I need to write my /public/ views from scratch?" feeling.
*) A much less painful time of it when it comes to trying to i18nize the
admin.

Thoughts?

Robert

[Adrian Holovaty]

On 9/19/05, Robert Wittams <rob...@wittams.com> wrote:
> My proposed solutions are:
>
> 1) add in some newlines!
> 2) change the whole output philosophy of the admin code to one of
> templates + extensive tag libraries. Minimal fruity output logic in the
> view code.
> 3) fix the manipulators/formfields so no work arounds are required.
> Virtual functions exist for a reason.

Hey Robert,

That sounds good to me. Patches are welcome!

Adrian

--
Adrian Holovaty
holovaty.com | djangoproject.com | chicagocrime.org

[mrelectron]

hi robert:

<snip>

"the admin views serve almost as a "false advertisement", because they
hack around the framework so much to achieve their impressive
aesthetic"

</snip>

that is quite worrying, and if a rewrite is the only solution,
consensus needs to be obtained from the community on the best way to
proceed. i am very happy to help.

thanks
mark

ps. you may want to put your letter on django-users, as it hints at a
fairly fundamental shortcoming in the framework.

[Adrian Holovaty]

On 9/19/05, mrelectron <mark.a...@gmail.com> wrote:
> that is quite worrying, and if a rewrite is the only solution,
> consensus needs to be obtained from the community on the best way to
> proceed. i am very happy to help.

I think you're misunderstanding. What Robert said was that the code
that generates admin views is complex and hard to understand. It
really has no effect on use of Django unless you plan to edit the
admin-view code for some reason.

[Andreas]

> I think you're misunderstanding. What Robert said was that the code
> that generates admin views is complex and hard to understand. It
> really has no effect on use of Django unless you plan to edit the
> admin-view code for some reason.

Don't you think it is a little worrying that an interface like the one
used in the admin view seems to be only achievable by, as Robert wrote,
"hacking around the framework"? If it is achievable within the limits
of the framwork, without workarounds -- who should be able to figure it
out if not the authors of the framework?

I don't think this is just an issue for those planning to edit the
admin-view code.

Andreas

[Robert Wittams]

Well, I'm not an author of the framework, but the admin code is very
hard to understand (imo). I'm sure the authors understand it fine, but
for open source to instill confidence clarity is key. The admin view
should be a great example to django users, but unfortunately right now
it isn't.

I do not believe that there is any fundamental limitation to the design
of django that would stop an interface very very similar to the admin
site using only "best practices". I do believe that there are a fair
number of deficiencies in the implementation currently. And you are
right - these deficiencies have certainly affected my project, and thus
I have been submitting tickets and patches, and will continue to do so.

My point was really that the admin view presents a really nice interface
by default, and it is a real shame that it doesn't achieve this by
utilising the standard recommended django bits. If it is "impossible" to
get something like the admin view without hacking, then the best course
of action is to make it possible.

I don't want to alarm anyone or put anyone off django. It is a great
framework, and that is why I'm using it. I just think we need to be
clear that it is currently not possible to make stuff similar to the
admin views without a fair bit of hackery. I hope to obviate this.

Robert

[Manuzhai]

> I don't want to alarm anyone or put anyone off django. It is a great
> framework, and that is why I'm using it. I just think we need to be
> clear that it is currently not possible to make stuff similar to the
> admin views without a fair bit of hackery. I hope to obviate this.

Could you explain what sort of hackery is currently used in the admin
interface stuff that is not currently easily available from the
"normal" framework?

Regards,

Manuzhai

[Jacob Kaplan-Moss]

On Sep 19, 2005, at 11:39 AM, Robert Wittams wrote:
> Well, I'm not an author of the framework, but the admin code is very
> hard to understand (imo). I'm sure the authors understand it fine, but
> for open source to instill confidence clarity is key. The admin view
> should be a great example to django users, but unfortunately right now
> it isn't.

No, it shouldn't -- the admin views are incredibly complex because
the admin interface is incredibly complex. Public views are almost
always going to be much much simpler. Even if the admin views get
cleaned up -- which they should be, of course -- they still won't be
a very good example. You wouldn't tell someone to read the Linux
kernel source to learn how to program in C; the admin views should be
thought of the same way.

> My point was really that the admin view presents a really nice
> interface
> by default, and it is a real shame that it doesn't achieve this by
> utilising the standard recommended django bits. If it is
> "impossible" to
> get something like the admin view without hacking, then the best
> course
> of action is to make it possible.
>
> I don't want to alarm anyone or put anyone off django. It is a great
> framework, and that is why I'm using it. I just think we need to be
> clear that it is currently not possible to make stuff similar to the
> admin views without a fair bit of hackery. I hope to obviate this.

Can you explain in more detail how you feel the admin uses "hacks"?
There's nothing the admin uses that you can't use in the public
views; it's just that the admin interface is very complex, and so the
code is very complex.

As far as I'm concerned, cleaning up the admin code would be a good
idea from the "code smell" point of view, but would otherwise give
very little practical benefit. Fact is, the admin interface works,
and the work required to clean it up without missing any of the small
subtleties will be intense. If the admin interface was actually
broken, I'd feel differently, but at the moment there are many more
pressing concerns than how some of the code "looks".

That said, patches are always welcome!

Jacob

[Robert Wittams]

Jacob Kaplan-Moss wrote:
>
>
> On Sep 19, 2005, at 11:39 AM, Robert Wittams wrote:
>
>> Well, I'm not an author of the framework, but the admin code is very
>> hard to understand (imo). I'm sure the authors understand it fine, but
>> for open source to instill confidence clarity is key. The admin view
>> should be a great example to django users, but unfortunately right now
>> it isn't.
>
>
> No, it shouldn't -- the admin views are incredibly complex because the
> admin interface is incredibly complex. Public views are almost always
> going to be much much simpler. Even if the admin views get cleaned up
> -- which they should be, of course -- they still won't be a very good
> example. You wouldn't tell someone to read the Linux kernel source to
> learn how to program in C; the admin views should be thought of the
> same way.

I can't help thinking that this boils down to : It is complex now, so it
must be complex. I honestly do not feel that the admin code will
necessarily be horrific.

>> My point was really that the admin view presents a really nice interface
>> by default, and it is a real shame that it doesn't achieve this by
>> utilising the standard recommended django bits. If it is "impossible" to
>> get something like the admin view without hacking, then the best course
>> of action is to make it possible.
>>
>> I don't want to alarm anyone or put anyone off django. It is a great
>> framework, and that is why I'm using it. I just think we need to be
>> clear that it is currently not possible to make stuff similar to the
>> admin views without a fair bit of hackery. I hope to obviate this.
>
>
> Can you explain in more detail how you feel the admin uses "hacks"?
> There's nothing the admin uses that you can't use in the public views;
> it's just that the admin interface is very complex, and so the code is
> very complex.

The first problem is the inexplicable concatenation of templates. This
is effectively layering two layers of logic. This makes the code hard to
follow - and it is very inconsistent as to where a particular bit of
logic goes. Even making the AST of the template dynamically would make
more sense (but I don't think it is necessary or desirable).

The parts which I consider to be real hacks are the bits which just
plain don't work in generic views, and require huge messing around in
the view functions. Eg select fields do not pick up their values (
http://code.djangoproject.com/ticket/338 ). The flatten_data stuff is
essential for certain things to work, but is only in the admin. There is
all kinds of special casing that is done, imo in the wrong place.

TBH, my point is not really fixing the admin. My point is making
manipulators and formfields work out of the box. This also needs the
admin to be fixed (as they seem fairly intertwined atm) so I started
working on that. Whilst doing so, I couldn't help thinking that the
whole thing needed a spruce up, which I'm willing to do.

Rob

[Ian Maurer]

On 9/19/05, Jacob Kaplan-Moss <ja...@jacobian.org> wrote:
>
>
> On Sep 19, 2005, at 11:39 AM, Robert Wittams wrote:
> > Well, I'm not an author of the framework, but the admin code is very
> > hard to understand (imo). I'm sure the authors understand it fine, but
> > for open source to instill confidence clarity is key. The admin view
> > should be a great example to django users, but unfortunately right now
> > it isn't.
>
> No, it shouldn't -- the admin views are incredibly complex because
> the admin interface is incredibly complex. Public views are almost
> always going to be much much simpler. Even if the admin views get
> cleaned up -- which they should be, of course -- they still won't be
> a very good example. You wouldn't tell someone to read the Linux
> kernel source to learn how to program in C; the admin views should be
> thought of the same way.

I think I know where people are coming from. When I first started
playing with Django, I was frustrated that there were no examples on
how to leverage the admin into my webapp. But, once I looked at the
code and read some of the tutorials it dawned on me that the admin
stuff was NOT to leveraged by my webapps. Once I accepted this, I
became very content with letting the admin code stay behind the
curtain and concentrated on building my site using the standard set of
tools.

I think most new developers to Django will go through this thought
process and either accept things the way they are or get frustrated
and move on to another framework.

So, I offer 3 suggestions:

1. Make it more clear that the admin tool is really a separate
applications and cannot really be leveraged into new apps easily. This
should be done in that first admin tutorial.

2. I really think the admin tool is very slick and it really is a
differentiator for Django. I also understand some serious wizardry and
magic went into making it, so once 1.0 is finished and in the can, I
would suggest looking at the Admin tool as the guide for future
enhancements or versions of the software. I would love to see the key
admin components (such as the auto data tables, filters, search, and
forms) abstracted in such a way that any app could use them easily. To
me, this would be another layer of Rapid App Development on top of
generic views that would make this framework even more productive
(especially for quick-and-dirty CRUD sites).

3. This last item is really out there... But, to me it would be nice
if the admin tool could be decoupled from the ORM library (using
interfaces, adapters and pyprotocols?), that way people could create
adapters for SQLObject or any other ORM tool and they wouldn't lose
out on using the Admin console. While I am very content with the ORM
library, a lot people might not use Django if it cannot play nice with
their existing models.

Sorry for contributing just ideas at this time... I hope I can start
contributing patches as I get comfortable with the actual code behind
the framework.

thanks again for the great framework,
ian

[Jacob Kaplan-Moss]

On Sep 19, 2005, at 1:42 PM, Robert Wittams wrote:
> The parts which I consider to be real hacks are the bits which just
> plain don't work in generic views, and require huge messing around in
> the view functions. Eg select fields do not pick up their values (
> http://code.djangoproject.com/ticket/338 ). The flatten_data stuff is
> essential for certain things to work, but is only in the admin.
> There is
> all kinds of special casing that is done, imo in the wrong place.

Yes; this should be considered a bug, and is on my list to fix.

Jacob

[HBTaylor]

> Can you explain in more detail how you feel the admin uses "hacks"?
> There's nothing the admin uses that you can't use in the public
> views; it's just that the admin interface is very complex, and so the
> code is very complex.

In my opinion, I don't disagree with the complexity argument. For me,
it is more about admin eating Django's own dog food. For example, there
are two tickets which come to mind
(http://code.djangoproject.com/ticket/337 and
http://code.djangoproject.com/ticket/338) which deal with things which
work correctly for firlds within admin but which don't for generic
views. There is a patch available for at least one of them, and the
discussion for these indicates that the admin classes are doing some
"magic" (note I didn't say "hacks"!) that isn't present to the
non-admin world. As I read this thread, what I read into some of the
messages was that it would be nice if the admin interface used
capabilities available to, for instance, generic views, instead of
having to replicate some of the same magic already present in the admin
views.

As for the complexity of admin, there are INDEED places where it is
more complex than non-admin views. Enumerating the available tables,
available fields, etc., is beyond what a lot of non-admin views need.
However, if I'm going to place a field on a page (form field or
otherwise), it is probably good for adoption and expansion of the
framework to make it as little code as possible in the view and to make
the code in the underlying classes as simple and straight-forward as
possible.

H.B.

[Eugene Lazutkin]

I would love to see an application, which looks at my models and provide
different Admin UI --- current Admin provides all necessary basic
functionality but it would be nice to extend it even more for some custom
apps. As soon as I have it, I'll compare it with existing Admin application
and form my opinion.

Most probably it would take a lot of time to develop it => I think it should
be an independent project: there are other more pressing issues in Django to
be fixed. But I am all for such parallel project. Regardless of final
outcome it will generate a list of Django public interface deficiencies,
which can be fixed.

The problem is availability of resources for such thing.

Thanks,

Eugene

"Ian Maurer" <ima...@gmail.com> wrote in
message news:adc55a7805091...@mail.gmail.com...

[Jacob Kaplan-Moss]

On Sep 19, 2005, at 3:21 PM, HBTaylor wrote:
> For example, there are two tickets which come to mind
> (http://code.djangoproject.com/ticket/337 and
> http://code.djangoproject.com/ticket/338) which deal with things which
> work correctly for firlds within admin but which don't for generic
> views. There is a patch available for at least one of them, and the
> discussion for these indicates that the admin classes are doing some
> "magic" (note I didn't say "hacks"!) that isn't present to the
> non-admin world.

That's only somewhat correct -- basically, the flatten/unflatten
functions that are now in the admin need to be refactored to be
methods of the manipulator, that's all. It's a bug, not a case of
"eating our own dog food". I'm really sorry this bug has taken so
long to fix; I've been turning in 70 hour weeks at work, and so I've
been less than excited about more Django hacking. I'll do my best to
get those fixes rolled in this week, but -- as always -- patches will
help.

> As for the complexity of admin, there are INDEED places where it is
> more complex than non-admin views. Enumerating the available tables,
> available fields, etc., is beyond what a lot of non-admin views need.
> However, if I'm going to place a field on a page (form field or
> otherwise), it is probably good for adoption and expansion of the
> framework to make it as little code as possible in the view and to
> make
> the code in the underlying classes as simple and straight-forward as
> possible.

I agree 100%.

Jacob

[Jacob Kaplan-Moss]

On Sep 19, 2005, at 3:56 PM, Eugene Lazutkin wrote:
> I would love to see an application, which looks at my models and
> provide
> different Admin UI --- current Admin provides all necessary basic
> functionality but it would be nice to extend it even more for some
> custom
> apps. As soon as I have it, I'll compare it with existing Admin
> application
> and form my opinion.

I'd love to see it!

Jacob

[Eugene Lazutkin]

"As soon as I have it" --- it doesn't mean that I am going to do it. It
means "as soon as I get my hands on such implementation", I'll compare it
with existing one and make a decision, which one to use.

And, yes, I'd love to see it too! :-)

Thanks,

Eugene

"Jacob Kaplan-Moss" <ja...@jacobian.org> wrote in
message news:6A49F8B0-C90A-47A0...@jacobian.org...

[Robert Wittams]

I'm tracking my changes here:
http://code.djangoproject.com/ticket/535

Currently it is at a very early stage so expect regressions.

Robert