Splitting out admin functionality

[tWoolie]

I'd like to propose a project to split out the table/list
functionality and the filtering functionality in the django admin into
a seperate contrib app. This is really core functionality which is
lacking from django (in terms of being locked to the admin app).
I believe this would be beneficial because currently there are a lot
of people who love using the admin interface but find building the
same functionality into their own applications daunting at best. This
leads a lot of people to try to abuse the admin app by allowing
regular users into there.

I propose moving a lot of the django-tables2 functionality into a
contrib app and augumenting it with filterspec. Then the admin can use
this app, or any custom subclass of Table/Filter. Currently, django-
tables2 is fantastic but has no support for filterspecs which would be
super super helpful, and would benefit from the level of [url]
abstraction that d-t2 provides.

I'm posting to this group to gather feedback and ask if anyone would
actually like to see this happen. I believe it's something i'd love to
contribute but only if the django devs would be interested in
incorporating it.

Thomas

[Yo-Yo Ma]

+1

I spent this Saturday morning implementing ad hoc table sorting in a
ListView's get_queryset. It felt very wrong, to say the least. An
abstracted sorting and filtering framework, along with a few of the
admin-ish features that are already in core (inline model formsets, et
al) provide some very powerful tools for those lone rangers (like
myself) who aren't able to use the admin for a project, but still need
to be able to emulate some or all of the admin's functionality.

I would be very interested in this, and feel it could add a lot of
value to the framework. I'd be willing to provide code, with some
guidance from members of the core team.

On Oct 9, 10:20 pm, tWoolie <thomas.woolf...@student.adelaide.edu.au>
wrote:

[Yo-Yo Ma]

To further note; while I second the OP's request, in its abstract, my
approach wouldn't be to add another contrib app. It would be to
implement a fairly low level API. Reuse of a third party app, or
abstracting some of the existing admin code, or any combination
thereof would be sufficient, but this should surely be in core.

[Russell Keith-Magee]

Broadly speaking, I agree with the motivation; I'm not 100% sure I
agree about the implementation plan.

Getting "admin-like" functionality into the mainstream was one of the
prime motivations behind the introduction of class-based generic
views. Django's admin contains a number of neat features (like
fieldsets, views with inline formsets, filtering and searching) that
can be very easily implemented as a set of class-based mixins or
extensions. I've got code in private repositories that implement some
of these features; I'd like to find the time to clean them up as
generic utilities.

To my mind, the end goal should be to replace most of the internals of
Django's admin with relatively light customizations of generic
form/list functionality. Admin's filters should be a specialization of
a generic capability, not something specific to the admin.

I haven't seen django-tables2 before, so I can't comment on whether
the implementation is suitable for core as-is. However, my immediate
reaction is that if we add this, it won't be as a contrib app. This is
a piece of core functionality of generic views, not an "optional
library implementing a defacto implementation of a common pattern"
(this is the yardstick against we measure contrib proposals).

I'd also say that django-tables2, by itself, appears to be only one
small part of a much larger solution. I am aware of several efforts to
build 'admin-like' view and form functionality on top of the CBV
framework introduced in 1.3. I'd like to see a lot more consolidation
in these projects before anything is proposed for trunk.

The approach taken by django-messages should be the model here. When a
new messages framework was proposed for Django in 1.2, an effort was
made to find out all the features that were desirable, and then pull
together all those features into a third party library that borrowed
code from all the available implementations. The same approach should
be taken here -- find out all the features that we want to have in
generic views (e.g., formsets, fieldsets, filtering, ordering, table
rendering), build an amalgam library that takes the best parts from
all the available 3rd party libraries, and develop that into a single
patch that can be applied to trunk.

Yours,
Russ Magee %-)

[tWoolie]

> Getting "admin-like" functionality into the mainstream was one of the
> prime motivations behind the introduction of class-based generic
> views. Django's admin contains a number of neat features (like
> fieldsets, views with inline formsets, filtering and searching) that
> can be very easily implemented as a set of class-based mixins or
> extensions. I've got code in private repositories that implement some
> of these features; I'd like to find the time to clean them up as
> generic utilities.

I think we all have code like this sitting around. It would be great
to standardise this code into a common package/platform

> To my mind, the end goal should be to replace most of the internals of
> Django's admin with relatively light customizations of generic
> form/list functionality. Admin's filters should be a specialization of
> a generic capability, not something specific to the admin.

Sorry if i wasn't clear but this is exactly what i want to do: namely
take the magic out of admin, and turn the admin into a thin wrapper
over standardised tables/filters

> I haven't seen django-tables2 before, so I can't comment on whether
> the implementation is suitable for core as-is.

Think of the django forms declarative syntax, but for tables. It's
really quite neat but in need of some refactoring that can't be done
in dt2 as it would seriously break backwards compatibility.

>However, my immediate
> reaction is that if we add this, it won't be as a contrib app. This is
> a piece of core functionality of generic views, not an "optional
> library implementing a defacto implementation of a common pattern"
> (this is the yardstick against we measure contrib proposals).

The problem I see here is that while tables are awesome, they are in
no way a view. and limiting a view to having a single table, or
filterset does not sit right with me. Class based views are great, but
are not a catch-all solution.

> I'd also say that django-tables2, by itself, appears to be only one
> small part of a much larger solution. I am aware of several efforts to
> build 'admin-like' view and form functionality on top of the CBV
> framework introduced in 1.3. I'd like to see a lot more consolidation
> in these projects before anything is proposed for trunk.

The inclusion of views that have tables "baked in" is a great idea,
but really the wrapper would be so thin as to be trivial.
The point of this effort is to consolidate declarative tables/filters,
not build generic views.

> The approach taken by django-messages should be the model here. When a
> new messages framework was proposed for Django in 1.2, an effort was
> made to find out all the features that were desirable, and then pull
> together all those features into a third party library that borrowed
> code from all the available implementations. The same approach should
> be taken here -- find out all the features that we want to have in
> generic views (e.g., formsets, fieldsets, filtering, ordering, table
> rendering), build an amalgam library that takes the best parts from
> all the available 3rd party libraries, and develop that into a single
> patch that can be applied to trunk.

I'm more than happy to build this as a third party app
(better_admin?), but part of the appeal for me is to rebuild admin
without the magic,
not introduce a competing admin that nobody knows about.

As far as CBV is concerned, this is a non-starter. these features
extend beyond generic views. perhaps as a convenience after the
majority of the work is done.

[Andrew Ingram]

On 10 October 2011 13:13, Russell Keith-Magee <rus...@keith-magee.com> wrote:

The approach taken by django-messages should be the model here. When a
new messages framework was proposed for Django in 1.2, an effort was
made to find out all the features that were desirable, and then pull
together all those features into a third party library that borrowed
code from all the available implementations. The same approach should
be taken here -- find out all the features that we want to have in
generic views (e.g., formsets, fieldsets, filtering, ordering, table
rendering), build an amalgam library that takes the best parts from
all the available 3rd party libraries, and develop that into a single
patch that can be applied to trunk.

On this point, I'd like to draw attention to my own set of class-based views which provide formsets functionality, mimicking the APIs of the existing CBVs as closely as possible. The ultimate goal was to attempt to reimplement some of Django's admin as a proof-of-concept. It doesn't support any higher-level functionality like fieldsets, filtering and sorting, and there's an issue with pagination (see this ticket: https://code.djangoproject.com/ticket/16432), but it's fairly feature-complete and could be a good starting point.

- Andrew

[Carl Meyer]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Russell,

On 10/10/2011 06:13 AM, Russell Keith-Magee wrote:
> Getting "admin-like" functionality into the mainstream was one of the
> prime motivations behind the introduction of class-based generic
> views. Django's admin contains a number of neat features (like
> fieldsets, views with inline formsets, filtering and searching) that
> can be very easily implemented as a set of class-based mixins or
> extensions. I've got code in private repositories that implement some
> of these features; I'd like to find the time to clean them up as
> generic utilities.

FWIW, TemplateResponse was introduced in the same release as the CBVs,
and it allows these same generic utilities (sorting, filtering,
pagination) to be easily implemented as reusable decorators on top of
TemplateResponse-using views. Having tried both approaches in different
projects, I find the code using TemplateResponse and decorators more
concise, readable and maintainable than the equivalent using CBV mixins;
there are not nearly as many subtle ordering and what-calls-what-when
issues, method names to memorize, and extraneous super() noise.

If you do get the CBV versions cleaned-up and public, I'd be happy to
put together the equivalent in decorator style for a head-to-head
comparison :-)

I also agree with tWoolie that as much of the functionality as possible
should be implemented separate from anything that's tied to views, so
it's more easily reusable for e.g. filtering two different lists in the
same view. The decorators and/or mixins should be a relatively light
glue layer between views and some kind of Sort/Filter objects (roughly
analogous to the Paginator object).

Carl
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk6TGP0ACgkQ8W4rlRKtE2coFACffzpv6SZ621Qe0TARJlNYP0bX
AdEAniAeOrZNHjxTtpxz04avwZ1vA/Nj
=t1A5
-----END PGP SIGNATURE-----

[Luke Plant]

On 10/10/11 17:10, Carl Meyer wrote:

> FWIW, TemplateResponse was introduced in the same release as the CBVs,
> and it allows these same generic utilities (sorting, filtering,
> pagination) to be easily implemented as reusable decorators on top of
> TemplateResponse-using views. Having tried both approaches in different
> projects, I find the code using TemplateResponse and decorators more
> concise, readable and maintainable than the equivalent using CBV mixins;
> there are not nearly as many subtle ordering and what-calls-what-when
> issues, method names to memorize, and extraneous super() noise.

I haven't tried using TemplateResponse much, but I have tried CBVs, and
I've also found with relatively small projects that mixin ordering etc
can be a real pain. Example: http://goo.gl/pzEVF
(I learnt about the type.mro() method, which was fun, but that kind of
thing should *not* be needed!)

Regards,

Luke

--
"My capacity for happiness you could fit into a matchbox without
taking out the matches first." (Marvin the paranoid android)

Luke Plant || http://lukeplant.me.uk/

[Julien Phalip]

Hi,

Making the admin's codebase more flexible and reusable is definitely something that we should aim for. In fact, there have been lots of group discussions about this during DjangoCon. Idan and I will try to post a detailed summary to this list soon. In a nutshell, the aim is to make the admin more lightweight and modular so that its various components can easily be replaced or used in different contexts. It's quite likely that a full rewrite will eventually be necessary, following the same path as oldforms/newforms: "newadmin".

However, I think at this stage one great way to start would be to refactor the current admin to make use of class-based views. This would be good for a number of reasons:

- Dogfooding: having a concrete real-world app in Django core would help drive improvements and extensions to the CBVs themselves.

- The admin codebase would reach a position where abstracting its core functionality out gets easier.

- This would help quickly anticipate issues (both from a technical and usability point of view) to do with ravioli code [1].

- A complete rewrite would likely take time and would realistically not ship in Django before the next 2-3 releases -- unless someone pulls out the work to make me lie :-). Adding CBVs would quickly open the door to a lot of possible improvements in oldadmin, which would in turn inform the parallel development of newadmin.

- This would make the future transition and upgrade path to newadmin easier.

If someone is willing to take this on then I'd be happy to provide some support by reviewing patches, giving feedback etc.

Cheers,

Julien

[1] http://lukeplant.me.uk/blog/posts/class-based-views-and-dry-ravioli/ 

[tWoolie]

> If someone is willing to take this on then I'd be happy to provide some
> support by reviewing patches, giving feedback etc.

Cheers. Would you be willing to review code that swaps out entire
swathes of admin functionality?

[Julien Phalip]

Yep. Could you outline your plan of attack for this?

Julien

[tWoolie]

> Yep. Could you outline your plan of attack for this?

1) create "newadmin" project to be a dropin replacement for admin.
(Clone current admin and slowly replace features/trim dead code)

2) Split out filterspec functionality into newadmin.filter.
- Give it a nicer wrapper such that a FilterSet can be declared and
instantiantiated much like newforms
- Move API for registering filters out of ModelAdmin and into
newadmin.filter

3) Split out table generating/sorting/display into newadmin.table
- clone django_tables2 and do some refactoring as necessary
- replace admin functionality with admin.table functionality

4) Develop CBV Wrappers for the core functionality
- replace admin views with CVB style views

[Yo-Yo Ma]

What would be the value in replacing the admin with a new version with
abstracted, reusable tools... that still resides in the new admin?
IMO, duplicating the admin's features would be better than that
(assuming you can't simply change the admin's guts in-place, which is
probably the case).


On Oct 10, 9:04 pm, tWoolie <thomas.woolf...@student.adelaide.edu.au>
wrote:

[Julien Phalip]

Actually, my proposal at this stage simply is to start with step #4 (i.e. Turn admin views into CBVs, in a backwards-compatible way and without changing existing functionality yet) with the hope that the core functionality would then be more easily extractable and could be made more generic.

I'd be reluctant to start a "newadmin" in core just yet without extensive design discussions first taking place on this topic. However, nothing prevents you or anyone who's interested to push this along by starting it outside of Django and then potentially merge it to core once it reaches a satisfactory state. I see that step #4 could be done in core regardless, though, and my point is that this work would benefit other efforts that are put into the full newadmin rewrite as it'd give us some empirical evidence of what does or doesn't work, quickly highlighting any major snag in the process.

Julien