Refactoring the documentation

18 views
Skip to first unread message

Jacob Kaplan-Moss

unread,
Mar 31, 2008, 2:45:18 PM3/31/08
to django-d...@googlegroups.com
I'm embarking on a mission to refactor Django's documentation; this is where
you give me feedback about how much crack I'm smoking. Keep in mind that most
of this post fits into the "heads-up" category: if I'm not asking for feedback
on some point it's because I've made up my mind. Only argue with those points
if you *really* think you need to.

I've identified five types of people reading Django documentation:

* New users looking to get started or to learn new techniques.
* Developers looking to solve a specific problem.
* Developers looking for reference to a particular API.
* Experienced developers looking to build other frameworks on top of
Django using extension points (middleware, signals, contribute_to_class,
etc.)
* Contributors looking for information about Django internals and
workflow.

This leads me to identify five types of documentation we need:

* Beginner material (tutorials, getting started)
* Cross-sectional "how-to" material (specific walkthroughs of particular
techniques)
* Topical guides ("caching", "middleware")
* API reference ("HttpResponse.__init__")
* Internals and meta-documentation.

(This leaves out misc. stuff like release notes; I don't really think we need
to worry about those beyond stuffing 'em in a "misc" section.)

My breakdown of stakeholders is one place I'm looking for feedback. Am I
missing anyone? Is there a "type" of documentation I've not thought of?
This'll become the high-level organization of the docs, so getting it right is
important.

My thoughts on how we do on each section and how we need improvement:

* Beginner material: we've got a good amount if this in the tutorials,
overview, and installation guide. Some bits of those documents --
especially the installation guide -- ought to be broken out into
reference material.

* Cross-sectional how-tos: we've only got a few of these currently. This
is the area where bloggers are doing a wonderful job these days. The
problem with this blog material is that it usually doesn't every get
updated, so it eventually becomes obsolete. The task here will be to
identify the best of these articles and make them official parts of the
documentation. If we can get the original author to maintain the doc,
even better.

* Topical guides: we're awesome here; most current docs are organized
around one particular topic covered in depth. The only thing to change
here is to move detailed reference out of these documents into dedicated
reference docs. This will improve the overall flow of the topical
guides.

* API reference: like above, we're pretty awesome here, too. Again, though
the API reference needs to move to documents linked from the topical
guides.

* Internals: not so hot; in fact, we've really only got the contributing
document. Help is needed here.

Again, am I missing anything here?

By now you probably want more details. OK, here we go.

If you hung out with my at the PyCon sprint this'll probably make sense. If
not apologies, 'cause I'm gonna do a bunch of hand-waving now. Ask questions
where it doesn't make sense.

The main goal in refactoring is to break everything up into manageable chunks.
The model API document is over 30 printed pages -- finding anything useful
there is rapidly becoming impossible. Even worse, it's hard to tell if
something's gonna be found there or in the DB API -- quick, where are primary
keys documented?

What I'm planning on doing is some radical chopping. I'd turn the model
documentation into a bunch of different docs, laid out something like the
following (where each bullet point is a separate doc file/page):

* High-level overview: defining and loading models.
* Model definition reference
* Field types
* Relationships
* Meta options
* Writing managers
* "Special" model methods
* DB API reference (CRUD basics)
* Manager methods
* QuerySets / QuerySet methods
* Filter lookup options
* Following relationships
* Custom SQL

This is only a rough sketch -- please don't even bother critiquing it; it'll
change. I only include it there to give you a rough idea of what I'm intending
to do. And of course I'm not going to limit my work to models; I plan to do
something similar to all the documentation.

This is the main bit I'm looking for feedback on. Is this a good idea, or am I
on crack?

In the end, the basic idea will be that topical information will "flow" like
good prose should, but will link down into detailed reference where
appropriate. Each little bit of reference will have a canonical "location" in
the docs, so that every like to HttpResponse will go to the same place.

Of course, this'll require better tooling -- we're going to need to make sure
these links all work and maintain them correctly. More documents means we need
better indexing, searching, etc. Luckily someone else already wrote the
perfect tool for the job; it's called Sphinx (http://sphinx.pocoo.org/). It'll
do everything I want, and more. I'm not interested in using different tools,
so let's not discuss that. Suffice to say I've sketched out the code needed
and it's perfect; all the hard work is going to be in actually
rewriting/moving around docs.

Right. How will I do this, you say? Here's my plan:

* Using the outline from the existing documentation
(http://toys.jacobian.org/django/docoutline-r7392/), make a new outline
and figure out where each bit fits into the new regime. This doesn't
need to be in any way perfect or 100% complete; just good enough to get
started from.

* Following that outline, break up the existing docs. Copy/paste is fine;
no rewriting yet.

* A few areas will need top-level outlines/indexes. Write those.

* Go through each document and mark it up with all the gory metadata and
links. This is the part that'll take a crazy amount of work.

* Rewrite the website documentation tools to work under the new regime
(they'll already build standalone thanks to Sphinx).

Am I missing anything here?

So feedback is welcome, please give it. Volunteers are welcome, too, but I'll
probably not be ready to distribute work just yet. I need to finish the
groundwork, and I need to figure out how to best to work without totally
fucking up the existing docs.

Jacob

alex....@gmail.com

unread,
Mar 31, 2008, 3:22:55 PM3/31/08
to Django developers
I like it, the problem becomes a bit more obvious if you look at
something like the templates for developers page, which is pretty
messy and contains a lot of different topics, or something like the
newoforms docs, which are just huge and could probably do to be broken
down a lot. And of course, internals docs are always awesome. My one
concern is that we have 2 branches that are approaching merging, and
while qs-rf won't invalidate much, because it is almost completely
backwards compatible, nfa is going to change quite a bit, to my
understanding most of the nfa docs need to be written, and therefore
it would be optimal if this could happen quickly enough that the nfa
stuff could be built on the new docs system, instead of writing them,
and then needing to refactor them(probably post merge).

Also, as a note, we currently have docs on the site for trunk, .96,
and .95 it would be awesome if we could have that for branches as
well, considering that it's all generated from .txts that are already
there, no extra work(besides setting it up).

On Mar 31, 1:45 pm, "Jacob Kaplan-Moss" <jacob.kaplanm...@gmail.com>
wrote:

Tim Chase

unread,
Mar 31, 2008, 3:29:12 PM3/31/08
to django-d...@googlegroups.com
> Am I missing anything here?
>
> So feedback is welcome, please give it.

Ah, the curse of backwards compatibility and permalinks. My one
concern (okay, given more time to think about it, perhaps more
will percolate to the surface) with this is the abundance of
(legacy) HTML-fragment references that reside in the ML archives,
such as
http://www.djangoproject.com/documentation/model-api/#primary-key

which will break in the new scheme.

That said, the above fragment-link is clearly a conceptual link
to the work you're undertaking, such as

http://www.djangoproject.com/documentation/model-api/primary-key/

Breaking it into more bite-sized chunks is good, IMHO.

But while we're still pre-1.0, growing pains are par for the
course :)

-tim


Fredrik Lundh

unread,
Mar 31, 2008, 4:59:27 PM3/31/08
to django-d...@googlegroups.com
Tim Chase wrote:

> Ah, the curse of backwards compatibility and permalinks. My one
> concern (okay, given more time to think about it, perhaps more
> will percolate to the surface) with this is the abundance of
> (legacy) HTML-fragment references that reside in the ML archives,
> such as
> http://www.djangoproject.com/documentation/model-api/#primary-key
>
> which will break in the new scheme.

unless you add a small amount of JavaScript to the page in question, of
course.

</F>

J. Cliff Dyer

unread,
Mar 31, 2008, 5:01:09 PM3/31/08
to django-d...@googlegroups.com
On Mon, 2008-03-31 at 13:45 -0500, Jacob Kaplan-Moss wrote:

> Right. How will I do this, you say? Here's my plan:
>
> * Using the outline from the existing documentation
> (http://toys.jacobian.org/django/docoutline-r7392/), make a new outline
> and figure out where each bit fits into the new regime. This doesn't
> need to be in any way perfect or 100% complete; just good enough to get
> started from.
>
> * Following that outline, break up the existing docs. Copy/paste is fine;
> no rewriting yet.
>
> * A few areas will need top-level outlines/indexes. Write those.
>
> * Go through each document and mark it up with all the gory metadata and
> links. This is the part that'll take a crazy amount of work.
>
> * Rewrite the website documentation tools to work under the new regime
> (they'll already build standalone thanks to Sphinx).
>
> Am I missing anything here?
>
> So feedback is welcome, please give it. Volunteers are welcome, too, but I'll
> probably not be ready to distribute work just yet. I need to finish the
> groundwork, and I need to figure out how to best to work without totally
> fucking up the existing docs.
>

Since all the documentation is already in subversion, it might make
sense to have a docs-refactor branch on the tree.

> Jacob

I volunteered to help out on the documentation at the sprint, and then
had to leave before anything got underway on it. I just want to put my
hat in the ring again. I'd love to help out, when the time comes for
work delegation.

Cheers,
Cliff

Jacob Kaplan-Moss

unread,
Mar 31, 2008, 5:02:42 PM3/31/08
to django-d...@googlegroups.com
On Mon, Mar 31, 2008 at 2:29 PM, Tim Chase
<django...@tim.thechases.com> wrote:
> Ah, the curse of backwards compatibility and permalinks. My one
> concern (okay, given more time to think about it, perhaps more
> will percolate to the surface) with this is the abundance of
> (legacy) HTML-fragment references that reside in the ML archives,
> such as
> http://www.djangoproject.com/documentation/model-api/#primary-key
>
> which will break in the new scheme.

I'll do my best to maintain these. Worst-case, we can turn an invalid
fragment into a search string and try to find the content that way.

Jacob

Jacob Kaplan-Moss

unread,
Mar 31, 2008, 5:05:14 PM3/31/08
to django-d...@googlegroups.com
On Mon, Mar 31, 2008 at 4:01 PM, J. Cliff Dyer <j...@sdf.lonestar.org> wrote:
> Since all the documentation is already in subversion, it might make
> sense to have a docs-refactor branch on the tree.

I'd rather not -- I don't want to get sucked into thinking I have a
long time for this. As long as the refactor results in something
structurally correct we can improve on the content continuously. As
I've been working more today I think I can set the groundwork pretty
well and then ask for volunteers to assist in quickly adding all the
metadata. Once that's in place we can make the swap and improve on the
content bit by bit.

> I volunteered to help out on the documentation at the sprint, and then
> had to leave before anything got underway on it. I just want to put my
> hat in the ring again. I'd love to help out, when the time comes for
> work delegation.

Awesome; I'll certainly ask here when I need some help.

Jacob

chris....@gmail.com

unread,
Mar 31, 2008, 5:36:27 PM3/31/08
to Django developers
I've been looking at using the sphinx tools for overhauling the
Satchmo documentation too. There's far far less documentation in that
project so it is easier but I am anxious to see how you do it. IMHO,
Django really has some best practices with documentation so I
appreciate the fact that you're keeping the process open. It should
help other people do the same thing with their projects.

One thing I'm trying to figure out in my small set of docs is how
frequently I want/plan to use the index and glossary functionality
that comes with sphinx. How much additional meta data do you hope to
add to the docs? Adding more index directives (for example) could be
quite time consuming and I'm not sure how useful it would be. The
glossary is another intriguing concept but I'm not sure how useful it
is. I'm curious what your approach might be with these types of sphinx
features.

Anyway, looking forward to the process - so I can "leverage" it for
Satchmo.

-Chris

Malcolm Tredinnick

unread,
Mar 31, 2008, 6:08:39 PM3/31/08
to django-d...@googlegroups.com

On Mon, 2008-03-31 at 13:45 -0500, Jacob Kaplan-Moss wrote:
[...]

> * Internals: not so hot; in fact, we've really only got the contributing
> document. Help is needed here.

There's http://code.djangoproject.com/wiki/DevModelCreation. It needs to
be updated when model inheritance lands, because the data flow in
__new__ has changed slightly, but it's still predominantly accurate.
I've started writing a similar page (just locally so far) for the
QuerySet and Query classes. They're useful for people wanting to
customise. I'd prefer stuff like that to be in subversion rather than
the wiki because it's not really appropriate fodder for just anybody to
edit.

The trick here is walking the line between what we want to commit to and
what is in existence at the moment. In the past we've used the lack of
documentation as a clear sign that something isn't stable, which isn't a
bad approach, but does make it annoying when you actually want to work
on that stuff and don't know how it works.

Prior to you posting this thread, I'd been meaning to send a query about
maybe having a docs/dev/ directory which would have the status of "stuff
in here is accurate, but not frozen and not public API". But now it's
all your problem and you can work it out. I like that plan better.
Mwahahaha...

Regards,
Malcolm

--
Success always occurs in private and failure in full view.
http://www.pointy-stick.com/blog/

Marty Alchin

unread,
Mar 31, 2008, 6:51:12 PM3/31/08
to django-d...@googlegroups.com
I won't bother responding to anybody in particular, because I'm in
agreement with so many on this thread so far. I'm glad to see this
refactoring happen, because a doc improvement can only make things
better for everybody.

I know you're not getting into details yet, but once you do get down
to it, I'll gladly volunteer to write, or at least edit, API reference
and internals docs. I already delve into all that anyway, so
documenting it sounds like a blast!

-Gul

flo...@gmail.com

unread,
Mar 31, 2008, 7:25:52 PM3/31/08
to Django developers
This all sounds really awesome!

>     * Cross-sectional how-tos: we've only got a few of these currently. This
>       is the area where bloggers are doing a wonderful job these days. The
>       problem with this blog material is that it usually doesn't every get
>       updated, so it eventually becomes obsolete. The task here will be to
>       identify the best of these articles and make them official parts of the
>       documentation. If we can get the original author to maintain the doc,
>       even better.

I'm wondering if something like http://django.reddit.com/ shouldn't
get some sort of official blessing for listing/searching those blog
posts which aren't of good enough quality for inclusion in the
official docs, but are still useful to some people.

Jacob Kaplan-Moss

unread,
Mar 31, 2008, 8:21:15 PM3/31/08
to django-d...@googlegroups.com
On Mon, Mar 31, 2008 at 4:36 PM, chris....@gmail.com
<chris....@gmail.com> wrote:
> One thing I'm trying to figure out in my small set of docs is how
> frequently I want/plan to use the index and glossary functionality
> that comes with sphinx. How much additional meta data do you hope to
> add to the docs? Adding more index directives (for example) could be
> quite time consuming and I'm not sure how useful it would be. The
> glossary is another intriguing concept but I'm not sure how useful it
> is. I'm curious what your approach might be with these types of sphinx
> features.

I'd take these questions (and any others about Sphinx) to the
(brand-new!) sphinx-dev mailing list (also on Google Groups). It's
just getting started, but I'd love to say a bunch of projects start
using and discussing using Sphinx.

Jacob

Jacob Kaplan-Moss

unread,
Mar 31, 2008, 8:25:02 PM3/31/08
to django-d...@googlegroups.com
On Mon, Mar 31, 2008 at 6:25 PM, flo...@gmail.com <flo...@gmail.com> wrote:
> I'm wondering if something like http://django.reddit.com/ shouldn't
> get some sort of official blessing for listing/searching those blog
> posts which aren't of good enough quality for inclusion in the
> official docs, but are still useful to some people.

Mmmm... I'm really skeptical of "consensus" documentation. I think
when it comes to prose a centralized group of editors does better than
"the masses" -- Wikipedia may be (and is!) technically superior to
Britannica, but Wikipedia's writing style doesn't even measure up to
Encarta. Technology helps in some places, but there's no replacement
for a good editor. And I'm *not* talking about Emacs.

Jacob

pbx

unread,
Mar 31, 2008, 8:47:32 PM3/31/08
to Django developers
On Mar 31, 8:25 pm, "Jacob Kaplan-Moss" <jacob.kaplanm...@gmail.com>
wrote:
> On Mon, Mar 31, 2008 at 6:25 PM, flo...@gmail.com <flo...@gmail.com> wrote:
> >  I'm wondering if something likehttp://django.reddit.com/shouldn't
> >  get some sort of official blessing for listing/searching those blog
> >  posts which aren't of good enough quality for inclusion in the
> >  official docs, but are still useful to some people.
>
> Mmmm... I'm really skeptical of "consensus" documentation. I think
> when it comes to prose a centralized group of editors does better than
> "the masses" -- Wikipedia may be (and is!) technically superior to
> Britannica, but Wikipedia's writing style doesn't even measure up to
> Encarta. Technology helps in some places, but there's no replacement
> for a good editor. And I'm *not* talking about Emacs.

First, this project is a great idea. Django's docs are great, but they
will only stay great if they're attended to. I'll be happy to help out
a bit once I finish up this (ahem) other pesky writing project I've
been working on.

Second, I take and agree with your point about editorial authority.
Leaving aside the question of outside resources, I wonder if some
pathway for user feedback or annotation of the official docs, in
addition to the ticket system, could be beneficial. A dynamic FAQ that
serves as an alternative TOC/index for the docs? A "did this answer
your question" type of feedback loop? Maybe I've been sniffing the Web
2.0 fumes too much.

Third, one stakeholder type I didn't see in your list is the refugee/
migrator coming from another web framework. This probably doesn't map
to another full-blown documentation type, since the topical and API
reference docs will serve pretty well. But one could make an argument
that a guide or guides along the lines of "Django for Blub-on-Boats
programmers" could help bring framework-switchers up to speed more
quickly.

pb

James Bennett

unread,
Mar 31, 2008, 8:48:21 PM3/31/08
to django-d...@googlegroups.com
On Mon, Mar 31, 2008 at 7:25 PM, Jacob Kaplan-Moss
<jacob.ka...@gmail.com> wrote:
> Technology helps in some places, but there's no replacement
> for a good editor. And I'm *not* talking about Emacs.

The next release will have 'M-x copyeditor' for that, actually...


--
"Bureaucrat Conrad, you are technically correct -- the best kind of correct."

Russell Keith-Magee

unread,
Apr 1, 2008, 8:41:28 AM4/1/08
to django-d...@googlegroups.com

On 01/04/2008, at 2:45 AM, Jacob Kaplan-Moss wrote:
>
> I'm embarking on a mission to refactor Django's documentation; this
> is where
> you give me feedback about how much crack I'm smoking. Keep in mind
> that most
> of this post fits into the "heads-up" category: if I'm not asking
> for feedback
> on some point it's because I've made up my mind. Only argue with
> those points
> if you *really* think you need to.
...

>
> Am I missing anything here?

I can't see anything obvious - the categories and user types you have
identified seem pretty much right to me.

Echoing (and agreeing with) a few other comments:

* I vastly prefer the single editor approach to the hivemind for
editing

* Existing permalinks need to be preserved in some fashion, if only
so that the archive of answers in django-users continues to be a
useful resource.

* +1 to an index/glossary page. However, I have a slight aversion to
completely automated indexing. My experience has been that automated
indexes fall into the same bucket as hivemind edited documents. A good
editor knows how to prune the index to remove useless keywords and
page references.

Yours,
Russ Magee %-)

Alex Myodov

unread,
Apr 1, 2008, 6:35:14 PM4/1/08
to Django developers
> So feedback is welcome, please give it.
Breaking the huge documents into smaller chunks for readability is
good... but please do leave the original huge ones as well! There are
cases where "a huge document" serves more help than "a bunch of
smaller ones" and really has its worth.
Opening /cache/ or /db-api/ gives you a single document that has
*everything* on the cache/ORM access topic, without need to jump over
urls here and there; having two or three tabs opened in the browser
and covering the topics of the most current importance to you is much
more convenient, than opening two or three dozens smaller ones and
trying to figure out which tab has the document you need. And the huge
size of the document is not that much of a problem in the browsers
that support Ctrl+F or "/" hotkeys... all of them, I believe, with the
different degrees of convenience though.

It is not that I am representing the view *against* splitting the huge
documents. I am just raising a hand in a case if "document is broken
into shorter sections" will mean that "no good old long-sectioned
documents are available anymore". The best what could be having two
"builds" of the documentation made from the same sources - one having
long unsplitted sections, and one having short splitted sections. Kind
of a "devil advocate" - with the sincere incline to the devil side of
the long documents.

On 31 мар, 22:45, "Jacob Kaplan-Moss" <jacob.kaplanm...@gmail.com>
wrote:

Nick

unread,
Apr 1, 2008, 6:48:29 PM4/1/08
to Django developers
On Mar 31, 7:45 pm, "Jacob Kaplan-Moss" <jacob.kaplanm...@gmail.com>
wrote:
> I'm embarking on a mission to refactor Django's documentation;

Great! I'll be happy to help out with some of the grunt work as and
when
I can find a spare moment.

> * Beginner material: we've got a good amount if this in the tutorials,
> overview, and installation guide. Some bits of those documents --
> especially the installation guide -- ought to be broken out into
> reference material.

+1 on breaking out the installation guide.

This is a good opportunity to think about those topics listed under
"Coming Soon" in Part 4 of the existing tutorial. Are they still
needed?
Do they fall into the Topical Guides category rather than the Beginner
Material category? etc...

> The main goal in refactoring is to break everything up into manageable chunks.
> The model API document is over 30 printed pages -- finding anything useful
> there is rapidly becoming impossible. Even worse, it's hard to tell if
> something's gonna be found there or in the DB API -- quick, where are primary
> keys documented?

This will be good to see. The quality of the docs has always been
high,
but usability is definitely an issue with the bigger documents. My
mouse's
scroll wheel always gets a good workout at djangoproject.com :)


Nick

Jacob Kaplan-Moss

unread,
Apr 1, 2008, 7:59:14 PM4/1/08
to django-d...@googlegroups.com
On Mon, Mar 31, 2008 at 7:47 PM, pbx <paul....@gmail.com> wrote:
> Second, I take and agree with your point about editorial authority.
> Leaving aside the question of outside resources, I wonder if some
> pathway for user feedback or annotation of the official docs, in
> addition to the ticket system, could be beneficial. A dynamic FAQ that
> serves as an alternative TOC/index for the docs? A "did this answer
> your question" type of feedback loop? Maybe I've been sniffing the Web
> 2.0 fumes too much.

If so, I've been sniffing the same fumes. I've got something in mind
that I get to develop as my "reward" when the hard work is done. It'll
rock your world :)

> Third, one stakeholder type I didn't see in your list is the refugee/
> migrator coming from another web framework. This probably doesn't map
> to another full-blown documentation type, since the topical and API
> reference docs will serve pretty well. But one could make an argument
> that a guide or guides along the lines of "Django for Blub-on-Boats
> programmers" could help bring framework-switchers up to speed more
> quickly.

I think I'd lump that in with the "cross-sectional how-to's" but
agreed that it's something we should be thinking about. A "Django for
Xers" doc actually would be pretty simple for someone who knows both X
and Django: such a doc would mostly consist of high-level overviews
with links to API reference where needed.

I'll set aside a space for these kinds of documents in my outline.

Thanks!

Jacob

Jacob Kaplan-Moss

unread,
Apr 1, 2008, 8:00:09 PM4/1/08
to django-d...@googlegroups.com
On Mon, Mar 31, 2008 at 7:48 PM, James Bennett <ubern...@gmail.com> wrote:
> The next release will have 'M-x copyeditor' for that, actually...

See, this is why I like to mention Emacs 'round here; I can always
count on James for the follow-up. I'll tell you what, though, "M-x
copyeditor" would *totally* convince me to switch!

Jacob

Jacob Kaplan-Moss

unread,
Apr 1, 2008, 8:04:38 PM4/1/08
to django-d...@googlegroups.com
On Tue, Apr 1, 2008 at 7:41 AM, Russell Keith-Magee
<freakb...@gmail.com> wrote:
> * Existing permalinks need to be preserved in some fashion, if only
> so that the archive of answers in django-users continues to be a
> useful resource.

It'll be tough. The worst part about anchors is that they don't show
up in server logs so it's hard to find what's been referenced where.
There'll likely be some breakage, but with some clever thinking it
shouldn't be too hard -- we could do something like building an index
of anchors from the current docs and then mapping those to their new
locations. It'd take some time, but probably worth it.

> * +1 to an index/glossary page. However, I have a slight aversion to
> completely automated indexing. My experience has been that automated
> indexes fall into the same bucket as hivemind edited documents. A good
> editor knows how to prune the index to remove useless keywords and
> page references.

One of the reasons I'm quickly falling in love with Sphinx, actually:
the indexing/glossary support is designed with exactly this kind of
tweaking in mind:
http://sphinx.pocoo.org/markup/para.html#index-generating-markup

Jacob

AmanKow

unread,
Apr 2, 2008, 11:47:51 AM4/2/08
to Django developers
Will the api and internals references be generated (at least in part)
via doc string extraction? That's a big plus to me.

On Mar 31, 2:45 pm, "Jacob Kaplan-Moss" <jacob.kaplanm...@gmail.com>
wrote:

James Bennett

unread,
Apr 2, 2008, 12:22:58 PM4/2/08
to django-d...@googlegroups.com
On Wed, Apr 2, 2008 at 10:47 AM, AmanKow <wwe...@si.rr.com> wrote:
> Will the api and internals references be generated (at least in part)
> via doc string extraction? That's a big plus to me.

I most sincerely hope not; unless you're like me and write short-form
novels in your docstrings, they tend to be an absolutely horrible
source of end-user documentation.

Marty Alchin

unread,
Apr 2, 2008, 12:26:57 PM4/2/08
to django-d...@googlegroups.com
On Wed, Apr 2, 2008 at 12:22 PM, James Bennett <ubern...@gmail.com> wrote:
> I most sincerely hope not; unless you're like me and write short-form
> novels in your docstrings, they tend to be an absolutely horrible
> source of end-user documentation.

def get_object_from_cache(self, ...):
"""
Get an object from the cache and return it.
""""
...

Jacob Kaplan-Moss

unread,
Apr 2, 2008, 12:40:56 PM4/2/08
to django-d...@googlegroups.com
On Wed, Apr 2, 2008 at 10:47 AM, AmanKow <wwe...@si.rr.com> wrote:
> Will the api and internals references be generated (at least in part)
> via doc string extraction? That's a big plus to me.

When appropriate we could; Sphinx does that, too:
http://sphinx.pocoo.org/ext/autodoc.html

Like James and Marty I'm skeptical that it'll be useful all that
often. docstrings are really meant for folks reading the source; they
make pretty shitty documentation for users. I suspect the autodoc
stuff will mostly come in handy when documenting internals.

Jacob

AmanKow

unread,
Apr 3, 2008, 8:02:18 AM4/3/08
to Django developers
Sorry, as usual, I wasn't clear. When I said "at least in part" I
didn't express my thoughts very well.

I don't mean that high level or in depth discussion should be taking
place in the doc strings, that's for your excellent blog posts (and
the docs, of course)!.

But it doesn't hurt to have class/function/method signatures in the
code, along with any interesting quirks and caveats mentioned. I
actually find it enormously helpful. For instance, heavy use of
keyword args can make it necessary to read the code sometimes to find
argument names. Not that the code isn't worth reading, but sometimes
you're looking for that hit and run from your code browser in a
hurry. As an added bonus, some code editors take advantage of the doc
strings as well, popping them up as help as you type.

My experience is that it is more likely that api changes/additions are
noted when the signature docs lives near the code.

Not advocating code as prose! I do indeed understand that code is
poetry!

Wayne


On Apr 2, 12:40 pm, "Jacob Kaplan-Moss" <jacob.kaplanm...@gmail.com>
wrote:

Fredrik Lundh

unread,
Apr 4, 2008, 10:04:34 AM4/4/08
to django-d...@googlegroups.com
Jacob Kaplan-Moss wrote:

>> I'm wondering if something like http://django.reddit.com/ shouldn't
>> get some sort of official blessing for listing/searching those blog
>> posts which aren't of good enough quality for inclusion in the
>> official docs, but are still useful to some people.
>
> Mmmm... I'm really skeptical of "consensus" documentation. I think
> when it comes to prose a centralized group of editors does better than
> "the masses"

A compromise would be centralized core documentation, but with links to
external sites in a clearly-marked "see also" section. The web *is* a
distributed place, after all.

We did work on mechanisms for this for python.org a very long time ago
(*), but that work stalled (and I'm not going to even try to work with
the python.org maintainers again until they've migrated to Django ;-)

</F>

*) see e.g.
http://mail.python.org/pipermail/python-list/2005-May/322224.html
and http://svn.python.org/view/sandbox/trunk/seealso/

an old format spec is here: http://effbot.org/zone/idea-seealso.htm
(including samples) but I guess you could use Atom for this these days.

Fredrik Lundh

unread,
Apr 4, 2008, 10:17:33 AM4/4/08
to django-d...@googlegroups.com
Alex Myodov wrote:

> Breaking the huge documents into smaller chunks for readability is
> good... but please do leave the original huge ones as well! There are
> cases where "a huge document" serves more help than "a bunch of
> smaller ones" and really has its worth.

I haven't checked if Sphinx already supports this, but one of the core
ideas for the Python AltLibRef project that partially inspired Sphinx
was a simple "target" concept that could be used to link to very
specific locations in a document, but still allow the authors work with
full pages. Here's an example from the PIL documentation:

http://effbot.org/tag/PIL.Image.open

(other important AltLibRef parts were a) authoring through the web, so
that "certified maintainers" could quickly fix minor things wiki-style,
and b) light-weight mechanisms for providing "drive-by" suggestions and
fixes. the django book site has excellent support for (b), and django
used to have decent support for (b), but that seems to have been removed.)

</F>

Fredrik Lundh

unread,
Apr 4, 2008, 11:06:30 AM4/4/08
to django-d...@googlegroups.com
Jacob Kaplan-Moss wrote:

> Like James and Marty I'm skeptical that it'll be useful all that
> often. docstrings are really meant for folks reading the source

nah, they're meant for folks using the "help" command when playing with
things via the command line (or using similar mechanisms in smart
IDE:s). It's pretty much like the difference between

command --help

and

man command

(docstrings occupy memory too, so if you're *only* writing for folks
reading the source, use comments.)

</F>

Jacob Kaplan-Moss

unread,
Apr 4, 2008, 11:36:40 AM4/4/08
to django-d...@googlegroups.com
On Fri, Apr 4, 2008 at 9:04 AM, Fredrik Lundh <fre...@pythonware.com> wrote:
> A compromise would be centralized core documentation, but with links to
> external sites in a clearly-marked "see also" section. The web *is* a
> distributed place, after all.

That's probably the best idea. We'd still need an editor to eye
submissions, remove outdated or missing material, brow-beat authors
into updating blog entries, etc.

Jacob

Jacob Kaplan-Moss

unread,
Apr 4, 2008, 11:42:39 AM4/4/08
to django-d...@googlegroups.com
On Fri, Apr 4, 2008 at 9:17 AM, Fredrik Lundh <fre...@pythonware.com> wrote:
> I haven't checked if Sphinx already supports this, but one of the core
> ideas for the Python AltLibRef project that partially inspired Sphinx
> was a simple "target" concept that could be used to link to very
> specific locations in a document, but still allow the authors work with
> full pages. Here's an example from the PIL documentation:
>
> http://effbot.org/tag/PIL.Image.open

Mmm, yeah -- that's a really big deal. PHP is the gold-standard here
-- URLs like http://www.php.net/mysqli_real_escape_string rock.

I don't think Sphinx supports this out of the box, but I'm not going
to be using the Sphinx web server parts anyway, so I'll make sure that
my views include this one. Sphinx already takes care of tracking
"canonical" locations for functions, modules, etc, so it shouldn't be
hard to write a view to redirect to one of these canonical locations.

> (other important AltLibRef parts were a) authoring through the web, so
> that "certified maintainers" could quickly fix minor things wiki-style,
> and b) light-weight mechanisms for providing "drive-by" suggestions and
> fixes. the django book site has excellent support for (b), and django
> used to have decent support for (b), but that seems to have been removed.)

For the record, we removed comments on the documentation because 99%
of them were usage questions better asked on django-users; we were
doing folks a disservice allowing them to post on the docs. I've got
some ideas for doing this better up my sleeve; stay tuned...

Jacob

Fredrik Lundh

unread,
Apr 4, 2008, 12:14:54 PM4/4/08
to django-d...@googlegroups.com
Jacob Kaplan-Moss wrote:

> That's probably the best idea. We'd still need an editor to eye
> submissions, remove outdated or missing material, brow-beat authors
> into updating blog entries, etc.

Just make sure you make it possible for external editors (=anyone) to
maintain link collections, and include stuff "on the fly" from trusted
collections. If a collection deteriorates, just drop it until the
author gets her/his act together. (And make that part of site
maintenance, not documentation maintenance.)

(initial sources could be trusted django bloggers willing to provide a
proper link feed, high-rated stuff from the snippets site, and perhaps
some collaborative effort using delicious or a similar site as a
suggestion box *and* collection source.)

You need some kind of target concept (see my earlier post) for this to
work, but that should be pretty straightforward to design; just add a
few "django:topic" overall targets, and use FQPOP [1] for the rest.

</F>

1) "fully qualified python object path", e.g. "PIL.Image.new",
"django.shortcuts.render_to_reponse", etc. also see "target space" on
this page: http://www.effbot.org/zone/pyref-targets.htm

AmanKow

unread,
Apr 5, 2008, 1:59:42 PM4/5/08
to Django developers

On Apr 2, 12:26 pm, "Marty Alchin" <gulop...@gamemusic.org> wrote:
> def get_object_from_cache(self, ...):
> """
> Get an object from the cache and return it.
> """"
> ...
hmm... the doc string equivalent of the infamous c inanity:
i++ // increment i

I have seen a bit of that in the code. IMO, those one liner doc
strings on already well named methods are indeed a waste of run time
memory, coding time real estate and coder time, both reading or
writing them. What ever the final disposition of doc strings in the
code base, those one liners are not useful and should be burned.

AmanKow

unread,
Apr 5, 2008, 2:02:14 PM4/5/08
to Django developers


On Apr 4, 11:06 am, Fredrik Lundh <fred...@pythonware.com> wrote:
> (docstrings occupy memory too, so if you're *only* writing for folks
> reading the source, use comments.)

And of course, a production environment can always optimize the doc
strings out.

holdenweb

unread,
Apr 13, 2008, 9:43:56 PM4/13/08
to Django developers
I've actually been looking for accepted documentation bugs as a
starting-point for my Django development work, but most of them seem
to be point issues. In the face of Jacob's promise (threat?) to
radically revise the structure and nature of the documentation I am
unsure what value these point edits might have.

How long do we have to wait for the "master plan" to be revealed?

regards
Steve

Malcolm Tredinnick

unread,
Apr 13, 2008, 9:48:04 PM4/13/08
to django-d...@googlegroups.com

On Sun, 2008-04-13 at 18:43 -0700, holdenweb wrote:
> I've actually been looking for accepted documentation bugs as a
> starting-point for my Django development work, but most of them seem
> to be point issues. In the face of Jacob's promise (threat?) to
> radically revise the structure and nature of the documentation I am
> unsure what value these point edits might have.
>
> How long do we have to wait for the "master plan" to be revealed?

You mean the plan that Jacob has posted on this list in the last week.
And that has been given a specific TODO list and has been having a
mini-sprint over the course of the past weekend (also mentioned on this
list)? For that plan ... about minus three or four days from now. :-)

For example, one of the links Jacob posted was
http://code.djangoproject.com/wiki/DocSprint12Apr

Malcolm

--
I just got lost in thought. It was unfamiliar territory.
http://www.pointy-stick.com/blog/

holdenweb

unread,
Apr 13, 2008, 10:17:21 PM4/13/08
to Django developers


On Apr 13, 9:48 pm, Malcolm Tredinnick <malc...@pointy-stick.com>
wrote:
> On Sun, 2008-04-13 at 18:43 -0700, holdenweb wrote:
[...]
> > How long do we have to wait for the "master plan" to be revealed?
[...]
> ... about minus three or four days from now. :-)
>
> For example, one of the links Jacob posted washttp://code.djangoproject.com/wiki/DocSprint12Apr
>
Aah, right. And I mulched the garden this weekend instead :-(

Anyway, I guess the bottom line is that the new structure is clarified
as soon as I read up on it. Thanks, Malcom.

regards
Steve

Jacob Kaplan-Moss

unread,
Apr 13, 2008, 11:09:51 PM4/13/08
to django-d...@googlegroups.com
Howdy folks --

FYI, I'm rapidly gaining ground on this big task. There's a lot to be
done, but I've got about half a dozen patches in my inbox, and I'm
going to keep pushing into next week. If anyone wants to start helping
tomorrow, here's some quick pointers:

How-to: http://tinyurl.com/68k7cb
Repository: http://toys.jacobian.org/hg/docs-refactor/
(Incomplete) TODO list: http://code.djangoproject.com/wiki/DocSprint12Apr

I'll be in #django-sprint tomorrow and until I wrap this up. I think
realistically I'm looking at about another solid day or two of work,
plus about a day to rewrite the frontend bits for the website.

Jacob

Reply all
Reply to author
Forward
0 new messages