Hi folks --

This is a proposed release process for Django going forward. It's been
discussed among the core developers to consensus, but is presented
here as a proposal -- we'd like to hear your feedback!

Like the 1.0 roadmap, this is presented in the future perfect tense,
though it is a draft. After feedback from the community, I'll post the
final version of this as an addition to the contributing guide.

My next email will be a draft timeline for 1.1 which follows this
procedure; if all you care about is 1.1, wait a moment for that.

As a quick refresher in semantics, be sure to read the "official
releases" bit of the guide
(http://docs.djangoproject.com/en/dev/internals/contributing/#official...)
for a reminder of how version numbers work.

Django's release process
========================

Django uses a time-based release schedule, with minor (i.e. 1.1, 1.2,
etc.) releases every 6-12 months, depending on features.

After each previous release (and after a suitable cooling-off period),
the core development team will examine the landscape and announce a
timeline for the next release. Most releases will be scheduled in the
6-9 month period, but if we have bigger features to development we
might schedule a longer period to allow for more ambitious work.

Release cycle
-------------

Each release cycle will be split into three periods, each lasting
roughly one-third of the cycle:

Phase one: feature proposal
~~~~~~~~~~~~~~~~~~~~~~~~~~~

The first phase of the release process will be devoted to figuring out
what features to include in the next version. This should include a
good deal of preliminary work on those features -- working code trumps
grand design.

At the end of part one, the core developers will propose a feature
list for the upcoming release. This will be broken into:

* "Must-have": critical features that will delay the release if not finished
* "Maybe" features: that will be pushed to the next release if not finished
* "Not going to happen": features explicitly deferred to a later release.

Anything that hasn't got at least some work done by the end of the
first third isn't eligible for the next release; a design alone isn't
sufficient.

Phase two: development
~~~~~~~~~~~~~~~~~~~~~~

The second third of the release schedule is the "heads-down" working
period. Using the roadmap produced at the end of phase one, we'll all
work very hard to get everything on it done.

At the end of phase two, any unfinished "maybe" features will be
postponed until the next release. Though it shouldn't happen, any
"must-have" features will extend phase two, and thus postpone the
final release.

Phase two will culminate with an alpha release.

Phase three: bugfixes
~~~~~~~~~~~~~~~~~~~~~

The last third of a release is spent fixing bugs -- no new features
will be added during this time. We'll release a beta release about
halfway through, and an rc complete with string freeze two weeks
before the end of the schedule.

Bug-fix releases
----------------

After a minor release (i.e 1.1), the previous release will go into
bug-fix mode.

A branch will be created of the form "/branches/releases/1.0.X" to
track bug-fixes to the previous release. When possible, bugs fixed on
trunk must *also* be fixed on the bug-fix branch; this means that
commits need to cleanly separate bug fixes from feature additions.
Each bug-fix branch will have a maintainer who will work with the
committers to keep them honest on backporting bug fixes.

Micro releases (i.e. 1.0.1) will happen roughly half-way between minor
releases -- or earlier, if enough bugs have been fixed to warrant a
release.

After the *next* minor release (i.e. 1.2), the previous bug-fix
release will enter security-fix mode: only security fixes will be
applied to the micro-branch, and new releases will only be done for
those fixes.

Feature branches
----------------

Development of major features between versions will be done on
branches. See the branch policy
(http://docs.djangoproject.com/en/dev/internals/contributing/#branch-p...)
for details.

[I'll be updating this policy shortly to indicate new procedures for
branches, including the use of DVCS tools --JKM].

How this all fits together
--------------------------

Let's look at a hypothetical example for how this all first together.
Imagine, if you will, a point about halfway between 1.1 and 1.2. At
this point, development will be happening in a bunch of places:

    * On trunk, development towards 1.2 proceeds with small additions, bugs
      fixes, etc. being checked in daily.

    * On the branch "/branches/releases/1.1.X", bug fixes found in the 1.1
      release are checked in as needed. At some point, this branch will be
      released as "1.1.1", "1.1.2", etc.

    * On the branch "/branches/releases/1.0.X", security fixes are made if
      needed and released as "1.0.2", "1.0.3", etc.

    * On features branches, development of major features is done. These
      branches will be merged into trunk before the end of phase two.

Again, feedback is welcome.

Jacob

Hi there,

I'm slightly concerned that people will be tempted to run off trunk or  
other branches since they would potentially have to wait up to 9  
months to see a feature make it to the stable version. Imagine if  
people had had to wait 9 months to get newcomments...
It would be their entire fault for being too impatient, but it's  
basically what was happening before 1.0, particularly in the last few  
months, and way too many people were running bug-ridden installs...

I also think you'll end up accumulating a lot of features with such a  
long release cycle. And although people will be really happy when a  
release goes out, they'll be really unhappy in between releases.

So, even though I consider the proposal perfect in every other sense,  
I think a shorter release cycle even if releases are smaller, would  
make people stick to stable versions a bit more.

Otherwise, great work.

Regards,

Cyril Doussin

On 15 Sep 2008, at 12:18, Jacob Kaplan-Moss wrote:

I doubt 6-9 months is a "long" release cycle. Note that when you do a
release, distributions have to package and distribute it, people have
to upgrade their code (which can be a pain in large deployments) and
(in theory) keep working on other things like their own applications!

So a shorter release cycle would mean that people would need to spend
most of their life upgrading their code (as old releases would go out
of security-fix mode sooner).

A side, 9 months/3 means: 3 months for planning, 3 for development, 3
for bug-squashing. Spending 3 months in planning the next schedule and
other 3 squashing bugs seems a bit too much time for me. Maybe 2/5/2 ?

Anyway, thanks Jacob for sharing so much details and asking for comments ;)

Regards,
Marc
--
http://www.marcfargas.com - will be finished someday.

Hey Cyril --

Thanks for the comments.

Keep in mind that we'll continue to make sure that the trunk is
reasonably stable; folks who want the latest and greatest can (and
probably will) continue to track trunk.

Also, I think if you look around you'll see that 6-9 months is
anything but a "long" release cycle -- GNOME has a six month cycle,
and I'm told it's nearly insanely quick. The thing we need to avoid is
a such a short cycle that larger features can't be considered.

Please also keep in mind that (as with most Open Source projects)
nearly everyone working on Django does so in his/her spare time; this
means you can't do the standard 40 hours/week calculation to figure
out how much time each person can put in.

That said, how quick a cycle would you like to see?

Jacob

Good point -- that's why I said "roughly" split into thirds: a longer
release cycle will, as you say, have more time in the middle.

Jacob

[...]

Well "planning" means that people can work on lots of different stuff
and work out if it's appropriate. So for that stuff, it's 6 months of
development. The idea, though, is that we don't have to worry about
"here's some feature that I'd like to see in the next release but isn't
really even started yet" messages popping up 4 months out from the
release. The answer in those cases is simple: "great! keeping working on
it and let's see what state it's in for the next round of feature
freezing." There's actually nine months of bug fixing in the nine month
schedule. It's only that in the final period it's more of a concentrated
focus.

Regards,
Malcolm

On Sep 15, 1:18 pm, "Jacob Kaplan-Moss" <jacob.kaplanm...@gmail.com>
wrote:

I wouldn't call "period to 2*period, depending on features" a time-
based
release schedule.
Obviously, I have no idea what has been discussed internally, but let
me
describe a real time-based schedule that is a lot simpler imo:

Releases are generally labeled MAJOR.MINOR. MAJOR.MINOR.TINY is only
used
for situations when there's a capital problem with a MAJOR.MINOR that
needs to
be fixed asap (e.g. security).

Everything that breaks backward compatibility or is just too plain
awesome goes
into a MAJOR.0 release. The core devs decide when such a release is
made
and what to include in it.

At a fixed date in every (or second or third) month there is a
MAJOR.MINOR release.
Those releases contain all bugfixes to the previous release and
smaller new
features that don't break existing code based on the previous MAJOR.0
release.

A real time-based release schedule has the advantage of removing the
decision
about what to include in which release. Feature devs work on their
stuff until it's
done, then it is released with the next version. A rule might be in
place to forbid
feature merges in the last week before a MAJOR.MINOR release to sort
out any
possible remaining bugs in those features.

What are the advantages over the model Jacob described?
1. more frequent releases, allowing users to move away from SVN
checkouts
if they want those nice small new features and bugfixes (!) more
quickly.
2. less decision-making, even though it seems to work pretty well
right now,
Django will grow and it might become a problem. I've seen how this
crippled
Gentoo.
3. MAJOR.0 releases can be perceived as API version bumps and make it
very
clear when rewriting code is required on the user side.

My schedule might have Django reaching version 7.0 a lot sooner than
most
people would expect, but personally I think there's nothing wrong with
that.
However, it may be necessary to prepend a 1. to my versioning scheme,
that will eventually be bumped to 2.MAJOR.MINOR when Django is
adapted
for Python 3.

After all, there is no absolute truth to release schedules, but
personally, I'd be a
lot more comfortable with the above concept.

P.S.: KDE has a monthly release cycle and Django is _a lot_ easier to
package
than KDE. So I don't think that there is a need to worry about those
distribution
issues.

Hi Jacob,

That's all fair enough.
I had in mind 6 months as a rough deadline for releases, not too sure  
why, apart from knowing other big projects like Gnome indeed are  
running on such a schedule. It seems to me like a good middle ground  
which could keep most people happy (taking a wild guess here...).

It's true that the ongoing stability of trunk is one of the strong  
points to consider and helps to alleviate the problems the most  
impatient of us could run into (to a degree).

I fully understand this is all benevolent work, don't worry. That's  
probably why even though Torsten's points about keeping to a strict  
schedule make sense, I think keeping a bit of bandwidth for just  
"getting in right", whilst still having a life, would be more than  
fair on the developers.

Anyway, I'm not too fuss, if you guys feel more comfortable with  
something longer than 6 months so be it, it'll just be one more  
incentive for folks like me to contribute and help making things move  
faster :)

Thanks for asking for feedback.

Cyril

On 15 Sep 2008, at 12:39, Jacob Kaplan-Moss wrote:

Hey Torsten --

Thanks for your input. I've got some specific thoughts about your
proposal, but first let me ask this: does any project *really* have
monthly feature releases? If so, how do they cope with the
alpha/beta/final run-up -- is four weeks really enough?

You said KDE has monthly releases, but from what I'm reading this is
far from true. According to their wiki
(http://techbase.kde.org/Schedules/Release_Schedules_Guide), each
feature release is at least five months, In practice this seems to not
be actually happening KDE 4.0 was released in August of 2006; KDE 4.1
is scheduled for November 2009
(http://techbase.kde.org/Schedules/KDE4/4.1_Release_Schedule), and
they claim that 4.2 is scheduled for Janurary 2009
(http://techbase.kde.org/Schedules/KDE4/4.2_Release_Schedule), though
that seems obviously wrong unless they plan to release 4.2 before 4.1

Or perhaps I'm reading the wrong content?

Even if I'm not, I think you've got a pretty major misunderstanding
about how we talk about and use releases here:

On Mon, Sep 15, 2008 at 1:29 PM, Torsten Rehn

This isn't at all how we do things here; I asked everyone to read
http://docs.djangoproject.com/en/dev/internals/contributing/#official...
as a refresher in how version numbers work in Django for a reason, it
clearly explains how the numbers work.

We don't release micro versions for "capital problems"; we do them for
bug fixes. In practice, this means we'll be releasing 1.0.1 in the
next month or so; there's been quite a few bug fixes since the 1.0
release. We may very well get to 1.0.3 by the time 1.1 ships.

Look, it's impossible to get a serious feature release out in just two
months. Seriously -- let's take a look at some milestones leading up
to a release, working backwards:

* String freeze: at some point, we have to freeze all marked strings
so translators have a chance to update their translations. A week is
an absolute minimum here.
* A release candidate. We were crazy in 1.0 and went from rc -> final
release in 48 hours, but that was an unusual situation and we rightly
got some criticism about it. A week would be more doable, but let's
say five days and be really tight.
* A beta release. Most large projects give about a month from beta to
release candidate; let's again be crazy and say only two weeks.
Remember, this is when most bugs get fixed -- too little time here and
releases are buggy.
* An alpha following feature freeze. Again, a month would be nice, but
let's say two weeks again.

OK, so that's six weeks from feature freeze to final. That leaves two
weeks for feature development in your system. See a problem here? I
do.

Again, bug fixes will be in the point releases, which will come out at
*least* once between minor releases, and possibly more often.

Can you clarify what problems you see here? Gentoo had a host of
leadership problems besides release scheduling; I don't think they're
a valid comparison. But if you do, please explain why.

Exactly, which is why this should happen as little as possible. Every
backwards-incompatible change leaves some users behind; do too many of
them and you end up annoyed users.

Jacob

I'd suggest codifying this "suitable cooling-off period" as
Phase 0 to stave off the expectations that 30 seconds after a
release N, you have to start planning for release N+1.  This
allows for a little breather/partying without the "hey, update
the roadmaps already" gripes :)

-tim

On Mon, Sep 15, 2008 at 13:39, Jacob Kaplan-Moss

I am very fond of release cycles that are strictly fixed in time, like
for example the OpenBSD project (www.openbsd.org). They do a release
every 6 months, on May 1, and on November 1, and then just keeps
cycling.

I understand that such a cycle may lead to problems with security
fixes for old releases (and how many of them to support), and that it
may put other types of strains on the developers, but from my
perspective (as a user of Django), I would appreciate such regularity,
and I would rather have releases doing small increments than big
leaps.

That said, I understand that the final say has to come form the actual
developers, based on what they feel is realistic.

Cheers, and keep up the great work!

johan

Indeed; I suspect that six months is an ambitious but fair goal. As
you can see from the 1.1 proposal, it's what I'm thinking in this
particular case, and I suspect we'll lean towards the shorter
timelines most of the time.

Again, the only reason to have some flex here is to allow for
development of bigger features. Or do you think it would be better to
simply cancel anything that can't be done in four months?

Jacob

For features that take longer than 4 months, isn't that one of the
main reasons for using branches? It seems to me that we could make
hard deadlines of "On May 1 and Nov 1 of every year, there will be a
new version" and if things aren't done, they stay in their branch. Is
this too short sided?

 -justin

On Mon, Sep 15, 2008 at 9:31 AM, Jacob Kaplan-Moss

--
Justin Lilly
Web Developer/Designer
http://justinlilly.com

Hi Jacob

since we are going to have new features in the ORM, I suggest to
include the ticket #7338 ( Method .cache(timeout) in QuerySet [2] ).

other 2 approaches about the same thig are the [2] and [3] solutions,
made by David Cramer and Jerzyk

[1] http://code.djangoproject.com/ticket/7338
[2] http://code.google.com/p/django-orm-cache/
[3] http://www.djangosnippets.org/snippets/502/

2008/9/15 Johan Liseborn <johan.liseb...@gmail.com>:

--
Marinho Brandão (José Mário)
http://marinho.webdoisonline.com/

On 15 Sep 2008, at 14:31, Jacob Kaplan-Moss wrote:

No, as you said, I'd rather see a certain degree of flexibility in the  
release schedule to accommodate for bigger features where it makes  
sense.
For example, if something like newforms-admin was being developed now  
and one more month was needed to ensure it's rock solid, extending the  
release cycle by one month is entirely reasonable in my view, and  
benefits everyone.
I think it would even be reasonable for big fat contrib apps (of the  
caliber of GeoDjango for example).

It all depends on feature lists and comes down to pragmatism in the end.
In this view, I think the plan you've outlined for 1.1 is good.

Cheers

Cyril

Hey Marinho --

Let's hold off talking about actually features right now; this thread
is for a more meta-discussion of releasing in general.

[But please add those things here:
http://code.djangoproject.com/wiki/Version1.1Features]

Jacob

Le 15 sept. 08 à 15:31, Jacob Kaplan-Moss a écrit :

Can't big features be planed for n+x release? This way you can release  
often and add big features too, it reminds me Ubuntu cycle (6 months +  
Long Term Support, even if it will probably be more a Big Feature  
Inclusion in our case). The best of two worlds.

David

For some projects (GNOME comes to mind) this works.

For us, however, I worry about branches getting stale. Any branch that
can't merge from trunk every couple of days is in danger of going off
the rails. This means that any branch that needs to persist across a
release cycle runs a major risk. The bug fixes in the run-up to the
release come fast and furious, and so the branch maintainer will have
to spend a huge amount of effort keeping things up-to-date. This
effectively takes that branch maintainer out of picture for that
cycle: s/he will have to spend most of the time just keeping the
branch up-to-date.

Further, there are often huge scads of bugs that only a major refactor
can fix -- think, for example, the old/newforms switch. Imagine, then,
that we had a similar task big: let's say next year some time Adrian
puts out "really-new-forms" which completely blows newforms away. It's
so awesome that we *have* to refactor forms again.

[Note to Adrian: don't do this. Please.]

If we're on a hard-assed time schedule and this task takes longer than
six months, we'll have two choices: release a version of Django with a
library we know to be broken or crappy, or release Django with a
half-finished new library.

I'd prefer, in a case like this, to have the flexibility to push a
longer release schedule.

Jacob

On Mon, Sep 15, 2008 at 12:18 PM, Jacob Kaplan-Moss

There seems to be a bit of consternation about the "6-12" part of
this, so I think I should clarify a bit, and explain a bit of
background.

I think we all agree that more frequent releases are good.

Six months is about as quick as we can do feature releases right now.
For reasons I've explained elsewhere, about the fastest we can go from
feature freeze to a release is two months; it'd be silly to spend more
than 1/3rd of our energy simply churning out releases -- we *do* need
time to develop Cool Shit!

Django is, above all, a framework for pragmatists: the things we add
to the framework are driven by real needs of real web developers.

We need the flexibility to respond to new changes in the
state-of-the-art; if we need to make a change that takes more than six
months, should we really avoid that change simply because it doesn't
fit into some arbitrary schedule?

Jacob

On Sep 15, 3:26 pm, "Jacob Kaplan-Moss" <jacob.kaplanm...@gmail.com>
wrote:

I'm not talking about big feature releases. KDE releases 4.1.x updates
every month with bugfixes and minor feature improvements. That's what
I called "minor" features. I'm not thinking about anything the scale
of nfa or qsrf, just some small new features that are easily tested
and very unlikely to break anything. Think of some function accepting
an optional extra parameter to reorder its output or whatever.
Additionally, let's assume there is a new shiny contrib app around
that is supposed to be included in a release. I don't see any reason
why this app couldn't be tested while it's still in its own branch and
merging it into a monthly release when it can be considered stable. I
don't really have any data on this, but is it really necessary to make
an alpha/beta/rc release with such an addition to get reasonable
testing?

I read that and my versioning scheme actually works pretty much like
that. The only difference is that minor releases are made on a more
frequent regular basis. In my opinion, backwards _compatible_ changes
should be tested in their branches and not in alpha/beta/rc releases,
especially when dealing with very modular software such as Django,
where new features are easily decoupled from the rest of the code. You
probably have way more insight in what happened during the alpha/beta/
rc releases prior to 1.0 but when concentrating more on indiviual big
features (say nfa) wouldn't it be possible to stabilize these without
aiming for a specific release date?

That's what I might do for what I called a major release.

My experiences in Gentoo have taught me to avoid the need for
consensus-based decisions like the plague. I have to say that so far,
the core devs seem to have been able to reach consenus (e.g. about new
features) easily. In Gentoo, the process of introducing a new API has
been the cause for way too much drama, drove developers away from the
project and lead to personal insults. It's probably best to ignore
this point of mine, as the Django community and developers generally
seem to be a nicer kind of people than where I come from (Gentoo). As
long as there is someone around with the authority to say what gets
done in what way, I don't see a problem.

Of course, but Django will eventually need some more of these changes.
My version scheme provides an easy way for users to say "This code is
written for Django 1.* and as long as I've got that major version
installed, I don't need to worry about anything" (note that this
implies to recognize feature deprecation as an API change, while with
the current scheme, upgrading from 1.0 to 1.1, to 1.2 etc. all the way
to 1.5 may break code - I know that there are deprecation warnings,
but you can't really expect users to watch out for these as they
update their servers). Coming from a distribution background, I'm very
concerned about _automatically_ upgrading some package (Django) and
breaking another (user code) in the process.

My main concern is that while getting 1.0 out got everyone excited and
motivated, future releases might not enjoy the same enthusiastic
anticipation. Time-based releases as I described them allow for
feature development to set its own, individual pace without having a
release deadline.

In another post in this thread you gave the example of a really-new-
forms refactor coming around and having to make a decision whether to
"release a version of Django with a library we know to be broken or
crappy, or release Django with a half-finished new library". In this
scenario Django would have had the supposedly crappy newforms all
along, one more release with it won't be the end of the universe.
With, say, a two-month release cycle, you could just wait for really-
new-forms to be done and stable, then merge it as really-new-forms. At
the next minor release, it is included under that name. At the next
major release, it replaces newforms as forms.

There's also the point that a shorter release cycle means the necessity
(or at least desirability) of maintaining more releases. If, for
example, Django were to go to a (IMHO ridiculously short) two-month
cycle then it would no longer be reasonable to suggest that the
four-month-old "two releases ago" would no longer get security fixes.

Six months seems ambitious to me, but short enough to keep most people
who aren't living right on the bleeding edge happy. If you want the new
features, then rely on the trunk's stability and use that, with
extensive integration testing before deployment.

regards
 Steve
--
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/

My views generally coincide with Torsten's. The huge bump from 0.96 to
1.0 was necessary, but now that the codebase is stable, I think Django
would benefit from more granular releases.

See Jacob's post on Django 1.1 schedule. Model validation and ORM
aggregation are nearly done, yet they won't be included in a stable
release for the next 6 months. A big part of the rationale for longer
cycles was the necessity to get the code production-ready. But if you
were to roll out 1.1 with just these two features, plus maybe several
small improvements that happen along the way, how long would it take
to do that? OTOH, if at some point you need to introduce a really big
set of changes, you can stretch the release time as needed.

Backwards-compatibility is another issue. With shorter release cycles,
you'd probably need to change the deprecate in .x => remove in .(x+3).
But if you maintain bacwards-compatibility longer, you could still
keep the security fixes policy. Once you're up to say 1.3, there
really is no need to backport security fixes to 1.1 - there'd be no
advantage in updating 1.1.4 -> 1.1.5 instead of 1.1 -> 1.3.

I guess that another big question is this: is Django's policy still to
encourage people to run on trunk, or the latest stable version? Longer
release cycles would lead to more people on trunk, and hence probably
better debugging etc; but shorter ones might just make Django more
attractive, especially for enterprise level usage (whatever the hell
that means).

Six months sounds about right to me, if you think about what goes into
landing a couple of "simple", "easy", "nearly done" large features and
producing something of the quality of a release from it.

--
"Bureaucrat Conrad, you are technically correct -- the best kind of correct."

On Mon, Sep 15, 2008 at 11:18 PM, Jacob Kaplan-Moss <

I'd be keen on this happening more regularly - that would take a dedicated
maintainer, but wouldn't be overly difficult. I'm thinking the guideline of
"3-5 months" as written is too long, and we should focus on the "when enough
bugs are fixed" - which could be 2 weeks if some key issues are identified
after a minor release, or 2-3 months if things are relatively quiet. So we
might get: 1.1.1 within 4 weeks of a release, fixing any obvious bugs that
slipped through the alpha/beta process. These would be things that would
have been fixed if they'd been found before 1.1. After that as soon as a few
more bugfixes pile up (like 10-20 or 2-3 key ones), or 2-3 months passes, do
another micro release.

It takes effort to backport fixes from trunk to a branch, so we may as well
push them out to the community quickly too. It also helps adoption - new
users get a really stable, non-broken release, and get a really good first
impression of Django.

The RFC also doesn't mention when backporting is stopping? Do we do a 1.1.3
at the same time 1.2 is released, just before the branch changes to
"security-fixes-only"? Or are they going to stop once 1.2alpha1 is released,
unless something really broken is found?

Rob :)

On Sep 15, 10:18 pm, "James Bennett" <ubernost...@gmail.com> wrote:

Whoa, when did I ever suggest that these features are simple or easy?
I said "nearly done" because those are the exact words that Jacob used
in his post. I have never seen the code for these branches and was
honestly enquiring how long would it take to finish them, incorporate
them into the trunk and get them stable. If it could be done in, say,
2 months, then it'd be worth considering doing a release based on
that. But if you know the code and say it's six months, then six
months it is.

Look, I realise that you guys have already discussed this at length
and that the chances of you changing your minds is ~0. But I'm still
free to voice my bona fide opinion, no? And you're still free to do
what you think is the best.