RFC: new backports policy

88 views
Skip to first unread message

Jacob Kaplan-Moss

unread,
Apr 19, 2011, 7:22:26 PM4/19/11
to django-developers
Hi folks --

Over the past few weeks, the core development team have been
discussing how we can streamline our process to get more work done
quicker. It's pretty clear that the bulk of the community wishes
Django could move forward a bit faster [1], and we'd like to be able
to deliver more awesomeness in less time. Ultimately, the goal will be
to put out releases quicker, hit our deadlines more accurately, and be
able to respond to community suggestions and patches in a more timely
manner.

[1] This isn't unique to Django, of course; almost every open
source project wishes they could release more features more quickly.

One way that we'd like to improve our speed is by modifying our
existing backport policy. To refresh your memories, right now we
backport any non-feature-containing patches to the previous release.
So if we fix a bug on trunk, we backport the bug fix to the 1.3.X
branch, where it would become part of the 1.3.1 release. This is a
fair bit of work: it basically means we have to fix each bug twice.

The core team has come to a rough consensus and we're planning to drop
this backport-everything policy. Instead, we'll only backport critical
patches. That is, we'd only backport patches for:

* Security issues.
* Data-loss bugs.
* Crashing bugs.
* Major functionality bugs in newly-introduced features.

In other words, we'd basically only backport bugs that would have
prevented a release in the first place.

Practically, this means that if there's a minor bug in 1.3 -- for
example, #15795, a bug with the representation of RegexURLPattern --
the fix *will not* be applied to the 1.3.X branch, and thus no 1.3.*
release will contain the fix, even if it's fixed in trunk. To get the
fix for these bugs, users will have to upgrade to 1.4.

The upside, of course, is that bug fixes will be quicker to apply, so
we can fix more bugs, so we can get that 1.4 release out sooner [2].
Additionally, this'll give users more of an incentive to upgrade to
the latest-and-greatest. This means that they get new features, and we
(the development community) get to focus more clearly on moving
forward, not maintaining older releases.

[2] Look for a proposed release date soon. Spoiler alert: it's
sooner than you think.

We'd like to make this change effective immediately, but we also don't
want to just issue this change by fiat. So we're requesting comments
and feedback on this change. Do you like the idea? Why or why not?
Will this policy make it more likely you'll contribute? Less likely?

Thanks!

Jacob

Tobias McNulty

unread,
Apr 19, 2011, 8:16:17 PM4/19/11
to django-d...@googlegroups.com
Hi All - 

In general I prefer to think about this the other way around: If we have a most current stable release of product X, it makes sense to fix any bugs that come up in that product.  For the sake of the next release, assuming the bug was not fixed or made obsolete by some other change, it then also makes sense to /forward port/ that bug fix to the current (unstable) development trunk.  Calling it "forward porting" rather than "back porting" makes it sound better, too. :-)

This workflow fits quite well if you're using one of the DVCS mirrors of the Django repository, as it can be as easy as merging the latest release branch into default/master on a regular basis.  There is obviously more to the story (running tests, dealing with the occasional merge conflict), but to me, it seems like the majority of the work is in fixing the original bug in the first place, not in applying that same fix to another branch of the code.  If my assumption is true, we might as well apply the fix in both places once it has been made.

Being a non-core-committer perhaps I don't fully understand the additional work involved, but I do appreciate the focus on stability in Django's releases and I'd rather be forced to upgrade due to new features that I want, rather than some silly bug that'll never be fixed -- especially if a stable release that includes the fix I need is not even available yet.  In other words, if a bug in a product has been fixed, it seems odd to me that no stable release of the product will include that fix.

What if the window for bug fixes was time boxed, so that bug fixes would be back/forward ported as usual up until 1-2 months prior to a new stable release (perhaps coinciding with the first release candidate)?  This way at least we have some semblance of a stable release destined to include all bug fixes at all points in time, and the actual window in which a bug fix might not make it into the current official stable release is reduced.

Just my 2 cents - hope that helps!

Tobias
--
Tobias McNulty, Managing Partner
Caktus Consulting Group, LLC
http://www.caktusgroup.com

Aymeric Augustin

unread,
Apr 19, 2011, 8:51:36 PM4/19/11
to django-d...@googlegroups.com
On 19 avr. 2011, at 21:22, Jacob Kaplan-Moss wrote:

> We'd like to make this change effective immediately, but we also don't
> want to just issue this change by fiat. So we're requesting comments
> and feedback on this change. Do you like the idea? Why or why not?
> Will this policy make it more likely you'll contribute? Less likely?

Hello Jacob,

I thought that the work required to backport a patch was relatively low compared to the work required to understand a ticket and review proposed changes. Not being a core developer, I'll take your word that it is "a fair bit of work".

As someone who manages development projects on a 1-2 year scale, I am +1 on this new policy. At some point, these projects stabilize and development turns into maintenance. We want to use the most recent release of Django at that point, and then stay on that branch:
- more frequent releases means we could use releases during the "build" (rather than trunk) and still get fixes within a decent timeframe;
- less changes in maintenance branches means it would be safer to update Django during the "run".

As someone who hacks on Django, the main factor preventing me from contributing more is the relatively low chance that someone will review or commit my patches. Reviewing patches written by others hits the same problem: more often than not, I upload an improved version, thus I can not mark the ticket as RFC. The backlog of unreviewing and uncommitted patches is quite discouraging. If this new policy helps core developers to review and commit patches faster, I'm also +1.

Best regards,

--
Aymeric Augustin.

Yishai Beeri

unread,
Apr 19, 2011, 9:13:17 PM4/19/11
to django-d...@googlegroups.com
Perhaps augment the new policy by stating that contributed backports for
bugs that are left out by this change will be favorably looked upon, and
committed to the branch unless there is a good reason no to commit them.
This still achieves the goal of making the routine bugfix commit cycle
faster, but allows people that depend on stable releases to "scratch their
itch", backport the important bugfixes, and get them as part of a
"blessed" stable branch release.


On Tue, 19 Apr 2011 22:22:26 +0300, Jacob Kaplan-Moss <ja...@jacobian.org>
wrote:

Daniel Moisset

unread,
Apr 19, 2011, 9:35:40 PM4/19/11
to django-d...@googlegroups.com
On Tue, Apr 19, 2011 at 4:22 PM, Jacob Kaplan-Moss <ja...@jacobian.org> wrote:
> Hi folks --
>
>
> The core team has come to a rough consensus and we're planning to drop
> this backport-everything policy. Instead, we'll only backport critical
> patches. That is, we'd only backport patches for:
>
> * Security issues.
> * Data-loss bugs.
> * Crashing bugs.
> * Major functionality bugs in newly-introduced features.
>
> In other words, we'd basically only backport bugs that would have
> prevented a release in the first place.

Is this a policy against backporting fixes by default, or against the
backports themselves?

Let me put it in another way with an example: you don't want to do the
backport, and it's your policy, and it sounds like a good idea. But
I'm using 1.3 in production and there's a bugfix I really want, so I
do the backport (and write the code, tests, docs). If I submit this to
the issue tracker, is there a chance my patch will get into the next
minor release, or you won't even consider it?

The answer for this should be included in the documentation, it's
useful for end-users and also for people reviewing patches

Regards,
D.

Ivan Sagalaev

unread,
Apr 19, 2011, 9:47:58 PM4/19/11
to django-d...@googlegroups.com
On 04/19/2011 02:35 PM, Daniel Moisset wrote:
> I'm using 1.3 in production and there's a bugfix I really want, so I
> do the backport (and write the code, tests, docs). If I submit this to
> the issue tracker, is there a chance my patch will get into the next
> minor release, or you won't even consider it?

Even with an accepted (or even ready-for-checkin) ticket containing all
what's needed committers have some non-trivial amount of work to do. As
I understand intention of the new policy it's this work that they want
to decrease.

In such situation at my company we just patch our internal Django
package. In fact we often patch an immediate problem locally first and
only then bring the patch to discussion upstream. Much less pressure for
everyone. I think you should either live with official stable releases
or be ready to maintain your own if you choose to live on trunk.

Luke Plant

unread,
Apr 19, 2011, 10:24:17 PM4/19/11
to django-d...@googlegroups.com
Another issue with regards to backporting bug fixes that Jacob didn't
mention is the fact that bug fixes often introduce subtle regressions
and backwards incompatibilities.

Personally, I find that backporting a fix very often takes only a
minute, by appropriate use of DVCS features (e.g. hg transplant) and
some scripting. (This doesn't include the time for running tests again,
and doesn't include cases where the merge fails or the patch has to be
effectively re-written). However, the problems caused by introducing
these regressions onto supposedly stable branches can be much more time
consuming and tricky.

By keeping backporting of bugs down to a minimum, we avoid these
problems, and help keep our stable branch genuinely stable. Not
introducing regressions is much more important than fixing existing
bugs, since most people will have worked around existing bugs where
necessary for their needs anyway.

Regards,

Luke

--
"In my opinion, we don't devote nearly enough scientific research
to finding a cure for jerks." (Calvin and Hobbes)

Luke Plant || http://lukeplant.me.uk/

Carl Meyer

unread,
Apr 19, 2011, 10:39:44 PM4/19/11
to django-d...@googlegroups.com
On 04/19/2011 05:24 PM, Luke Plant wrote:
> Another issue with regards to backporting bug fixes that Jacob didn't
> mention is the fact that bug fixes often introduce subtle regressions
> and backwards incompatibilities.
>
> Personally, I find that backporting a fix very often takes only a
> minute, by appropriate use of DVCS features (e.g. hg transplant) and
> some scripting. (This doesn't include the time for running tests again,
> and doesn't include cases where the merge fails or the patch has to be
> effectively re-written). However, the problems caused by introducing
> these regressions onto supposedly stable branches can be much more time
> consuming and tricky.
>
> By keeping backporting of bugs down to a minimum, we avoid these
> problems, and help keep our stable branch genuinely stable. Not
> introducing regressions is much more important than fixing existing
> bugs, since most people will have worked around existing bugs where
> necessary for their needs anyway.

+1 to all of this. IMO the bigger issue is the regressions that
inevitably get introduced when we backport all bugfixes, not the extra
committer time to do backports (which I also agree I haven't found
problematic).

Carl

Paul McMillan

unread,
Apr 20, 2011, 7:56:03 PM4/20/11
to django-d...@googlegroups.com
+1, I agree with Carl and Luke. The issue here is that for
non-showstopper bugs, users have probably found (or may even be
relying on!) the existing behavior. Keeping the "stable" branch more
stable by only changing things when there's a serious issue seems to
be a positive improvement.

I think this point means that even community supplied patches to minor
issues probably don't belong in the stable release. It's easy enough
to maintain a patched local version for your own particular itches.
Better stability and fewer regressions from patches related to minor
issues seems like a fair tradeoff.

I am in favor of a more open policy regarding backporting docs fixes.
We expect users to be developing using the stable docs, and if we have
identified a deficiency there, we should fix it so that they will
benefit before the next major release.

-Paul

Markus Gattol

unread,
Apr 20, 2011, 8:32:41 PM4/20/11
to Django developers
That's certainly a move in the right direction so +1 from me too. The
problem of
backporting correlates with how much time passes between any release
i.e. long time between releases gives bigger pita with backporting.
Even more so because you have several version control systems etc.
etc. (not going to rant about that fact :). What if we'd move release
dates so close that maybe backporting isn't an issue anymore?

What's really stopping us from ditching all but one DVCS and have
commit IDs/tags/whatever on a weekly basis that we advertise as "good"
or "stable" or whatever? Is Django so big and users are so cemented in
how to think about and use it that we must at all cost carry forward
the several decades old notion of having a fat release every now and
then (even if it goes down to several weeks, the procedures won't
change, just the backport pita might get a little smaller imo)? Don't
know, just thinking aloud ...
Message has been deleted
Message has been deleted

Jeremy Dunck

unread,
Apr 20, 2011, 8:52:07 PM4/20/11
to django-d...@googlegroups.com
On Wed, Apr 20, 2011 at 3:32 PM, Markus Gattol <markus...@gmail.com> wrote:
> That's certainly a move in the right direction so +1 from me too. The
> problem of
> backporting correlates with how much time passes between any release
> i.e. long time between releases gives bigger pita with backporting.
> Even more so because you have several version control systems etc.
> etc. (not going to rant about that fact :). What if we'd move release
> dates so close that maybe backporting isn't an issue anymore?

There is a lot of effort in making a release -- ensuring that it is
well-tested and stable. While it might be possible to shorten cycles,
I don't think that removes the need for support of old versions.
Anyway, a faster release cycle while keeping the existing support
policy would effectively shorten the time to upgrade and place
pressure on downstream cycles.

> What's really stopping us from ditching all but one DVCS and have
> commit IDs/tags/whatever on a weekly basis that we advertise as "good"
> or "stable" or whatever? Is Django so big and users are so cemented in
> how to think about and use it that we must at all cost carry forward
> the several decades old notion of having a fat release every now and
> then (even if it goes down to several weeks, the procedures won't
> change, just the backport pita might get a little smaller imo)? Don't
> know, just thinking aloud ...

It's funny, early in the history of the project, we tried to argue
that position and regularly got requests for "stable" releases despite
protests that trunk was, in fact, stable. Even so, the need for
releases is more about community cohesion and less about technical
limits. Support gathers around releases. And there are many people
running django in prod who are not active contributors -- the answer
to these people can, with releases, be "read the release notes".
Without releases, what is a reasonable answer?

Tobias McNulty

unread,
Apr 20, 2011, 9:37:23 PM4/20/11
to django-d...@googlegroups.com
On Wed, Apr 20, 2011 at 3:56 PM, Paul McMillan <pa...@mcmillan.ws> wrote:
+1, I agree with Carl and Luke. The issue here is that for
non-showstopper bugs, users have probably found (or may even be
relying on!) the existing behavior. Keeping the "stable" branch more
stable by only changing things when there's a serious issue seems to
be a positive improvement.

Okay, I do think the regression issue makes a much stronger argument than the developer time issue.

I'd be more comfortable if the policy stated that any new bugs introduced by the last release would be backported to that release.  It's possible that "major functionality bugs in newly-introduced features" will equate to virtually the same thing, but I'm not clear what would constitute a major functionality bug (it sounds big, and like it might be a difficult criterion to meet).

On the other hand, if we've all been living with a trivial bug since 1.0, we can probably live with it for just a little longer, and backporting certainly doesn't sound worth the potential risk of a regression.

Cheers,

Carl Meyer

unread,
Apr 20, 2011, 9:51:33 PM4/20/11
to django-d...@googlegroups.com

On 04/20/2011 04:37 PM, Tobias McNulty wrote:
> I'd be more comfortable if the policy stated that any new bugs
> introduced by the last release would be backported to that release.
> It's possible that "major functionality bugs in newly-introduced
> features" will equate to virtually the same thing, but I'm not clear
> what would constitute a major functionality bug (it sounds big, and like
> it might be a difficult criterion to meet).

I like this. In general we treat regressions as release blockers anyway,
so this fits with the "backport if it would have blocked the release"
spirit of the proposal.

Carl

Luke Plant

unread,
Apr 20, 2011, 10:24:13 PM4/20/11
to django-d...@googlegroups.com
On 20/04/11 22:37, Tobias McNulty wrote:

> Okay, I do think the regression issue makes a much stronger argument
> than the developer time issue.
>
> I'd be more comfortable if the policy stated that any new bugs
> introduced by the last release would be backported to that release.
> It's possible that "major functionality bugs in newly-introduced
> features" will equate to virtually the same thing, but I'm not clear
> what would constitute a major functionality bug (it sounds big, and like
> it might be a difficult criterion to meet).

Hmm, Jacob didn't specifically mention regressions, though in our
discussions on django-core we did include them. Was that a deliberate
change Jacob? Or perhaps regressions are covered by this sentence in
your email:

> In other words, we'd basically only backport bugs that would have
> prevented a release in the first place.

which certainly does include regressions introduced since the last release.

Luke

--
"I regret I wasn't born with opposable toes." (Calvin and Hobbes)

Luke Plant || http://lukeplant.me.uk/

Jacob Kaplan-Moss

unread,
Apr 20, 2011, 10:39:43 PM4/20/11
to django-developers
On Wed, Apr 20, 2011 at 5:24 PM, Luke Plant <L.Pla...@cantab.net> wrote:
> Hmm, Jacob didn't specifically mention regressions, though in our
> discussions on django-core we did include them.

Yup, sorry - was moving too fast. Regressions, clearly, get backported
-- if something worked in 1.2, it should work in 1.3 unless we've
deliberately removed it.

Jacob

Reply all
Reply to author
Forward
0 new messages