[Trac-dev] Proposal: RoadMap strategy details

[mrelbe]

Hi all, and especially Christian

The RoadMap wiki page, together with all tickets, was updated
yesterday to convey a new planning strategy. The new approach seems
easier and clearer to grasp, I think.

To make things somewhat even more clear to all of us, perhaps we
should define what we mean with "major releases" and "minor releases"
on the RoadMap wiki page. Currently, a hint is expressed in the
description of milestone 0.12.1: "enhancements should go directly into
0.13"

That's all clear and fine, but I would like that to be explained in
general on the RoadMap page. I also think the TracTicketTriage also
need to be updated with some info, or hints.

I know I could just jump in and start editing the wiki pages, but I
hesitate in doing that since I am not involved in the planning.

I would also feel more confident in actually taking ownership of
tickets and changing milestones if I understood the strategy, some
milestones currently express details regarding that as well: 0.12.1
again states "Move tickets from next-minor-0.12.x to here once you
know you're going to fix them for the 0.12.1 release".

What do you think? Could the RoadMap wiki page be the place to express
a general planning strategy to help me in understanding what I should
or shouldn't do regarding planning of tickets?

With best regards
Mikael

--
You received this message because you are subscribed to the Google Groups "Trac Development" group.
To post to this group, send email to trac...@googlegroups.com.
To unsubscribe from this group, send email to trac-dev+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/trac-dev?hl=en.

[Christian Boos]

Hello Mikael,

On 5/3/2010 8:22 AM, mrelbe wrote:
> Hi all, and especially Christian
>
> The RoadMap wiki page, together with all tickets, was updated
> yesterday to convey a new planning strategy. The new approach seems
> easier and clearer to grasp, I think.
>

Thanks - this is still in the making though, as you've noticed.

> To make things somewhat even more clear to all of us, perhaps we
> should define what we mean with "major releases" and "minor releases"
> on the RoadMap wiki page.

Minor releases are maintenance releases happening on a given line of
development (0.11.1, 0.11.2, ...).
Major releases correspond to new lines of development (0.11, 0.12, 0.13,
...).

But I suppose the real question was "what's the differentiation between
a minor and major release", or more precisely, "what makes a ticket go
to either a minor or a major release"?

The answer is two fold: how we did things so far, how do we intend to do
it from now on.

For 0.11, our minor releases were subtitled "Bug fixes and minor
enhancements", and when it was not depending on new code found only in
trunk, a feature or a bug fix was likely first implemented on
0.11-stable, then ported to trunk for 0.12. However, big new features
did go to trunk exclusively (e.g. i18n, timeline user filtering, custom
query rework, wiki editor and syntax improvements, multirepos). So you
had two Trac versions with two speed of development paces, as both
0.11-stable and 0.12dev were improving over time.

The idea behind that was to allow the users to benefit as soon as
possible from the new features, and this worked somehow.

The only downside was that the real big new features took a really long
time before hitting the average users. Granted, we tried our best to
always keep trunk in a good shape, and I think the users who trusted us
to always be on the tip of the trunk were not disappointed.
Nevertheless, it took a bit more than two years before 0.12 could be
more released "to the masses" (0.11-stable was forked end of April 2008).

We would like to shorten this cycle, possibly quite drastically, hence
the new strategy.

After 0.12, we'd like to deliver major releases at a higher pace, so
that the users could benefit from the big new features earlier, instead
of having those big features "waiting" on trunk for a long time, until
all possibly unrelated work in progress has reached a good level. Note
that with the former model, the risk could even be that we never reach a
"good enough" level and that the major release keep being postponed for
ever. We manage to avoid this the last few years, but the risk is
nevertheless present.

As we see it, there are several ways to achieve this goal.

The first is to really focus on the next major version (0.13 here), and
consider the stable branch (0.12.x here) as really stable, i.e. frozen
feature wise and only getting critical fixes, instead of being polished
continuously and incrementally. This is simply because it's a real time
sink to do so and we don't have that many resources to spread around
(otherwise we would still happily maintain 0.10.x, for the many happy
users of that version ...).

The second improvement is to plan ahead more realistically and more
conservatively: instead of the 200+ tickets heavy milestones we had in
the past for the major releases, we'd like to put on a given defined
release (like 0.12.1 and 0.13) only what we feel confident we can
implement for that release, targeting say no more than 80 tickets. To
get an idea, 0.9 had 400 tickets and took 1 year, 0.10 had 300 tickets
and took 1 year, 0.11 had 555 tickets and took 1.5 year, and finally
0.12 had 300 tickets (0.12 + 0.12-multirepos milestones) and took 2
years. So 100 tickets in 6 months looks realistic. Of course, this
highly depends on the nature of the tickets, and 0.12 had many
challenging tasks in it, hence the lower "velocity" compared to the
other releases despite the fact there has been assuredly *more* things
done in 0.12 than in the previous releases.

This more realistic planning is also a change in mind: we don't put in
the next milestone "what would be nice to have", or "what users want
most", but, as honestly as we can, "what we know will be worked on and
done". This means for a large part that we'll move there only tickets
from the pools that are already quite advanced, those not only with a
patch but with active feedback and discussion between contributors.

The last avenue of improvement is to make a better use of feature
branches. While nothing is yet finalized, we're inclined to start using
the DVCS quite more, with Mercurial and git mirrors of our SVN trunk as
a basis. That way, we could create topic branches for tickets and even
repository clones for the more ambitious changes. There could also be
temporary integration branches, where everything gets merged together
(in the spirit of the proposed update "pu" branch in git development).
If this model (or similar) proves to be successful, in the longer term
we could think about a migration. But this is not a near term goal, as
Subversion does the job adequately for us (well, except for feature
branches).

> Currently, a hint is expressed in the
> description of milestone 0.12.1: "enhancements should go directly into
> 0.13"
>

This probably needs clarification (as I said, this whole reorganization
is in the making, so expect a few inconsistencies here and there). *If*
a ticket should go to 0.12.1, this means that it already fills the
criteria for a defined milestone (active patch or commitment from a
developer). If this is the case and in addition it's an *enhancement*
and not a *defect*, then it should rather go to 0.13, because of the
feature freeze for the stable branch discussed earlier. So I should
rephrase this simply into "don't consider adding new features to 0.12.1".

> That's all clear and fine, but I would like that to be explained in
> general on the RoadMap page. I also think the TracTicketTriage also
> need to be updated with some info, or hints.
>

Sure, I picked RoadMap because it's directly reachable from the
WikiStart and I think is the right place for an overview. The details
and discussion belong to TracTicketTriage.

> I know I could just jump in and start editing the wiki pages, but I
> hesitate in doing that since I am not involved in the planning.
>

Once the rules are better defined, we will gladly appreciate help in
triaging, and you're welcome to help us define those rules. The main
idea is to have the milestones and tickets be a support for helping us
making Trac go forward, and not an hindrance, overwhelming us with
wishes that are never going to be fulfilled. Our goal is to shape the
vast amount of ideas and expectations concerning Trac in a coherent
vision that we're going to realize, in the coming years. If this is
getting clearer for us, I suppose this will also become clearer for a
wider audience outside of the core development team.

In order to make this vision more palatable, we decided to go away with
the 1.0 and 2.0 milestones, which had the bad connotation of "never"
anyway in people's mind. Instead, we decided to split valid tickets in
two groups. Those which are part of the vision we want to make happen,
and for which we reasonably expect to be able to involve ourselves (be
it coding it, helping a patch to get finalized) will go on
"next-major-0.1X", and those which don't. The latter will go to the
"unscheduled" milestone, meaning that as nice those enhancements
suggestions were, or as mind-boggling the reported defects are, they
simply didn't have caught our interest (we can't possibly do everything,
and meet every direction, right?).

Now I'm sure that at this point, the big question you have is: "who's
that *we*"? Well, plain simple, the people who are maintaining Trac on a
regular basis and who enjoy making it better day after day, month after
month, release after release. That's currently Remy and me, but there
have been many others in the past, and we're quite open to new
contributors, which means this can be you in the future. The rule is the
usual one in open source teams, show us the code, participate to the
development process, and after a while, if we're confident we can work
well together in a constructive way, you're in.

> I would also feel more confident in actually taking ownership of
> tickets and changing milestones if I understood the strategy, some
> milestones currently express details regarding that as well: 0.12.1
> again states "Move tickets from next-minor-0.12.x to here once you
> know you're going to fix them for the 0.12.1 release".
>

The dust is not settled, things are just being reorganized *now*. The
idea is to have all of the former 1.0 and 2.0 tickets currently in the
"triaging" milestone being dispatched to "unscheduled", one by one.
Parallel to that, we move away from "next-milestone-0.1X" the tickets
which no longer belongs there, putting them in "unscheduled" as well.
Once this first phase is over, we'll move the remaining "triaging"
tickets to "next-milestone-0.1X" (as you can imagine, if we moved
directly a ticket from "triaging" to "next-milestone-0.1X", we would
have to consider it twice). Note that I didn't move the
"next-milestone-0.1X" to "triaging" on the assumption than most tickets
on "next-milestone-0.1X" will probably remain there, though I'm not
really sure about that, quantitatively speaking.

In the end, mid term, you'll see the following :
- the tickets on "next-major-0.1X" are tickets we're interested in and
define the goals we want to reach; highest priority are the short term
goals and lowest priority the long term ones. The severity will be
indicative of the impact of the ticket on the project. For example, the
important but long term goals will have a low priority and high
severity. The priority ranking will give hints, but will not be binding,
we expect things to change depending on the dynamics of the contributions
- once a ticket from "next-major-0.1X" is sufficiently advanced, or
there's really a high motivation to dedicate the time necessary to make
it happen, it goes to the next defined major milestone ("0.13" here)
- the tickets on "unscheduled" simply didn't make it, sorry. We had
some vivid internal discussions whether those should simply be closed as
wontfix, but finally, the idea that someone could turn an "unscheduled"
ticket into one for "next-major-0.1X" by becoming an active member of
the Trac team was found more appealing ;-)
- the newly created tickets could go on "triaging"

Symmetrical to next-major-0.1X and 0.13, we have "next-minor-0.12.x" and
"0.12.1". This should only concern bug fixes and those which don't imply
major changes more fitting to a major release. Note that in these
conditions, we could even manage to have time-based minor releases (say
every 2 months).

As before, we also have a number of ancillary milestones with clearly
defined scopes: "translations", "not applicable", "plugin - mercurial"
and "plugin - spam-filter".

Thank you for reading that far, as that was a rather long mail, which I
hope clarifies all the questions you had.

-- Christian

[Itamar O]

Hi Christian,

In opensource projects you usually see a 3-level versioning scheme that follows the "Major.Minor.Patch" concept.
According to your detailed description, Trac follows a "shifted" scheme: "0.Major.Minor".
So I must wonder - what is the purpose of "0." if according to the strategy there will never be a "1."?
Or differently phrased - what needs to happen in Trac in order to go "1."?

Itamar O.

[Christian Boos]

On 5/4/2010 5:55 AM, Itamar O wrote:
> Hi Christian,
>
> In opensource projects you usually see a 3-level versioning scheme
> that follows the "Major.Minor.Patch" concept.
> According to your detailed description, Trac follows a "shifted"
> scheme: "0.Major.Minor".

Correct.

> So I must wonder - what is the purpose of "0." if according to the
> strategy there will never be a "1."?

Historical reasons, the first public release was 0.5, and at 0.9 we felt
we were too far away of a finished product to feel confident to go with
a "1.0", so that 0.10 was picked and from there the sequence of major
releases just went on.

> Or differently phrased - what needs to happen in Trac in order to go "1."?

Think about it differently: what would be the advantage? If we find a
role and a need to use the 1.0 version, then we'll probably use it. But
having the number as a goal in itself, when no real meaning stands
behind, is pointless. Hence the removal of the 1.0 milestone.

[Noah Kantrowitz]

On May 3, 2010, at 11:59 PM, Christian Boos wrote:

> On 5/4/2010 5:55 AM, Itamar O wrote:
>> Hi Christian,
>>
>> In opensource projects you usually see a 3-level versioning scheme that follows the "Major.Minor.Patch" concept.
>> According to your detailed description, Trac follows a "shifted" scheme: "0.Major.Minor".
>
> Correct.
>
>> So I must wonder - what is the purpose of "0." if according to the strategy there will never be a "1."?
>
> Historical reasons, the first public release was 0.5, and at 0.9 we felt we were too far away of a finished product to feel confident to go with a "1.0", so that 0.10 was picked and from there the sequence of major releases just went on.
>
>> Or differently phrased - what needs to happen in Trac in order to go "1."?
>
> Think about it differently: what would be the advantage? If we find a role and a need to use the 1.0 version, then we'll probably use it. But having the number as a goal in itself, when no real meaning stands behind, is pointless. Hence the removal of the 1.0 milestone.

Most FOSS projects treat 1.0 as a symbolic gesture of stability. It indicates the developers feel the project is mature and the internals aren't likely to undergo the kind of big changes we still face. The details are naturally project-specific, but you can think of it is that kind of event. Unfortunately Trac isn't at that point in its lifecycle yet, it will get there, but it isn't a specific milestone to plan for as with other releases.

--Noah

[mrelbe]

Hello Christian and thank you for your apprehensive reply.

You pretty much answered all my questions, and you also answered
questions I didn't realize I had :)

Your thoughts and strategies makes a lot of sense, and would be very
useful if written down somehow among the Trac pages, which I am sure
will be done. As a newcomer to the Trac project on t.e.o, this was
very enlightening to read.

I have followed the Trac development activities for some years now,
and I am aware of the long time periods between major releases. The
new approach, to decrease time between major releases, should increase
the interest in taking part of the development among users like
myself. I am definitely all for it.

Trac is a huge project, and it's not easy to get acquainted with the
code since it's large, and the nature of Python using dynamic bindings
makes it an even harder effort.

No, don't read this as I think its bad, I am saying this because my
own background in software development is based on heavily typed
languages as Ada, for military applications. I find Python and Trac
very thrilling since I am deeply involved in pushing it out to
development teams in industrial organisations, and I know what good it
can do there.

My interest in taking part of the development here on t.e.o is to gain
a better understanding of the internals of Trac, to understand the
culture of open-source development, to understand "the supplier" of
Trac, and to understand how to expand the functionality.

I therefore appreciated reading your thoughts about getting involved.
There are a few hints here and there on Trac for newcomers like me,
but a one-pager on Trac could perhaps be a good idea explaining things
like when it's ok to take ownership of a ticket though not being part
of the core team, when it's ok to set a milestone on a ticket, how to
actually produce patches (this is based on my observation that you
guys seem to check out the code from SVN, and then use Mercurial
locally, perhaps making several clones locally to manage your own
development and to produce accurate patches -- this may be natural to
many, but not for industry people using commercial VCS...).

I also noticed that milestones 1.0 and 2.0 were removed, and some have
already commented about the fact that Trac does not denote versions as
the mainstream does. I would like to make a comment about this.

When it comes to introducing tools like Trac to the industry, my
experience is that the main obstacle is not the product itself; it's
about confidence in it (maintenance, quality, ... you know). The
biggest problem I face when I talk to managers in development
organisations is the fact that the version is 0.x and not 1.x. Now,
this may seem silly, but since most people are not aware of Trac, even
less aware of the maturity of Trac, it's quite hard to explain the
version numbers. I think that this could be overcome by the help of a
formal statement made on a Trac wiki page, containing pretty much what
has been stated in this discussion thread. The industry have to get
used to that each open-source project is driven by its own culture and
personal interests, which must be handled with care of course. But we
(which includes myself) should make some effort in explaining this, to
provide arguments and facts to be used by Trac advocates, like myself,
when facing presumptive, professional, users.

Once again, thanks for your reply (and yes, I read it all, and will do
again many more times)

Sincerely yours
Mikael Relbe

[Christian Boos]

On 5/4/2010 10:56 AM, mrelbe wrote:
> Hello Christian and thank you for your apprehensive reply.
>

> ...

>
> There are a few hints here and there on Trac for newcomers like me,
> but a one-pager on Trac could perhaps be a good idea explaining things
> like when it's ok to take ownership of a ticket though not being part
> of the core team, when it's ok to set a milestone on a ticket, how to
> actually produce patches (this is based on my observation that you
> guys seem to check out the code from SVN, and then use Mercurial
> locally, perhaps making several clones locally to manage your own
> development and to produce accurate patches -- this may be natural to
> many, but not for industry people using commercial VCS...).
>

Yes, this is a good idea. We simply need to expand on HowToContribute.

About the VCS usage, I think it's indeed getting more and more common to
mirror locally a centralized source tree using DVCS tools, for keeping
track of local changes and patches while staying in sync with upstream.
We're going to make that even easier by setting up official git and hg
mirrors (#9235).

> I also noticed that milestones 1.0 and 2.0 were removed, and some have
> already commented about the fact that Trac does not denote versions as
> the mainstream does. I would like to make a comment about this.
>
> When it comes to introducing tools like Trac to the industry, my
> experience is that the main obstacle is not the product itself; it's
> about confidence in it (maintenance, quality, ... you know). The
> biggest problem I face when I talk to managers in development
> organisations is the fact that the version is 0.x and not 1.x. Now,
> this may seem silly, but since most people are not aware of Trac, even
> less aware of the maturity of Trac, it's quite hard to explain the
> version numbers. I think that this could be overcome by the help of a
> formal statement made on a Trac wiki page, containing pretty much what
> has been stated in this discussion thread. The industry have to get
> used to that each open-source project is driven by its own culture and
> personal interests, which must be handled with care of course. But we
> (which includes myself) should make some effort in explaining this, to
> provide arguments and facts to be used by Trac advocates, like myself,
> when facing presumptive, professional, users.
>

I can understand this. Combined with the remark of Itamar that we're
actually using a "shifted" scheme, and the fact that in practice between
us it's common to talk about "version eleven", "version twelve" and more
seldom about "version zero dot eleven", I wonder if we couldn't simply
"unshift", drop the 0. prefix and directly go with the next version as
being version 12? That would at the same quiet down the concerns about
maturity that corporate users could have, yet change nothing in essence
to our actual sequence of major releases (0.10, 0.11, 12.0, 13.0, etc.).
As Noah said, changing that sequence to come suddenly with a "1.0" adds
some connotation of stability of the interface. We simply don't want to
be encumbered with that notion of freeze and the long term maintenance
this could imply. Trac will continue to evolve, and will do so
incrementally. I think that if we would have had a 1.0 version, this
should have been what we called 0.10 and we would now be talking about
releasing 1.2.

But in the end, it's not *that* important. If we can make Trac
advocates' life easier by picking a release numbering scheme reflecting
more adequately the level of confidence we have in our product (like in,
using major version numbers for our major versions...), I'd personally
be OK to do it. If for some reasons there are resistances to change, I
wouldn't spend much time on this, continuing with 0.13, 0.14, etc. until
the day we feel it would make sense to call it 1.0 also works for me ;-)

-- Christian

[Rowan]

I love the idea of a git mirror, as for versioning if someone said to
me version 10, 11, 12 I'd assume major differences between them, if
someone said 1.0, 1.1, 1.2 I'd assume minor differences. So for track
the 10,11,12 makes more sense to me. I'm looking forward to seeing the
up coming changes in how trac is managed and hope to start
contributing patches in a few months (once I'm less busy, life just
has to make itself get in the way all my fun!). Keep up the good work
everyone :)

~Rowan

[Eli]

On Tuesday 04 May 2010 4:40:34 am Christian Boos wrote:
> I can understand this. Combined with the remark of Itamar that we're
> actually using a "shifted" scheme, and the fact that in practice between
> us it's common to talk about "version eleven", "version twelve" and more
> seldom about "version zero dot eleven", I wonder if we couldn't simply
> "unshift", drop the 0. prefix and directly go with the next version as
> being version 12? That would at the same quiet down the concerns about
> maturity that corporate users could have, yet change nothing in essence
> to our actual sequence of major releases (0.10, 0.11, 12.0, 13.0, etc.).

I'd rather keep the 0.x releases going for the foreseeable future; with talk
about another templating engine change and the like, a version >= 1.0 is, I
think, a bit... presumptuous? misleading? erm... something. Makes _me_ a bit
nervous anyway. ;)

And for me, I talk about "dot ten" and "dot eleven", etc. but I'm pedantic
that way. :)

Eli
------------------. "If it ain't broke now,
Eli Carter \ it will be soon." -- crypto-gram
retr...@gmail.com `-------------------------------------------------

[Remy Blank]

Christian Boos wrote:
> But in the end, it's not *that* important. If we can make Trac
> advocates' life easier by picking a release numbering scheme reflecting
> more adequately the level of confidence we have in our product (like in,
> using major version numbers for our major versions...), I'd personally
> be OK to do it. If for some reasons there are resistances to change, I
> wouldn't spend much time on this, continuing with 0.13, 0.14, etc. until
> the day we feel it would make sense to call it 1.0 also works for me ;-)

I'm ok with either way. To me, version numbers are just that: version
numbers, and don't carry any other meaning than "we've made some
significant progress".

-- Remy

[Remy Blank]

Christian Boos wrote:
> But in the end, it's not *that* important. If we can make Trac
> advocates' life easier by picking a release numbering scheme reflecting
> more adequately the level of confidence we have in our product (like in,
> using major version numbers for our major versions...), I'd personally
> be OK to do it. If for some reasons there are resistances to change, I
> wouldn't spend much time on this, continuing with 0.13, 0.14, etc. until
> the day we feel it would make sense to call it 1.0 also works for me ;-)

How about just shifting by one digit? The next release after 0.12 could
be 1.3 instead of 0.13. This would avoid the psychological burden of the
1.0 release, and would be a way of saying: "Oh, we've been stable for
quite some time, actually."

;-)

-- Remy

[Noah Kantrowitz]

Except for that whole "Do we want to change template systems again?" issue ;-)

--Noah

[Josh Godsiff]

On 6/5/2010 1:43 PM, Noah Kantrowitz wrote:
> On May 5, 2010, at 10:50 AM, Remy Blank wrote:
>
>
>> Christian Boos wrote:
>>
>>> But in the end, it's not *that* important. If we can make Trac
>>> advocates' life easier by picking a release numbering scheme reflecting
>>> more adequately the level of confidence we have in our product (like in,
>>> using major version numbers for our major versions...), I'd personally
>>> be OK to do it. If for some reasons there are resistances to change, I
>>> wouldn't spend much time on this, continuing with 0.13, 0.14, etc. until
>>> the day we feel it would make sense to call it 1.0 also works for me ;-)
>>>
>> How about just shifting by one digit? The next release after 0.12 could
>> be 1.3 instead of 0.13. This would avoid the psychological burden of the
>> 1.0 release, and would be a way of saying: "Oh, we've been stable for
>> quite some time, actually."
>>
> Except for that whole "Do we want to change template systems again?" issue ;-)
>
> --Noah
>
>

As a professional web-developer who's just started playing around with
Trac's code, I have to say that Genshi is one of the nicer templating
systems I've come across. There's certainly some work that could be done
in regards to the interface between Trac and Genshi (it's often not
obvious where, what or how things are being passed into the templates),
but other than that, I'd say keep it.

- Josh
-- http://www.oxideinteractive.com.au/

[Remy Blank]

Noah Kantrowitz wrote:
> Except for that whole "Do we want to change template systems again?" issue ;-)

Templating system? Peanuts! ;-)

-- Remy