What's gone wrong on the road to Trac 0.11
Christopher Lenz
when I went on my honeymoon in December of last year, I thought we
might be able to do the unthinkable and get 0.11 out in spring, only
about half a year after 0.10. That optimism was based on the great
progress that had already been made at that point. Now it's
September, and if anyone asks me when 0.11 will be released, I can
only say that I have no idea. And that disturbs me big time.
So here's my highly subjective analysis on how things went off track
this year.
1) Too many changes with too much impact
There've been numerous changes, such as the introduction of
trac.context and the flexible permissions system, that have gone in
without ever having been completed on a branch, or without having
gotten a solid consensus among the team. trac.context in particular
has been an extremely broad change that has spread through the code
base in an almost viral manner. And now we're at a point where
there's actually a consensus (AFAIK) that the design should have been
different, that there should be a separation between "resource
descriptors" and "wiki rendering context". So what we have now is a
newly introduced API that is already used all over the Trac code that
noone actually thinks is properly designed. Personally, I don't feel
like shipping that API and maintaining it for a few more major
releases, trying to put a proper API into place while still providing
some sort of backwards compatibility. IMHO We really need to fix this
before releasing 0.11. And that won't be a trivial undertaking.
2) Too many API changes without much benefit
At least in part based on the (controversial) introduction of
trac.context, there have been some API changes, such as the
introduction of a new ITimelineEventProvider API. Again, there's
already a consensus that that API should look quite different, and
the API isn't even released yet. When we make API changes, they need
to be solid. And if there is no consensus, or even no review
feedback, they really *should not go in*. Changes like this shouldn't
go in just because noone actively objected. You need other developers
to say that they are a good idea! These things really need more care.
There's also the incomplete refactoring of trac.wiki, with the split
between parser and formatter. The only thing those changes have done
were to move lines of code around, AFAICT. Also, I don't recall them
being discussed explicitly on this list. The discussion may have
happened somewhere across several tickets, wiki pages, and IRC, but
let's be honest: that's not the only kind of communication that
should be happening for a project this size when changing APIs around.
3) Added bloat
There are a number of features currently in trunk that I think should
have never made there way in. Ticket cloning is one example, and I
know that many others share the sentiment that this shouldn't be in
the core. I don't care whether there's no appropriate plugin API for
implementing it outside of core; instead of just adding it, come up
with the extension API! We don't even have ticket deletion in core,
but we're adding cloning?!?
Similar deal with "ticket history": I don't recall those changes
being properly presented or discussed here. They really should have
been.
There's also things like the "coloring" of directory listings, which
I would've vetoed, if (a) we had real policies on vetoing, and (b) I
hadn't been on my honeymoon at the time. I don't really understand
how anyone thinks this coloring by age is needed, but if there was a
need, add a friggin extension API instead! Let's call it
"IDirectoryListingAnnotator" or something.
As the last example, there's the auto-linking of all timestamps back
to the timeline. I personally think this is quite distracting, there
are links in many more places now, and it isn't really obvious what a
datetime can possibly link to. I argued back then on IRC that I don't
think anyone actually needs this, but Christian thought it was a good
idea, so he implemented it and checked it in. I don't remember anyone
else ever asking for such a feature. Please let's handle such
"enhancements" with a *lot* more care, especially if they require
rather dirty hacks to implement.
That's just a couple of examples, I could go on here, and probably
will in a future mail. The important point here is: adding
functionality is a slippery slope; we've adopted a plugin
archictecture because we really want Trac to be a minimalistic and
lightweight system out of the box, so adding features should be done
very carefully, and only with a broad consensus in the team.
4) Not enough testing
The unit test suite has seen many prolonged phases of being broken. A
lot of functionality has been added, but there are relatively few new
tests. This really needs to change. Now I know Eli is working on
functional testing on a branch, but that doesn't replace a good and
expanding unit test suite. Functional testing is on top of unit
tests, not a replacement.
To be honest, this is one of the reasons I started working on Bitten
again. I want us all to see how rapidly our test coverage is
approaching zero on trunk.
5) Not enough communication
The trac-dev mailing list has been extremely quiet. A lot of
discussion is taking place exclusively on IRC and on tickets, and
that means that anyone who doesn't have the bandwidth to follow those
channels (which are both very high traffic, and often with a bad
signal-to-noise ratio) is out of the loop. Let's *please* get
development discussions on this list.
Now, I'll definitely take some blame for being rather unresponsive on
the list this year. I could say I didn't have the time, and as that's
a reason that's not going to upset anyone, I'll just leave it at
that ;) I hope I'll do better from now on, and I hope you'll all join
me in breathing life back into this list.
So that was me venting. I hope I didn't upset anyone too much, and
that I made my point of view reasonably clear.
What do I think can be done? Well, first I think we need to realize
that 0.11 is further off than some had thought. We need to organize
around fixing the biggest problems, and for the future we need to
figure out ways to avoid such situations and "stay on Trac".
I also think we really need to get away from these feature-packed
releases, and move towards releasing often and early. And we need to
stop adding features over features to the core, and instead
concentrate on getting the infrastructure in place to be able to
release a 1.0 version sometime this millenium.
Cheers,
Chris
--
Christopher Lenz
cmlenz at gmx.de
http://www.cmlenz.net/
Luis Matos
well ... i am nobody here. I am just helping the packaging of trac and
trac plugins into debian and we reached somekind of crossroad.
People is ancious for the 0.11 release, because it has, like it was
said, too many new features. Many people is already running 0.11 from trunk.
The release when ready is not the way in the opensource community
anymore, but the "release often" because you can get more testing and
users can get on your side and comment the changes.
My take in here is to say that users expect an 0.11 release, even if
buggy, so bugs can be corrected to create a new stable release.
I know there are some trac API's or implementations that are not right
or not yet completed, but still ... you should release, at least a
preview of 0.11 that people can really use, that distros can package and
that plugin developers can develop new plugins or update the old ones.
Releasing an alpha or beta release would give more focus on trac, better
testing and a new wave of development with user feedback.
thanks for developing trac
Luis Matos
Lucas Stephanou
I'm using 0.11 trunk in production ( Yes, I'm crazy) but that it quite stable after all.
Chris, I'm not a trac developer, but my experience with others projects , I can say: All that you said is very normal.
To resolve this situation I think that someone( are you?) must directly list changes that DON'T come in 0.11 and focus on a stable release, like Luis has been said.
A example is i18n branch, initial was planned to 0.11 and after some time moved to 0.12.
After 1 mount working in production with 0.11 trunk, I think that is very close of a stable version.
I hope that this message is helpfull.
Luis Matos
... there is always something more to include and something that is not
quite right.
catching the subject, i18n (internationalization) is very important and
mistreated by trac devs for a long time. Even if incomplete, you should
include it in 0.12 ... users will thank, even if it is incomplete. Oh,
and translators can have their job made easier.
Lucas Stephanou wrote:
> I agree with you Luis,
> I'm using 0.11 trunk in production ( Yes, I'm crazy) but that it quite
> stable after all.
>
> Chris, I'm not a trac developer, but my experience with others
> projects , I can say: All that you said is very normal.
> To resolve this situation I think that someone( are you?) must
> directly list changes that DON'T come in 0.11 and focus on a stable
> release, like Luis has been said.
> A example is i18n branch, initial was planned to 0.11 and after some
> time moved to 0.12.
>
> After 1 mount working in production with 0.11 trunk, I think that is
> very close of a stable version.
>
> I hope that this message is helpfull.
>
> On 9/10/07, *Luis Matos* <ga...@otiliamatos.ath.cx
Manuzhai
> 4) Not enough testing
> 5) Not enough communication
I have to say I agree with a lot of your points. In the past, I've
expressed similar opinions on the mailing list and in IRC; most
specifically that we should release sooner instead of adding more to
the release. I also agree with the bloat/useless API's, specifically
the whole context/security thing which I have looked at but haven't
been able to grasp. Also, one of the important reasons I stopped
maintaining the community buildbot for Trac was that the unit tests
were always failing. Lastly, in my experience it's largely impossible
to follow the development. I peruse the Timeline through RSS and track
all the checkins from there, but there is often too little context to
follow the high-level reasoning behind the changes. I'm assuming that
some of that is discussed on IRC and I started idling in the channel,
but there's just too much noise/friendly talk/discussion to keep up.
I hope the Trac development can refocus, get the communication on the
mailing list so more people can contribute, and that we can release
0.11 soon.
Cheers,
Manuzhai
osimons
On Sep 10, 1:44 pm, "Lucas Stephanou" <dom...@gmail.com> wrote:
> I agree with you Luis,
> I'm using 0.11 trunk in production ( Yes, I'm crazy) but that it quite
> stable after all.
>
> close of a stable version.
>
I agree with you on the stability - from a user perspective 0.11dev is
looking very nice, with the new features and the old mostly working as
expected. Normally I would have no problem recommending/accepting a
beta - it could have been made months ago if the end-user features
where all that mattered.
However, I do feel most of the issues raised by Christopher deals with
the internals - used by the other Trac developers, plugin developers
and those customising Trac for their own purposes in various ways.
Having through the last couple of months worked to convert my solution
to 0.11dev (15+ own customisations/plugins, 5+ external ones) I have
bumped into a number of issues - some are fixed, some core issues that
are still open, and some are issues that sort of works but just feels
awkward or half-finished (like the context, security and timeline api
examples mentioned).
With the Trac focus of minimalim, component archtecture, active plugin
development, and flexibility for customisation and re-use, the most
important criteria for starting a new release cycle should be that the
internals (core entities, mechanisms and interfaces) are as we want
them - the building bloks for making features. An official beta is a
signal for a lot of people to start requesting, converting and
supporting their code/plugins for 0.11, and out of respect for the
time and effort of developers it should not be shipped knowing they
might have to redo it before official 0.11.
So, personally - and from an external developer perspective - I'm -1
on any 'official' release until some core issues have been sorted out
(even though I have no vote :-). 'Sorted out' could be as easy as the
core developers agreeing to leave it as it is, or as complicated as re-
writing a number of internals. I encourage you all to reach some sort
of consensus on what needs to be done, as I (like others) would
obviously like to see a release soon.
I suppose this discussion also nicely underlines the need for more
formal and review based implementation of features - using branches,
having complete patches, and achieving consensus on important
changes...
:::simon
http://www.coderesort.com
Erik Huelsmann
> The trac-dev mailing list has been extremely quiet. A lot of
> discussion is taking place exclusively on IRC and on tickets, and
> that means that anyone who doesn't have the bandwidth to follow those
> channels (which are both very high traffic, and often with a bad
> signal-to-noise ratio) is out of the loop. Let's *please* get
> development discussions on this list.
Looking from the outside in - and being used to the way Subversion is
developed - I've *always* been amazed by the fact that sincere
requests for information are nearly always redirected to IRC. First of
all because it's really hard (much harder so than mail traffic) to
build an archive out of if, but also because reading mail backlog is a
lot easier than reading IRC backlog.
With the Subversion community we have the rule 'IRC is fine, but it
isn't official until it hits the development mailing list'. Which has
worked really great for us.
I hope these remarks help you in some way.
bye,
Erik.
Christian Boos
> ...
>
> So here's my highly subjective analysis on how things went off track
> this year.
>
First, thank you for bringing the discussion on a good fact-based
level... makes it easier for me to mostly agree with your analysis.
Let's go through some details though, and see where we can go from there.
> 1) Too many changes with too much impact
> There've been numerous changes, such as the introduction of
> trac.context and the flexible permissions system, that have gone in
> without ever having been completed on a branch, or without having
> gotten a solid consensus among the team. trac.context in particular
> has been an extremely broad change that has spread through the code
> base in an almost viral manner. And now we're at a point where
> there's actually a consensus (AFAIK) that the design should have been
> different, that there should be a separation between "resource
> descriptors" and "wiki rendering context". So what we have now is a
> newly introduced API that is already used all over the Trac code that
> noone actually thinks is properly designed. Personally, I don't feel
> like shipping that API and maintaining it for a few more major
> releases, trying to put a proper API into place while still providing
> some sort of backwards compatibility. IMHO We really need to fix this
> before releasing 0.11. And that won't be a trivial undertaking.
>
The WikiContext related changes seemed to me a good idea at first, as I
was able to address a few long-standing issues with this approach
(http://trac.edgewall.org/wiki/WikiContext#FixedIssues). But after some
time, I realized by myself a few of the drawbacks already pointed out in
the initial review by Christopher, and I think that a resource
descriptor / rendering context split can achieve the same results in a
cleaner way. So right, these concerns will be top-prio for me, and I'll
try to address them in a new branch, along the lines already outlined in
that other mail (1).
As for too many changes, well, yes, the scope of 0.11 could certainly
have been narrowed down.
> 2) Too many API changes without much benefit
> At least in part based on the (controversial) introduction of
> trac.context, there have been some API changes, such as the
> introduction of a new ITimelineEventProvider API. Again, there's
> already a consensus that that API should look quite different, and
> the API isn't even released yet. When we make API changes, they need
> to be solid. And if there is no consensus, or even no review
> feedback, they really *should not go in*. Changes like this shouldn't
> go in just because noone actively objected. You need other developers
> to say that they are a good idea! These things really need more care.
>
> There's also the incomplete refactoring of trac.wiki, with the split
> between parser and formatter. The only thing those changes have done
> were to move lines of code around, AFAICT. Also, I don't recall them
> being discussed explicitly on this list. The discussion may have
> happened somewhere across several tickets, wiki pages, and IRC, but
> let's be honest: that's not the only kind of communication that
> should be happening for a project this size when changing APIs around.
>
Some of these changes are perhaps wrong or not as good as they could
have been, others are "work in progress" (or merely just started, like
the wiki split). Sure.
The point is that at the time when I was working on those changes,
nobody really seemed to be interested enough to give feedback, so trunk
was for a while a working area which I used to build "a better Trac" in
an incremental way. It's now easy to say "if there were no feedback,
then you shouldn't have integrated the changes", but this would have
applied to /most/ of the changes. Often the distinction between a newly
introduced feature and a bug fix is difficult to make (e.g. the whole
WikiContext thing was introduced in the first place to address existing
issues), so never introducing any change which didn't get any feedback
would have led to a status quo, which I didn't want. Instead, I
preferred to make the changes, at the risk of taking criticism afterwards.
Mea culpa, I certainly went down that slippery slope of adding features
in a too liberal way, or rather, in a way that *I* thought would benefit
the user. That leads to the question of who ultimately has the last say
on each and every big and little design issue in Trac. If someone (you)
thinks that e.g. the timeline links are distracting and useless and
someone else (me) thinks that they have a real value (like the ability
to put a change in context, in this case), whose opinion should prevail?
Not to mention other issues where the disagreement comes down to a
choice of colors (ok, I slightly exaggerate here, but not by much).
Adding a configuration switch is not an answer, neither can "just create
another plugin" be one, as we are in the end talking about things that
are done in the core.
So we have different taste and judgment... So what? Do we really have to
come down to some middle ground that would presumably satisfy none of
us? Is there any other solution? Probably. I wouldn't mind creating my
own "feature rich" branch (even outside of t.e.o, though I'd prefer have
it stored there for ease of merging and comparing) if that could help to
make the development process go smoother.
What I won't do is to "give up" on my ideas and what I think are worthy
goals for Trac, stated quite a long time ago in TracObjectModelProposal,
and in (2). Those longer term goals are probably not all shared by other
core developers, but it's not what makes them less valuable in my eyes;
I see enough echo of my ideas in suggestions from various people in
tickets or mails so that I know I'm not completely loony ;-) If there's
a consensus saying that t.e.o is not the right place to develop such a
vision, so let it be and I'll find another way to eventually make it happen.
On the other hand, what I also won't do is to leave trunk in a messy
state, and what the consensus says should go away will go away. So keep
assured that I'll actively work toward releasing 0.11 the way the *team*
wants it to be. I'll also be more conservative about what goes into
trunk in the future /and/ play the plugin game a bit more.
> 4) Not enough testing
> The unit test suite has seen many prolonged phases of being broken. A
> lot of functionality has been added, but there are relatively few new
> tests. This really needs to change. Now I know Eli is working on
> functional testing on a branch, but that doesn't replace a good and
> expanding unit test suite. Functional testing is on top of unit
> tests, not a replacement.
>
> To be honest, this is one of the reasons I started working on Bitten
> again. I want us all to see how rapidly our test coverage is
> approaching zero on trunk.
>
> 5) Not enough communication
> The trac-dev mailing list has been extremely quiet. A lot of
> discussion is taking place exclusively on IRC and on tickets, and
> that means that anyone who doesn't have the bandwidth to follow those
> channels (which are both very high traffic, and often with a bad
> signal-to-noise ratio) is out of the loop. Let's *please* get
> development discussions on this list.
>
I agree that the signal-to-noise ratio argument is a good reason to
discuss things on trac-dev.
> Now, I'll definitely take some blame for being rather unresponsive on
> the list this year. I could say I didn't have the time, and as that's
> a reason that's not going to upset anyone, I'll just leave it at
> that ;) I hope I'll do better from now on, and I hope you'll all join
> me in breathing life back into this list.
>
> So that was me venting. I hope I didn't upset anyone too much, and
> that I made my point of view reasonably clear.
>
... and it's a reasonable point of view to have.
Now I think this analysis misses one big part of the picture: what's
also gone wrong on the road to Trac 0.11 was that there could really
have been a bit more contributions given to the project by its team
members. I certainly don't claim having done all the work and assuredly
some of it would have been better spent in a more focused / inspired
way, but all in all, if at least someone else had put an equivalent
amount of work in ticket triaging, troubleshooting, bug fixing
(including for the 0.10 line), giving feedback, testing, then Trac 0.11
would already have been released since quite some time.
> What do I think can be done? Well, first I think we need to realize
> that 0.11 is further off than some had thought. We need to organize
> around fixing the biggest problems, and for the future we need to
> figure out ways to avoid such situations and "stay on Trac".
>
> I also think we really need to get away from these feature-packed
> releases, and move towards releasing often and early. And we need to
> stop adding features over features to the core, and instead
> concentrate on getting the infrastructure in place to be able to
> release a 1.0 version sometime this millenium.
> \
>
Yep, agreed.
-- Christian
(1)
http://groups.google.com/group/trac-dev/browse_frm/thread/f6fb241963be9178/95b255ce192ba5c5#95b255ce192ba5c5
(2) http://trac.edgewall.org/wiki/ChristianBoos#NextSteps
Alec Thomas
> without ever having been completed on a branch, or without having
I feel I have to defend my decision to merge only the Wiki part of the
permission system. Given the same situation I would make the same
decision again, so I'm not apologising, I'm justifying :)
The primary reason was that keeping a branch updated against Trac trunk,
particularly one that due to its very nature touches a large part of
the code base, is a draining experience. This is mainly due to the high
rate of change on trunk.
The second reason was that even though I waited two weeks for review, I
would have liked more input. With that in mind, if anybody had raised
major concerns after the merge, reverting just the Wiki changes would
have been relatively easy.
With my excuses^Wreasoning out of the way, I think the recently proposed
changes to how we develop Trac (development occuring exclusively in
branches, a decent review process, etc.), would have helped massively
with both of these problems. C'est la vie.
> 2) Too many API changes without much benefit
> 3) Added bloat
> 4) Not enough testing
All agreed.
> 5) Not enough communication
> The trac-dev mailing list has been extremely quiet. A lot of
> discussion is taking place exclusively on IRC and on tickets, and
> that means that anyone who doesn't have the bandwidth to follow those
> channels (which are both very high traffic, and often with a bad
> signal-to-noise ratio) is out of the loop. Let's *please* get
> development discussions on this list.
This is a very good point, and one echoed by several others already in
this thread. I agree wholeheartedly.
> So that was me venting. I hope I didn't upset anyone too much, and
> that I made my point of view reasonably clear.
I think the venting was quite coherent :)
> I also think we really need to get away from these feature-packed
> releases, and move towards releasing often and early. And we need to
> stop adding features over features to the core, and instead
> concentrate on getting the infrastructure in place to be able to
> release a 1.0 version sometime this millenium.
I can't agree more. I think the wider Internet has shown that releasing
often and early helps keep projects vibrant and healthy. And I think the
rot that's setting in to Trac illustrates what happens when this doesn't
happen.
--
Evolution: Taking care of those too stupid to take care of themselves.
Luis Matos
> So, personally - and from an external developer perspective - I'm -1
> on any 'official' release until some core issues have been sorted out
> (even though I have no vote :-). 'Sorted out' could be as easy as the
> core developers agreeing to leave it as it is, or as complicated as
> re-
> writing a number of internals. I encourage you all to reach some sort
> of consensus on what needs to be done, as I (like others) would
> obviously like to see a release soon.
if there where releases out there and more people using trac 0.11, more
developers would try to devel to trac 0.11 and propose for API
changes/improvements. If there is no release, trac 0.11 continues to be
a bunch of features, where you could improve lots of stuff yet ... and
get all the pretty stuff in ... and then ... never release.
Jeroen Ruigrok van der Werven
>A example is i18n branch, initial was planned to 0.11 and after some time
>moved to 0.12.
Sorry, you are quite mistaken about this I think.
From the moment I created the various i18n tickets I have always set the
milestone to 0.12. 0.11 never even came into the picture and I doublechecked
with Chris on this before I assigned the milestones.
--
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
イェルーン ラウフãƒãƒƒã‚¯ ヴァン デル ウェルヴェン
http://www.in-nomine.org/ | http://www.rangaku.org/
Dream, a noise, the wind awakes me, and you're already here...
Christopher Lenz
> well ... i am nobody here. I am just helping the packaging of trac and
> trac plugins into debian and we reached somekind of crossroad.
>
> People is ancious for the 0.11 release, because it has, like it was
> said, too many new features. Many people is already running 0.11
> from trunk.
>
> The release when ready is not the way in the opensource community
> anymore, but the "release often" because you can get more testing and
> users can get on your side and comment the changes.
Your opinion on this is certainly respected, and I guess that my
message was bad news for all users who were hoping that 0.11 was just
around the corner. But there's a reason the subject of this thread
goes "What's gone wrong ...", and the mail was an analysis on why
there is no 0.11 yet from my point of view. We're in a situation we
ideally shouldn't be in, for a number of reasons.
I maintain that I don't think that the current codebase is in a state
that we should be releasing it. This has less to do with whether or
not Trac trunk works okay from the perspective of the user, but a
whole lot more with a number of team members (and I hope others will
speak up on this thread) being unhappy with some internals and added/
changed APIs. And as Trac is now really a platform that many
developers add additional functionality to in the form of plugins,
things that aren't necessarily user-visibe have become very
important. I'm ready to put work into moving things forward, as are
others, but it'll take time.
And as I said, we really need to look into moving towards release
often and early. It's been said many times before, but somehow we've
always messed it up with recent releases.
Luis Matos
> I maintain that I don't think that the current codebase is in a
> state
> that we should be releasing it. This has less to do with whether or
> not Trac trunk works okay from the perspective of the user, but a
> whole lot more with a number of team members (and I hope others will
> speak up on this thread) being unhappy with some internals and added/
> changed APIs. And as Trac is now really a platform that many
> developers add additional functionality to in the form of plugins,
> things that aren't necessarily user-visibe have become very
> important. I'm ready to put work into moving things forward, as are
> others, but it'll take time.
i read your email with caution ... i am subscribed to trac-dev and i
know the change that is 0.10->0.11 ...
I know the API problems ... the other core problems (+/-) ...
What i meant on my email was to tell you (trac devs) that you should do
less changes between releases ... or that bunch of changes in one
time ... you have some api problems, so, to work around them, you built
in some functionality ... But the changes are so huge and so wide, that
you will not be able to release trac 0.11 soon because there a lots of
loose edges.
what i was saying in my email was an attempt for you to release some
alpha/beta version as feedback for trac users.
once again ... thanks for your effort.
Luis Matos
Christopher Lenz
first off, thanks a lot for this great response!
Am 10.09.2007 um 16:26 schrieb Christian Boos:
> Christopher Lenz wrote:
>> 1) Too many changes with too much impact
[snip]
> The WikiContext related changes seemed to me a good idea at first,
> as I
> was able to address a few long-standing issues with this approach
> (http://trac.edgewall.org/wiki/WikiContext#FixedIssues). But after
> some
> time, I realized by myself a few of the drawbacks already pointed
> out in
> the initial review by Christopher, and I think that a resource
> descriptor / rendering context split can achieve the same results in a
> cleaner way. So right, these concerns will be top-prio for me, and
> I'll
> try to address them in a new branch, along the lines already
> outlined in
> that other mail (1).
That sounds good. We should get together and discuss what the right
way forward should be in this regard, on a separate thread. I must
admit I'm still a bit lost here, due the pervasiveness of the Context
stuff in the code, so I think your support with refactoring this will
be really appreciated.
>> 2) Too many API changes without much benefit
[snip]
> Some of these changes are perhaps wrong or not as good as they could
> have been, others are "work in progress" (or merely just started, like
> the wiki split). Sure.
Yeah, that's what my impression was. IMHO, things such as the wiki
formatting/parsing split should have gone on a branch, and be
proposed/discussed here. I don't think that releasing a version
sporting "half refactored" APIs is a good idea, as usually you only
find out whether an API actually works when you put it to serious
use. When you try to anticipate how an API *should* look in a future
version, chances are that the actual API will need to look different.
> The point is that at the time when I was working on those changes,
> nobody really seemed to be interested enough to give feedback, so
> trunk
> was for a while a working area which I used to build "a better
> Trac" in
> an incremental way. It's now easy to say "if there were no feedback,
> then you shouldn't have integrated the changes", but this would have
> applied to /most/ of the changes. Often the distinction between a
> newly
> introduced feature and a bug fix is difficult to make (e.g. the whole
> WikiContext thing was introduced in the first place to address
> existing
> issues), so never introducing any change which didn't get any feedback
> would have led to a status quo, which I didn't want. Instead, I
> preferred to make the changes, at the risk of taking criticism
> afterwards.
I fully understand that the lack of involvement of team members like
myself plays an important role in how we got here. In my defense I
can only say that I was frustrated at the time about some of the
changes being made. Sometimes frustration paralyzes, and that of
course doesn't help at all. This is basically me breaking out of that
paralysis and trying to start being constructive again :P
That said, I think there were still too many changes done too quickly
without seriously trying to trigger discussion on this list. And
complaining about a change after it's been made is harder than when
it's still a proposal. So for the future, that's something that we
should try to avoid.
But let's not talk about spilled milk...
>> 3) Added bloat
[snip]
> Mea culpa, I certainly went down that slippery slope of adding
> features
> in a too liberal way, or rather, in a way that *I* thought would
> benefit
> the user. That leads to the question of who ultimately has the last
> say
> on each and every big and little design issue in Trac. If someone
> (you)
> thinks that e.g. the timeline links are distracting and useless and
> someone else (me) thinks that they have a real value (like the ability
> to put a change in context, in this case), whose opinion should
> prevail?
> Not to mention other issues where the disagreement comes down to a
> choice of colors (ok, I slightly exaggerate here, but not by much).
> Adding a configuration switch is not an answer, neither can "just
> create
> another plugin" be one, as we are in the end talking about things that
> are done in the core.
I think in these cases we (a) need to keep the Trac motto of being
minimalistic in mind, and (b) only add features for which there's a
consensus in the team (+0), or even active support by other members
(+1). If there's a feature that someone else on the team thinks
should not be added, there needs to be discussion, and if in the end
there is no agreement, it should not be added. I don't see how we as
a project can work otherwise.
On a related note, we've now evidently reached a size and state where
we need to write down some basic rules on this decision making process.
> So we have different taste and judgment... So what? Do we really
> have to
> come down to some middle ground that would presumably satisfy none of
> us? Is there any other solution? Probably. I wouldn't mind creating my
> own "feature rich" branch (even outside of t.e.o, though I'd prefer
> have
> it stored there for ease of merging and comparing) if that could
> help to
> make the development process go smoother.
>
> What I won't do is to "give up" on my ideas and what I think are
> worthy
> goals for Trac, stated quite a long time ago in
> TracObjectModelProposal,
> and in (2). Those longer term goals are probably not all shared by
> other
> core developers, but it's not what makes them less valuable in my
> eyes;
> I see enough echo of my ideas in suggestions from various people in
> tickets or mails so that I know I'm not completely loony ;-) If
> there's
> a consensus saying that t.e.o is not the right place to develop such a
> vision, so let it be and I'll find another way to eventually make
> it happen.
I'm glad that you raise this concern. As you know, I've always been
skeptical about the TracObject ideas, which basically go back to the
xref branch; the main problem I see is premature/over-ambitious
generalization, and a design that is not user-oriented. But we
shouldn't discuss that here, it's obvious that we're in disagreement.
And the problem is that it's disagreement over vision and long-term
goals.
I personally would encourage you to explore this vision in a separate
branch or fork. This will give you the advantage of being able to
make faster progress with fewer tiresome discussions over what
amounts to a conflicting vision. If you choose to follow that path, I
think you would really want to run the project on a separate system
and repository (using svnsync or somesuch), enabling you to "eat your
own dogfood" and all that. I also think that having a branch
following a decidely different direction than Trac proper would be
confusing to our users
But that's not to say that there shouldn't be synergy; for example,
it's rather unfortunate how DrProject has lost any connection to the
Trac project since it was forked. I think keeping up healthy
communication benefits both sides of a fork (for example by trying to
keep at least parts of the APIs and technologies in sync).
> On the other hand, what I also won't do is to leave trunk in a messy
> state, and what the consensus says should go away will go away. So
> keep
> assured that I'll actively work toward releasing 0.11 the way the
> *team*
> wants it to be. I'll also be more conservative about what goes into
> trunk in the future /and/ play the plugin game a bit more.
Sounds great.
>> So that was me venting. I hope I didn't upset anyone too much, and
>> that I made my point of view reasonably clear.
> ... and it's a reasonable point of view to have.
>
> Now I think this analysis misses one big part of the picture: what's
> also gone wrong on the road to Trac 0.11 was that there could really
> have been a bit more contributions given to the project by its team
> members. I certainly don't claim having done all the work and
> assuredly
> some of it would have been better spent in a more focused / inspired
> way, but all in all, if at least someone else had put an equivalent
> amount of work in ticket triaging, troubleshooting, bug fixing
> (including for the 0.10 line), giving feedback, testing, then Trac
> 0.11
> would already have been released since quite some time.
I would like the emphasize that I think the amount of work and time
you've put into Trac has always been a tremendous boon to the
project. I don't think anyone doubts that.
The other side of the medal, though, is that with one person able to
contribute a seemingly infinite amount of time and resources to an
open-source project, getting involved may at least seem harder for
others (committers and potential contributors alike). The same could
be said of myself when we were around the 0.9 mark. It's probably no
coincendence that we're having this discussion at a point when you've
seriously scaled back your activity on Trac ;-)
Jonas Borgström
> 1) Too many changes with too much impact
Agreed. Hopefully the development policy we discussed earlier on trac-
dev in combination with more focus "release early release often" will
help us avoid similar problems in the future.
> 2) Too many API changes without much benefit
> 3) Added bloat
This one is a bit tricky, we have for a while now been spoiled by an
excellent component system allowing Trac to be extended and customized
in many different ways. So now most of our users expect the "Trac
platform" to be usable for almost any task just by writing a plugin or
two.
I think in order to keep being able to call Trac a "minimalistic
approach" we need not only to think twice about adding new concrete
features but also new apis and configuration options. A too complex api
and too many config options is also bloat.
> What do I think can be done? Well, first I think we need to realize
> that 0.11 is further off than some had thought. We need to organize
> around fixing the biggest problems, and for the future we need to
> figure out ways to avoid such situations and "stay on Trac".
Agreed. Even though from a user's point of view trunk might be usable
enough for at least a beta release already there are a lot of things we
really need to sort out first (both in the code itself and the project)
before 0.11 should be released.
Cheers,
Jonas
Jani Tiainen
Christopher Lenz kirjoitti:
Cat's can drink it still.. :D
Few things comes to my mind to prevent this: planning and sticking with
a plan. See down...
Decision making is really important thing. Someone needs to be in
charge, even this is open project.
A bit more mature projects have usually regular meetings - there final
decision making team (usually main contributors) decides (votes) what
will be included. And sticks with that.
For help of decisions, some ticketing systems have end user vote for
critical enhancements. Like you can always have three tickets you want
contributors to concentrate.
One "problem" I see is that over time features were added, added and
even more added. And release date is moved always to forward future.
Still, planning what will be done for releases and specially sticking
with that decision helps to reach goal.
We here at work have few very mature products that are still developed
actively. Twice a year we have user meeting where we show proposal about
several enhancements (usually around 10 to 15) what would be in next
release some of them are our own fancy ideas, some of them are requests
from end users. End users are allowed to select (or vote) top five or
four they want to have. And then we do it. Rinse and repeat.
About twice a month someone in our support department goes through all
open enhancement/bug reports. Reports are always handled by someone
immediately when they arrive, but action might not be done. Like if it's
feature request we might just ask for more info, illustrations and such
and later produce some proposal.
Democratic voting. After vote is passed, results are recorded.
As an alien in Trac world it seems that most of API decisions, new
features were decided in behind closed doors or even ad hoc manner.
Someone just felt it would be cool and in it goes.
>>> So that was me venting. I hope I didn't upset anyone too much, and
>>> that I made my point of view reasonably clear.
>> ... and it's a reasonable point of view to have.
>>
>> Now I think this analysis misses one big part of the picture: what's
>> also gone wrong on the road to Trac 0.11 was that there could really
>> have been a bit more contributions given to the project by its team
>> members. I certainly don't claim having done all the work and
>> assuredly
>> some of it would have been better spent in a more focused / inspired
>> way, but all in all, if at least someone else had put an equivalent
>> amount of work in ticket triaging, troubleshooting, bug fixing
>> (including for the 0.10 line), giving feedback, testing, then Trac
>> 0.11
>> would already have been released since quite some time.
>
> I would like the emphasize that I think the amount of work and time
> you've put into Trac has always been a tremendous boon to the
> project. I don't think anyone doubts that.
>
> The other side of the medal, though, is that with one person able to
> contribute a seemingly infinite amount of time and resources to an
> open-source project, getting involved may at least seem harder for
> others (committers and potential contributors alike). The same could
> be said of myself when we were around the 0.9 mark. It's probably no
> coincendence that we're having this discussion at a point when you've
> seriously scaled back your activity on Trac ;-)
This is one real problem in any open source project. Projects tend to
grow and grow and becomes an endless timesink for contributor(s).
Finally happens that major (or even only one) contributor will step down
and recession begins. It seems really inevitable.
--
Jani Tiainen
Eli Carter
> Am 10.09.2007 um 16:26 schrieb Christian Boos:
> > So we have different taste and judgment... So what? Do we really
> > have to
> > come down to some middle ground that would presumably satisfy none of
> > us? Is there any other solution? Probably. I wouldn't mind creating my
> > own "feature rich" branch (even outside of t.e.o, though I'd prefer
> > have
> > it stored there for ease of merging and comparing) if that could
> > help to
> > make the development process go smoother.
> I personally would encourage you to explore this vision in a separate
> branch or fork. This will give you the advantage of being able to
> make faster progress with fewer tiresome discussions over what
> amounts to a conflicting vision. If you choose to follow that path, I
> think you would really want to run the project on a separate system
> and repository (using svnsync or somesuch), enabling you to "eat your
> own dogfood" and all that. I also think that having a branch
> following a decidely different direction than Trac proper would be
> confusing to our users
I for one would rather see Christian's efforts in a branch than in a seperate
system. While it sounds like we'll see significant divergance between trunk
and his "feature rich" branch, it makes it clearer that we're on friendly
terms, and can help with sharing bugfix load and the like.
Besides, I expect that Christian will be proven "right" on some of his large
controversial changes, so we should keep the door open to steal those. ;)
Eli
Eli Carter
> There are a number of features currently in trunk that I think should
> have never made there way in. Ticket cloning is one example, and I
> know that many others share the sentiment that this shouldn't be in
> the core. I don't care whether there's no appropriate plugin API for
> implementing it outside of core; instead of just adding it, come up
> with the extension API! We don't even have ticket deletion in core,
> but we're adding cloning?!?
I have a ticket clone plugin (that works differently from what's in core).
I'll try to push that out to trac-hacks.
> There's also things like the "coloring" of directory listings, which
> I would've vetoed, if (a) we had real policies on vetoing, and (b) I
> hadn't been on my honeymoon at the time. I don't really understand
> how anyone thinks this coloring by age is needed, but if there was a
> need, add a friggin extension API instead! Let's call it
> "IDirectoryListingAnnotator" or something.
Hm. I like the coloring, personally. "Need"? Well, I'd say that 'file
annotate' is a needed feature for the source browser. The coloring of the
directory listing is the same thing, but at a file/directory granularity
instead of per-line.
> 4) Not enough testing
> The unit test suite has seen many prolonged phases of being broken. A
> lot of functionality has been added, but there are relatively few new
> tests. This really needs to change. Now I know Eli is working on
> functional testing on a branch, but that doesn't replace a good and
> expanding unit test suite. Functional testing is on top of unit
> tests, not a replacement.
Agreed. I took the functional testing approach because I saw it as a way to
get the broadest test coverage I could. Shallow testing, but broadly cover
the user-visible features.
Eli
Erik Bray
> I for one would rather see Christian's efforts in a branch than in a seperate
> system. While it sounds like we'll see significant divergance between trunk
> and his "feature rich" branch, it makes it clearer that we're on friendly
> terms, and can help with sharing bugfix load and the like.
>
> Besides, I expect that Christian will be proven "right" on some of his large
> controversial changes, so we should keep the door open to steal those. ;)
I've been hesitant to join in on this discussion, but the mention of
moving work that's already been done on 0.11 into a separate branch
worries me. I have already done a lot of work based on 0.11 as it is
currently in trunk (not necessarily by my own choice--most of this
work was started back when 0.11 seemed closer to release).
Fortunately, I don't have *too* much riding on this--if anything gets
changed I can adjust. The thing I'm most concerned about is the
permission system. I have been maintaining a somewhat complex
fine-grained permissions system for Trac, which used to depend on
numerous small patches. With trac.context, plus the improvements
merged in from the security branch--the IPermissionPolicy interface
and the redone PermissionCache built around them--my fine-grained
permission system fit in almost perfectly. I just had to modify the
ticket.html template slightly to support ticket permissions better.
At any rate, I agree with that rendering contexts and resource
descriptors should be separate, and that it could probably be done
better (though I haven't seen any of the discussion about that. Was
it on IRC?) But whatever ends up happening the existing support for
more customized permission systems is already a huge benefit in 0.11.
As for other features, I'm mostly apathetic. One point I would make,
in defense of dates/times linking to the timeline, is that when I was
showing off 0.11 to my program manager he discovered that (I didn't
even know about it) and really liked it. Why? I don't know. He also
wants HTML e-mails which I don't care about. But regardless he likes
it. And I don't see it as harmful.
Erik
Alec Thomas
> Fortunately, I don't have *too* much riding on this--if anything gets
> changed I can adjust. The thing I'm most concerned about is the
> permission system. I have been maintaining a somewhat complex
The new fine grained permissions will not be disappearing, fear not.
The only real change will be that something other than a Context object
will be passed to the policy plugins. What exactly this is, is
undecided, but the changes to policy plugins are very likely to be
syntactical only.
Erik Bray
>
> On 9/13/07, Erik Bray <hyugar...@gmail.com> wrote:
> > Fortunately, I don't have *too* much riding on this--if anything gets
> > changed I can adjust. The thing I'm most concerned about is the
> > permission system. I have been maintaining a somewhat complex
>
> The new fine grained permissions will not be disappearing, fear not.
> The only real change will be that something other than a Context object
> will be passed to the policy plugins. What exactly this is, is
> undecided, but the changes to policy plugins are very likely to be
> syntactical only.
>
That's a relief to hear, though I figured it would be the case. I'm
all for splitting up the Context objects, and regardless of what gets
passed around in their place I'm sure it won't be difficult to adjust.
Thanks,
Erik
David Abrahams
on Mon Sep 10 2007, Christopher Lenz <cmlenz-AT-gmx.de> wrote:
> Hey folks,
>
> when I went on my honeymoon in December of last year, I thought we
> might be able to do the unthinkable and get 0.11 out in spring, only
> about half a year after 0.10. That optimism was based on the great
> progress that had already been made at that point. Now it's
> September, and if anyone asks me when 0.11 will be released, I can
> only say that I have no idea. And that disturbs me big time.
Wow, that sounds really familiar. Boost.org recently went through a
1.5 year release cycle; it was agony. We've been working hard to
revamp our release tools and procedures so that we can release
quarterly.
So, excuse me for butting in here as I've only contributed a very
little bit to Trac development, but I have a big stake in Trac's
future and I have been thinking about similar issues a lot recently.
I hope I won't ruffle any feathers by injecting my opinions without
further apology.
> So here's my highly subjective analysis on how things went off track
> this year.
>
> 1) Too many changes with too much impact
> There've been numerous changes, such as the introduction of
> trac.context and the flexible permissions system, that have gone in
> without ever having been completed on a branch, or without having
> gotten a solid consensus among the team. trac.context in particular
> has been an extremely broad change that has spread through the code
> base in an almost viral manner. And now we're at a point where
> there's actually a consensus (AFAIK) that the design should have been
> different, that there should be a separation between "resource
> descriptors" and "wiki rendering context". So what we have now is a
> newly introduced API that is already used all over the Trac code that
> noone actually thinks is properly designed. Personally, I don't feel
> like shipping that API and maintaining it for a few more major
> releases, trying to put a proper API into place while still providing
> some sort of backwards compatibility. IMHO We really need to fix this
> before releasing 0.11. And that won't be a trivial undertaking.
The need to interrupt the spiral of never releasing is much more
important than releasing exactly the right thing. If I were in your
shoes I would release it with that API in its current state, but
"deprecated." That is, you make a loud announcement that it's going
to go away very soon. It was never in an official release, so nobody
can really complain too loudly.
Sometimes, unfortunately, one needs to get some usage experience with
an API before you can tell that it's wrong. If it hadn't been so long
since an official Trac release, I think you could afford to take some
time to fix it. However, you also probably wouldn't have so much
experience with it, so you might not know that it's wrong yet, or at
least what the right interface is. The bottom line: you have to
expect that occasionally you'll release design errors and you need a
procedure/mechanism for dealing with them. Deprecation is one good
approach.
Another point: as odious as some people may find it to make and follow
rules, I think you need some policies in place that will prevent
half-baked (e.g. never completed on a branch or not having consensus)
checkins. Trac has probably become too big and important to be run as
informally as it has up 'till now.
> 2) Too many API changes without much benefit
> At least in part based on the (controversial) introduction of
> trac.context, there have been some API changes, such as the
> introduction of a new ITimelineEventProvider API. Again, there's
> already a consensus that that API should look quite different, and
> the API isn't even released yet. When we make API changes, they need
> to be solid. And if there is no consensus, or even no review
> feedback, they really *should not go in*. Changes like this shouldn't
> go in just because noone actively objected. You need other developers
> to say that they are a good idea! These things really need more care.
Again, you need to write policies for this stuff, IMO. Since the
project is growing quite large, I strongly suggest you consider ways
to spread responsibility across the team so that you don't end up
having to act personally as a gatekeeper for everything.
Another point about these APIs: you can gain a lot of power by
distinguishing documented interfaces from what's in the code. I know
it's very common that people crawl through source (and are often
expected to) to find out what they can take advantage of, but if you
make it very clear that the official Trac API is the one you've
documented, you can make a choice to release before cleaning up
ugly/mistaken "APIs" as long as the code works and you either don't
document, or deprecate, the mistakes.
That's a good start on a policy. I think you ought to define
precisely what you mean by broad consensus before this is over.
> 4) Not enough testing
> The unit test suite has seen many prolonged phases of being broken. A
> lot of functionality has been added, but there are relatively few new
> tests. This really needs to change. Now I know Eli is working on
> functional testing on a branch, but that doesn't replace a good and
> expanding unit test suite. Functional testing is on top of unit
> tests, not a replacement.
>
> To be honest, this is one of the reasons I started working on Bitten
> again.
Hooray!
You can bet I'm going to try that out!
> What do I think can be done? Well, first I think we need to realize
> that 0.11 is further off than some had thought.
I strongly suggest you consider compromises that can bring it closer.
Such huge divergence of the development branch from the state of the
latest released software is really harmful, especially in a world
where so much functionality comes in from plugins. If, instead of
planning one release out, you carefully plan the next two or three
releases, an imperfect 0.11 release doesn't seem so bad. You can
chart a course that brings the world up-to-date, lets you fix design
errors, and still make progress on new features.
> We need to organize around fixing the biggest problems, and for the
> future we need to figure out ways to avoid such situations and "stay
> on Trac".
>
> I also think we really need to get away from these feature-packed
> releases, and move towards releasing often and early.
Hear hear!
Thanks,
--
Dave Abrahams
Boost Consulting
http://www.boost-consulting.com
The Astoria Seminar ==> http://www.astoriaseminar.com
rupert....@gmail.com
> Hey folks,
>
> when I went on my honeymoon in December of last year, I thought we
> might be able to do the unthinkable and get 0.11 out in spring, only
> about half a year after 0.10. That optimism was based on the great
> progress that had already been made at that point. Now it's
> September, and if anyone asks me when 0.11 will be released, I can
> only say that I have no idea. And that disturbs me big time.
could you pls clearly state what needs to be done iyo until release?
backport genshi and release 0.10.6?
-rupert.