Re: Thunderbird plans for after Tb 3.0
Dan Mosedale
newsgroup versions of dev-apps-thunderbird and dev-apps-calendar.
On 11/23/09 7:02 PM, Dan Mosedale wrote:
> [Note that followups & replies have been aimed at the
> dev-apps-thunderbird mailing list]
>
> Part of the plan for Thunderbird is to move our development process in a
> more agile direction. Now that we've just about got Tb3 out the door,
> I've drafted some proposed process changes to help that along:
> <https://wiki.mozilla.org/User:Dmose/Tb_Post-3.0_Scratchpad>.
>
> Feedback is encouraged either here or on the wiki Talk page.
>
> More concrete documents for detailed 3.1 discussion are around the corner.
>
> Dan
Ben Bucksch
Below, I call 4-6 months "long", and 1-2 months "short".
Advantages:
- Pace: Users see new improvements faster
- Checkin pressure: Developers don't have the pressure to make a
given release.
Missing a release, when the next one is 6 months away, is a bummer. When it's only 1-2 months, it's only a minor bummer. So, as you say, there's less pressure to ship almost-finished (but not entirely finished) work. - Branching: With longer releases, there's likely to be a significant time from branching to release, also due to the previous problem: features stuffed in, then need baking/fixing time on branch. With shorter releases, you simply cut from trunk at a given time, and do only small fixes on branch for the release.
- Branch maintenance: We would not maintain branches for very long. Users are expected to upgrade to the newer step release. Ideally, we wouldn't have have to maintain any release branch for TB (unless there's an urgent hotfix): A security problem in TB 3.1 would be fixed by 3.2 (which branched directly from comm-central trunk), there's no 3.1.1.
- Testing: Doing so many releases sounds scary from a testing
perspective, but it may turn out to be an advantage, we can potentially
get *better* testing coverage. If trunk is always in good shape (see
below), more users are going to use the test releases (and we can often
ship betas without much effort), so we'd get faster feedback and better
cover than now where only few people use nightlies and betas.
- Branching (see above): This requires that the trunk is always in good condition. That's good practice anyways. It can be achieved with feature branches (ideally one branch per bug), and it gets merged to trunk only when it's working well. Not sure how realistic this is, both for merging and testing. It works well for Linux, though.
- The previous point is esp. hard for features which take long to develop and touch a lot of code.
Dan Mosedale
On 11/25/09 4:59 PM, Ben Bucksch wrote:
> I'd propose releases much faster than 4-6 months. More like 1-2 months.
>
> Below, I call 4-6 months "long", and 1-2 months "short".
How practical this actually is, I'm unsure. That said, over the longer
term, it's at least worth doing this as a gedankenexperiment and perhaps
even taking it further than that.
There are a couple of other disadvantages I can imagine to what you're
proposing, but I don't think it makes sense to dive into them now. The
reason being that I think we need to demonstrate that we can walk before
we start planning to run.
In particular, even 4-6 month cycles are going to have a bunch of
branching and scaling problems. My belief is that our current
infrastructure and processes are so far from handling the level of
branching and change we'd need for 1-2 month cycles that it doesn't make
a lot of sense to try and have that discussion until we've experienced
the issues first-hand once by scaling down to 4-6 months.
Dan
Kent James
>
> In particular, even 4-6 month cycles are going to have a bunch of
> branching and scaling problems.
>
>
I'm glad to hear you talking about trying to understand the underlying
root causes of slow cycles. So I'd like to make sure that the review
process issues get considered along with others here. I know that you
have heard me whine about the review process before, but let me mention
it in the context of these numbers. I want to emphasize that I am
talking about the process here and not the individuals, as generally the
cooperation that I have had with reviews from individuals has been great.
My last non-essential patches were landed in the first week of
September, and my feeling at the time was that they were quite late for
TB3. The formal and informal freezes and slowdowns associated with the
release process have largely been in effect since then, at least
according to my perceptions as an outsider. Here we are approaching the
end of November, and only now do I feel that the review window may be
opening again for 3.1. Add to that 11 week delay another 2-3 weeks for
the reviews themselves, and we are pushing 14 weeks of formal and
informal freezes for review work for non-blockers (with the bar set
quite high for blocking status) near a release. 14 weeks is a pretty
major chunk of the 4 - 6 months that you are trying to achieve.
Looking back, I can't complain about any decisions or actions that were
taken by individuals during this period. But that is the whole point of
process review, to see that even when everyone is doing a great job, the
process is forcing a slowness that can only be changed through
understanding and changing the formal processes, plus trying to affect
the decisions made by "rumor and tradition" (which is my generic term
for the informal process) that cause slowness.
I hope that you will consider what can be done to shorten this freeze
cycle during your discussions, particularly as experienced by someone
like me who is not part of the inner circle.
rkent
Simon Paquet
> I'd propose releases much faster than 4-6 months. More like 1-2
> months.
>
> Below, I call 4-6 months "long", and 1-2 months "short".
Looking at your advantages and disadvantages list it's pretty easy
to notice that you're a developer. But (un)fortunately a release
encompasses far more than just the development work, e.g.
* Localization
* Marketing
* Release Engineering
* Feature documentation
* PR
* Community relations (e.g. with add-on developers)
Doing all of this within a 4-6 month cycle seems already hard enough
and I agree with Dan, that we should first try to show to the world
that we can handle such a cycle before we move to an even more
aggressive one.
Cya
Simon
--
Thunderbird/Calendar Localisation (L10n) Coordinator
Thunderbird l10n blog: http://thunderbird-l10n.blogspot.com
Calendar website maintainer: http://www.mozilla.org/projects/calendar
Calendar developer blog: http://weblogs.mozillazine.org/calendar
Patrick Brunschwig
> I'd propose releases much faster than 4-6 months. More like 1-2 months.
>
> Below, I call 4-6 months "long", and 1-2 months "short".
Looking at it from an add-on developer I don't think that this is a very
good idea. Looking back, it was often challenging enough to keep up to
date with TB development during hot phases (changed APIs, UI etc.). If
Thunderbird would release every 2 months, I'd assume that many add-on
developers would give up because they would get tired running after a
permanently moving target.
-Patrick
Ben Bucksch
> On 11/25/2009 07:59 PM, Ben Bucksch wrote:
>> I'd propose releases ... [every] 1-2 months.
>
> How much of that time would actually be open for non-blocker checkins?
All of it. Trunk is *always* open. Only solid stuff gets on trunk. And
releases branch at random times from it.
> string freeze
... would be on the release branch only.
> there's significantly less time to let complex features bake on trunk
Features are not baked on trunk, but on feature branches, as my post
said, and only checked into trunk when they are better than trunk.
Only problem I see there is to get testers who take a close look at
these feature branch builds.
Ben Bucksch
It took "half an eternity" from 2.0 to 3.0. Therefore, it takes 1/20 of
an eternity from 3.0 to 3.1.
JoeS
>
>> there's significantly less time to let complex features bake on trunk
>
> Features are not baked on trunk, but on feature branches, as my post
> said, and only checked into trunk when they are better than trunk.
>
> Only problem I see there is to get testers who take a close look at
> these feature branch builds.
Most testers that I know don't build, so what would you do for these feature branches.
Nightly Tryserver builds ?
--
JoeS Using TB3
http://kb.mozillazine.org/Thunderbird_3.0_-_New_Features_and_Changes
https://developer.mozilla.org/en/Thunderbird/Thunderbird_Binaries
Ron K.
Yesterday when I first read Dans post I thought the shorter 4-6 month cycle
is more optimal. I am believing a short cycle could be more flexible. There
is need for infrastructure to support more frequent releases, more in terms
of human resources than hardware, to make it work.
Here is what I thought initially. Plan for 4 month cycles with a hard
policy that release creep will not exceed 6 months. New policy guidelines
for use of priority to aid cycle targeting may help. The flexibility I
envisioned comes from a person deciding how many cycles will be needed to
produce a solid feature. This could translate to Priority 1 means One
Cycle, 2 means Two Cycles, etc. This can tie into Roadmap and other Tb
management processes.
Also related to short cycles is a reduced maintainers work load. We gain
from branches expiring more often. This combines with moving to Gecko point
releases when security issues crop up.
Where we can gain is relieve of two pressures. First is the crunch to
polish for a looming deadline. This I think generates a risk for mistakes
at worst or less than best solutions. A great strength of Peer Review is it
mitigates risk of error or sub-optimal solutions.
The other pressure I perceive leads from long cycles. There I see: "If I
miss this release, it will be close to a year before users get to see this
in action". One of the issues we discussed months ago was related to
Netbooks and there small screens. My comprehension then was TbNext was the
target for this emerging user appliance. So What happened? We were
impacted by evolution of the User Marketplace that put Tb into a stern
chase where We don't have the speed to catchup. Long cycles produce risk
that We will be out of step more often.
Some of what I hope results from short cycles is expansion of the QA Tester
community. The gain here could be more mentoring opportunities.
--
Ron K.
Who is General Failure, and why is he searching my HDD?
Kernel Restore reported Major Error used BSOD to msg the enemy!
Wayne Mery
> On 11/26/2009 10:14 AM, Ben Bucksch wrote:
>>
>>> there's significantly less time to let complex features bake on trunk
>>
>> Features are not baked on trunk, but on feature branches, as my post
>> said, and only checked into trunk when they are better than trunk.
>>
>> Only problem I see there is to get testers who take a close look at
>> these feature branch builds.
>
> Most testers that I know don't build, so what would you do for these
> feature branches.
> Nightly Tryserver builds ?
>
additionally, there would need to be a stronger announcement mechanism
for new features, so that we get as much testing as possible, and not
rely on serendipity for people to learn of them. preferably something
automated, single authoritative source. something like planet comes to
mind, but not sure if planet is well suited to that. maybe a planet for
announcing to testers?
Andrew Sutherland
> end of November, and only now do I feel that the review window may be
> opening again for 3.1. Add to that 11 week delay another 2-3 weeks for
> the reviews themselves, and we are pushing 14 weeks of formal and
> informal freezes for review work for non-blockers (with the bar set
> quite high for blocking status) near a release. 14 weeks is a pretty
> major chunk of the 4 - 6 months that you are trying to achieve.
Out of interest, what kind of patches were these?
I think a current trend and planned future direction has been to
encourage initial development of features as extensions first and
recognizing that some things should perhaps only exist as extensions.
You have been doing an exemplary job of this, so I think it is very
relevant to the conversation to know what efforts of yours have been
stymied by being unable to land on an active common development branch.
Andrew
David Ascher
> additionally, there would need to be a stronger announcement mechanism
> for new features, so that we get as much testing as possible, and not
> rely on serendipity for people to learn of them. preferably something
> automated, single authoritative source. something like planet comes
> to mind, but not sure if planet is well suited to that. maybe a
> planet for announcing to testers?
I like that idea. I don't think we need a planet, we could have a
multi-author blog aimed at testers and maybe add-on developers. Or a
mailing list, or both.
dev-apps-thunderbird is not currently targeted enough. It could have
served that purpose, but at this point there are too many "side"
conversations going on there.
--david
Kent James
> On 11/25/2009 11:06 PM, Kent James wrote:
>
>
A good example of what for me was a "late" patch that just made it
before the informal freeze is Bug 127250 - "Body" filter for IMAP
messages downloaded for offline use. Submitted for review on 2009-07-31,
checked in 2009-09-04. Each bug has its quirks, that one had a lower
benefit to risk nature than most, which slowed it down some.
> I think a current trend and planned future direction has been to
> encourage initial development of features as extensions first and
> recognizing that some things should perhaps only exist as extensions.
> You have been doing an exemplary job of this, so I think it is very
> relevant to the conversation to know what efforts of yours have been
> stymied by being unable to land on an active common development branch.
>
I was not stymied by the process, but I took advantage of a 2 year
release cycle that we all agree is not acceptable. My point was that the
processes associated with reviews and landing, both formal and informal,
insert very long freezes into the development process. These processes
need to be shortened if you are going to shorten the cycle, and leave
room for outsiders to contribute innovation that is outside of the
"blocker" list established by drivers. If you just follow your existing
management and release strategies, the natural tendency would be to
shorten the thawed state to an unacceptably short length.
On the issue of extensions, I keep meaning to do a blog posting on
"extension driven development" to discuss some of my experiences. But in
the context of the current discussion, if you had some sort of status of
a program feature that was intermediate between a current "extension"
and a current "core feature", then you could reduce the required
churning in the core code. That could both help reduce the development
cycle time, as well as avoid the sort of negative experiences that some
existing users have with the introduction of new features (such as gloda
and the new header) that are "forced" on them in a sometimes raw state.
Kent James
Mark Banner
> On 11/26/2009 8:54 AM, Andrew Sutherland wrote:
>> On 11/25/2009 11:06 PM, Kent James wrote:
>>
>> ... Out of interest, what kind of patches were these?
>>
>
> A good example of what for me was a "late" patch that just made it
> before the informal freeze is Bug 127250 - "Body" filter for IMAP
> messages downloaded for offline use. Submitted for review on 2009-07-31,
> checked in 2009-09-04. Each bug has its quirks, that one had a lower
> benefit to risk nature than most, which slowed it down some.
Agreed that was slow, but my gut instinct tells me it wasn't a typical
period (though stats could prove me wrong).
I think we the TB 3.0 cycle it has been quite difficult on the reviewer
front. IMO one of the issues was that we started off with a very small
team, and an even smaller team of reviewers. Unfortunately adding new
reviewers overnight is very difficult because of the necessary learning
curve.
However we've seen that as we went through the 3.0 cycle we have added a
few extra reviewers and I wouldn't be surprised if we add more soon.
Added to that we're also starting to think about adopting the changes to
the sr requirement that gecko has done (as mentioned in Dan's post), I
think this will help a lot of smaller patches as they don't have to go
through the smaller set of people required to do sr and we can share the
load more (which in turn will then help the bigger patches move through).
We also want to set up something so that we can track review times
better and analyse them, we just haven't had the time to do that yet.
> If you just follow your existing
> management and release strategies, the natural tendency would be to
> shorten the thawed state to an unacceptably short length.
Having thought about it a bit, I think one of the main problems here is
that the drivers had to do a significant proportion of the blocker work
on the branch and most of the reviews.
I think that as we grow and more contributors get knowledge of the code
base, we'll be able to get more reviewers who are non-drivers which will
help with the general review load overall.
Standard8
Kent James
>
> Agreed that was slow, but my gut instinct tells me it wasn't a typical
> period (though stats could prove me wrong).
>
That was longer than most, which I pointed out in my post.
> I think we the TB 3.0 cycle it has been quite difficult on the reviewer
> front.
I'm sure it must have been. The few times I was asked to do reviews, I
found it difficult and disruptive. I can imagine how challenging it must
be to do lots of reviews, as well as pushing forward critical code yourself.
>
> Having thought about it a bit, I think one of the main problems here is
> that the drivers had to do a significant proportion of the blocker work
> on the branch and most of the reviews.
>
Indeed, that is the main issue - and that is what would have to change.
How could that possibly ever happen? Only with some radical changes. One
approach would be an extension-based development process, where new
features bake in extensions for a release cycle before getting added to
core. Then the focus on the release cycle would be getting out a new
stable base, and the vast majority of critical developer time would go
into extension development which is not correlated to the core release
schedule. There are lots of issues with that approach of course.
rkent
Axel Hecht
Mark Banner
> On 11/26/2009 2:58 PM, Mark Banner wrote:
>> Having thought about it a bit, I think one of the main problems here is
>> that the drivers had to do a significant proportion of the blocker work
>> on the branch and most of the reviews.
>>
> Indeed, that is the main issue - and that is what would have to change.
> How could that possibly ever happen?
Did you read the next paragraph (maybe I should have done it as one)? As
we grow more people contributing (which we have over the TB 3 cycle and
I see no reason that will change at the moment), we'll be able to get
more people with enough knowledge to be able to do reviews. Most of
those won't become drivers and so we'll be able to share the load out
better and maybe redirect reviews from overloaded people.
> Only with some radical changes. One
> approach would be an extension-based development process, where new
> features bake in extensions for a release cycle before getting added to
> core. Then the focus on the release cycle would be getting out a new
> stable base, and the vast majority of critical developer time would go
> into extension development which is not correlated to the core release
> schedule. There are lots of issues with that approach of course.
I think we agree that an extension based development process would help.
If you look at the 3.1 page referenced from Dan's 3.0 page then we are
thinking about this.
One issue with extension (and branch) based development is that the
reviews for getting it into the code base are typically on a large piece
of code which makes the review complex, harder and much longer -
although there is probably an overall saving. We need to try and work
out a better way of simplifying the final review, and we'll look at that
as we start experimenting with extension based development.
I'd also like to bring in the other point in Dan's 3.0 page, which is
testing - providing automated tests for a patch does typically (IMO)
make a reviewer's life simpler as it shows various cases have already
been thought about and tried out, hence requiring less need for the
reviewer to go and thrash it out as much as possible.
Standard8
Pavol Mišík
simpler compare iid of interface that I want to use then do complicated
checking. :-)
I do a binary extension that is compatible with Tb2 & Tb3 with debug and
release build of Thunderbird and it is only dll.
Try to use https://bugzilla.mozilla.org/show_bug.cgi?id=477166 to check
them. It is better then nothing. It would bi fine to check on night
build or before alfa, beta, rc release.
PM-
Ben Bucksch
> Out of interest, what kind of patches were these?
>
> I think a current trend and planned future direction has been to
> encourage initial development of features as extensions first and
> recognizing that some things should perhaps only exist as extensions.
> You have been doing an exemplary job of this, so I think it is very
> relevant to the conversation to know what efforts of yours have been
> stymied by being unable to land on an active common development branch.
ACL edit UI <https://bugzilla.mozilla.org/show_bug.cgi?id=522955>, but
it can't work without the IMAP backend code changes in
<https://bugzilla.mozilla.org/show_bug.cgi?id=522954>.
Ben Bucksch
> On 11/26/2009 10:14 AM, Ben Bucksch wrote:
>> Features are not baked on trunk, but on feature branches, as my post
>> said, and only checked into trunk when they are better than trunk.
>>
>> Only problem I see there is to get testers who take a close look at
>> these feature branch builds.
>
> Most testers that I know don't build, so what would you do for these
> feature branches.
> Nightly Tryserver builds ?
>
Yes, something like that.
Ideally, we'd have a field in bugzilla, much like the attachments thing
now, where we'd put the hg branch URL, and the server would find it, see
when the branch changed, make a new build, and put the URL of the builds
(all 3 platforms) next to the branch URL. The new builds are also
automatically published to some webpage, so that testers can find the
"new hot stuff".
For now, we'd do that manually, putting the hg branch URL in a comment,
and when we made significant code changes, trigger tryserver builds and
put the results in the bug as comments, maybe at the same time when we
describe the changes we made.
I think the manually published test/build blog/planet that David A.
suggested is a good idea. That would allow to describe changes for
testers and allow them to focus on particular areas and to ask them to
test specific functionality or parts of the product. That's something we
don't have at all right now and I think that'd be useful even without
feature branches.
Ben Bucksch
To expand on this, on one thing I haven't made explicit, but should:
The meaning of "trunk" changes: It is still the "trunk" and "central"
where everything branches from, the "mainline", where you can follow the
progress of the project over long-term history and look up why a certain
function has been added 3 years ago,
But trunk/comm-central is *no* longer the active development branch.
Development does not happy on trunk anymore, but on branches. Branches
are brought to a release-worthy state, and only then get checked into
trunk. Trunk is always release-worthy, at any given time, at least
that's the goal.
Of course, that won't be *always* true, there will be unforeseen
regressions occasionally, and they can be fixed with followups. If a
release happened to branch after such a regression-causing check-in but
before the regression fixes, the latter can be ported to the release
branch, cherry-picked by drivers. The process of that could work much
like devs apply for branch approval for patches now, just that now
hopefully much less patches would be needed to be ported.
---
So, technically, trunk would always be open, there'd be no freezes on
trunk at any time, new work can always be added and it will appear in
the next release, which is < 2 months away, at any given time, no matter
which release train the check-in catches.
As for human resources, who can also be a bottleneck, there'd be 1 or 2
dedicated release drivers, who take care of the release branch, track
and manage its stability, approve patches. This wouldn't have to be
dmose, it'd be a person not focused on dev, but on release stability and
quality, somebody who's a bit of dev, manager and QA person in one. The
release drivers would work pretty much independently from the dev
process and people, but would have the authority to drag devs in as
needed with high priority when they find regressions which need
hotfixes, or when it's not fixable. The latter would be the only
bottleneck that releases cause for dev, and maybe testers more focused
on release than feature branches.
Ben
Ben Bucksch
I think we agree that an extension based development process would help.
They both have strengths and weaknesses, though:
Extension based dev:
- [+] Natural dev style for XUL/JS devs coming from outside
extension community
- [+] No need to know hg
- [-] Changes / development progress not tracked in hg
- [-] Extension hookups (element IDs, boxen etc.) missing, can't
add in the extension. It's good to notice that, but the concrete
extension can't be realized/installed until core changes have happened.
Process splits.
- [-] Can't do C++ changes to existing classes.
- [-] Additional work to change the code from extension to core
code. (Particularly for smaller changes, the overhead is big.)
- file location on harddisk
- file URLs
- Source (JS, DTD, .properties) location - new file (extension)
or merged into existing file (core code)
- XUL overlays or merged into existing code.
Feature branch based dev:
- [+] Can do all necessary changes: C++, XUL changes
- [+] Can make them in proper form, ready for check-in
- [+] Proper and full dev and project history in hg, very fine-grained
- [+] This includes regression fixes which happen after the first
merge. They are checked into the feature branch and then merged to
trunk. This allows to associate, in a machine-readable way, the
regression fix with the feature, and allows to easily back out the
whole feature including regression fixes on top of it when needed.
- [+] The idea is that all changes of all kinds required for a
certain feature are in one place, one form, tracked and easy to see.
- [+] One dev process for all kinds of changes (C++ and XUL changes),
which potentially means smaller learning curve for testers and better
project tools coverage like bugzilla integration. E.g. with bugzilla-hg
integration you could see whenever the dev made changes to the branch,
without having to attach patches or whole xpi files to bugs. Try server
could be integrated with bugzilla, too, so you can install directly
from bugzilla, even for C++ changes.
- [-] Harder to try out. Need try server builds or self-made builds.
- [-] Can't merge several different changes unless you build
yourself or make a custom try server build.
When I look at the number of [+] and [-] above, it's obvious that I'm biased. A lot of that bias may steem from me typically doing core changes, including C++. Extension devs will have a different view. For them, the "one process for all" argument goes the other way, as they need to work with extensions anyways, and there are tools for that.
One issue with extension (and branch) based development is that the reviews for getting it into the code base are typically on a large piece of code which makes the review complex, harder and much longer - although there is probably an overall saving.
I'd also like to bring in the other point in Dan's 3.0 page, which is testing - providing automated tests for a patch does typically (IMO) make a reviewer's life simpler as it shows various cases have already been thought about and tried out, hence requiring less need for the reviewer to go and thrash it out as much as possible.
This kind of thinking is exactly what must *not* happen with tests, and what terribly scares me. They do *not* allow to reduce the amount of thinking of potential problems, not by any involved party.
Ben
Andrew Sutherland
>> I'd also like to bring in the other point in Dan's 3.0 page, which is
>> testing - providing automated tests for a patch does typically (IMO)
>> make a reviewer's life simpler as it shows various cases have already
>> been thought about and tried out, hence requiring less need for the
>> reviewer to go and thrash it out as much as possible.
>
> The reviewer should still do that. Chances are he'll think of something
> or run into corner cases that the coder did not. If the coder thought of
> it, he'd have fixed it, no?
>
> This kind of thinking is exactly what must *not* happen with tests, and
> what terribly scares me. They do *not* allow to reduce the amount of
> thinking of potential problems, not by any involved party.
I think Standard8's point was that patches should come with unit tests,
not that a reviewer should use the presence of unit tests as an excuse
to slack off.
Tests show that the coder not only thought about the corner case but
also dealt with it. (Or at least tried to deal with it; it's very
possible to write a test that doesn't test what it thinks it is testing.
This is one reason it's nice when the situation allows for tests that
fail meaningfully without the test and pass with it.)
In my mind, a functional (non-aesthetic/documentation) patch that is not
simply addressing a currently failing test should come with tests.
Otherwise, how do you help guarantee that the functionality doesn't
break in the future or change back to how it was before the patch?
Andrew
Kent James
>
> more or less the same idea.
>
They are quite different if I use my definition of "extension based
development". I guess I really need to write up my thoughts on that, but
the best examples that I have are my two extensions "FiltaQuilla" and
"JunQuilla". Both are based on extensive backend work in C++ that
enabled new features, but the features were primarily expressed in
extensions - and always will be.
I don't have a very positive attitude about branch based development.
Delaying adding changes to trunk until the extent of the change has
accumulated to a massive scale does not seem to me to have any
advantages, unless the changes are clearly experimental with a less than
50% chance of ever landing. One good application in my work would have
been the experimental replacement of the .mdb databases with a
generalized format, with parallel sqlite and mork implementations.
> They both have strengths and weaknesses, though:
>
> Extension based dev:
>
> * [+] Natural dev style for XUL/JS devs coming from outside
> extension community
> * [+] No need to know hg
> * [-] Changes / development progress not tracked in hg
> * [-] Extension hookups (element IDs, boxen etc.) missing, can't add
> in the extension. It's good to notice that, but the concrete
> extension can't be realized/installed until core changes have
> happened. Process splits.
> * [-] Can't do C++ changes to existing classes.
> * [-] Additional work to change the code from extension to core
> code. (Particularly for smaller changes, the overhead is big.)
> o file location on harddisk
> o file URLs
> o Source (JS, DTD, .properties) location - new file
> (extension) or merged into existing file (core code)
> o XUL overlays or merged into existing code.
>
I want to criticize this point by point, but it is not really fair since
we are working from two different definitions of "extension based
development". Let me just point out that I am primarily viewed as a C++
developer in the community - but also as primarily an extension
developer. My extensions are managed through a private installation of
hg and bugzilla. So virtually nothing that you wrote above is true of me
or my extensions. Frankly, I can't imagine trying to do extensions
without also being free to incorporate needed changes or features in the
core code (as Ben has already observed can be critical to success). But
fortunately the owners of the core code are quite willing to accept
patches that are primarily intended as hooks for extensions.
>
>> One issue with extension (and branch) based development is that the
>> reviews for getting it into the code base are typically on a large
>> piece of code which makes the review complex, harder and much longer -
>> although there is probably an overall saving.
>
> You can, if you want to, review steps in between. That's easy with
> feature branches, they're just like trunk in this respect, because
> there's a clear state and easy diffs, but also possible with extension
> development.
>
The answer to this is that the extensions that I am talking about are
never incorporated into the core code, so the need to review a massive
change to core never happens. But as I said, I really need to write a
coherent description of my thoughts rather than argue pointlessly when
we have not even agreed on definitions. I'm going to claim the term
though (the top two Google hits on "extension based development", of 10
total, are from this current thread, where I first used the term.)
rkent
Eddy Nigg
Dan, what I am greatly missing is the vision and guide to which product
Thunderbird (aka Mozilla Messaging) is supposed to evolve. Quite a while
ago when Thunderbird was split from the Firefox Mozilla team, a small
group of people made suggestions and wrote down ideas about how
Thunderbird could look down the road. I fail to see in which direction
Thunderbird wants to go generally.
Looking today at Google Wave I can clearly see how Thunderbird could
have evolved instead - Google just does it in the browser what the group
thought could/would/should happen with the application called
Thunderbird. What is the road map of Thunderbird,
https://wiki.mozilla.org/Thunderbird:Mission_and_Roadmap doesn't reveal
much...where is it heading?
--
Regards
Signer: Eddy Nigg, StartCom Ltd.
XMPP: star...@startcom.org
Blog: http://blog.startcom.org/
Twitter: http://twitter.com/eddy_nigg
Ben Bucksch
> My extensions are managed through a private installation of hg ... I
> can't imagine trying to do extensions without also being free to
> incorporate needed changes or features in the core code (as Ben has
> already observed can be critical to success).
I guess we're in violent agreement, then. What you do is pretty much
what I mean with 'feature branch development' (just that you use
extensions in addition to that).
> The answer to this is that the extensions that I am talking about are
> never incorporated into the core code
Yup, and the changes I have in mind are typically intended for core
code, that's probably the only difference. So, it seems we're in violent
agreement ;-).
Joshua Cranmer
> I don't have a very positive attitude about branch based development.
> Delaying adding changes to trunk until the extent of the change has
> accumulated to a massive scale does not seem to me to have any
> advantages, unless the changes are clearly experimental with a less than
> 50% chance of ever landing. One good application in my work would have
> been the experimental replacement of the .mdb databases with a
> generalized format, with parallel sqlite and mork implementations.
The problem with keeping large, separate branches for changes is where
feature development starts squashing on others' toes. You especially
seem confident in this strategy for major underlying rewrites, but
keeping those changes up-to-date and preventing them from affecting
others would be a very big hassle. I discovered that keeping something
as small as the changes to morkreader in sync required a lot of rebasing
patches--and this is an API used only in two or three places.
Core rewrites that greatly affect API usage are not fun in any system,
but if you have two or three of those going on as "semi-official"
branches, that makes merges very painful.
Ben Bucksch
> On 11/28/2009 11:23 PM, Kent James wrote:
>> I don't have a very positive attitude about branch based development.
>> Delaying adding changes to trunk until the extent of the change has
>> accumulated to a massive scale does not seem to me to have any
>> advantages, unless the changes are clearly experimental with a less than
>> 50% chance of ever landing
>
> feature development starts squashing on others' toes. You especially
> seem confident in this strategy for major underlying rewrites, but
> keeping those changes up-to-date and preventing them from affecting
> others would be a very big hassle.
I am not talking about large, long-lived branches only. It's the SVN
thinking that branches are mostly for that.
I am talking about doing *everything* in branches. There are no more
"check-ins" to trunk, only merges. Branches are no longer a private
little side project, but the main development platform where everybody
plays.
Yes, if - in some, exceptional cases - you end up with a branch with
many changes which lives over a long time, you have a problem with
merging. It's not easy or fun, but I think it's manageable, if other
devs are aware of it and avoid conflicts or help resolving them, the hg
merge helps a bit, etc.. The branches wouldn't be "semi-official" as you
say and is the case now, but they'd be official. We can collaborate.
Remember, in effect, the feature branches are the new development
branch(es) where we all work, not trunk.
The current dev style, where we check these "massive changes" into trunk
early, has the result of destabilizing trunk, and that we can't release
unless the things are fixed before. Often, we find that it all takes
longer than expected, and we're forced to ship something which just
barely meets the "kind of usable" bar, and is often in rather raw state
forced on users.
Big changes aren't the common case, though. Typically, people work in
different parts, and most things don't need more than 1-2 weeks to
develop, review, test, possibly regression-fix, and then merge. The only
difference in these cases is that test and regression fix happens
/before/ check-in to trunk.
> required a lot of rebasing patches
We need a hg or hg use pattern which can make it easy to have branches
which follow trunk, yes.
> Core rewrites that greatly affect API usage are not fun in any system
Right, indeed.
I just want to stress that "feature branch developer" is not just for
big core rewrites, but for *all* changes.
Ben