Branching for Firefox releases in Mercurial

9 views
Skip to first unread message

Ben Hearsum

unread,
Aug 5, 2008, 9:20:15 AM8/5/08
to
Hi All,

With implementation beginning on release automation for Mercurial it's
time to make a decision on how we will branch for releases in Mercurial.

I will outline a few options here. Options not listed here are welcome.

1) Clone a new repository for each release. This is close to the
subversion model. We would have a top-level
http://hg.mozilla.org/releases that would have sub-directories like:
'firefox-3.1a1', 'firefox-3.1b2', 'firefox-3.1rc1', etc.

Advantages: Simple, requires no changes to existing repository hooks.
Disadvantages: Release branch landings, version bumps will NOT be
visible in mozilla-central. Slower (Onumber-of-changesets). Unclear how
automation would clone a new repository (currently, only IT has access
to do this). Easy, but very slow to land release branch fixes on.

2) Create a named branch for each release inside of mozilla-central.
This would create one new head per release.

Advantages: Release branch history+changesets live inside of
mozilla-central. Fast. Easy to land fixes on (proviso: user knows how to
use Mercurial branches).
Disadvantages: Things that land on the release branch and 'default'
branch of mozilla-central will show up twice in the changelog.

3) Create an unnamed branch for version bumps, release branch landings,
and merge it back to 'default' before pushing. This is would we did (by
hand) for Firefox 3.1a1.

Advantages: Fast, requires no changes to existing repository hooks.
Release branch history+changesets live inside of mozilla-central.
Disadvantages: A little hacky, things that land on the release branch
will still show up twice in the changelog. More difficult to land
release branch fixes on.


Based on the above and offline (IRC, summit) conversations I've had and
heard, I think 2) (Create a named branch inside of mozilla-central) is
the best option here. Option 1 feels pretty heavy and it's not
immediately clear how we'll implement it. Option 3 is pretty hacky, and
as stated above, more difficult to land fixes on if need be.

From a RelEng standpoint it doesn't matter much what we do, as long as
it can be automated.

I'll open this up to discussion now. I'd like to make a decision about
this before the end of the week.

- Ben

Benjamin Smedberg

unread,
Aug 5, 2008, 10:29:48 AM8/5/08
to
Ben Hearsum wrote:

> 2) Create a named branch for each release inside of mozilla-central.
> This would create one new head per release.
>
> Advantages: Release branch history+changesets live inside of
> mozilla-central. Fast. Easy to land fixes on (proviso: user knows how to
> use Mercurial branches).
> Disadvantages: Things that land on the release branch and 'default'
> branch of mozilla-central will show up twice in the changelog.

The other disadvantages of this option are:

1) would require changes to the require-singlehead hook to require a single
head per-branch...
2) would require some changes, perhaps significant, to the pushlog, so that
pushes to the release branch did not show up on the main tinderbox

Of these, I'm more concerned about the second one, because it doesn't sound
trivial to map heads to branches in a persistent way.

There is another way to do this: have a single repository (not
mozilla-central) on which all releases live. e.g.

hg.mozilla.org/mozilla-releases (or firefox-releases?)

This repository would have the multiple heads, release branches, and tagged
releases of option 2) above, but only the build automation would push to it.
This has the advantage of not requiring any changes to existing buildbots,
while also not requiring IT to create a new repo for each release.

--BDS

Sergey Yanovich

unread,
Aug 5, 2008, 10:30:15 AM8/5/08
to
Ben Hearsum wrote:
> 1) Clone a new repository for each release. This is close to the
> subversion model. We would have a top-level
> http://hg.mozilla.org/releases that would have sub-directories like:
> 'firefox-3.1a1', 'firefox-3.1b2', 'firefox-3.1rc1', etc.

This option looks best.

> Advantages: Simple, requires no changes to existing repository hooks.
> Disadvantages: Release branch landings, version bumps will NOT be
> visible in mozilla-central. Slower (Onumber-of-changesets). Unclear how
> automation would clone a new repository (currently, only IT has access
> to do this). Easy, but very slow to land release branch fixes on.

Extra advantages: minimal size of mozilla-central checkout, minimal size
of repository.

> 2) Create a named branch for each release inside of mozilla-central.
> This would create one new head per release.
>
> Advantages: Release branch history+changesets live inside of
> mozilla-central. Fast. Easy to land fixes on (proviso: user knows how to
> use Mercurial branches).
> Disadvantages: Things that land on the release branch and 'default'
> branch of mozilla-central will show up twice in the changelog.

Extra disadvantages: huge size of repository, big size of checkout.

> 3) Create an unnamed branch for version bumps, release branch landings,
> and merge it back to 'default' before pushing. This is would we did (by
> hand) for Firefox 3.1a1.
>
> Advantages: Fast, requires no changes to existing repository hooks.
> Release branch history+changesets live inside of mozilla-central.
> Disadvantages: A little hacky, things that land on the release branch
> will still show up twice in the changelog. More difficult to land
> release branch fixes on.

Extra disadvantages: big size of repository, huge size of checkout, all
commits on release branch will appear twice in the changelog (either
normally, or in reverted form).

In addition, this option may turn out to not be doable without manual
intervention.

> Based on the above and offline (IRC, summit) conversations I've had and
> heard, I think 2) (Create a named branch inside of mozilla-central) is
> the best option here. Option 1 feels pretty heavy and it's not
> immediately clear how we'll implement it. Option 3 is pretty hacky, and
> as stated above, more difficult to land fixes on if need be.

I'll try to explain my estimations:

Mercurial keeps full history for each file in each unmerged head. So
option #2 will add up full history of every file touched on any release
branch.

Both options #2 and #3 will also add every commit on every release
branch to every checkout be every Mozilla hacker.

Finally, option #3 will require that each release-specific commit is
reverted after its unnamed head is merged back. This will most probably
require some manual interventions to tell release-specific commits (like
version changes, build config changes and so on) from code fixes.

I also have some use cases:

Linux kernel is using a strategy that closely resembles option #1, and
they have rather slim mainstream repository. After 3 years of using git,
their repository weights around 180 Mb for a project that roughly equals
Mozilla in size.

Samba is using something similar to option #2. They keep at least 6
branches (3.0-stable, 3.0-test, 3.2-stable, 3.2-test, 4.0-stable,
4.0-test) inside a single repository. After 2 years of development,
their repository is ~400Mb, although they are ~8 times smaller than
linux kernel.

--
Sergey Yanovich

Ben Hearsum

unread,
Aug 5, 2008, 11:18:15 AM8/5/08
to Sergey Yanovich
Sergey Yanovich wrote:
> Ben Hearsum wrote:
>> 1) Clone a new repository for each release. This is close to the
>> subversion model. We would have a top-level
>> http://hg.mozilla.org/releases that would have sub-directories like:
>> 'firefox-3.1a1', 'firefox-3.1b2', 'firefox-3.1rc1', etc.
>
> This option looks best.
>

What do you think of bsmedberg's proposed option?


>> 3) Create an unnamed branch for version bumps, release branch
>> landings, and merge it back to 'default' before pushing. This is would
>> we did (by hand) for Firefox 3.1a1.
>>
>> Advantages: Fast, requires no changes to existing repository hooks.
>> Release branch history+changesets live inside of mozilla-central.
>> Disadvantages: A little hacky, things that land on the release branch
>> will still show up twice in the changelog. More difficult to land
>> release branch fixes on.
>
> Extra disadvantages: big size of repository, huge size of checkout, all
> commits on release branch will appear twice in the changelog (either
> normally, or in reverted form).
>
> In addition, this option may turn out to not be doable without manual
> intervention.
>

Not sure what you mean here. It certainly requires manual intervention
to land fixes before a respin. The automation is going to take an input
of a changeset that's signed off on, though, and will branch/tag from
there (only branching for the first build of a release, of course). Once
the tags are applied, the rest of the process simply clones and updates
to the tag. Not sure what type of manual intervention would be needed.

>> Based on the above and offline (IRC, summit) conversations I've had
>> and heard, I think 2) (Create a named branch inside of
>> mozilla-central) is the best option here. Option 1 feels pretty heavy
>> and it's not immediately clear how we'll implement it. Option 3 is
>> pretty hacky, and as stated above, more difficult to land fixes on if
>> need be.
>
> I'll try to explain my estimations:
>
> Mercurial keeps full history for each file in each unmerged head. So
> option #2 will add up full history of every file touched on any release
> branch.
>

I expected this will be minimal. Every release will have 3 commits, at
least. One for version bumps, two for tags (_BUILD# tag, and _RELEASE
tag). More will be necessary if we have to respin.

> Both options #2 and #3 will also add every commit on every release
> branch to every checkout be every Mozilla hacker.
>
> Finally, option #3 will require that each release-specific commit is
> reverted after its unnamed head is merged back. This will most probably
> require some manual interventions to tell release-specific commits (like
> version changes, build config changes and so on) from code fixes.
>

This would be handled as part of the automation. If we go this option
we'd require the tree be closed while this is happening to avoid getting
a third head. Assuming no-one else pushes between our clone to version
bump, tag, etc, and our push back it's a very predictable process. Of
course, we guarantee someone won't push while we're doing this.

> I also have some use cases:
>
> Linux kernel is using a strategy that closely resembles option #1, and
> they have rather slim mainstream repository. After 3 years of using git,
> their repository weights around 180 Mb for a project that roughly equals
> Mozilla in size.
>
> Samba is using something similar to option #2. They keep at least 6
> branches (3.0-stable, 3.0-test, 3.2-stable, 3.2-test, 4.0-stable,
> 4.0-test) inside a single repository. After 2 years of development,
> their repository is ~400Mb, although they are ~8 times smaller than
> linux kernel.
>

These are interesting.

Sergey Yanovich

unread,
Aug 5, 2008, 1:11:31 PM8/5/08
to Ben Hearsum
Ben Hearsum wrote:
> What do you think of bsmedberg's proposed option?

The idea not to pollute mozilla-central is great. However, if security
branches are split off the proposed all-in-one repository, all downsides
of original option #2 will be felt by those who work on security branches.

> I expected this will be minimal. Every release will have 3 commits, at
> least. One for version bumps, two for tags (_BUILD# tag, and _RELEASE
> tag). More will be necessary if we have to respin.

> This would be handled as part of the automation. If we go this option

> we'd require the tree be closed while this is happening to avoid getting
> a third head. Assuming no-one else pushes between our clone to version
> bump, tag, etc, and our push back it's a very predictable process. Of
> course, we guarantee someone won't push while we're doing this.

Why not do those 3 commits on the mozilla-central, then? No branches, no
heads...

This is exactly the way linux kernel repository is managed. The tag is
later used as branching point for dot-releases.

--
Sergey Yanovich

Ben Hearsum

unread,
Aug 5, 2008, 1:20:30 PM8/5/08
to Sergey Yanovich
Sergey Yanovich wrote:
> Ben Hearsum wrote:
>> What do you think of bsmedberg's proposed option?
>
> The idea not to pollute mozilla-central is great. However, if security
> branches are split off the proposed all-in-one repository, all downsides
> of original option #2 will be felt by those who work on security branches.
>

What do you mean by "security branch"? For example, when 3.1 is released
the 3.0.x would be a "security branch"? If that's the case, I think
you've misinterpreted what I've said. This discussion in only about
release "minibranches". These branches are where we do version bumps
(eg. 3.0.2pre -> 3.0.2) and land fixes in the case of a respin. This is
explicitly NOT like the MOZILLA_1_8_BRANCH in CVS, but rather, like the
GECKO* branches.

What we do when 3.1 is released isn't relevant to this discussion -- but
you make a good point (if I'm interpreting it correctly!).

>> I expected this will be minimal. Every release will have 3 commits, at
>> least. One for version bumps, two for tags (_BUILD# tag, and _RELEASE
>> tag). More will be necessary if we have to respin.
>
>> This would be handled as part of the automation. If we go this option
>> we'd require the tree be closed while this is happening to avoid
>> getting a third head. Assuming no-one else pushes between our clone to
>> version bump, tag, etc, and our push back it's a very predictable
>> process. Of course, we guarantee someone won't push while we're doing
>> this.
>
> Why not do those 3 commits on the mozilla-central, then? No branches, no
> heads...
>
> This is exactly the way linux kernel repository is managed. The tag is
> later used as branching point for dot-releases.
>

The problem here is that we have to close the tree between the last
checkin for a release and when we do the version bumps and tagging.
There can be lag time here and it's not good to close the tree during
that. Option 3) proposes something like this, though, where we do a
temporary branch from the last checkin for a release, version bump it,
tag it, then merge it back to the default branch with the new version
numbers, all checkins that happened after the branches parent, etc.

Does that make sense?

Dave Townsend

unread,
Aug 5, 2008, 2:10:56 PM8/5/08
to

Can you elaborate on this? If you don't have multiple heads how do you
have the mainline trunk with bleeding edge code and the release point
with your three changesets against an earlier point in trunk?

Dave Townsend

unread,
Aug 5, 2008, 2:13:20 PM8/5/08
to
Ben Hearsum wrote:
> The problem here is that we have to close the tree between the last
> checkin for a release and when we do the version bumps and tagging.
> There can be lag time here and it's not good to close the tree during
> that. Option 3) proposes something like this, though, where we do a
> temporary branch from the last checkin for a release, version bump it,
> tag it, then merge it back to the default branch with the new version
> numbers, all checkins that happened after the branches parent, etc.
>
> Does that make sense?

If you are using a separate named branch for checkins there why would
the main tree need to be closed? Maybe just for sanity I guess but it
isn't like anyone should be checking in on the release branch.

Sergey Yanovich

unread,
Aug 5, 2008, 2:38:37 PM8/5/08
to Ben Hearsum
Ben Hearsum wrote:

> Sergey Yanovich wrote:
>> The idea not to pollute mozilla-central is great. However, if security
>> branches are split off the proposed all-in-one repository, all
>> downsides of original option #2 will be felt by those who work on
>> security branches.
>>
>
> What do you mean by "security branch"? For example, when 3.1 is released
> the 3.0.x would be a "security branch"? If that's the case, I think
> you've misinterpreted what I've said. This discussion in only about
> release "minibranches". These branches are where we do version bumps
> (eg. 3.0.2pre -> 3.0.2) and land fixes in the case of a respin. This is
> explicitly NOT like the MOZILLA_1_8_BRANCH in CVS, but rather, like the
> GECKO* branches.

Yeah, we use the same term for the same thing. My points were:
* there is no easy way to drop branches from Mercurial;
* all-in-one repository will be the only place to branch 3.1.x from;
* as a result, branch 3.1.x will also contain a head for every 3.0.x
release made before 3.1 ships.

>> Why not do those 3 commits on the mozilla-central, then? No branches,
>> no heads...
>

> The problem here is that we have to close the tree between the last
> checkin for a release and when we do the version bumps and tagging.
> There can be lag time here and it's not good to close the tree during
> that.

IIRC, when firefox 3.0 was about to go out, only commits with explicit
driver's approval were allowed. Having a shipment done is usually the
highest priority task.

> Option 3) proposes something like this, though, where we do a
> temporary branch from the last checkin for a release, version bump it,
> tag it, then merge it back to the default branch with the new version
> numbers, all checkins that happened after the branches parent, etc.
>
> Does that make sense?

If each minibranch has only 3 commits, option 3) is just fine. In this
case, the biggest question is really whether it is fine to keep the tree
closed for the period from the last commit of the planned release to the
point when those 3 commits are done. If the answer is positive, you can
avoid all that branching/merging work.

--
Sergey Yanovich

Sergey Yanovich

unread,
Aug 5, 2008, 2:48:56 PM8/5/08
to
Dave Townsend wrote:

> Sergey Yanovich wrote:
>> Why not do those 3 commits on the mozilla-central, then? No branches, no
>> heads...
>>
>> This is exactly the way linux kernel repository is managed. The tag is
>> later used as branching point for dot-releases.
>
> Can you elaborate on this? If you don't have multiple heads how do you
> have the mainline trunk with bleeding edge code and the release point
> with your three changesets against an earlier point in trunk?

Sure. Linux kernel (and all other projects I ever used VCS on) has its
release point in the mainstream (trunk) repository. Making a shipment is
usually a highest priority task.

After shipment is done, the shipment point become root of the security
branch. [1] has a separate repository with a single head for each
"stable" branch.

1. http://git.kernel.org/

--
Sergey Yanovich

Ben Hearsum

unread,
Aug 5, 2008, 3:10:22 PM8/5/08
to Dave Townsend

If we're using a separate, permanent, named branch we do not need to
close the tree. Option 3 proposes a short lived nameless branch that
gets merged back to default. If someone else pushes while we're version
bumping, tagging, etc., we'll end up with weird heads when we push back.
To avoid that, we'd want to close the tree during this process if we go
this route.

Ben Hearsum

unread,
Aug 5, 2008, 3:13:49 PM8/5/08
to Sergey Yanovich
Sergey Yanovich wrote:
> Ben Hearsum wrote:
>> Sergey Yanovich wrote:
>>> The idea not to pollute mozilla-central is great. However, if
>>> security branches are split off the proposed all-in-one repository,
>>> all downsides of original option #2 will be felt by those who work on
>>> security branches.
>>>
>>
>> What do you mean by "security branch"? For example, when 3.1 is
>> released the 3.0.x would be a "security branch"? If that's the case,
>> I think you've misinterpreted what I've said. This discussion in only
>> about release "minibranches". These branches are where we do version
>> bumps (eg. 3.0.2pre -> 3.0.2) and land fixes in the case of a respin.
>> This is explicitly NOT like the MOZILLA_1_8_BRANCH in CVS, but rather,
>> like the GECKO* branches.
>
> Yeah, we use the same term for the same thing. My points were:
> * there is no easy way to drop branches from Mercurial;
> * all-in-one repository will be the only place to branch 3.1.x from;
> * as a result, branch 3.1.x will also contain a head for every 3.0.x
> release made before 3.1 ships.
>

Ahh..I see what you mean now. I'm not particularly concerned about that
situation, but I can see the argument.

>>> Why not do those 3 commits on the mozilla-central, then? No branches,
>>> no heads...
>>
>> The problem here is that we have to close the tree between the last
>> checkin for a release and when we do the version bumps and tagging.
>> There can be lag time here and it's not good to close the tree during
>> that.
>
> IIRC, when firefox 3.0 was about to go out, only commits with explicit
> driver's approval were allowed. Having a shipment done is usually the
> highest priority task.
>

This is true. This is more of a problem with Alphas, where we want to
get the tree open as soon as possible. We really don't want to rely on
the tip of the tree being our release changeset. Mistakes can happen,
and with Mercurial we have increased flexibility to open the tree back
up before Build even starts the release process.

>> Option 3) proposes something like this, though, where we do a
>> temporary branch from the last checkin for a release, version bump it,
>> tag it, then merge it back to the default branch with the new version
>> numbers, all checkins that happened after the branches parent, etc.
>>
>> Does that make sense?
>
> If each minibranch has only 3 commits, option 3) is just fine. In this
> case, the biggest question is really whether it is fine to keep the tree
> closed for the period from the last commit of the planned release to the
> point when those 3 commits are done. If the answer is positive, you can
> avoid all that branching/merging work.
>

See above

igor.b...@gmail.com

unread,
Aug 6, 2008, 5:11:35 AM8/6/08
to
On Aug 5, 8:38 pm, Sergey Yanovich <ynv...@gmail.com> wrote:
> My points were:
...

> * all-in-one repository will be the only place to branch 3.1.x from;

This is not so. All-in-one repository like mozilla-releases can be
used only for release handling with multiple source repositories.

When time comes for 3.1, a new repository like firefox-3.1 can be
created for security and stability work straight from mozilla-central.
After that point mozilla-releases will just pool from 2 repositories,
mozilla-central and firefox-3.1. This way neither mozilla-central nor
firefox-3.1 would contain any noise from the release check-ins.

Regards, Igor

Sergey Yanovich

unread,
Aug 6, 2008, 5:35:24 AM8/6/08
to
ig...@mir2.org wrote:
> On Aug 5, 8:38 pm, Sergey Yanovich <ynv...@gmail.com> wrote:
>> My points were:
> ...
>> * all-in-one repository will be the only place to branch 3.1.x from;
>
> This is not so. All-in-one repository like mozilla-releases can be
> used only for release handling with multiple source repositories.
>
> When time comes for 3.1, a new repository like firefox-3.1 can be
> created for security and stability work straight from mozilla-central.

This is the original option 1), which I believe is the best approach in
the long run.

Cheers,
--
Sergey Yanovich

igor.b...@gmail.com

unread,
Aug 6, 2008, 9:20:24 AM8/6/08
to
On Aug 6, 11:35 am, Sergey Yanovich <ynv...@gmail.com> wrote:

> i...@mir2.org wrote:
>
> > When time comes for 3.1, a new repository like firefox-3.1 can be
> > created for security and stability work straight from mozilla-central.
>
> This is the original option 1), which I believe is the best approach in
> the long run.

I interpret the original option 1 as to clone a repo for each and
every alpha-beta-rc etc. releases. This is different from the idea of
having only one single repository that pulls from mozilla-central,
firefox-3.1 etc. as necessary.

Regards, Igor

Dave Townsend

unread,
Aug 6, 2008, 9:36:03 AM8/6/08
to

Right, so presumably that means the main "trunk" has to remain closed to
checkins until the versions have been updated, the release spun,
possibly respun with fixes, baked and released? Given that that can be
up to a week or more that seems to block development a lot. I guess we
don't do the whole distributed thing so well yet though.

Dave

Ben Hearsum

unread,
Aug 6, 2008, 9:40:50 AM8/6/08
to igor.b...@gmail.com

That's right. However, bsmedberg proposed that option in a previous comment.

Ben Hearsum

unread,
Aug 6, 2008, 9:42:54 AM8/6/08
to Dave Townsend

Yeah, that's my presumption, too. And at this point it is surely not a
route we want to go. And given that there is an easy way to NOT close
the tree between those times I'm not sure we ever want to do this. Not
being able to check-in means losing almost all test coverage.

That's not to say we shouldn't revisit this when we get better at being
distributed, though :).

Rob Helmer

unread,
Aug 6, 2008, 6:24:24 PM8/6/08
to
Maybe this is a dumb question, but instead of checking in three files,
can't the version just be set in the mozconfig, just like branding or
whatever? Why does the version number need to be checked in?

Axel Hecht

unread,
Aug 6, 2008, 7:18:32 PM8/6/08
to

One of the pieces that grabs version information from the code are l10n
repacks. They use the checked in version to figure out which builds they
should repack. In particular for folks doing that locally, putting it
into a config file would be tough.

Axel

Ben Hearsum

unread,
Aug 7, 2008, 8:10:25 AM8/7/08
to Rob Helmer

Even if we did do this we'd still need/want a release branch for respin
fixes. TBH, I feel a little more comfortable with the version number in
the code, with a specific revision. I realize it's still pretty trivial
to change it, but being able to pull revision X and claim it's any
version without changing the code makes me uncomfortable.

Ben Hearsum

unread,
Aug 7, 2008, 2:09:45 PM8/7/08
to
I just had a short conversation with bsmedberg about this.

To be clear, the suggestion here is to do the following for each release:
hg clone http://hg.m.o/mozilla-central
hg push ssh://hg.m.o/mozilla-releases
hg clone http://hg.m.o/mozilla-releases
hg branch GECKO191_20080815_RELBRANCH
[bump the version numbers, create tags]
hg push ssh://hg.m.o/mozilla-releases

This would end up giving us a repository that looks a lot like
mozilla-central, but contains a lot of named that have version bumps on
them.

I think this idea is OK in principle, but a couple things make this hard:
* We'd need a separate releases repository for other products (eg,
mobile-browser, or comm-central).
* We'd probably need separate release repositories for each locale, too.

Based on this knowledge I'd like to suggest that we go with option 2) or
3), with 2) still being my choice.

- Ben

Ben Hearsum

unread,
Aug 8, 2008, 9:39:11 AM8/8/08
to
There haven't been comments from others for a day or so, so I'd like to
try and wrap this discussion up. I'll re-iterate what I said in my
latest post and give people a chance to weigh in.

--

Taking option 1), or bsmedberg's proposed option would end up giving us

a repository that looks a lot like mozilla-central, but contains a lot

of named branches that have version bumps on them.

I think this idea is OK in principle, but a couple things make this hard:
* We'd need a separate releases repository for other products (eg,
mobile-browser, or comm-central).
* We'd probably need separate release repositories for each locale, too.

Based on this knowledge I'd like to suggest that we go with option 2) or
3), with 2) still being my choice.

--

If there are no objections by Tuesday, August 12th I'm going to assume
silent acceptance of option 2).

- Ben

Mike Shaver

unread,
Aug 8, 2008, 9:59:16 AM8/8/08
to Ben Hearsum, dev-pl...@lists.mozilla.org
On Fri, Aug 8, 2008 at 9:39 AM, Ben Hearsum <bhea...@mozilla.com> wrote:
> There haven't been comments from others for a day or so, so I'd like to
> try and wrap this discussion up. I'll re-iterate what I said in my
> latest post and give people a chance to weigh in.
>
> --
>
> Taking option 1), or bsmedberg's proposed option would end up giving us
> a repository that looks a lot like mozilla-central, but contains a lot
> of named branches that have version bumps on them.
>
> I think this idea is OK in principle, but a couple things make this hard:
> * We'd need a separate releases repository for other products (eg,
> mobile-browser, or comm-central).
> * We'd probably need separate release repositories for each locale, too.

Why does that make things hard? If there's anything computers are
good at, it's doing the same thing many times over -- seems like a
very small investment in scripting makes those cases virtually
identical to the one in which we have a single repository to worry
about. Named branches are a model we understand very well, and there
is certainly value to familiarity. (It's sort of too bad that we
don't permit them in mozilla-central, since they were useful to us in
CVS; removing that capability seems sort of like a regression?)

Mike

Ben Hearsum

unread,
Aug 8, 2008, 10:26:56 AM8/8/08
to
Since I've started a new thread I'll summarize the options again:

Option 1) Create a new repository for each release under the
http://hg.mozilla.org/mozilla-releases directory.

Option 1a) A variant of 1); Create a corresponding 'releases' repository
for each code/locale repo (mozilla-central-releases,c
omm-central-releases, l10n/de-releases, etc). These repositories would
end looking very similar to their counterparts, but containing named
branches for each release that have version bumps, tag commits, and
possible respin fixes on them.

Option 2) Create a named branch for each release inside of each
code/locale repository.

Option 3) Do NOT create a branch for releases, create temporary,
anonymous branches that are merged back to default before being pushed
to central repositories.

--

Axel Hecht

unread,
Aug 8, 2008, 10:32:14 AM8/8/08
to

IMHO, Sergey's comments on repository size rule out option 2.

Axel

Ben Hearsum

unread,
Aug 8, 2008, 10:32:20 AM8/8/08
to
Mike Shaver wrote:
> On Fri, Aug 8, 2008 at 9:39 AM, Ben Hearsum <bhea...@mozilla.com> wrote:
>> There haven't been comments from others for a day or so, so I'd like to
>> try and wrap this discussion up. I'll re-iterate what I said in my
>> latest post and give people a chance to weigh in.
>>
>> --
>>
>> Taking option 1), or bsmedberg's proposed option would end up giving us
>> a repository that looks a lot like mozilla-central, but contains a lot
>> of named branches that have version bumps on them.
>>
>> I think this idea is OK in principle, but a couple things make this hard:
>> * We'd need a separate releases repository for other products (eg,
>> mobile-browser, or comm-central).
>> * We'd probably need separate release repositories for each locale, too.
>
> Why does that make things hard? If there's anything computers are
> good at, it's doing the same thing many times over -- seems like a
> very small investment in scripting makes those cases virtually
> identical to the one in which we have a single repository to worry
> about.

It doesn't necessarily make things hard, but I feel like it makes them
messier. Every time we create a locale we'll need a corresponding
'release' repository. Especially in the locales case, where we rarely
have things land on the release branch this seems unnecessary. Even in
the case of mozilla/comm-central we would not end up with _that_ many
extra commits.

> Named branches are a model we understand very well, and there
> is certainly value to familiarity. (It's sort of too bad that we
> don't permit them in mozilla-central, since they were useful to us in
> CVS; removing that capability seems sort of like a regression?)

I'm not clear on your stance here. With adjustments to pushlog and
commit hooks we _can_ do named branches inside of mozilla-central.

I totally agree on your points here, though.

Ben Hearsum

unread,
Aug 8, 2008, 10:35:16 AM8/8/08
to Axel Hecht

I commented on this a bit in my reply to shaver. To summarize, the
amount of commits that happen on a release branch (esp. in the l10n
case) is minimal.

Additionally, doesn't creating a 'release' repository for everything end
up using _double_ the amount of space we have to (admittedly, it does
not increase the size of mozilla-central itself)?

Mike Shaver

unread,
Aug 8, 2008, 10:51:40 AM8/8/08
to Axel Hecht, dev-pl...@lists.mozilla.org
On Fri, Aug 8, 2008 at 10:32 AM, Axel Hecht <l1...@mozilla.com> wrote:
> Ben Hearsum wrote:
>> Based on this knowledge I'd like to suggest that we go with option 2) or
>> 3), with 2) still being my choice.

I agree with option 2 (named branches in m-c and friends).

> IMHO, Sergey's comments on repository size rule out option 2.

I disagree: I don't think that projected repository sizes (and
projected issues with them) should be ruling out workflow now. We can
change what we do when the size _is_ much larger and it _is_ a
problem. If too-big-repo is a problem it'll be a problem we have to
solve for everyone, and I don't think that localizers are going to
have a EREPOTOOBIG any time soon regardless, since their locale repo
will only have changesets for changes to their locale files.

Mike

Sergey Yanovich

unread,
Aug 8, 2008, 11:23:37 AM8/8/08
to Ben Hearsum, Axel Hecht
Ben Hearsum wrote:

> Axel Hecht wrote:
>> IMHO, Sergey's comments on repository size rule out option 2.
>
> Additionally, doesn't creating a 'release' repository for everything end
> up using _double_ the amount of space we have to (admittedly, it does
> not increase the size of mozilla-central itself)?

No. A new repository should be taking much space. Recommended way to
clone a local Mercurial repository is:

$ cp -al old new

The result will take space only for dirs and changed files. It is very
close to the cost of a hg named branch.

--
Sergey Yanovich

Sergey Yanovich

unread,
Aug 8, 2008, 11:38:50 AM8/8/08
to Mike Shaver, Axel Hecht, dev-pl...@lists.mozilla.org
Mike Shaver wrote:
> I disagree: I don't think that projected repository sizes (and
> projected issues with them) should be ruling out workflow now. We can
> change what we do when the size _is_ much larger and it _is_ a
> problem. If too-big-repo is a problem it'll be a problem we have to
> solve for everyone [...]

IIRC, Mozilla 2 workflow assumes frequent releases. So after a period of
time, list of named branches will require less pipe to be seen.

There isn't a clean way to drop a branch from hg, except to merge a
head. There can be numerous solution to the problem of mozilla-central
noise. My point is having things done right the first time saves resources.

In this regard, either 1), 1a) or 3) is fine for me.

--
Sergey Yanovich

Ben Hearsum

unread,
Aug 8, 2008, 11:39:39 AM8/8/08
to Sergey Yanovich, Axel Hecht

OK, I didn't know about this. I have a question though: If we clone
mozilla-central to mozilla-central-releases on hg.m.o with that method
can we pull to and push from platforms that don't support hardlinks (eg,
Windows/MSYS)? I did a quick search and found a mention[1] to them
breaking when pulled from (albeit, it's from December 2007)

Guess I'll try this for myself.

- Ben


[1] http://www.dribin.org/dave/blog/archives/2007/12/30/why_mercurial/

Sergey Yanovich

unread,
Aug 8, 2008, 11:46:03 AM8/8/08
to Ben Hearsum, Axel Hecht
Ben Hearsum wrote:
> OK, I didn't know about this. I have a question though: If we clone
> mozilla-central to mozilla-central-releases on hg.m.o with that method
> can we pull to and push from platforms that don't support hardlinks (eg,
> Windows/MSYS)?

First, Windows supports hardlinks. At least, I was told so discussing
Mercurial on this list in Spring 2008. I was thinking the opposite
before, of course.

Second, this sounds completely like server side choice, which shouldn't
affect clients in any way.

--
Sergey Yanovich

Ben Hearsum

unread,
Aug 8, 2008, 12:31:26 PM8/8/08
to

Out of curiosity, why is 3) fine for you but not 2)? Both share the same
disk usage characteristics, and named branches can be merged back just
like the anonymous ones in 3).

Ben Hearsum

unread,
Aug 8, 2008, 12:37:12 PM8/8/08
to

I just did a local test of this and it looks like you're right -- cp -al
negates most of the disk use downsides. It still uses about 10% more
than options 2) or 3) - but certainly not double.

Sergey Yanovich

unread,
Aug 8, 2008, 1:36:24 PM8/8/08
to
Ben Hearsum wrote:
> Sergey Yanovich wrote:
>> In this regard, either 1), 1a) or 3) is fine for me.
>>
>
> Out of curiosity, why is 3) fine for you but not 2)? Both share the same
> disk usage characteristics, and named branches can be merged back just
> like the anonymous ones in 3).

Hg named branches are like GM weeds. They will stay forever, once in. I
have also noticed that a named branch has a separate manifest (that is,
full history for each file it changes is added to disk space usage).

In addition, 3) is more likely to transform into in-place releases,
which are traditional for source distribution ;). Finally, having tags
for branch points would be a good thing, even for 1) and 1a). So after
discussing in depth, my opinion is 3) is best, 1) and 1a) are fine, and
2) can be lived with, but not too good.

--
Sergey Yanovich

Justin Wood (Callek)

unread,
Aug 8, 2008, 1:56:53 PM8/8/08
to

for what little its worth from me (not being a release eng guy, nor a
person who usually works on stable/release branches) I like this option
the best.

I can clone from the 3.1 stability branch if I want; or I can clone to a
specific revision of a release etc. based on my needs at the time; and
not have an insanely bloated mozilla-central.

--
~Justin Wood (Callek)

Justin Wood (Callek)

unread,
Aug 8, 2008, 2:03:10 PM8/8/08
to

For release driving why not clone static copies of the locales to a
"master" l10n-releases repo, and import changesets into that somehow.
To eliminate the need for multiple l10n release repo's, the l10n teams
themselves can probably get away with simply tagging each release in
their source.

[disclaimer: I'm not a person who actively works on l10n, so if my
answer is not feasible because of any technical/social flaw; sorry]

> > Named branches are a model we understand very well, and there
>> is certainly value to familiarity. (It's sort of too bad that we
>> don't permit them in mozilla-central, since they were useful to us in
>> CVS; removing that capability seems sort of like a regression?)
>
> I'm not clear on your stance here. With adjustments to pushlog and
> commit hooks we _can_ do named branches inside of mozilla-central.
>
> I totally agree on your points here, though.

--
~Justin Wood (Callek)

Zack Weinberg

unread,
Aug 8, 2008, 2:00:35 PM8/8/08
to dev-pl...@lists.mozilla.org
Sergey Yanovich <ynv...@gmail.com> wrote:

> Ben Hearsum wrote:
> >
> > Out of curiosity, why is 3) fine for you but not 2)? Both share the
> > same disk usage characteristics, and named branches can be merged
> > back just like the anonymous ones in 3).
>
> Hg named branches are like GM weeds. They will stay forever, once in.

Of course named branches stay forever. Isn't that exactly what we
want? We might want to do point releases off the 1.9.1 branch, for
instance, months or even years later... Could you explain why you see
this as a showstopper problem?

> I have also noticed that a named branch has a separate manifest (that
> is, full history for each file it changes is added to disk space
> usage).

Disk space isn't that important to me, but even if it were, I
wouldn't think we should be making VCS layout decisions based on
technical limitations of the current version of hg. I can think of
several ways the hg developers could make this problem go away that
would be totally transparent to us.

> In addition, 3) is more likely to transform into in-place releases,
> which are traditional for source distribution ;)

I don't see how (3) could possibly work at all, which means I probably
don't understand what Ben means by it, but even if I did, I think this
statement would be opaque to me. Mind unpacking a bit?

-- My preferred choice would be (2), for what it's worth, followed by
(1a), and then (1) a very distant third, and (3) not even on the map.

zw

Sergey Yanovich

unread,
Aug 8, 2008, 2:59:02 PM8/8/08
to Zack Weinberg
Zack Weinberg wrote:

> Sergey Yanovich <ynv...@gmail.com> wrote:
>> Hg named branches are like GM weeds. They will stay forever, once in.
>
> Of course named branches stay forever. Isn't that exactly what we
> want? We might want to do point releases off the 1.9.1 branch, for
> instance, months or even years later... Could you explain why you see
> this as a showstopper problem?

First, it ain't exactly showstopper.

Second, named branches are useful for normal development workflow. Using
them for releases and keeping them all in m-c will effectively render
them useless. In relatively short time, you'll be using less or grep to
find your branches in the ocean of releases.

Third, it will waste some disk space. I oppose waste in any avoidable
form. Let's be green ;)

>> In addition, 3) is more likely to transform into in-place releases,
>> which are traditional for source distribution ;)
>
> I don't see how (3) could possibly work at all, which means I probably
> don't understand what Ben means by it, but even if I did, I think this
> statement would be opaque to me. Mind unpacking a bit?

Up in the thread, we were discussing that most free open-source projects
do releases in source form. They usually use VCS tags for that. Once the
tag is committed, the release is made. That's not the case with Mozilla
today (tag, build, bake, test, declare release), but I hope it will be
in the future.

--
Sergey Yanovich

Axel Hecht

unread,
Aug 8, 2008, 5:40:29 PM8/8/08
to

I don't think it's feasible to put tagging work onto the localization
teams. The risk in failing that is far greater than our expectations on
their technical skills.

Axel

Zack Weinberg

unread,
Aug 8, 2008, 6:05:41 PM8/8/08
to dev-pl...@lists.mozilla.org
Sergey Yanovich <ynv...@gmail.com> wrote:
> Zack Weinberg wrote:
> > Sergey Yanovich <ynv...@gmail.com> wrote:
> >> Hg named branches [...] will stay forever, once in.

> > Could you explain why you see this as a showstopper problem?
> First, it ain't exactly showstopper.

Ok.

> Second, named branches are useful for normal development workflow.
> Using them for releases and keeping them all in m-c will effectively
> render them useless. In relatively short time, you'll be using less
> or grep to find your branches in the ocean of releases.

So what? I don't see that as a problem _at all_. If it really bugs
you, I'm sure it would be possible to hack some sort of filter into
"hg branches" itself, so you don't even have to remember the "grep"
part.

I think the additional difficulty of backporting a patch from
moz-central to the release repository (or repositories, even worse) if
they aren't the same thing is much more important.

> Third, it will waste some disk space. I oppose waste in any avoidable
> form. Let's be green ;)

Like I said, the right fix for that is inside hg. I won't speculate on
how that could be done as I don't know hg internals at all, but I'm
sure they'd be happy to take patches that improved many-named-branches
disk usage.

> Up in the thread, we were discussing that most free open-source
> projects do releases in source form. They usually use VCS tags for
> that. Once the tag is committed, the release is made. That's not the
> case with Mozilla today (tag, build, bake, test, declare release),
> but I hope it will be in the future.

It might be nice, yes, but I don't see how that's easier with _any_ of
the available options. It's entirely a matter of when you decide to do
the "tag" command...

zw

Mike Shaver

unread,
Aug 9, 2008, 9:50:19 AM8/9/08
to Justin Wood (Callek), dev-pl...@lists.mozilla.org
On Fri, Aug 8, 2008 at 1:56 PM, Justin Wood (Callek) <Cal...@gmail.com> wrote:
> I can clone from the 3.1 stability branch if I want; or I can clone to a
> specific revision of a release etc. based on my needs at the time; and
> not have an insanely bloated mozilla-central.

Have you observed "insane bloat" (not quite sure what that means), or
seen a trend in measurements that will clearly lead to the "insane
bloat" threshold? Is multi-cloning repositories the only available
prophylactic? I fear we have people picking their choices here based
on the fuzzy spectre of an unboundedly-large tree at some
indeterminate future point, when there are many many options available
to us for dealing with that problem when it's actually upon us.

We should pick the workflow we want for today; it's inconceivable to
me that we will actually stick with this workflow forever, given what
novices we are (collectively, and almost all of us individually) with
hg and the distributed development model, so we're very likely to make
changes later as we learn more.

Mike

Justin Wood (Callek)

unread,
Aug 9, 2008, 6:40:54 PM8/9/08
to
Mike Shaver wrote:
> On Fri, Aug 8, 2008 at 1:56 PM, Justin Wood (Callek) <Cal...@gmail.com> wrote:
>> I can clone from the 3.1 stability branch if I want; or I can clone to a
>> specific revision of a release etc. based on my needs at the time; and
>> not have an insanely bloated mozilla-central.
>
> Have you observed "insane bloat" (not quite sure what that means), or
> seen a trend in measurements that will clearly lead to the "insane
> bloat" threshold?

What I mean by bloat is basically repository size/usage. And no I have
not seen any evidence of such, my comments here were based on Sergey
Yanovich's <ynv...@gmail.com> comments from Message-ID:
<RZSdnUTDucyW-QXV...@mozilla.org>

> Is multi-cloning repositories the only available
> prophylactic? I fear we have people picking their choices here based
> on the fuzzy spectre of an unboundedly-large tree at some
> indeterminate future point, when there are many many options available
> to us for dealing with that problem when it's actually upon us.

My main reason for my opinion here is the added pressure on Hg hooks and
tooling required as also mentioned elsewhere in this thread, but if this
option is deemed best and the added overhead on the tooling side is ok,
then "so be it" (I'm by far not the most knowledgeable in this area)

> We should pick the workflow we want for today; it's inconceivable to
> me that we will actually stick with this workflow forever, given what
> novices we are (collectively, and almost all of us individually) with
> hg and the distributed development model, so we're very likely to make
> changes later as we learn more.

I mostly agree; we should pick what we want for today, but we shouldn't
forgo the impact it will have on the future or we risk ending up in some
situation that is hard[er] to fix much sooner.

--
~Justin Wood (Callek)

Ben Hearsum

unread,
Aug 14, 2008, 8:09:04 AM8/14/08
to
It's time to wrap this one up (for real this time).

I've gone over all of the comments again and I still feel strongly about
using named branches inside of our repositories for "release branches".

The biggest argument against it was permanent named branches keeping
full history for every file they touch. After doing some tests of my own
and speaking with a Mercurial developer I have confirmed that this is
not the case, and therefore not an issue.

The biggest pros here are: No need to have clones of 60+ repositories on
hg.m.o, all history for. No need for developers to clone a second
repository when landing on a release branch. And in my opinion, a
simpler overall structure.

If there are *strong* objections and arguments against this, voice them now.

Axel Hecht

unread,
Aug 14, 2008, 8:42:53 AM8/14/08
to

Not an objection, just a question for my clarity, does this require a
change to the single-head hook?

Axel

Ben Hearsum

unread,
Aug 14, 2008, 8:50:33 AM8/14/08
to
Axel Hecht wrote:
> Not an objection, just a question for my clarity, does this require a
> change to the single-head hook?
>

(whoops, meant to mention this in my previous post.)

Oh, yes, it does. I've got a patch for it ready to go.

Justin Wood (Callek)

unread,
Aug 14, 2008, 10:11:23 PM8/14/08
to

Ben, just as a "remember" thing here, can you please apply the hook
changes to comm-central as well, as we would like to adopt the same
release methods as m-c, iirc.

--
~Justin Wood (Callek)

Ben Hearsum

unread,
Aug 15, 2008, 8:03:44 AM8/15/08
to Justin Wood (Callek)

I'll make sure this happens.

Jonas Sicking

unread,
Aug 15, 2008, 2:06:52 PM8/15/08
to

What is the strategy for long lived branches, such as the 1.9.1 branch
(once it happens)?

/ Jonas

Ben Hearsum

unread,
Aug 15, 2008, 2:35:09 PM8/15/08
to
Jonas Sicking wrote:
> What is the strategy for long lived branches, such as the 1.9.1 branch
> (once it happens)?

It hasn't been officially decided but I'm working on the assumption that
we clone mozilla-central to another repository. We could also just land
in a named branch in mozilla-central...

Benjamin Smedberg

unread,
Aug 15, 2008, 2:49:06 PM8/15/08
to

I tend to think we should use a named branch in mozilla-central, since we've
already pretty much decided to use named minibranches. If you only want the
trunk history, you can only clone that part:

hg clone -rdefault https://hg.mozilla.org/mozilla-central

--BDS

Ben Hearsum

unread,
Aug 15, 2008, 2:59:45 PM8/15/08
to Benjamin Smedberg

wfm, we should have this conversation is a separate thread at some
point, though

Robert Kaiser

unread,
Aug 16, 2008, 1:30:07 PM8/16/08
to
Rob Helmer wrote:
> Maybe this is a dumb question, but instead of checking in three files,
> can't the version just be set in the mozconfig, just like branding or
> whatever? Why does the version number need to be checked in?

Because if we hand a snapshot of a source tree to someone, say, a
distributor, it should have the correct version when being built?

Robert Kaiser

timeless

unread,
Aug 20, 2008, 4:32:06 AM8/20/08
to mozilla.dev.planning group
Ben Hearsum wrote:
> I've gone over all of the comments again and I still feel strongly about
> using named branches inside of our repositories for "release branches".

i'm actually in favor of this, as it'll help w/ my pet project,
however i'm going to need some advice about how it should actually
work.

currently mxr checks out each root individually, unless there's some
magical intervention involved. (mozillasvn v. mozillasvn-all is an
example)

if there's a single unified repo, then ideally mxr should be able to
have a single copy of the .hg/ store, and only pull it once, and just
use update in the individual roots.

it'd be nice if someone contacted me / reed w/ enough time for us to
make sure we have some decent design for this before it happens.

I was given a vm to play w/, and I've spent 40gb out of 95gb already.
I'm fairly certain that I'd like to save space in that vm and also for
the real mxr and mxr-stage if we can, and this plan *should* enable
that.

i think i'd kinda like for mxr to be able to detect and automatically
provide additional roots for hg branches at least of mozilla-central,
advice on how such hooks might work will also be appreciated (queue it
via email, it shouldn't be urgent).

Reply all
Reply to author
Forward
0 new messages