branch management in git
Brian Warner
Myk and I spent some time today talking about how to manage branches and
releases within Git, in particular how to map from the patterns we've
used in Hg into the git world. Here are some of our conclusions:
* Hg tended to enforce a certain shape to the revision graph for
releases, since the branch name was an immutable property of each
revision. You usually wound up with a straight-line "trunk" with each
release branch forking off to the side. In git, a "branch" is really
just a mutable pointer to some revision, and you can merge easily from
one place to another, so we could wind up with different shapes
(typically much more tangled than what you'd get in Hg). We actually
like the old shape: when looking at the graph later, it's easy to tell
where the "goal" of the branch diverged from the mainline. So we'd
like to use Git in such a way that we will still see this shape in the
Git revision graph. As a result:
** we should cherrypick (rather than merge) changes from the release
branch into trunk (or vice versa). Merges would confuse/tangle the
two branches, and we don't expect to see a lot of work on the release
branch, so the total number of patches that need migration should be
low.
** we're willing to make somewhat artificial commits in order to provide
a distinct branch shape. In particular, the first commit on the new
branch will do nothing but change the various ".version" markers (in
the README, in a couple of docs files), and will contain no code
changes. (if we did not make these otherwise-useless commits, it
could be hard to identify where the release process changed when we
look at the revision graph later: a very very fast release cycle
could result in a single revision on the main line having both 0.9rc1
and 0.9 (final) tags, and then having trunk revisions as descendants).
* branches should be short-lived. Each branch corresponds to a specific
task, like "let's make an 0.10 release of the SDK". When the task is
complete, the deliverable is a tag (probably named "0.10"), and the
branch is deleted. (Note that keeping the branch reference around
doesn't add any value, since git branches don't record any more
history than the resulting tag does). All branch references on the
http://github.com/mozilla/addon-sdk repository should correspond to
active projects, for which the deliverable will either be to produce a
release tag or to merge some functionality into some other branch
(like the permanent "master" branch).
** If we've tagged e.g. 1.0, and proceeded to work on 2.x, and then
later discover we need to produce a 1.0.1: we'll create a new
"branch-1.0.1" starting from the 1.0 release tag, work on it until we
create the 1.0.1 release tag, then delete the branch. We defer the
creation of release branches until we decide to start working on that
task.
* branches and tags should have distinctive names, since git accepts
either when running commands like "git diff" or "git log". When
developers pass around diffs, we'd like to quickly tell whether their
diff involves a specific revision (like the "0.9" release tag) or the
"latest" version of some branch (like the "branch-0.9" branch leading
up to it). When a diff or bug report involves a branch, the revision
is ambiguous, and you need to know exactly *when* the command was run
to resolve the ambiguity. (I think of tags and revisions as points,
and branches as lines or rather geometric vectors: it might be the
case that branch X and tag Y currently point to the same thing, but
tomorrow that branch may point to something else).
** the straw-man proposal is to use e.g. "branch-1.0" to name the branch
on which 1.0 work is being done. When the 1.0 project is complete, a
tag named "1.0" will be created, and branch-1.0 will be deleted. This
will result in small (and easy-to-type) tag names for specific
releases, and distinctive branch names.
** tags should be treated as immutable. (even though Git lets you delete
and move tags around, we should refrain from using that feature). The
"1.0" tag will not be created until 1.0 is released, and then it will
never be assigned to any other revision.
** we won't bother with "signed tags" for now, however release
announcements will include the SHA1 revisionid of the final release,
which is sufficient to reliably and securely reconstruct the correct
source tree
* the tree will contain a ".version" file that will be used to
introspect the version (used in documentation and a hypothetical 'cfx
--version' command, etc). In practice we have something like 8 places
where the version is recorded, mostly in docs, and we don't yet have a
".version" file, but we'll probably add one soon. For the purposes of
this note, ".version" is a stand-in for all those 8 places.
** in the final release tarball, .version will obviously contain the
final release version, e.g. "0.9" or "1.0beta1" or "1.0"
** on all trunk revisions, .version will contain the anticipated next
release version plus a "pre" suffix. For example, after 0.8 has been
released, .version will contain "0.9pre".
** on the release branch, .version will contain the upcoming final
release version, e.g. "0.9". It will never contain an rcN suffix. The
idea is to make the code on the release branch ready for imminent
release at all times.
* the branch/release process will look like this (e.g. after an 0.8
release):
** between releases, all development occurs on the trunk ("master") as
usual. Large projects are spun off onto topic branches as appropriate
and merged when they are ready. Trunk should pass unit tests and be
usable at all times. .version will contain the anticipated next
release number plus a "pre" suffix, e.g. "0.9pre".
** The contents of .version are an indication of which features should
be allowed in: "0.9pre" means that features which are destined for
0.10 should not be landed yet.
** when a freeze is declared, all developers switch into "get ready to
release" mode: all attention is focused on resolving blockers, etc.
No special checkins are made. Commit policy changes and approvals may
be required.
** when the frozen trunk is ready, the following steps occur:
- a new branch reference is created named "branch-0.9", based upon the
latest trunk revision
- a special commit is made on branch-0.9 which updates .version to
"0.9". The code on branch-0.9 is now ready to be released: if no
other problems were found, we could ship the latest revision of
branch-0.9 as the 0.9 release.
- a special commit is made on master which updates .version to
"0.10pre". We choose a conservative next-version here: even if we've
scheduled 1.0b1 or 1.0 (final) to follow 0.9, we'll pretend that
we're merely expecting 0.10. We want the .version values on trunk to
be monotonically increasing, and this would be violated if we
prematurely jumped to 1.0b1pre and then (for whatever reason) were
forced to insert an 0.10 or 0.11 release before 1.0b1. This causes
the trunk to be declared "unfrozen", and developers are allowed to
resume general checkins on trunk (and finally get to start landing
features scheduled for after the 0.9 release).
** at any time, the revision pointed to by the branch-0.9 reference
(i.e. the "branch-0.9 tip") may be given a "0.9rcN" tag (for some
monotonically-increasing value of N). We do this when the code on
this branch is solid enough to use as a testing candidate. Any given
release branch may wind up with zero or more rc tags.
** commits to the release branch are strictly either release-related
(i.e. changing .version) or code-related. All code-related changes
are also copied to trunk (and/or copied from trunk into the release
branch, depending upon where the work first landed) by using
git cherry-pick COMMITID
This causes the release branch and trunk to contain parallel edges of
the revision graph, rather than visually entangling them with merges.
It also avoids the danger of accidentally merging release-related
changes into trunk.
** at any time, the branch may be released. When this happens, the
current branch-0.9 tip is tagged as "0.9", and the branch-0.9
reference is deleted:
git tag 0.9 branch-0.9
git branch -d branch-0.9
Then a tarball is created, with something like:
git archive --output addon-sdk-0.9.zip --prefix addon-sdk-0.9/ 0.9
and the other steps in https://wiki.mozilla.org/Labs/Jetpack/Release are
followed, culminating in the high-five and/or fist jab checklist
Make sense?
Also, some crazy ideas for the future:
- publish the frozen/unfrozen state of trunk on a server somewhere (or
better yet inside the git repo) and add a post-commit hook which uses
that info to warn developers if they commit something to trunk
without the appropriate a= approval. Bonus points for knowing the
current list of approved blocker bugs and not warning on commits for
those.
- replace an explicit .version file with runtime code to compute it
from 'git describe HEAD' and a build-tarball step which inserts a
static .version file into .git-less released tarballs. This would fix
the uncomfortable situation that e.g. 0.10rc1 produces XPIs that
claim to have been produced by 0.10 (final), and 0.10rc1's "cfx
version" will misrepresent itself. Unfortunately it would require
code to be run during the release process (you could no longer just
ask github for a tarball and get one with the right .version file)
cheers,
-Brian
James Burke
> Myk and I spent some time today talking about how to manage branches and
> releases within Git, in particular how to map from the patterns we've
> used in Hg into the git world. Here are some of our conclusions:
we recently discussed a git workflow. We ended up going with the Git
Flow[1] model, and it helps that there are some tooling to help with
some of the mundane branch work[2].
We just recently started to use it, so we have not had a lot of time
with it, but so far I like it. Using a feature branch that is easy to
create and merge/discard has been useful -- we have been able to work
on about 4 features in separate branches without having to break
things with people working off of "develop", the more stable-but-
cutting-edge line. It helps though that the 4 features are somewhat
unrelated though. I can see if we go too long on feature branches
merging will be more trouble, so we have a vested interest in keeping
the feature branches small or do incremental merges.
Some miscellaneous things:
* Knowing what branch you are on is *really* helpful. I updated my
shell to always print out the branch in the command line prompt[3].
* It would be nice to have some automated step that stamps the version
for a release branch. I wonder if that can be handled either as some
sort of plugin to the git flow tools or maybe has a git hook (not sure
if that is possible).
* I would expect to see people that do forks and ask for pull requests
do them against "develop".
I'm mentioning all this just to provide a data point for git
workflows. In many ways we are a much smaller project than the addon-
sdk, so our needs are probably not the same. Hopefully sharing our
real world experiences will lead to some best practices we eventually
could leverage across teams.
James
[1] http://nvie.com/posts/a-successful-git-branching-model/
[2] http://jeffkreeftmeijer.com/2010/why-arent-you-using-git-flow/
[3] http://www.jonmaddox.com/2008/03/13/show-your-git-branch-name-in-your-prompt/
After doing the changes in [3], start a new terminal tab and see
the glorious colors! cd to a git directory and see the branch.
I looked at this blog post to find the END value, because I wanted the
end of my prompt to be in black instead of GREEN:
http://www.erat.org/shell.html
I defined a local END="\[\033[0m\]" then replaced the last GREEN with
END.
Brian Warner
>
> I work on Mozilla F1, a project that is also using git and GitHub, and
> we recently discussed a git workflow. We ended up going with the Git
> Flow[1] model, and it helps that there are some tooling to help with
> some of the mundane branch work[2].
Yeah, we've looked at that a bit (FlightDeck is using it, and I like how
Piotr renamed the branch names in [1]). From my perspective, it's a bit
more appropriate for a server-based deployment than a client-based SDK.
In particular, being able to make a hotfix and "release" it quickly is
more useful on a live site than on an SDK with a 4- or 6- week release
cycle.
[1]: https://wiki.mozilla.org/Labs/Jetpack/FlightDeck/Code_Workflow
Server-side software is a funny thing, there's a "release", and then
there's "deployment", and in a lot of cases there's only ever one
instance of the server (e.g. you could, theoretically, run your own
FlightDeck server, independent of AMO, but there's not much value to it,
so nobody does). So the "release" step is an extra hurdle that nobody
really pays attention to, while the "deploy" step is the one that
everyone really cares about.
The presence of a branch named "production" in that model is a dead
giveaway: that's a great way to let everyone know what's currently
supposed to be running on the live servers. (I think FlightDeck is also
using a branch named "staging" for a similar purpose). In the ideal
case, your Deployment Manager (like a release manager, but they're the
person who decides when to make changes to the production servers)
expresses their decisions by committing a change to the "staging" or
"production" branches, and Post-commit hooks take care of the rest.
I'm still exploring, though. I use feature/topic branches all the time,
and having a release branch to separate freeze-time policy is a good
plan. If the codebase includes embedded version markers, then merging
from the release branch back into trunk is problematic, as well as
making it a bit harder to visually spot the release branch when looking
at the full revision graph.
> Some miscellaneous things:
>
> * Knowing what branch you are on is *really* helpful. I updated my
> shell to always print out the branch in the command line prompt[3].
That's cool! I obsessively update my "gitx" view, especially just before
I do a merge or push changes upstream, to make sure that I'm changing
the right thing. If you're on OS-X, I highly recommend gitx:
http://GitX.frim.nl/ . Install the CLI tools (which adds /usr/bin/gitx)
and then run 'gitx' from inside your tree to pop up a viewer window.
It's the same basic information as 'git show-branch' or gitk, but
presented very clearly.
> * It would be nice to have some automated step that stamps the version
> for a release branch. I wonder if that can be handled either as some
> sort of plugin to the git flow tools or maybe has a git hook (not sure
> if that is possible).
Which stamp would be updated? Git has lots of hooks; the post-commit
hook is probably the most useful, but it could be easier to just include
some scripts in a misc/ directory and have the release manager run them
as part of the start-a-new-release-branch process.
thanks!
-Brian
Atul Varma
getting rid of the 8-different-places schism that we currently have.
I've actually been liking the merge-instead-of-rebase strategy that I've
been using lately for commits to the trunk. It does make looking at
history a bit more tangled, but I like that the tangle actually
represents real development decisions, including mistakes that were made
and so forth. Github's Network Graph view also looks really rich and
informative this way.
It also seems eminently possible to make a simplified view of a
project's development by e.g. only looking at merges into master and
ignoring all the individual commits on each branch.
It sounds like the branching you're talking about here is more for
individual versions of the SDK rather than just the master/trunk, and
cherry-picking to enforce a particular shape for those does seem to make
sense. Let me know if I am totally misinterpreting things, though!
- Atul
Irakli Gozalishvili
Myk and I spent some time today talking about how to manage branches and
releases within Git, in particular how to map from the patterns we've
used in Hg into the git world. Here are some of our conclusions:
* Hg tended to enforce a certain shape to the revision graph for
releases, since the branch name was an immutable property of each
revision. You usually wound up with a straight-line "trunk" with each
release branch forking off to the side. In git, a "branch" is really
just a mutable pointer to some revision, and you can merge easily from
one place to another, so we could wind up with different shapes
(typically much more tangled than what you'd get in Hg). We actually
like the old shape: when looking at the graph later, it's easy to tell
where the "goal" of the branch diverged from the mainline. So we'd
like to use Git in such a way that we will still see this shape in the
Git revision graph. As a result:
** we should cherrypick (rather than merge) changes from the release
branch into trunk (or vice versa). Merges would confuse/tangle the
two branches, and we don't expect to see a lot of work on the release
branch, so the total number of patches that need migration should be
low.
I prefer merge vs cherry-pick-ing since it's less error prone, it also gives provide more info by giving distinguishing person introducing changes and person making performing merge. Also I'd like to quote Linus (Author of git & not only :) "Rebasing doesn't result in cleaner history. It just results in **incorrect** history that looks simpler." - http://bit.ly/fMdylH
--
You received this message because you are subscribed to the Google Groups "mozilla-labs-jetpack" group.
To post to this group, send email to mozilla-la...@googlegroups.com.
To unsubscribe from this group, send email to mozilla-labs-jet...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mozilla-labs-jetpack?hl=en.
Atul Varma
I prefer merge vs cherry-pick-ing since it's less error prone, it also gives provide more info by giving distinguishing person introducing changes and person making performing merge. Also I'd like to quote Linus (Author of git & not only :) "Rebasing doesn't result in cleaner history. It just results in **incorrect** history that looks simpler." - http://bit.ly/fMdylH
- Atul
Irakli Gozalishvili
Will this work for you ?
git log --merges --graph --oneline --color
- Atul
Atul Varma
git log --merges --graph --oneline --color
Now if there was just an easy way to see that kind of thing through Github, or that newbies could know how to do without getting totally confused by our repo's commit history. I guess the network graph on Github is pretty good at making sense of the history...
- Atul