version-string management: abolish .version, use git-describe
Brian Warner
I've been working on some changes to the way we handle version strings
inside the SDK. In particular, I've been looking for a way to remove the
manual "update .version" step from the release process:
https://wiki.mozilla.org/Jetpack/Release_Process#Spin_Test_Builds
and to make sure that the only time we ever type in a version string is
when the release manager does "git tag". I want this to work both in
local git checkouts, and in the tarballs/zipfiles generated by "git
archive" (which is both part of the Release Process and the command run
by github when you push the "give me a tarball" button).
To support this, I've got a (separately-maintained) generic project
named "python-versioneer", useful for any distutils-based Python
package, hosted at:
https://github.com/warner/python-versioneer
and a set of patches for jetpack itself in:
https://github.com/warner/addon-sdk/tree/704519-versioning
These patches start by adding python-lib/cuddlefish/_version.py , with a
get_versions() function that's called each time 'cfx' does anything
(including a new 'cfx version' command that just prints the version).
_version.py has two ways to figure out a version:
* _version.py includes some magic strings like $Format:%d$ and
$Format:%H$ . These are expanded by git-archive when it makes a
tarball, because of some "export-subst" settings in .gitattributes.
If _version.py notices that these strings were expanded, then it
knows we're in an unpacked tarball, and gets the version data from
there.
* if they aren't expanded, then we're probably in a git checkout, so
_version.py runs "git describe --tags --always --dirty", which uses
local .git metadata to compute a version string.
get_versions() returns both the full hex revisionid and a short version
string, which is the tag (if available), otherwise it looks like this:
0.9-1669-g93c7e14-dirty
which means that 0.9 is the most recent tagged ancestor it found, 1669
patches have been applied since then, the abbreviated revisionid is
93c7e14, and the tree is "dirty" (it contains uncommitted changes).
This works best when we merge the stabilization and release branches
back into master on a regular basis: if we'd merged 1.2 back into master
after it was released in October, this would look more like
"1.2.1-400-g646b94b". If we merged stabilization back into master after
each tag, it would look like "1.3rc1-328-g1c3e54c". That keeps the patch
count from growing too large, and gives a better lower bound on the age
of the tree.
So here's the question for everybody: are we ok with version strings
like that? What version string would you expect a trunk tree to report,
and when would you expect it to change from saying 1.2.something to
1.3.something?
Personally, I'd usually want 'cfx version' on trunk to give me some idea
about which era this tree is from: something like "somewhere after we
forked off the 1.3 release branch, but before we forked off 1.4". The
abbreviated revid is enough to reconstruct any tree which came from the
official repo, but it's usually nice to know more than that. Mostly you
want to answer questions like "is feature/bugfix XYZ in this tree or
not", and knowing that the tree contains at least everything that was in
1.2.1 is often a good proxy for questions like that.
There are a few quirks with this code to keep in mind:
* in git checkouts, it will run "git describe" each time you run "cfx".
This is generally really fast (the revisionid part is O(1), but
checking timestamps for the --dirty flag depends upon an effective
filesystem cache). If "git" is not on your path (windows comes to
mind), it should fail gracefully with a version of "unknown".
* in unpacked tarballs, git only expands the abbreviated ref names, so
the code gets a list like "(master, HEAD, 1.3)", and has to guess
which one might be a tag (and not a branch name). The heuristics only
accept things with a digit. So if you ask github for a tarball from a
branch named e.g. 685378-repacker, _version.py may mistakenly think
the short version of the resulting tree is "685378-repacker".
My versioning branch also has some patches to remove "version"
properties from all package.json files (so they don't get out-of-date),
as well as removing the top-level .version file. It also adds some
changes to the docs generation code, to expand any $SDKVERSION strings
in the .md files to the current version: this seemed useful for marking
the generated docs with the SDK version they describe. The last part is
just a strawman: I'd like Will to decide where such a version string
should go, if at all.
Let me know what you think!
cheers,
-Brian
Brian Warner
> This works best when we merge the stabilization and release branches
> back into master on a regular basis:
One reason we *haven't* been merging release branches back to trunk is
the minVersion/maxVersion settings inside the SDK. Each release knows
what range of Firefox versions it's compatible with, and we don't want
trunk's behavior to change abruptly when we merge the release or
stabilization branch back in.
Should we consider putting a table into the SDK, and having it check its
own sdkVersion to decide what minVersion/maxVersion should be written
into generated addons? At the moment, current trunk is declared to
support 8.0 through 11.0a1, and the stabilization branch declares
support for 8.0 through 9.* . Everything on the stabilization branch
reports itself as 1.3.something. Given the versions that will be
reported by our various branches, could we come up with a rule like "if
your sdkVersion is 1.3.something, support (8.0-9.*), but if it's
1.4.something, support (8.0-11.0a1)"?
uncertainly,
-Brian
Brian Warner
> On 11/22/11 12:36 PM, Brian Warner wrote:
>
>> This works best when we merge the stabilization and release branches
>> back into master on a regular basis:
>
> One reason we *haven't* been merging release branches back to trunk is
> the minVersion/maxVersion settings inside the SDK. Each release knows
> what range of Firefox versions it's compatible with, and we don't want
> trunk's behavior to change abruptly when we merge the release or
> stabilization branch back in.
Oh, one trick to mention: we could just *pretend* to merge the release
branches back into the trunk, but use:
git merge --strategy=ours stabilization
and not actually change any of the code on the master side. If we were
pretty confident that all of the stabilization changes were just
cherry-picked from master anyways (except for the min/maxVersion
change), this should be safe. And then the master branch would get
sensible version numbers.
cheers,
-Brian
Will Bamberg
It also adds some changes to the docs generation code, to expand any $SDKVERSION strings in the .md files to the current version: this seemed useful for marking the generated docs with the SDK version they describe. The last part is just a strawman: I'd like Will to decide where such a version string should go, if at all.
Will Bamberg
(currently the SDK incorrectly claims it's compatible with Firefox 4.0
onwards), and if we want to get away from having to update this sort of
information manually when we make a release, then it sounds as if we do
need something like this.
Will
Brian Warner
> This works best when we merge the stabilization and release branches
> back into master on a regular basis
Dave Townsend just suggested an even simpler idea: just after we start
the release process for e.g. 1.4 (and merge from master into the
'stabilization' branch), we add a tag to master named "1.5-dev". This
indicates that all work that lands on master after this tag is intended
to show up in the 1.5 release. We wouldn't merge from stabilization or
release back into master. Apparently, this is how mozilla-central
handles the "Aurora" release process.
The result is that anyone using the SDK from master will get version
strings like "1.5-dev-5-gabc123", meaning "revision abc123, which is 5
patches since 1.4 was branched, and anything here will show up in 1.5".
The only downside is a slight increase in the number of tags we wind up
adding. At the moment we're using about 6 tags per release (three betas,
a few release candidates, and one final), and this would make that 7.
Not a big deal.. tags are cheap, the only place where having lots of
them is a drag is when you use 'git tag' to list all of them, and then
have to pick out the ones that you care about (and that's what "grep" is
for).
cheers,
-Brian
Myk Melez
>> This works best when we merge the stabilization and release branches
>> back into master on a regular basis
> Dave Townsend just suggested an even simpler idea: just after we start
> the release process for e.g. 1.4 (and merge from master into the
> 'stabilization' branch), we add a tag to master named "1.5-dev".
https://github.com/mozilla/addon-sdk/tree/1.5-dev
Thus we can try out this idea and iterate as needed.
-myk