version-string management: abolish .version, use git-describe

144 views
Skip to first unread message

Brian Warner

unread,
Nov 22, 2011, 3:36:39 PM11/22/11
to mozilla-la...@googlegroups.com
Greetings Rocketeers!

I've been working on some changes to the way we handle version strings
inside the SDK. In particular, I've been looking for a way to remove the
manual "update .version" step from the release process:

https://wiki.mozilla.org/Jetpack/Release_Process#Spin_Test_Builds

and to make sure that the only time we ever type in a version string is
when the release manager does "git tag". I want this to work both in
local git checkouts, and in the tarballs/zipfiles generated by "git
archive" (which is both part of the Release Process and the command run
by github when you push the "give me a tarball" button).

To support this, I've got a (separately-maintained) generic project
named "python-versioneer", useful for any distutils-based Python
package, hosted at:

https://github.com/warner/python-versioneer

and a set of patches for jetpack itself in:

https://github.com/warner/addon-sdk/tree/704519-versioning

These patches start by adding python-lib/cuddlefish/_version.py , with a
get_versions() function that's called each time 'cfx' does anything
(including a new 'cfx version' command that just prints the version).
_version.py has two ways to figure out a version:

* _version.py includes some magic strings like $Format:%d$ and
$Format:%H$ . These are expanded by git-archive when it makes a
tarball, because of some "export-subst" settings in .gitattributes.
If _version.py notices that these strings were expanded, then it
knows we're in an unpacked tarball, and gets the version data from
there.
* if they aren't expanded, then we're probably in a git checkout, so
_version.py runs "git describe --tags --always --dirty", which uses
local .git metadata to compute a version string.

get_versions() returns both the full hex revisionid and a short version
string, which is the tag (if available), otherwise it looks like this:

0.9-1669-g93c7e14-dirty

which means that 0.9 is the most recent tagged ancestor it found, 1669
patches have been applied since then, the abbreviated revisionid is
93c7e14, and the tree is "dirty" (it contains uncommitted changes).

This works best when we merge the stabilization and release branches
back into master on a regular basis: if we'd merged 1.2 back into master
after it was released in October, this would look more like
"1.2.1-400-g646b94b". If we merged stabilization back into master after
each tag, it would look like "1.3rc1-328-g1c3e54c". That keeps the patch
count from growing too large, and gives a better lower bound on the age
of the tree.

So here's the question for everybody: are we ok with version strings
like that? What version string would you expect a trunk tree to report,
and when would you expect it to change from saying 1.2.something to
1.3.something?

Personally, I'd usually want 'cfx version' on trunk to give me some idea
about which era this tree is from: something like "somewhere after we
forked off the 1.3 release branch, but before we forked off 1.4". The
abbreviated revid is enough to reconstruct any tree which came from the
official repo, but it's usually nice to know more than that. Mostly you
want to answer questions like "is feature/bugfix XYZ in this tree or
not", and knowing that the tree contains at least everything that was in
1.2.1 is often a good proxy for questions like that.

There are a few quirks with this code to keep in mind:

* in git checkouts, it will run "git describe" each time you run "cfx".
This is generally really fast (the revisionid part is O(1), but
checking timestamps for the --dirty flag depends upon an effective
filesystem cache). If "git" is not on your path (windows comes to
mind), it should fail gracefully with a version of "unknown".
* in unpacked tarballs, git only expands the abbreviated ref names, so
the code gets a list like "(master, HEAD, 1.3)", and has to guess
which one might be a tag (and not a branch name). The heuristics only
accept things with a digit. So if you ask github for a tarball from a
branch named e.g. 685378-repacker, _version.py may mistakenly think
the short version of the resulting tree is "685378-repacker".

My versioning branch also has some patches to remove "version"
properties from all package.json files (so they don't get out-of-date),
as well as removing the top-level .version file. It also adds some
changes to the docs generation code, to expand any $SDKVERSION strings
in the .md files to the current version: this seemed useful for marking
the generated docs with the SDK version they describe. The last part is
just a strawman: I'd like Will to decide where such a version string
should go, if at all.


Let me know what you think!

cheers,
-Brian

Brian Warner

unread,
Nov 22, 2011, 3:49:40 PM11/22/11
to mozilla-la...@googlegroups.com
On 11/22/11 12:36 PM, Brian Warner wrote:

> This works best when we merge the stabilization and release branches
> back into master on a regular basis:

One reason we *haven't* been merging release branches back to trunk is
the minVersion/maxVersion settings inside the SDK. Each release knows
what range of Firefox versions it's compatible with, and we don't want
trunk's behavior to change abruptly when we merge the release or
stabilization branch back in.

Should we consider putting a table into the SDK, and having it check its
own sdkVersion to decide what minVersion/maxVersion should be written
into generated addons? At the moment, current trunk is declared to
support 8.0 through 11.0a1, and the stabilization branch declares
support for 8.0 through 9.* . Everything on the stabilization branch
reports itself as 1.3.something. Given the versions that will be
reported by our various branches, could we come up with a rule like "if
your sdkVersion is 1.3.something, support (8.0-9.*), but if it's
1.4.something, support (8.0-11.0a1)"?

uncertainly,
-Brian

Brian Warner

unread,
Nov 22, 2011, 5:04:27 PM11/22/11
to mozilla-la...@googlegroups.com
On 11/22/11 12:49 PM, Brian Warner wrote:
> On 11/22/11 12:36 PM, Brian Warner wrote:
>
>> This works best when we merge the stabilization and release branches
>> back into master on a regular basis:
>
> One reason we *haven't* been merging release branches back to trunk is
> the minVersion/maxVersion settings inside the SDK. Each release knows
> what range of Firefox versions it's compatible with, and we don't want
> trunk's behavior to change abruptly when we merge the release or
> stabilization branch back in.

Oh, one trick to mention: we could just *pretend* to merge the release
branches back into the trunk, but use:

git merge --strategy=ours stabilization

and not actually change any of the code on the master side. If we were
pretty confident that all of the stabilization changes were just
cherry-picked from master anyways (except for the min/maxVersion
change), this should be safe. And then the master branch would get
sensible version numbers.

cheers,
-Brian


Will Bamberg

unread,
Nov 23, 2011, 12:05:37 PM11/23/11
to mozilla-la...@googlegroups.com
On this:


It also adds some
changes to the docs generation code, to expand any $SDKVERSION strings
in the .md files to the current version: this seemed useful for marking
the generated docs with the SDK version they describe. The last part is
just a strawman: I'd like Will to decide where such a version string
should go, if at all.


I think we should add the version string to the header or footer, so it's always clear which version of the SDK you're looking at, wherever you are. I'm kind of ambivalent about whether we should _also_ document the current version in the .md files, as we currently do in "Installation", although if we do choose to, your changes would definitely be useful.

Will Bamberg

unread,
Nov 23, 2011, 12:09:45 PM11/23/11
to mozilla-la...@googlegroups.com
We have to document the set of supported Firefox versions in the SDK
(currently the SDK incorrectly claims it's compatible with Firefox 4.0
onwards), and if we want to get away from having to update this sort of
information manually when we make a release, then it sounds as if we do
need something like this.

Will

Brian Warner

unread,
Nov 29, 2011, 5:46:52 PM11/29/11
to mozilla-la...@googlegroups.com

> This works best when we merge the stabilization and release branches
> back into master on a regular basis

Dave Townsend just suggested an even simpler idea: just after we start
the release process for e.g. 1.4 (and merge from master into the
'stabilization' branch), we add a tag to master named "1.5-dev". This
indicates that all work that lands on master after this tag is intended
to show up in the 1.5 release. We wouldn't merge from stabilization or
release back into master. Apparently, this is how mozilla-central
handles the "Aurora" release process.

The result is that anyone using the SDK from master will get version
strings like "1.5-dev-5-gabc123", meaning "revision abc123, which is 5
patches since 1.4 was branched, and anything here will show up in 1.5".

The only downside is a slight increase in the number of tags we wind up
adding. At the moment we're using about 6 tags per release (three betas,
a few release candidates, and one final), and this would make that 7.
Not a big deal.. tags are cheap, the only place where having lots of
them is a drag is when you use 'git tag' to list all of them, and then
have to pick out the ones that you care about (and that's what "grep" is
for).

cheers,
-Brian

Myk Melez

unread,
Nov 30, 2011, 2:50:59 AM11/30/11
to mozilla-la...@googlegroups.com, Brian Warner
On 2011-11-29 14:46, Brian Warner wrote:
>> This works best when we merge the stabilization and release branches
>> back into master on a regular basis
> Dave Townsend just suggested an even simpler idea: just after we start
> the release process for e.g. 1.4 (and merge from master into the
> 'stabilization' branch), we add a tag to master named "1.5-dev".
I like this idea and have added the 1.5-dev tag to master:

https://github.com/mozilla/addon-sdk/tree/1.5-dev

Thus we can try out this idea and iterate as needed.

-myk

Reply all
Reply to author
Forward
0 new messages