Ok, first of all is merging perlbal-doc into perlbal desirable? Although
it's good to ensure the documentation is available, the perlbal-doc
package is slightly bigger than perlbal itself, so it's actually
doubling the package size.
If they're going to be merged, then 'perlbal' should obsolete
'perlbal-doc' so they don't conflict upon install. Installation
of the newer 'perlbal' should silently remove 'perlbal-doc'.
About the version numbers, I agree that the debian/changelog file
should at least say version 1.70. I don't have any tags in my git
repo; don't know why (I might have cloned it after converting it).
Beyond that, ("+svn784-1") I don't know. It would be nicest, I think,
for the version number in the package to be derived automatically
from the released version number (0.70) plus something about the
repository state (in my case, a git commit ID). When I build a
linux kernel from git, the kernel name looks like this:
vmlinuz-2.6.25-rc8-printk-00250-g489cd5b
In http://www.jukie.net/~bart/blog/20061020145437 there is an example
to get a version number out of git, with the upstream revision number
(1.70), local build number (1 .. n) and git commit ID (gXXXXXXXX).
BRANCH=`git-describe | awk -F'-g[0-9a-fA-F]+' '{print $1}'`
COMMIT=`git-rev-parse HEAD | awk '{print substr($1,0,8)}'`
INCREMENT=`git-rev-list $BRANCH..HEAD | wc -l | awk '{print $1}'`
TGZ="openswan-$BRANCH-($INCREMENT)-g$COMMIT.tgz"
I expect something similar could be done for subversion.
But that article doesn't explain how to set the debian package
name (since it's talking about making a tarfile).
Actually "git describe" does basically all of the above without
any awk commands required.
At http://honk.sigxcpu.org/projects/git-buildpackage/manual-html/gbp.releases.html
I found a description of git-buildpackage ... looks like it updates
the debian/changelog file automatically.
git-buildpackage looks pretty cool but what about Subversion users?
$ svn info | grep '^Revision:' | awk ' { print $2 } '
784
So ... given that the build tools (at least the git one) do their thing
by editing the changelog file, my opinion is that the changelog should
only contain the main version number (1.70) not the svn revision -
otherwise the changelog file will need a manual edit every time a new
revision is checked into svn (or if not edited it will become out of date
and useless).
Nick.
Ok, I'll take Andrew's patch, make the version number just 1.70 (the -1
is only needed by debian when they change a package downstream)
and build/test.
Nick.
Well, it represents the first debian package built by a debian
maintainer. If that person subsequently makes a debian specific
change and makes a new package, they will update it to -2. In
the interests of stability the debian people will take a base
revision and backport security fixes to it for a while, and they
need a revision number they can increment without affecting the
upstream version number.
> Ah. Here. From debian policy manual, regarding the upstream version:
> "It is optional; if it isn't present then the upstream_version may not
> contain a hyphen. This format represents the case where a piece of
> software was written specifically to be turned into a Debian package,
> and so there is only one "debianisation" of it and therefore no
> revision indication is required."
I read the same thing, and interpreted it to mean "the -1 is added by
a debian maintainer if/when that person makes debian specific
(downstream) changes". I'm not a debian maintainer; I'm upstream -
and so somebody else will be doing the submitting and it's up to
that person to add suffixes to the package version. I would hope they
don't need to make any downstream changes - but if they do, that
suffix is for their use.
If they don't need to make any changes at all, the suffix isn't
required.
Now regarding the +svn784 in the version number, I wouldn't want to
see that in the debian package revision; a debian downstream person
should remove it (which probably necessitates them adding the -1
suffix). What I'd like to see is, for somebody who builds their
own package, that the build process should automatically plug in
the revision number. If they build from an svn checkout it should
say svn, if they build from a git checkout it should say git. I
already found what technique to use for git and I'm sure I can
figure out the corresponding technique for svn with a few minutes
of playing around.
If +svn784 is in the version number somebody will have to keep it
up to date from time to time. If we did 100 commits and no version
number increase how many times should the +svn be changed? I think
if it's added automatically by a build tool, no need to commit it
into svn.
How's this software being released anyway? CPAN only, no tarballs?
Every time Brad does a new release to CPAN he has to increment the
version number ... 1.70, 1.71, 1.72. The svn revision will be
meaningless to somebody who installs from CPAN.
Conversely, people can use the svn repo (or clone my git one) to
build it themselves, so any build tool will have the svn or git
revision details there to add automatically.
Finally there's the debian packaging. When a debian maintainer
person takes a snapshot of the source code to package up (with
any debian downstream changes) and put into the repository, if
they take the exact tagged revision then the debian revision
number can be just "1.70", if they take some intermediate
commit not yet released by Brad, they should use "1.70+svnXXX"
and those are the only two choices I see. Basically it's up
to the debian person to put the specific revision they took
into the changelog file; we don't need to do so.
Nick.
Maintainers without upload permission ...
> Most build tools (and all automated ones used to submit packages) use
> the version number from the changelog, not one that's generated from a
> revision.
The way it works seems to be that the changelog is generated
from the SCM log and then the build tools use whatever is in
the changelog.
> Removing the suffix means that the package the is built is
> debian-native, which neither perlbal nor mogilefs is (see "What's the
> difference between a native Debian package and a non-native package?"
> in http://people.debian.org/~mpalmer/debian-mentors_FAQ.html)
I'll need to read that document in more detail. I don't necessarily
agree with its assertion that "You should only use a native Debian
package when it is clear that the package would only ever be of use in
Debian." ... what we've got with mogilefs and perlbal looks like a
debian-native package because the debian directory is in the repo
itself and there's documented support for building debian packages,
but there's also support for building RPMs. It can't be both debian
native and RedHat native so perhaps there's a problem with the concept
of debian native.
> Also, if we always keep the version at whatever the current CPAN is--
> say perlbal-1.70-- what do we put in the changelog when we check in a
> change into the package (like, say, merging the perlbal-doc and
> perlbal packages)? If we keep the version the same, then we break
> debian policy.
I was thinking about that earlier. One possibility is to put the _next_
version number into it (1.71) and add to that changelog entry until
1.71 is released.
> Here's an example (from mogilefs):
> Say a user builds the mogilefs debian packages using the build-all-
> debian-sh script. dpkg-buildpackage generates a package _base on
> debian/changelog_. The user installs it. Later we, realize we forgot
> something in the debian package (maybe a dependency!). The user checks
> out the changes from svn, builds a new deb file, and tries to install
> -- but won't be able to since the version of the package hasn't
> changed. That's the whole reason for adding entries to the changelog.
That's where you'd want support from git-buildpackage (I believe there
are also svn-buildpackage and cvs-buildpackage with the same or similar
functionality) to generate a new changelog each time.
> I'm not saying that we need to make sure that every time we make a
> change to source we have to keep the debian changelog up-to-date. But:
>
> **if we're checking in changes to /debian, we need to add an entry to
> the changelog, and the version needs to change.**
Seems reasonable.
> That said, a proposal:
>
> We really shouldn't have a /debian directory checked into the upstream
> at all (and especially not in releases). See "What's wrong with
> upstream shipping a debian/ directory?" in
> http://people.debian.org/~mpalmer/debian-mentors_FAQ.html
>
> I propose _removing_ the debian directory from the source control and
> putting into publicly accessible, separate maintainer's branch for
> debian (and another for ubuntu). Perhaps hosted via git or launchpad.
> Source changes can be automatically or manually propagated to the
> branch, and changes to the packaging can be checked into the branch
> itself. We link from the wiki of both perlbal and mogilefs.
That's certainly something to think about. The buildpackage tools
would have a repo for the debian directory and would take a tar.gz
file containing the pristine sources, apply changes to them and
generate a debian package. Changes to the packaging work would
occur in the debian repo and changes to the perlbal daemon itself
would occur in its own repo. But what about RedHat? Do they have
to have a completely separate build process too?
For my own selfish needs (and this is a bit of a digression), I really
want to be able to work on the development version of these tools _and_
package them up for installation on a box. I like that only debian
packages should be installed in the system directories, otherwise
too much cruft accumulates and file owners can't be located. If
there's a package in CPAN and a corresponding package in debian, I'll
choose the debian one every time. And if I want to run a development
version or hack the source then I want to do it through a repository
so my own changes will be tracked and mergeable with outside changes.
The debian/ directory is, to me, the "make install" part of the package.
Separating the debian directory might (or might not) make the building
or hacking process more cumbersome. Distributed SCM systems have
really made hacking code for personal requirements much easier,
cause I can checkin my changes locally and merge them with upstream
whenever I want. Without distributed SCM my local hacks to qpsmtpd
(Thanks Ask!) meant that I couldn't upgrade easily and I got stuck
on version 0.28 for a long time.
> Nick, I noticed you already have a launchpad team for perlbal, and I
> think that'd be a natural place to start hosting the ubuntu branch
> (plus, then I can start moving all the perlbal/mogile packages out of
> my PPA).
It wasn't me. I looked, and it seems that launchpad automates
the repository conversion. That's nice, except it converts to
bzr (launchpad.net as a gateway drug to bzr use). It seems that
active developers are going to have to become familiar with all
the distributed SCMs. At least there's already some support for
bidirectional syncing between heterogeneous repositories. Anyway my
name is there only because I did the last commits into SVN.
Github has a similar interesting effect. Somebody uploaded the linux
2.6.x kernel repository to github and some recent commits from me
through the normal linux development process were automatically linked
to my github account.
Nick.
> > The way it works seems to be that the changelog is generated
> > from the SCM log and then the build tools use whatever is in
> > the changelog.
>
> Only if you use a tool that is specifically aware of the SCM. Many
> people use the standard package builders: dpkg-buildpackage, pdebuild,
> debuild. As I mentioned before, if a user builds debs from the
> mogilefs script, it will use dpkg-buildpackage. We should be able to
> support whatever build tools the user feels comfortable with.
These tools will always work. When I say "generates the changelog" what
I think happens is that the tool (e.g. git-dch) figures out what commits
occurred since the last packaged version and prepends a section to the
changelog containing all those commit messages (and computed current
version number). From the git-dch manpage:
git-dch reads git commit messages and generates the debian changelog
from it. If no arguments are given git-dch starts from the last tagged
debian package version up to the current tip of the current branch. If
the distribution of the topmost section in debian/changelog is UNRE‐
LEASED the changelog entries will be inserted into this section. Other‐
wise a new section will be created.
Now if the user isn't using any kind of packaging tool, only
dpkg-buildpackage, then the changelog file is whatever was checked out
from the repository and the version of the package will be whatever
is the first one in the changelog. It's up to the user to change the
number themselves if they build the package more than once.
> That's exactly my point. It's NOT debian native. That's why we
> shouldn't drop the -1 version.
Ok then. Not debian-native, so we keep the -1 version. And whenever
a change is made under /debian that's a packaging change, so the -1
has to be incremented by us.
> > I was thinking about that earlier. One possibility is to put the _next_
> > version number into it (1.71) and add to that changelog entry until
> > 1.71 is released.
>
> So what happens when 1.71 is released? What happens to the changelog?
What would happen is that the changelog would change to 1.72. Let me
demonstrate:
--> 1.70 is released
packages built (_1.70-1_all.deb)
changelog updated to 1.71
commit
commit ... over time
commit
--> 1.71 is released
packages built (_1.71-1_all.deb)
changelog updated to 1.72
[and repeat]
The concept is "all commits made after 1.70 is released are 1.71
prerelease" and the version number on any home built package would
be greater than the distro one. Of course that doesn't change the
package number on every build. And some user's home built package
will be named the same as the later official 1.71, which isn't so
great, so I think we don't need to consider this idea any further.
> What happens if we need to commit two changes to the package in
> between CPAN revisions?
Non-/debian changes, already committed into the master repo? You'd
probably not want to do it, just ask Brad to release another version.
My impression is the shipit tool makes that pretty painless for him.
Alternately if they are /debian changes or if it involves some
debian-specific hack not committed into the repo, then the debian
revision increments to -2, -3 etc.
> > That's where you'd want support from git-buildpackage (I believe there
> > are also svn-buildpackage and cvs-buildpackage with the same or similar
> > functionality) to generate a new changelog each time.
>
> Does git-buildpackage not support the changelog format of
> perlbal-1.70+svnXXX-1? or perlbal-1.70+svn2008XXXX-1? I'd think that
> if it's generating a changelog, what's already in there should be moot
> -- but I'm really not familiar with it.
Perhaps I phrased it badly. I think when using git-buildpackage
(actually git-dch to generate the changelog) the previous contents of
the changelog don't matter. And it will figure out a unique version
string based on the latest tag, number of commits made after that tag,
and the git repo HEAD identifier (SHA1 hash).
In a project of mine, "git describe --tags" currently returns:
v1.0-3-g182998a
I built a debian package (1.0) from it, tagged that commit as "v1.0",
made 3 subsequent commits and the HEAD is currently 182998a43[etc].
So that's the kind of id which git-dch would output if I were using
it, except it would be in compliance with debian versioning rules.
> I'm sorry , I should have been more clear. The package branches would
> look exactly like the master branch does now: source code + /debian.
> The master branch just wouldn't have the /debian folder.
And frequent merges from master branch into debian branch? I'm not
sure that would be a nice workflow. There would potentially be as
many merges as commits. Branches are great for point releases (the
1.70.1 branch can make any necessary commits on top of the 1.70
tag) but the arrangement you mention seems like a lot of work just
to keep the debian directory out of the master branch.
If any debian-specific hacks are required, we'd probably be better
off to manage them using the git-buildpackage method (where the debian
specific stuff is in a different repo). This has the advantage that
we're using somebody else's (presumably well known and tested) workflow
rather than inventing a new one ourselves.
I'm going to go off and investigate git-buildpackage and see what is
necessary to start using it. The basic concept is that the /debian
directory and all debian-specific changes (in our case, none) are
managed in a separate repository (git, svn or cvs) and the tool
manages the package build process.
I'll finish this email with some more ranting:
Distinction between developers and users - developers are people
who use the repository (even if they don't send any patches); users
are people who install pre-built packages. We should build packages
for users (ok, you have already) even before they become part of the
official distro.
We should move to a DSCM as soon as possible. SVN doesn't do merges
properly, doesn't do tags properly, doesn't do branches properly. Due
to some weird stuff in the perlbal or mogilefs SVN repos, the conversion
of perlbal to git didn't bring across any tags.
Apropos to the previous point, I was a bit surprised to see perlbal
in bzr in ubuntu, but I think that's likely to happen more and more
frequently - for any package which gets popular, people will convert
it into the DSCM of their choice. I've done it a few times myself. Best
to make it easy to accept code from other places.
Nick.