ANNOUNCE: Debian build scripts on a public Monotone server
Ludovic Brenta
download them from Debian's many mirrors, change them, and run them to
build binary packages. Now, you can also follow their development
almost in real-time thanks to Monotone, a powerful distributed version
control system.
About Debian source packages
----------------------------
A Debian source package consists of three files:
* {package}_{version}.orig.tar.gz - the pristine upstream sources.
The directory tree in the tarball always starts from
{package}-{version}.orig (note: _ in the tarball name but - in the
directory name).
* {package}_{version}-{revision}.diff.gz - a compressed patch which
applies to the above directory tree extracted from the tarball.
This patch brings in all of the Debian build scripts,
Debian-specific files, and patches. The {revision} is specific to
Debian, too, and changes with every upload. The most important file
brought by the patch is debian/rules, which is executable and builds
the package. Usually (and always for the Ada packages),
debian/rules is a Makefile.
* {package}_{version}-{revision}.dsc - a short text file containing
the MD5 sums of the two above files, and which is signed with the
maintainer's private GPG key.
In order to build the set of binary packages for a source package, one
must therefore:
$ tar xzf {package}_{version}.orig.tar.gz
$ zcat {package}_{version}-{revision}.diff.gz | patch -p0
$ mv {package}-{version}.orig {package}-{version}
$ cd {package}-{version}
$ debian/rules binary
All these steps, and then some, are automated by apt-get:
$ apt-get source --build {package}
So, what's the problem?
-----------------------
With the above system, you can download and rebuild from source any
Debian package from your current distribution (stable, testing,
unstable, or experimental). You can change the package itself, or the
build scripts. But you cannot:
- use older versions of the build scripts
- use newer build scripts that have not yet been published to the
Debian archives
- submit your changes to the maintainer in a clean, efficient and
automated way (the official way is by opening a bug report).
These problems become critical for the many packages that are
maintained by teams rather than individuals.
A version control system allows the in-development build scripts to
become public, so that team maintenance is possible. For the Ada
packages, I would like to encourage people to look at the scripts and
propose improvements; maybe even form a team and benefir from each
other's experience?
Many Debian packages have project pages on Alioth[1], and use one of
Subversion[2], GNU Arch[3], Bazaar-NG[4] or GIT[5] as their version
control system.
[1] http://alioth.debian.org/
[2] http://subversion.tigris.org/
[3] http://www.gnu.org/software/gnu-arch
[4] http://bazaar-vcs.org/
[5] http://git.or.cz/
Since I do most of my Debian work on the train and without any network
connection, I require a _distributed_ version control system. After
evaluating[6] several candidates, I settled on Monotone[7] several
months ago.
[6] http://www.ada-france.org/debian/distributed_version_control.html
[7] http://venge.net/monotone
Why Monotone?
-------------
I believe that Monotone is the Ada of version control systems, so it
is only appropriate that I use it for my Ada work. Monotone is safe,
correct and powerful _by design_. It uses cryptographic keys to
authenticate changes. It is written by elite programmers who, despite
using C++, have the "Ada attitude": no pointers, one assert() every 9
lines of code, massive use of generics (templates), and not a single
critical bug in 3 years. The slides at [8] and my own tests convinced
me to switch from Meta-CVS several months ago for my Debian packages,
as well as for other work.
A Monotone database consists of one single file; this is very
convenient for maintenance. A Monotone database takes only a fraction
of the disk space required for an equivalent database in any other
system I've tried (Subversion, Bazaar-NG, Mercurial, CVS), which is
also an important consideration for me.
(I like to think that...
CVS is the "C" of version control systems,
Subversion is the "C++" designed to replace the "C",
GIT is the "assembly language" who needs cogito to be useable,
Bazaar-NG is the "perl", grossly inefficient and completely baroque,
Mercurial is the "Eiffel" i.e the second best,
Monotone is the "Ada", i.e. the best, even if not perfect)
(I particularly dislike Subversion and its distributed derivative,
SVK. I do not recommend them because their working model is
inherently broken, IMHO. A branch is NOT a directory, and a tag is
NEITHER a branch NOR a directory. And Subversion does not even try to
keep track of merges; just like C++ does not even try to multitask.)
What's in the Ada-France database?
----------------------------------
The database that I just published on Ada-France is a replica of the
one I work on every day. It contains one branch (sometimes a couple
of branches, actually) for each package I work on. Each published
upload of each package also has a tag. You can browse the whole
history of all changes, with comments. The size of the database is
about 1.7 Mb.
As of today, "mtn list branches" says:
org.debian.adacontrol
org.debian.asis
org.debian.asis-doc
org.debian.asis.2005
org.debian.gnat-gdb
org.debian.gnat-glade
org.debian.gnat-gps
org.debian.libaunit
org.debian.libaws
org.debian.libflorist
org.debian.libgtkada2
org.debian.libopentoken
org.debian.libtemplates-parser
org.debian.libtexttools
org.debian.libxmlada1
org.debian.libxmlada2
The list of tags ("mtn list tags") would be too boring for this post.
Each branch contains a "debian" directory and, in most cases, a
"patches" directory. I use Quilt[9] to manage the patches.
[9] http://savannah.nongnu.org/projects/quilt
If you extract an upstream source tarball, and checkout from Monotone
into the source tree, you're ready to build the package.
The database does not contain the upstream tarballs (.orig.tar.gz);
these are available from Debian mirrors anyway or, if not yet
published in the Debian archive, from
http://www.ada-france.org/debian/pool .
How to use Ada-France's Monotone database
-----------------------------------------
If you want to *read* from the database:
1. Install Monotone, version 0.26 or later (the server is currently
running 0.28).
2. Create a key pair:
$ mtn genkey yo...@email.address
3. Create a new, local database:
$ mtn --db=debian.mtn db init
(I like to keep all my databases in /var/lib/monotone, but you can
place your database anywhere; remember: it's a single file anyway.
You can, of course, have as many databases as you want.)
4. Pull all branches starting with "org.debian":
$ mtn --db=debian.mtn pull www.ada-france.org 'org.debian.*'
5. Create a working copy:
$ tar xzf {package}_{version}.orig.tar.gz
$ mv {package}-{version}.orig {package}-{version}
$ cd {package}-{version}
$ mtn --db=../debian.mtn checkout --branch=org.debian.{package} .
6. Now you can build the package, change the scripts, add new patches
and whatnot. More importantly, you can *check in* into your local
database, keeping track of your own changes. You can even create
sub-branches at will, if you wish your changes to remain separate.
(the above commands have abbreviations, e.g. -d for --db=, -b for
--branch=, etc.. Also, --db= and --branch= are unnecessary in a
working copy, because the working copy remembers the database and
branch it was checked out of.)
If you want to *write* into Ada-France's database:
You can "push" your changes directly to the Ada-France database, too.
But I will allow you to do so only if you identify yourself :-) You
will need a GPG[10] keypair, signed and part of the Debian Web of
Trust[11], in addition to your Monotone keypair. You need not be a
Debian Developer; you only need to be authenticated. (The recent
debates about copyright law made it plain that anonymous contributions
are quite dangerous in fact).
[10] http://www.gnupg.org/
[11] http://www.debian.org/events/keysigning.html
1. Extract your public key from your Monotone database:
$ mtn --db=debian.mtn pubkey {yo...@email.address} > your_public_key
(your_public_key will be a short plain-text file).
2. Send me your public key in a GPG-signed email (signed with your
public GPG key, that is).
3. Wait until I reply to tell you you're good to go.
4. Push your changes:
$ mtn --db=debian.mtn push
(the database remembers where it was "pulled" from, so by default
Monotone will "push" there. You can change that if you wish).
Enjoy!
--
Ludovic Brenta.
Steve Whalen
> Now, you can also follow their development
> almost in real-time thanks to Monotone, a powerful distributed version
> control system.
> Since I do most of my Debian work on the train and without any network
> connection, I require a _distributed_ version control system. After
> evaluating[6] several candidates, I settled on Monotone[7] several
> months ago.
> I believe that Monotone is the Ada of version control systems, so it
> is only appropriate that I use it for my Ada work. Monotone is safe,
> correct and powerful _by design_. It uses cryptographic keys to
> authenticate changes. It is written by elite programmers who, despite
> using C++, have the "Ada attitude": no pointers, one assert() every 9
> lines of code, massive use of generics (templates), and not a single
> critical bug in 3 years. The slides at [8] and my own tests convinced
> me to switch from Meta-CVS several months ago for my Debian packages,
> as well as for other work.
> (I particularly dislike Subversion and its distributed derivative,
> SVK. I do not recommend them because their working model is
> inherently broken, IMHO. A branch is NOT a directory, and a tag is
> NEITHER a branch NOR a directory. And Subversion does not even try to
> keep track of merges; just like C++ does not even try to multitask.)
Thanks for the access to the Debian scripts AND the information on
Monotone.
Just looking at Monotone's web site and how organized it is (along with
your recommendation), is going to make me quit thinking of Subversion
as the "successor" to CVS.
A quick glance at Monotone's change control history looks like they try
to ensure that existing projects can move forward through their
improvements. Do you consider Monotone as "safe" for production
environments, even though they're not at "release 1.0" yet? Would you
recommend them as an alternative to CVS for new projects needing a
version control system, but wanting to be comfortable that they'll not
regret entrusting their project to Monotone?
Comparing Monotone to Ada to me is high praise. If I understood you
correctly, you seem to be saying that even though Monotone is only at
version 0.30, Monotone's "standards" and professionalism make their
v0.30 "better" than a lesser tools' 1.0+ version... Is that true?
Bringing us back to Ada: Is Monotone relatively Ada friendly? Does it
handle Ada "projects" and source code naturally?
Steve
Brian May
Ludovic> (I particularly dislike Subversion and its distributed
Ludovic> derivative, SVK. I do not recommend them because their
Ludovic> working model is inherently broken, IMHO. A branch is
Ludovic> NOT a directory, and a tag is NEITHER a branch NOR a
Ludovic> directory. And Subversion does not even try to keep
Ludovic> track of merges; just like C++ does not even try to
Ludovic> multitask.)
To be fair, subversion was only ever meant to be a "better CVS", and
as such carries across some of the broken decisions (even if I don't
remember all of them right now). At least it isn't as broken as CVS...
There is a (biased?) comparison at
http://bazaar-vcs.org/RcsComparisons
Sounds like I should get familiar with monotone sometime....
--
Brian May <b...@snoopy.apana.org.au>
Ludovic Brenta
> Thanks for the access to the Debian scripts AND the information on
> Monotone.
:)
> Just looking at Monotone's web site and how organized it is (along with
> your recommendation), is going to make me quit thinking of Subversion
> as the "successor" to CVS.
Cool!
> A quick glance at Monotone's change control history looks like they try
> to ensure that existing projects can move forward through their
> improvements. Do you consider Monotone as "safe" for production
> environments, even though they're not at "release 1.0" yet? Would you
> recommend them as an alternative to CVS for new projects needing a
> version control system, but wanting to be comfortable that they'll not
> regret entrusting their project to Monotone?
Yes, definitely. They've had zero data-loss bugs in their entire
history, despite changing the database schema a couple of times, and
the netsync protocol another couple of times. So, yes.
> Comparing Monotone to Ada to me is high praise. If I understood you
> correctly, you seem to be saying that even though Monotone is only at
> version 0.30, Monotone's "standards" and professionalism make their
> v0.30 "better" than a lesser tools' 1.0+ version... Is that true?
Yes, it is true. Their emphasis on correctness is simply astounding.
It's the attitude, man :)
> Bringing us back to Ada: Is Monotone relatively Ada friendly? Does it
> handle Ada "projects" and source code naturally?
Yes; it handles renames and complex directory structures well, even
across merges. It also handles file attributes (e.g. executable), and
there is even a monotone.el for us Emacs fans.
--
Ludovic Brenta.
Ludovic Brenta
> To be fair, subversion was only ever meant to be a "better CVS", and
> as such carries across some of the broken decisions (even if I don't
> remember all of them right now). At least it isn't as broken as CVS...
Right, that's why I compared CVS to C and Subversion to C++ :)
--
Ludovic Brenta.
Ludovic Brenta
> Steve Whalen writes:
> > A quick glance at Monotone's change control history looks like they try
> > to ensure that existing projects can move forward through their
> > improvements. Do you consider Monotone as "safe" for production
> > environments, even though they're not at "release 1.0" yet? Would you
> > recommend them as an alternative to CVS for new projects needing a
> > version control system, but wanting to be comfortable that they'll not
> > regret entrusting their project to Monotone?
>
> Yes, definitely. They've had zero data-loss bugs in their entire
> history, despite changing the database schema a couple of times, and
> the netsync protocol another couple of times. So, yes.
I'd like to add that my confidence in Monotone is not just because I
like the developers' attitude; it is also because:
- Monotone stores the SHA1 sum of everything in the database, and
verifies the sum when checking out, updating, merging, or syncing
between databases. Any data corruption _will_ be detected, and
Monotone _will_ crash rather than corrupt data (never happened to me
though). (indeed, revision IDs are SHA1 sums, not "monotonic" numbers
that would make no sense in the context of a distributed version
control system).
- all commits to a database are signed by a crypto key, and so
authenticated in a tamper-proof way.
- the database is an SQLite database; one can always retrieve the data
using SQL commands, if things came to worst. In fact, I have already,
and successfully, tampered with some trial databases using SQL
commands. That's the almost only way one can delete data, BTW.
Now Monotone is not perfect, and not 1.0 yet. The areas where perhaps
it might be weak is scalability and performance. In my experience, it
shines both for small projects and for large projects with short
histories ( few revisions). But, apparently, as the number of revisions
grows, Monotone scales less than linearly. They're working on it at the
moment, with the first notable scalability improvements in the latest
version, 0.30.
--
Ludovic Brenta.
Ludovic Brenta
> Steve Whalen writes:
> > to ensure that existing projects can move forward through their
> > improvements. Do you consider Monotone as "safe" for production
> > environments, even though they're not at "release 1.0" yet? Would you
> > recommend them as an alternative to CVS for new projects needing a
> > version control system, but wanting to be comfortable that they'll not
> > regret entrusting their project to Monotone?
>
> Yes, definitely. They've had zero data-loss bugs in their entire
> history, despite changing the database schema a couple of times, and
> the netsync protocol another couple of times. So, yes.
I'd like to add that my confidence in Monotone is not just because I
Georg Bauhaus
> What's in the Ada-France database?
> ----------------------------------
>
...
> 1. Install Monotone, version 0.26 or later (the server is currently
> running 0.28).
A project that hasn't silently faded away. Excellent.
I noticed, though, that with Monotone you need synchronous
communication. Either between two developers or between
a Monotone server and a Monotone client. Is this true?
(The alternative could be privacy guarded patch emails.)
-- Georg
Ludovic Brenta
> > 1. Install Monotone, version 0.26 or later (the server is currently
> > running 0.28).
>
> A project that hasn't silently faded away. Excellent.
and 0.30 is out, but has not reached Debian etch yet :)
> I noticed, though, that with Monotone you need synchronous
> communication. Either between two developers or between
> a Monotone server and a Monotone client. Is this true?
Monotone needs synchronous communications only during "mtn sync" or its
two halves, "mtn push" and "mtn pull", as per the Law of Least
Astonishment :) The rest of the time, you're offline and talking to
your local database.
I did mention that I required the ability to work off-line on the
train, didn't I?
> (The alternative could be privacy guarded patch emails.)
A very poor alternative, very work-intensive and error-prone, and
without support for merging. It requires a human to merge manually,
keeping track of the ordering of the patches. That's how I sent most of
the far-reaching changes for gnat-4.1 to the Debian maintainer before I
became a full DD. Now I use svk for gcc/gnat in Debian, and it's much
better than emails, GPG-signed or not, even though I still hate svk.
There's also the issue that GPG only creates a trusted relationship
between a signature and a person (i.e. authentication); not between the
signature and the person's worthiness or trustability as regards code
contributions (i.e. authorisation). That's why Monotone uses its own
keys, separate from the GPG keys. The FAQ has a blurb on that, too.
--
Ludovic Brenta.
Georg Bauhaus
> Monotone needs synchronous communications only during "mtn sync" or its
> two halves, "mtn push" and "mtn pull",
Yes, I meant this kind of synchronous communication.
> I did mention that I required the ability to work off-line on the
> train, didn't I?
You did.
> > (The alternative could be privacy guarded patch emails.)
>
> A very poor alternative, very work-intensive and error-prone, and
> without support for merging.
I wasn't clear enough about what I mean by "email changeset".
The same content that is needed by mtn sync
could be passed in a mailbox style, decoupling operations.
(Going from rendezvous to something else, in Ada terms.)
I wasn't thinking of sending a standard patch in some email,
I was thinking of using SMTP as a "Monotone sync packet wrapper".
Monotone merging wouldn't have to be changed I think, if you follow
the "commit first" policy that Monotone recommends.
The GPG keys could be associated with the permission settings in
your Monotone configuration. Then if mtn receives an "email sync
message" the software knows what to do with it, just like when it
operates in "normal synchronous sync mode".
Just an idea, to overcome some issues:
1/ For mtn sync between two people,
you need both ends at the same time.
2/ For mtn sync between you and a server,
you need both ends at the same time.
3/ For 2, you need a server that runs a Monotone server process.
If your ISP/company/... offers 3, that's perfect.
-- Georg
Ludovic Brenta
> I wasn't clear enough about what I mean by "email changeset".
> The same content that is needed by mtn sync
> could be passed in a mailbox style, decoupling operations.
> (Going from rendezvous to something else, in Ada terms.)
> I wasn't thinking of sending a standard patch in some email,
> I was thinking of using SMTP as a "Monotone sync packet wrapper".
> Monotone merging wouldn't have to be changed I think, if you follow
> the "commit first" policy that Monotone recommends.
Ah, now I understand what you meant. That would be much less efficient.
The benefit of synchronous communications is that each side can ask the
other what it is missing, and only send the necessary information for
both sides to come up to sync. With asynchronous comms, the sending
side has to make assumptions about what the receiving side already has
in its database. That's unsafe.
Monotone did use to support syncing over HTTP and NNTP, but dropped
that for efficiency.
> The GPG keys could be associated with the permission settings in
> your Monotone configuration. Then if mtn receives an "email sync
> message" the software knows what to do with it, just like when it
> operates in "normal synchronous sync mode".
>
> Just an idea, to overcome some issues:
>
> 1/ For mtn sync between two people,
> you need both ends at the same time.
> 2/ For mtn sync between you and a server,
> you need both ends at the same time.
> 3/ For 2, you need a server that runs a Monotone server process.
That's reasonable; it's just not implemented now in Monotone. If you
insist on asynchronous updates, you can always send patches by email
and have a human commit them manually on the other end, or investigate
Darcs.
> If your ISP/company/... offers 3, that's perfect.
Yes, Ada-France does now :) and a couple of others, see
http://www.venge.net/monotone/wiki/Hosting
--
Ludovic Brenta.
Pascal Obry
> (I particularly dislike Subversion and its distributed derivative,
> SVK. I do not recommend them because their working model is
> inherently broken, IMHO. A branch is NOT a directory, and a tag is
> NEITHER a branch NOR a directory. And Subversion does not even try to
> keep track of merges; just like C++ does not even try to multitask.)
Well, well! Tracking merges is planed for version 2.0. We need to give
time to Subversion. It used to be a CVS "like" project for compatibility
issued. Subversion 2.0 is on the way and the developers are going to do
things as it should have been done now... So Subversion 2.0 will
probably break compatibility with current version for good!
Let's not dismiss such a nice tool so easily ;)
Pascal.
--
--|------------------------------------------------------
--| Pascal Obry Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--| http://www.obry.net
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver wwwkeys.pgp.net --recv-key C1082595
Ludovic Brenta
> Well, well! Tracking merges is planed for version 2.0. We need to give
> time to Subversion. It used to be a CVS "like" project for compatibility
> issued. Subversion 2.0 is on the way and the developers are going to do
> things as it should have been done now... So Subversion 2.0 will
> probably break compatibility with current version for good!
>
> Let's not dismiss such a nice tool so easily ;)
You're kidding, right? If it takes them 4 versions (or more) and a
major compatibility-breaking rewrite to get things "as [they] should
have been done", then I don't see how Subversion is "a nice tool".
--
Ludovic Brenta.
Pascal Obry
> You're kidding, right? If it takes them 4 versions (or more) and a
> major compatibility-breaking rewrite to get things "as [they] should
> have been done", then I don't see how Subversion is "a nice tool".
I'm not kidding. This is explained into Subversion web site. As I said
the first goal was compatibility (as suppose from a user point of view)
with CVS for smooth migration. This step had a great success as seen by
the number of projects that have switched. Now let's see what the future
version will bring.
Brian May
Pascal> Well, well! Tracking merges is planed for version 2.0. We
Pascal> need to give time to Subversion. It used to be a CVS
Pascal> "like" project for compatibility issued. Subversion 2.0 is
Pascal> on the way and the developers are going to do things as it
Pascal> should have been done now... So Subversion 2.0 will
Pascal> probably break compatibility with current version for
Pascal> good!
So maybe Subversion will improve in the future...
However, right here and now, Monotone already does what Ludovic's
wants, and I must say it does look very interesting.
I find it interesting in Monotone you can commit changes even if your
tree is not up-to-date, so you only commit your changes first go, and
then you do the merge. As opposed to being forced to do both in the
one commit (I never liked this model).
There are still some thing on this I am curious about though, for
example, what happens if two people try to merge conflicting trees at
the same time?
It also seems to have Windows support (a weak point in gnuarch/bazaar
only addressed in bazaar-ng - yes Windows clients exist in the earlier
versions, but my understanding is that they are very much hacks).
--
Brian May <b...@snoopy.apana.org.au>
Alex R. Mosteo
> Now Monotone is not perfect, and not 1.0 yet. The areas where perhaps
> it might be weak is scalability and performance. In my experience, it
> shines both for small projects and for large projects with short
> histories ( few revisions). But, apparently, as the number of revisions
> grows, Monotone scales less than linearly. They're working on it at the
> moment, with the first notable scalability improvements in the latest
> version, 0.30.
Scaling not constantly, but not even linearly, is a major flaw for a
versioning system. How is that this hasn't been done right from the start?
(Just kidding.)
Samuel Tardieu
Steve> Just looking at Monotone's web site and how organized it is
Steve> (along with your recommendation), is going to make me quit
Steve> thinking of Subversion as the "successor" to CVS.
I also suggest that you look at Mercurial then
(http://www.selenic.com/mercurial). As of today, this is by far my
preferred distributed VCS. It is fast, reliable and very practical to
use. And its queue manager (mq) makes tracking versionned local
modifications of an upstream software very easy.
And it scales very well. I use it every day to track my changes to the
Linux kernel.
Sam
--
Samuel Tardieu -- s...@rfc1149.net -- http://www.rfc1149.net/
Ludovic Brenta
> So maybe Subversion will improve in the future...
>
> However, right here and now, Monotone already does what Ludovic's
> wants,
Well put.
> and I must say it does look very interesting.
>
> I find it interesting in Monotone you can commit changes even if your
> tree is not up-to-date, so you only commit your changes first go, and
> then you do the merge. As opposed to being forced to do both in the
> one commit (I never liked this model).
That's funny because it took me a little while to get used to the
"commit first, merge then" model. I wasn't used to it. Now I like it
quite a lot.
> There are still some thing on this I am curious about though, for
> example, what happens if two people try to merge conflicting trees at
> the same time?
They cannot merge conflicting trees at the same time, because they
operate on two distinct databases.
- if they both arrive at the same result, identified by its SHA1
(which is the common case thanks to automatic merge), then they end
up with the exact same version in their databases. When they next
sync, nothing happens since they are already in sync.
- if they arrive at different results, perhaps because they resolved
conflicts differently, then the branch has two heads, just as
before. When they next sync, both will see both heads, and merge if
they want to. Monotone sees nothing wrong with a branch having two
or more heads.
--
Ludovic Brenta.
Ludovic Brenta
> I also suggest that you look at Mercurial then
> (http://www.selenic.com/mercurial). As of today, this is by far my
> preferred distributed VCS. It is fast, reliable and very practical
> to use. And its queue manager (mq) makes tracking versionned local
> modifications of an upstream software very easy.
Mercurial is second best after Monotone, IMHO. And a worthy runner-up
at that. The reason is that I dislike its model of "a repository is a
branch is a working copy". I insist on having my repository separate
from my working copies, and I insist on having as many branches as I
need in one repository.
> And it scales very well. I use it every day to track my changes to the
> Linux kernel.
It scales performance-wise, yes, but not in terms of disk space
requirements. Branches are expensive, especially if you change many
files in the working copy. By contrast, Monotone does that very well.
--
Ludovic Brenta.
Samuel Tardieu
Ludovic> Mercurial is second best after Monotone, IMHO. And a worthy
Ludovic> runner-up at that. The reason is that I dislike its model of
Ludovic> "a repository is a branch is a working copy". I insist on
Ludovic> having my repository separate from my working copies, and I
Ludovic> insist on having as many branches as I need in one
Ludovic> repository.
When I started using GNU Arch and Darcs, I thought like you that a
separate repository was best for backups purpose. As the time went, I
started to like the "repository = branch = working copy" model, as it
makes it easy for me to clone a repository on any machine I'm working
on without having to create a db, get the keys and so on.
Ludovic> It scales performance-wise, yes, but not in terms of disk
Ludovic> space requirements. Branches are expensive, especially if
Ludovic> you change many files in the working copy. By contrast,
Ludovic> Monotone does that very well.
What version did you test? Mercurial uses hard links when you work on
the same file system, making cloning a very cheap operation. Moreover,
since Mercurial 0.9 is out, the disk usage has been cut by 40%.
Martin Krischik
>> I find it interesting in Monotone you can commit changes even if your
>> tree is not up-to-date, so you only commit your changes first go, and
>> then you do the merge. As opposed to being forced to do both in the
>> one commit (I never liked this model).
>
> That's funny because it took me a little while to get used to the
> "commit first, merge then" model. I wasn't used to it. Now I like it
> quite a lot.
That's exactly was I was always looking for. I just hate how most vcs
force me to mess up my perfectly good code by forcing me to merge when I
am not yet ready for merge. Worse: I won't have a backup copy of the
pre-merge code.
Yes, those vcs will mess up my working copy with ".#" files - but that
is not the same as having a backup inside the repository.
The other option I am often missing is a "golden code" option. What I
have prepared inside the working copy is right and propper - no merge
needed - thank you very much.
Martin
Ludovic Brenta
> What version did you test? Mercurial uses hard links when you work on
> the same file system, making cloning a very cheap operation.
I know[1]. The advantage disappears as soon as you modify a file:
Mercurial then breaks the hard link and duplicates the entire history
for the file. In my trials with GCC, many files were changed between
each version, so the space advantage was almost completely lost, even
if the changes to each file were actually small.
The use of hard links is a poor kludge to minimise the impact of
unshared histories.
[1] http://www.ada-france.org/debian/distributed-version-control-systems.html
> Moreover, since Mercurial 0.9 is out, the disk usage has been cut by
> 40%.
I tried 0.8.1 and my test consumed 686 megabytes. If I take your word
and reduce it by 40%, I get 411.6 Mb, which is still much, much more
than Monotone's 183 megabytes (in Monotone 0.24) or 166 megabytes (in
0.26).
I stand by my opinion that Mercurial is only second best, after
Monotone.
--
Ludovic Brenta.
Brian May
Brian> There is a (biased?) comparison at
Brian> http://bazaar-vcs.org/RcsComparisons
Definitely biased, if not wrong.
e.g. for Monotone "Cheap branching Anywhere", it has "No".
Hmmm...
--
Brian May <b...@snoopy.apana.org.au>
Brian May
Martin> That's exactly was I was always looking for. I just hate
Martin> how most vcs force me to mess up my perfectly good code by
Martin> forcing me to merge when I am not yet ready for
Martin> merge. Worse: I won't have a backup copy of the pre-merge
Martin> code.
Same here - occasionally at my previous job (they used subversion) it
was almost like a race to commit my changes first so I wouldn't have
to deal with the conflicts. Didn't always work as planned though, as
often the second person would accidently revert my changes by doing an
update operation with the file still open in the editor. "Why did you
revert my changes? The bug I fixed came back again. Did I break
something?" "I didn't revert your change!" "Yes you did, in revision
XYZ!"
Creating a fork, and then dealing with the merge operation, later,
when ready, would appear to be a much better strategy.
--
Brian May <b...@snoopy.apana.org.au>
Brian May
Samuel> When I started using GNU Arch and Darcs, I thought like
Samuel> you that a separate repository was best for backups
Out of curiosity: How does Darcs compare with other systems?
Samuel> purpose. As the time went, I started to like the
Samuel> "repository = branch = working copy" model, as it makes it
Samuel> easy for me to clone a repository on any machine I'm
Samuel> working on without having to create a db, get the keys and
Samuel> so on.
Earlier I use to find exact details like this important, but not so
much any more.
Having said this, I find it annoying if I get lots of unwanted results
if I try to do something like:
find . -type f | xargs grep abc
or
grep -r abc .
I believe this is a problem with subversion, bazaar (possibly OK if
you use revision libraries), bazaar-ng, as they keep copies of old
files available under a subdirectory for caching.
Yes, some programs will generate a list of files, but this can cause
other problems, e.g. slowness, filenames with spaces, filtering out
directories, etc.
Monotone looks a lot better...
--
Brian May <b...@snoopy.apana.org.au>
Samuel Tardieu
Brian> Out of curiosity: How does Darcs compare with other systems?
Its model is brilliant. Its implementation is sloooooooow.
n...@pobox.com
> I wasn't clear enough about what I mean by "email changeset".
> The same content that is needed by mtn sync
> could be passed in a mailbox style, decoupling operations.
> (Going from rendezvous to something else, in Ada terms.)
> I wasn't thinking of sending a standard patch in some email,
> I was thinking of using SMTP as a "Monotone sync packet wrapper".
> Monotone merging wouldn't have to be changed I think, if you follow
> the "commit first" policy that Monotone recommends.
Merging is a bit of a red-herring here; the reason monotone uses
bidirectional communication is more subtle.
The simplest way to explain it is to imagine a hypothetical example.
Say you have some files on one or more remote computers, and you want
to make them match the files you have on a local computer. (Basically
a simple mirroring problem.) Because bandwidth is expensive, and you
perform this operation often, you can't just send a whole tarball over
there. So you have basically two options.
One is to somehow guess what the other side has -- perhaps by
remembering what they had the last time you talked to them, plus
assuming they've received any changes you've sent since then. With
this information, you can calculate some sort of diff that needs to be
applied to the remote files, and send that diff off into the aether.
Some possible problems: You might not have a convenient way to keep
track of what the other side has. Even if you do, you might get it
wrong. If you get it wrong, then you might send too little
information, making your patch useless, or you might send too much
information, making your patch wasteful. You need some external
mechanism to recover from such loss-of-sync if it does occur. Losing
sync is pretty darn easy -- all it takes is some patch getting lost in
the mail.
The other option is to use rsync, which does a bit of algorithmic
tricksiness (requiring bidirectional communication), and figures out
what changes need to be transmitted on the fly, for each remote host.
This is always reasonably efficient, and completely immune to
loss-of-sync issues. If some connection gets lost, you just hit
up-arrow and run the same command again, it'll magically figure out how
to start up again where it left off. If something happens to the
remote site -- maybe its disk gets wiped, maybe it gets
a bunch of information from someone else that you no longer have to
send yourself -- then no problem.
It is not a coincidence that monotone calls its network protocol
"netsync" :-). The main algorithm we use isn't actually the rsync
algorithm (because we need to synchronize sets, not sequences), but it
has similar properties (and is actually asymptotically more efficient).
Doing this just elimates whole swathes of failure modes and thus makes
the overall tool much more robust (and means that users get to spend
less time worrying about these sorts of obscure failure modes and how
to avoid them, too -- a regrettably common way to spend one's brain
cells when using traditional systems, I think...).
For a concrete example... say your project server's disk melts.
Solution: drop an empty db on a host somewhere, start serving out of
it. The first person to use it will automatically push up whatever old
history they have available locally. If they weren't quite up to date,
no problem; as everyone else hits the server in the normal course of
their work, they'll fill in whatever parts are missing.
It also helps that, since one generally stores all of the history
related to a project in a single database, when you get online you run
just one command to sync everything related to that
project. (Compare to systems like mercurial, where each branch
generally needs to be pushed separately, to a separate destination --
assuming you remember which branches you
changed, and thus need to push.) Also, syncing is always a legal
operation -- in particular, you never have to merge-before-push.
(Again, compare to systems like mercurial,
where you may have to do this.) The biggest reason I know of to use
email is that we all already have a lot of infrastructure to make it
convenient, so even when you're offline you can just "queue and
forget". But we've tried to make "hit the button and forget" about as
convenient.
-- Nathaniel
(I don't normally read this group, so CC's on replies appreciated.)
Brian May
njs> For a concrete example... say your project server's disk
njs> melts. Solution: drop an empty db on a host somewhere, start
njs> serving out of it. The first person to use it will
njs> automatically push up whatever old history they have
njs> available locally. If they weren't quite up to date, no
njs> problem; as everyone else hits the server in the normal
njs> course of their work, they'll fill in whatever parts are
njs> missing.
Hmmm. Could get interesting. Consider the case of a user not using a
given computer account for several years, and suddenly they wake up
and do a sync operation.
"Where did this fork from??? Everything in it is obsolete!"
Just curious: Is it possible to somehow terminate a dead tree head
without merging it?
(sidenote: I have had a dead/obsolete DECstation/Ultrix computer at
another campus suddenly "wake up" and start sending the occasional
packet with my own IP address; fortunately it didn't seem to response
to incoming packets; hence I think the above is possible).
However not a show stopper by any means.
--
Brian May <b...@snoopy.apana.org.au>
n...@pobox.com
> Hmmm. Could get interesting. Consider the case of a user not using a
> given computer account for several years, and suddenly they wake up
> and do a sync operation.
>
> "Where did this fork from??? Everything in it is obsolete!"
>
> Just curious: Is it possible to somehow terminate a dead tree head
> without merging it?
You could merge it, and override the merge algorithm to say "resolve
all conflicts in my favor". Basically you end up with a revision
graph that looks like:
a old
\ /
a (again)
We don't quite have a convenient UI for that right now, but we're
working on an overhaul of the UI for merging in general, and one of the
things that should fall out is a natural way of expressing this. In
general merging is much better than simply marking a head as dead,
because it provides a path forward for any existing work that might be
based off of that head. (I.e., maybe it wasn't a head at all, but
because it's a distributed system you didn't know that.)
In general, I am biased much more towards worrying about the data loss
scenario, than the accidental data survival scenario :-). Bits, for
all their infinite replicability, are tenuous, fragile things, and it's
a lot easier to ignore something you've preserved, then to use
something you've lost. So our first priority is making data immortal;
overpopulation is an easier problem :-). I actually find the scenario
of some bits being lost for years, then accidentally promiscuously
replicated and saved for posterity to be heart-warming :-).
That's just me, of course, and overall, yeah, there are ways that kind
of scenario can be dealt with.
-- Nathaniel
Martin Krischik
> You could merge it, and override the merge algorithm to say "resolve
> all conflicts in my favor".
Yet another feature which I was allways looking for. Not very often - but
sometimes - I have a perfect version prepated in the working directory and
then (for whatever reason) the vcs (falsely) thinks I need a merge. And
then the trouble beginns...
Martin
--
mailto://kris...@users.sourceforge.net
Ada programming at: http://ada.krischik.com
Samuel Tardieu
Brian> Just curious: Is it possible to somehow terminate a dead tree head
Brian> without merging it?
Nathaniel> You could merge it, and override the merge algorithm to say
Nathaniel> "resolve all conflicts in my favor". Basically you end up
Nathaniel> with a revision graph that looks like:
A modified file in the dead branch only wouldn't result in a
conflict. Would those modified files be changed anyway during this
kind of merge?
n...@pobox.com
> >>>>> "Nathaniel" == njs <n...@pobox.com> writes:
>
> Brian> Just curious: Is it possible to somehow terminate a dead tree head
> Brian> without merging it?
>
> Nathaniel> You could merge it, and override the merge algorithm to say
> Nathaniel> "resolve all conflicts in my favor". Basically you end up
> Nathaniel> with a revision graph that looks like:
>
> A modified file in the dead branch only wouldn't result in a
> conflict. Would those modified files be changed anyway during this
> kind of merge?
[Sorry for missing this until now.]
In the UI I am thinking of, yes.
Create an on-disk directory tree matching GOODREV:
$ mtn checkout -r GOODREV my-workspace; cd my-workspace
Merge in the revision I want to kill. After this, my directory tree
contains a mix of edits from both sides, some conflicts, whatever, and
has both GOODREV and BADREV as parents:
$ mtn merge -r BADREV
Throw away all local changes, to make my local directory tree exactly
match that seen in GOODREV (but this only effects directory contents,
it doesn't affect the parent links):
$ mtn revert -r GOODREV
Commit my new tree, with its two parents:
$ mtn commit --message="Clobbered BADREV"
You could do something else at any step, of course -- this is just
composing primitives.
-- Nathaniel
Brian May
njs> $ mtn checkout -r GOODREV my-workspace; cd my-workspace
njs> Merge in the revision I want to kill. After this, my directory tree
njs> contains a mix of edits from both sides, some conflicts, whatever, and
njs> has both GOODREV and BADREV as parents:
njs> $ mtn merge -r BADREV
My (test) usage of merge so far as resulted in the result being
automatically committed. Is the above usage different?
Which brings me to a dislike of monotone, so often I see changelog messages like:
merge of '646c7d2cca0e8dd0e97e0d8f3cc73e5a3d39ab8d'
and '87c207f4bee6cecb68e9bcc9c65c37a93c99934c'
Unfortunately, this doesn't tell me anything.
A manual description would be better, for example:
Merge in bug fix for XYZ into main tree.
I believe the parents are already mentioned in the revision anyway.
I have tried to register with the BTS
https://savannah.nongnu.org/account/register.php, so I can send bug
reports, however, so far all I get is errors. Is it possible to check
if I have already registered?
njs> Throw away all local changes, to make my local directory tree exactly
njs> match that seen in GOODREV (but this only effects directory contents,
njs> it doesn't affect the parent links):
njs> $ mtn revert -r GOODREV
Have to keep that command in mind - was wondering about it.
njs> Commit my new tree, with its two parents:
njs> $ mtn commit --message="Clobbered BADREV"
Looks good.
--
Brian May <b...@snoopy.apana.org.au>
Ludovic Brenta
> My (test) usage of merge so far as resulted in the result being
> automatically committed. Is the above usage different?
>
> Which brings me to a dislike of monotone, so often I see changelog messages like:
>
> merge of '646c7d2cca0e8dd0e97e0d8f3cc73e5a3d39ab8d'
> and '87c207f4bee6cecb68e9bcc9c65c37a93c99934c'
You can avoid these messages with "mtn log --no-merges".
> Unfortunately, this doesn't tell me anything.
True. I also would prefer --no-merges to be the default. I think you
can make it so in your ~/.monotone/monotonerc, but I haven't looked
into it yet.
> A manual description would be better, for example:
>
> Merge in bug fix for XYZ into main tree.
For non-trivial merges where you resolve conflicts in a creative way,
this may indeed be nice to have. Maybe "merge_into_dir" or
"explicit_merge" already allow you to add your own comments?
But in the case of trivial, automated merges, such functionality would
in fact be harmful. For one thing, a trivial merge guarantees that you
have all the changes from both parents and nothing else; so a custom
comment would not say anything useful. OTOH, it would alter the SHA1 of
the resulting revision and make it impossible for other people to
obtain the exact same revision ID (which is precisely the SHA1) if
they, too, merge automatically.
> I believe the parents are already mentioned in the revision anyway.
Yes.
--
Ludovic Brenta.
Brian May
Ludovic> OTOH, it would alter the SHA1 of the resulting revision
Ludovic> and make it impossible for other people to obtain the
Ludovic> exact same revision ID (which is precisely the SHA1) if
Ludovic> they, too, merge automatically.
Hmmm.
Somewhere in the documentation I read something along similar lines
(deleting a certificate is not possible unless there are no child
revisions).
However, this really confuses me.
I thought a revision only had the SHA1 of the previous revision
details. Each certificate contains the SHA1 of the revision, not the
other way around.
As such, surely it should be OK to add as many different certificates
as you want without any of the certificates effecting any subsequent
revisions. Or delete certificate too, although they may get copied
back again.
In which case, wouldn't both merge's still result in the same
revision-id, but with two set of certificates?
Or have I misunderstood something?
--
Brian May <b...@snoopy.apana.org.au>
n...@pobox.com
> njs> $ mtn checkout -r GOODREV my-workspace; cd my-workspace
> njs> Merge in the revision I want to kill. After this, my directory tree
> njs> contains a mix of edits from both sides, some conflicts, whatever, and
> njs> has both GOODREV and BADREV as parents:
> njs> $ mtn merge -r BADREV
>
> My (test) usage of merge so far as resulted in the result being
> automatically committed. Is the above usage different?
Yes, that's what I meant about overhauling the merge UI :-). We're
working on implementing in-workspace merge now, which should make a lot
of these things more ergonomic. At the moment, merges just go through
memory straight to the database, like you said. (Mostly because when
we wrote the original merge code, we were still working out how to make
sophisticated merging work at all, and the code was not clever enough
to continue after it found a conflict. A few versions back we rewrote
all of that and the new merge code is so shiny you can see your face in
it, but rewriting the UI to take advantage of the new capabilities is a
volunteer-time-bound process :-).)
> Which brings me to a dislike of monotone, so often I see changelog messages like:
>
> merge of '646c7d2cca0e8dd0e97e0d8f3cc73e5a3d39ab8d'
> and '87c207f4bee6cecb68e9bcc9c65c37a93c99934c'
>
> Unfortunately, this doesn't tell me anything.
>
> A manual description would be better, for example:
>
> Merge in bug fix for XYZ into main tree.
Well, in the case you cite, the manual description would always be
"merge up trunk" or something like that :-). The case where you're
merging in a bug fix, you'd currently get an automated message like:
propagate from branch 'com.foo.bugfix-blah' (head
9edbe23f03c59cf5da6991d11ba42a364edf9ef6)
to branch 'com.foo' (head
1efba07ddd208be868a707565a26b6743d8b21ea)
which is at least somewhat better, especially without the forced line
breaks that google groups inserts (though it could still certainly be
improved).
When merges go through the workspace, of course, then the same commit
message machinery will be used for merges as for normal commits.
(Probably with the default being something useful, so you don't have to
type "merge up trunk" over and over again.)
> I believe the parents are already mentioned in the revision anyway.
Yes, it is a bit silly.
> I have tried to register with the BTS
> https://savannah.nongnu.org/account/register.php, so I can send bug
> reports, however, so far all I get is errors. Is it possible to check
> if I have already registered?
I stuck your name in the search box and it didn't return anything :-).
Other than that I'll have to shrug and suggest you talk to the savannah
admins.
monoton...@nongnu.org is a reasonable place for bug reports too.
> njs> Throw away all local changes, to make my local directory tree exactly
> njs> match that seen in GOODREV (but this only effects directory contents,
> njs> it doesn't affect the parent links):
> njs> $ mtn revert -r GOODREV
>
> Have to keep that command in mind - was wondering about it.
Note that the "-r" switch to revert is also a todo, not actually
supported at this moment. But the actual functionality is easy to add,
it's just blocked on making our "apply arbitrary changes to a
workspace" logic more robust against all the crazy states workspaces
can get into; revert needs to be totally bulletproof, so it currently
uses a hand-coded and less flexible approach. Again, just one of those
volunteer-bound processes.
> njs> Commit my new tree, with its two parents:
> njs> $ mtn commit --message="Clobbered BADREV"
>
> Looks good.
Cool. :-)
-- Nathaniel
n...@pobox.com
> Brian May wrote:
> > Unfortunately, this doesn't tell me anything.
>
> True. I also would prefer --no-merges to be the default. I think you
> can make it so in your ~/.monotone/monotonerc, but I haven't looked
> into it yet.
We made --no-merges the default for a while on mainline (and even in
one of the 0.26 pre-releases) and it caused lots of complaints, so we
put things back again. It looks like the real solution is to make the
log output for merges more useful. E.g., here is some discussion of
another possible format for log output:
http://article.gmane.org/gmane.comp.version-control.monotone.devel/8344
There isn't any way to make --no-merges the default ATM.
> > A manual description would be better, for example:
> >
> > Merge in bug fix for XYZ into main tree.
>
> For non-trivial merges where you resolve conflicts in a creative way,
> this may indeed be nice to have. Maybe "merge_into_dir" or
> "explicit_merge" already allow you to add your own comments?
>
> But in the case of trivial, automated merges, such functionality would
> in fact be harmful. For one thing, a trivial merge guarantees that you
> have all the changes from both parents and nothing else; so a custom
> comment would not say anything useful.
Not all automated merges are trivial -- "land backend rewrite" might
describe a clean merge that required no human intervention, yet that
description is useful :-).
Simple merging up of a branch, the sorts of merges that in cvs would
just be done with "cvs up" and have their existence thrown away
entirely, there isn't much that can usefully be said about them. There
a default commit message is definitely useful.
-- Nathaniel
n...@pobox.com
> Somewhere in the documentation I read something along similar lines
> (deleting a certificate is not possible unless there are no child
> revisions).
Probably you are thinking of the command to delete a revision, "db
kill_rev_locally", which does have such a comment in its documentation.
There is no command to delete an individual cert, though you can do it
straightforwardly enough with another of the "open up the hood and
start smashing stuff" commands, "db execute" :-).
> However, this really confuses me.
>
> I thought a revision only had the SHA1 of the previous revision
> details. Each certificate contains the SHA1 of the revision, not the
> other way around.
Correct.
> As such, surely it should be OK to add as many different certificates
> as you want without any of the certificates effecting any subsequent
> revisions. Or delete certificate too, although they may get copied
> back again.
Yes.
> In which case, wouldn't both merge's still result in the same
> revision-id, but with two set of certificates?
Exactly right.
> Or have I misunderstood something?
Nope.
-- Nathaniel
(hoping actual group regulars will forgive any inappropriate
off-topicness...)