The Linux Kernel Project Management System (INITIAL PROPOSAL)
Jordan Mendelson
This is just an initial proposal and some basic design information. I feel
that it is necessary that kernel development evolve past the current model.
With all the growth of Linux, more and more kernel related problems will arise
along with more and more kernel developers and soon I feel the source will be
unmanagable.
Questions and comments are welcome.
-----------
Linux Kernel Project Management System
by Jordan Mendelson (jo...@wserv.com) 19990927
Overview
The Linux kernel is unlike most open source projects. Developers of the kernel
source do not have the direct ability to write to the tree. Instead, patches
must be submitted to the maintainers and undergo a peer review process. This
system of management works very well for large projects with a large number of
developers.
The official process for patch submission is to test your patch on your
machine, have your patch tested by several other people and then finally
submit your patch to the maintainer. This provides a good method for peer
review, but all too often patches are submitted directly to Linus or Alan with
little or no peer review process.
Bug reporting and tracking is done by hand. Bug reports are submitted to the
maintainers or to the linux-kernel mailing list containing information about
the crash or deadlock. These reports almost always are missing information
necessary to diagnosing the problem. It is also difficult to track trends
where people with the same hardware are all experiencing similar trouble. A
bug tracking system would provide a good method for doing this.
The Linux kernel source is not currently maintained in an official central
repository like other open source projects. CVS has become a very popular
method of distributing and maintaining up-to-date versions of project source
on various developers' machines. Unfortunately, CVS lacks the functionality
necessary to maintain changes as separate entities and a simple method of
distributing source among several repositories. The alternative solution is
written by Peter Miller and is known as Aegis. Aegis supports all the features
required for large project development including distributed repositories,
change sets, and simultaneous active branches.
Documentation about internal kernel changes is sparse at best. During some of
the larger changes in the 2.3.x series, the only documentation on these
changes was scattered posts on the linux-kernel and other mailing list.
Several features of the kernel source have not been exploited due to lack of
documentation; people simply don't know they exist. With a management system
in place, a self-documenting system of changes would provide a method of
viewing these sort of changes in a patch-by-patch basis. This is not an ideal
system for documenting the kernel, but it is better than what currently
exists.
Detail
The idea is to create a single point where a developer or user could report
bugs, view bug reports, submit patches, comment on existing patches, and
receive information about synchronizing their kernel source directly with one
of the repositories.
In my opinion, the best method of doing this would be web-based. Viewing
patches and source online with the aid of tools such as LXR has proven
invaluable.
Access to the Linux Kernel Project Management System should be limited with
accounts. Read-only access should not require any type of account and free
access should be given in this situation. It would provide you with the
ability to synchronize your local source tree with a repository, view
unofficial patches, and view bug reports.
User accounts would be required for anyone needing the ability to submit
patches, comment on bug reports or existing patches.
Because Linus and Alan are the only ones to my knowledge that can directly
release an official kernel tree, it may be acceptable to provide direct Aegis
write access for them allowing direct manipulation using the Aegis
command-line tools.
The bug reporting and tracking system can be a modified version of Bugzilla.
Bugzilla has proven itself to be a very adaptable and usable system. The bug
reporting system should allow bug reporting directly from the console without
a web browser via an automated program that would gather system information,
however the bulk of bug reports should be done with a web browser. The bug
reports should contain all critical information including system components,
crash information, etc.
The peer review system should be web based. The web provides an efficient
means of managing something as complex as the Linux kernel. Patches should be
sorted into individual sets, new patches, accepted patches, and denied
patches. You should be able to comment on the patches. Patch comments should
be delivered via email by the author if requested. It should be as tightly
integrated with Bugzilla as possible as Bugzilla already implements most of
the features required for the peer review system.
Jordan
--
Jordan Mendelson : http://jordy.wserv.com
Web Services, Inc. : http://www.wserv.com
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majo...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
Larry McVoy
> distributing source among several repositories. The alternative solution is
> written by Peter Miller and is known as Aegis. Aegis supports all the features
> required for large project development including distributed repositories,
> change sets, and simultaneous active branches.
Cool. So explain to us how Aegis handles the following:
. disconnected operation with updates to all repositories
. undo - including things you don't do but incorporated
. multiple views, each of which include some subset of the others
. compressed repositories
. disk and/or file systems which corrupt the underlying files
. low bandwidth links
--
---
Larry McVoy l...@bitmover.com http://www.bitmover.com/lm
Jordan Mendelson
>
> Check out http://www.bitkeeper.com
>
> It's continuing effort is to facilitate large scale projects like The
> Kernel...
I actually did take a look at it a while back, but I sort of like Aegis better
because it allows a push (and pull) model of source distribution and
distributed repositories.
Also Aegis is under GPL and has over 8 years of development behind it.
Jordan
--
Jordan Mendelson : http://jordy.wserv.com
Web Services, Inc. : http://www.wserv.com
-
Shawn Leas
It's continuing effort is to facilitate large scale projects like The
Kernel...
-Shawn
-----------
Overview
distributing source among several repositories. The alternative solution is
written by Peter Miller and is known as Aegis. Aegis supports all the
features
required for large project development including distributed repositories,
change sets, and simultaneous active branches.
Documentation about internal kernel changes is sparse at best. During some
Detail
Larry McVoy
> Shawn Leas wrote:
> > It's continuing effort is to facilitate large scale projects like The
> > Kernel...
>
> because it allows a push (and pull) model of source distribution and
> distributed repositories.
BK has a distributed repository model with more than 8 years of proving
at Sun.
Furthermore, it seems that the push/pull model is supported by BK (in
fact, that's the only model supported):
$ bk help pull
USAGE
bk pull [-iqv]
The pull command is used to update your repository as painlessly as
possible. The default behaviour is to pull changes from your parent
repository and apply them with the minimal amount of human interaction.
If there are no over lapping changes (on a per line basis, not a per
file basis), then the changes are applied automatically. This may
not always produce the semantically correct result.
OPTIONS
-i force the resolution of changes to be interactive, do not
apply changes automatically
-q be quiet
-v be verbose (these stack, the more you do the more verbose it
gets, the default -is -vv)
pull also accepts all options valid for resync.
SEE ALSO
bk help parent
bk help pull
bk help resync
$ bk help push
USAGE
bk push [-qv]
The push command is used to update your parent repository with
your work. The push command will never create conflicts in the
parent repository, instead it first does an implicit pull to make
sure that your repository is a superset of your parent. If there
was new work in the parent, the push command will exit, to give
you a chance to rebuild your tree and verify that the pull worked.
If there is no new work in the parent, then the push command
updates the parent and applies the changes.
OPTIONS
-q be quiet
-v be verbose (-vv, -vvv, ... for even more detail)
push also accepts all options valid for resync.
SEE ALSO
bk help parent
bk help pull
bk help resync
Cheers,
--lm
Jordan Mendelson
>
> On Mon, Sep 27, 1999 at 04:32:13PM -0400, Jordan Mendelson wrote:
> > written by Peter Miller and is known as Aegis. Aegis supports all the features
> > required for large project development including distributed repositories,
> > change sets, and simultaneous active branches.
>
>
> . disconnected operation with updates to all repositories
> . undo - including things you don't do but incorporated
> . multiple views, each of which include some subset of the others
> . compressed repositories
> . disk and/or file systems which corrupt the underlying files
> . low bandwidth links
Peter Miller (the author of Aegis), submitted a proposal in March which
provides a much better overview of Aegis. What I provides was meant to glue
the different peices of a source management backend, a bug tracking system and
a peer review system to give people a better grasp on how project management
should work.
Aegis is open source and doesn't seem to have the licensing restrictions of
bitmover (unless the current licensing on the bitmover web site isn't
current). Take a look at it, see how it compares. I have found it to be a
pretty complete management system.
Anyway, his (long) post is available on MARC:
http://www.progressive-comp.com/Lists/?l=linux-kernel&m=92241232516116&w=2.
Here is the summary:
Summary for the Impatient
Source management is not enough. The Linux kernel is more than
the aggregation of its source files. A tool which supports the
the software development process for large teams is required.
Aegis supports large teams and large projects. Aegis is designed
around change sets. Aegis is designed around repository security
(availability, integrity and confidentiality). Aegis' distributed
development uses this existing mature functionality to keep two
or more repositories synchronized.
Aegis supports multiple repositories, multiple lines of
development, multiple distributed copies of repositories,
disconnected operation, and is security conscious.
Aegis is licensed under the GNU GPL.
Aegis is mature software. It is 8 years old. It has users all
around the world. It is actively being maintained and enhanced.
Aegis is easy to use. Is -is- big, it -does- have a lot of
functionality, but the essential process can be learned in less
than a day.
Aegis is available from
http://www.canb.auug.org.au/~millerp/aegis/
Please download it, plus one of the template projects, to get a
feel for the environment. If you would like more information,
there is also a Reference Manual and User Guide available from
the same place.
Jordan
--
Jordan Mendelson : http://jordy.wserv.com
Web Services, Inc. : http://www.wserv.com
-
Peter Miller
Larry McVoy writes:
> On Mon, Sep 27, 1999 at 04:32:13PM -0400, Jordan Mendelson wrote:
> > distributing source among several repositories. The alternative
> > solution is written by Peter Miller and is known as Aegis.
> > Aegis supports all the features required for large project
> > development including distributed repositories, change sets,
> > and simultaneous active branches.
I posted a proposal about this a few months ago.
(Fri 26 Mar 1999 "Proposal: Aegis to manage Linux kernel development")
It went into some detail, folks may wish to read the article from
the archives, and refresh their memories (that way this reply is
kept smaller).
Please note that Aegis models the change lifecycle, it isn't just
a file babysitter. It sets out to do a LOT more than simple file versioning.
> Cool. So explain to us how Aegis handles the following:
>
> . disconnected operation with updates to all repositories
You have a local copy, of course (just like folks do now, just like
any other tool requires). Except, instead of hammering on it
directly, you work in a change set.
> . undo - including things you don't do but incorporated
I'm assuming here that Larry is refering to change sets, or parts
thereof, which arrived from a "foreign" source and have subsequently
been incorporated into the repository. A rather artificial
distinction from change sets produced by other local staff.
Aegis doesn't have uncomditional commit/incorporate. However, if
you did decide to incorporate a "foreign" change set, or any other
change set for that matter, and later wish to remove it, you create
another change set, add the historical versions of the relevant
file (i.e. before the unwanted change) and proceed as normal.
Pretty boring stuff, really, same as all systems do.
Note: Aegis *doesn't* magically whisk the change set out of the
sequence, and reproject the repository. Repository transactions,
like any database transaction, are NOT COMMUTATIVE in the general
case. Aegis requires that all changes be validated, including
undoing.
> . multiple views, each of which include some subset of the others
Each branch is "overlayed" on its parent branches. Files which
are not modified by a branch are inherited from the parent branch,
so if the parent branch changes, these changes "propagate" into
the child. Various "insulation" schemes are possible on top of
this basic mechanism.
For multiple views, use multiple branches - without insulation.
> . compressed repositories
The history tool is decoupled from Aegis. You can use whatever
you wish. You can put a compressing wrapper shell script around
RCS or SCCS if that is what you need. Aegis doesn't care, just so
long as it can get the info back out again.
There is an option, but it defaults to off, to compress Aegis'
database files. This is transparent. It typically is much smaller
than the history files, so the savings are much smaller, in
comparison.
With the cost of disk plummeting faster than developers can crank
out code, the extra CPU overhead rarely seems worth it. But if you
want to, you can.
> . disk and/or file systems which corrupt the underlying files
This is Larry's hobby horse. Aegis leaves operating system
reponsibilities with the operating system. If your file system
corrupts files, it has a BUG. Fix the OS, not all of the apps
which -> depend <- on it. Sheesh.
> . low bandwidth links
Aegis decouples the transmition of the change set. You can use
just about anything: email, ftp, web, sneaker not, just tell Aegis
the command you want executed. BUT: that means Aegis doesn't know
or care about the bandwidth of your link. It does, by default,
gzip|mpack the change set, but you can turn either or both off
(e.g. no mpack when I'm distributing using FTP.)
Also, if you want to use PGP, add it to the command you tell Aegis
for how to send change sets.
Aegis supports a huge variety of distribution topologies: it provides
a change set, you distribute it however you want. Aegis supports
whatever granularity you want: you can distribute individual change
sets, aggregated change sets (there is a hierarchy, pick the level),
or the whole shebang; you choose.
Two *really* big deals:
1. The reception mechanism includes trojan detection.
Well: detection of potential trojan vectors: things like the
change set modifies the Makefile (you build you get infected)
or shell sripts (you run you get infected) or test scripts (you
test you get infected). This list of vector filename patterns is
per-project configurable.
If a potentian trojan file is present in a change set, Aegis
stops the incorporation of the change set earlier than usual,
to be checked by a human before it builds or tests (usually,
Aegis automatically builds it and tests it, and leaves is waiting
for a human reviewer).
(You can also turn it off trojan check in your procmail filter
script, if, say, it come from Linus and has a valid PGP signature.)
2. No UNCONDITIONAL commits. Ever. The whole commit portion of
other tools is broken into pieces, with validation prerequisites.
Aegis' goal is to keep the repository working (see the Aegis User
Guide for more on this, it's long). That means *no* validation
exceptions - not even when stuff arrives from Linus Himself.
No way any incoming patch from IDontKnowYou is going into my
repository without any preconditions! (Really simple preconditions
like "the repository still builds" and "the repository still
tests out OK")
3. Aegis is open source. Bitmover is ajar source.
Regards
Peter Miller E-Mail: mil...@canb.auug.org.au
/\/\* WWW: http://www.canb.auug.org.au/~millerp/
Disclaimer: The opinions expressed here are personal and do not necessarily
reflect the opinion of my employer or the opinions of my colleagues.
Larry McVoy
> Please note that Aegis models the change lifecycle, it isn't just
> a file babysitter. It sets out to do a LOT more than simple file versioning.
That's nice. Are you implying that everyone else is just doing RCS?
BK certainly does more than that, you get full changesets, complete
reproducability in a distributed environment, etc. So what's your
point?
> > Cool. So explain to us how Aegis handles the following:
> >
> > . disconnected operation with updates to all repositories
>
> You have a local copy, of course (just like folks do now, just like
> any other tool requires). Except, instead of hammering on it
> directly, you work in a change set.
What if I don't want to do that?
> > . undo - including things you don't do but incorporated
>
> Aegis doesn't have uncomditional commit/incorporate.
Pity for it. BK does. BK is a strict superset of Aegis in this respect.
With the difference being that we don't encode our policy into the
mechanism. Even though I happen to agree, in principle, with the policies
which Aegis enforces, that doesn't mean that they are right. Sticking
your meethodology in my face isn't likely to get me to use your system.
I want to make that choice for myself. Which is what BK lets you do.
> However, if
> you did decide to incorporate a "foreign" change set, or any other
> change set for that matter, and later wish to remove it, you create
> another change set, add the historical versions of the relevant
> file (i.e. before the unwanted change) and proceed as normal.
> Pretty boring stuff, really, same as all systems do.
Bzzt. It's a one line command in BK. Once again BK is a superset.
We can do the anti-changeset as you describe (and generate it
automatically, without adding anything to the size of the repository,
try that in RCS), or we can just make the changeset completely
disappear. Your choice.
> Note: Aegis *doesn't* magically whisk the change set out of the
> sequence, and reproject the repository. Repository transactions,
> like any database transaction, are NOT COMMUTATIVE in the general
> case. Aegis requires that all changes be validated, including
> undoing.
How nice for it. Wonderful in theory, horrible in practice. Once
again, sticking your methodology in my face.
Look, I have strong opinions too, opinions which are quite similar
to yours, Peter. But they are OPINIONS, not facts. We would both
like to believe that our way is the only correct way but we are
bothh full of shit. The world likes making up its own mind and a
tool which tries and be Daddy just isn't going to be accepted.
Especially by a crowd of people who for 8 years have managed to
do better than anyone in the history of computing using nothing
but tar. Kinda makes your methodology look a little silly by
comparison. Not wrong, just silly.
> > . multiple views, each of which include some subset of the others
>
> Each branch is "overlayed" on its parent branches. Files which
> are not modified by a branch are inherited from the parent branch,
> so if the parent branch changes, these changes "propagate" into
> the child. Various "insulation" schemes are possible on top of
> this basic mechanism.
Great. I have 1000 different branches, with 10,000 different changes.
I want to include some subset all changes from all branches in each
branch. How big is your revision history file? About 9,000 times
bigger than a BK file doing the same thing.
> > . compressed repositories
>
> The history tool is decoupled from Aegis. You can use whatever
> you wish. You can put a compressing wrapper shell script around
> RCS or SCCS if that is what you need. Aegis doesn't care, just so
> long as it can get the info back out again.
That's nice. It's also completely unreasonable. There are a number
of operations which have to plow through the metadata to do anything.
BK takes the approach of gzipping the non-meta data portion, while
leaving the metadata uncompressed. We get space savings and performance.
Gzipping the whole mess would be completely unreasonable for performance.
> There is an option, but it defaults to off, to compress Aegis'
> database files.
Nifty. Database files next to my repository files. What happens
when the DB gets corrupted?
> > . disk and/or file systems which corrupt the underlying files
>
> This is Larry's hobby horse. Aegis leaves operating system
> reponsibilities with the operating system. If your file system
> corrupts files, it has a BUG. Fix the OS, not all of the apps
> which -> depend <- on it. Sheesh.
It is a _fact_ that disks go bad and that file systems can corrupt
files. It is a _fact_ that billons of dollars of IP are put into
source control systems. And you think it is OK to just say "fix
the OS"? Bully for you but hell would freeze over before I'd let
you near my source.
Any source system which doesn't detect corruption immediately should
be tossed out the door. My personal feeling is that BK should be
more paranoid about this, not less.
> > . low bandwidth links
>
> Aegis decouples the transmition of the change set. You can use
> just about anything: email, ftp, web, sneaker not, just tell Aegis
> the command you want executed. BUT: that means Aegis doesn't know
> or care about the bandwidth of your link. It does, by default,
> gzip|mpack the change set, but you can turn either or both off
> (e.g. no mpack when I'm distributing using FTP.)
Nicely side stepping the question. If I have a changeset which has
N bytes of data changed, where N is measured as "diff -n | wc -c"
of the changeset, how much data does aegis send across the wire to
propogate that change?
And how much data was sent across the wire to figure out what to
send?
And while we're at it, suppose I have 2 repositories with 100,000
files in them. Suppose that they are uptodate. How long does it
take Aegis to figure that out?
> 2. No UNCONDITIONAL commits. Ever.
Jeeze. BK is quite happy to stick whatever you ask it into a
repository. The reason that it is happy to do so is because it
can undo that change with a single command.
> No way any incoming patch from IDontKnowYou is going into my
> repository without any preconditions! (Really simple preconditions
> like "the repository still builds" and "the repository still
> tests out OK")
And what, pray tell, is wrong with optimizing for the common case, which
is the changeset is just fine? It seems to me that you are making your
users do extra work that they don't need to do.
> 3. Aegis is open source. Bitmover is ajar source.
BitKeeper is as open as it can be. You get source, you get to wack
it and redistribute it for free. The license isn't as open as you
want but that's because we need to make money in order to support BK
and move on to developing the next generation of tools: bug tracking,
binary file revisioning, project management, etc. It's wonderful that
your tool is open source, but I'm a little concerned with your roadmap.
I'd really like to know when it will solve the same set of problems BK
can solve (making it a clearly better product), and when you are going
to integrate bug tracking in, mailing list and web server support, etc.
And how are you going to pay for all that development?
These are serious questions. If you have a good set of answers, I'd just
love to hear them. But if you don't, if you aren't going to solve 100%
of the problem and give people the assurances that they need that Peter,
Inc. is going to stand behind Aegis for the next 20 years, then why are
you here? Don't you think it is a little unfair to get people interested
in infrastructure that you have no intention of supporting at commercial
levels? What happens when you get interested in something else? I watch
the Aegis source tree and I'm not seeing this huge wave of development
coming in from the free software crowd, so that means I need to depend
on you. Are you going to make this work on all platforms? Where's the
NT, Windows/98, Mac, etc., ports? Etc.
--
---
Larry McVoy l...@bitmover.com http://www.bitmover.com/lm
-
Edward S. Marshall
> But if you don't, if you aren't going to solve 100% of the problem and
> give people the assurances that they need that Peter, Inc. is going to
> stand behind Aegis for the next 20 years, then why are you here?
No offense, but hearing you politely tell him to fuck off makes my skin
crawl, especially because noone here has any assurance that -YOU- will
stand behind Bitmover for the next 20 years. Businesses go under, are
acquired, and have their focus change.
> Don't you think it is a little unfair to get people interested in
> infrastructure that you have no intention of supporting at commercial
> levels? What happens when you get interested in something else?
Do you think any of us here care? Many of us are here, supporting a
non-commercial project, at non-commercial levels. I think you just
insulted most of us.
> I watch the Aegis source tree and I'm not seeing this huge wave of
> development coming in from the free software crowd, so that means I
> need to depend on you.
Get a few core developers here behind it (which would happen by necessity,
since people here would need improvements and modifications too), and I
see this as a very insignificant problem.
> Are you going to make this work on all platforms? Where's the NT,
> Windows/98, Mac, etc., ports? Etc.
Refer to my previous comment about caring. I doubt many people doing
serious Linux development give a damn if there's a Windows 98/NT/2000
port.
Or did you miss the original subject (you know, that part about a -Linux-
project management system?), and choose to simply plow on with your
marketing spiel whilst belittling someone who produces a competing product
to yours for free?
--
Edward S. Marshall <emar...@logic.net> [ What goes up, must come down. ]
http://www.logic.net/~emarshal/ [ Ask any system administrator. ]
Larry McVoy
> On Mon, 27 Sep 1999, Larry McVoy wrote:
> > But if you don't, if you aren't going to solve 100% of the problem and
> > give people the assurances that they need that Peter, Inc. is going to
> > stand behind Aegis for the next 20 years, then why are you here?
>
> No offense, but hearing you politely tell him to fuck off makes my skin
> crawl, especially because noone here has any assurance that -YOU- will
> stand behind Bitmover for the next 20 years. Businesses go under, are
> acquired, and have their focus change.
Yes, businesses do go under. Look at Cyclic. We've spent more on BK
development than they made in their entire history.
If _you_, my friend, and all of your buddies had been throwing money at
Cyclic, then perhaps they would have had the resources to solve the hard
problems (and believe me, they are hard).
CVS has had 10 years or so to get it right and they didn't. And you didn't
help, in spite of your assertions that it is "a very insignificant problem".
> > Don't you think it is a little unfair to get people interested in
> > infrastructure that you have no intention of supporting at commercial
> > levels? What happens when you get interested in something else?
>
> Do you think any of us here care? Many of us are here, supporting a
> non-commercial project, at non-commercial levels. I think you just
> insulted most of us.
Uh huh. Sure.
> > I watch the Aegis source tree and I'm not seeing this huge wave of
> > development coming in from the free software crowd, so that means I
> > need to depend on you.
>
> Get a few core developers here behind it (which would happen by necessity,
> since people here would need improvements and modifications too), and I
> see this as a very insignificant problem.
I'm sure you do. Perhaps that is some indication of your incredible
grasp of the difficulties involved.
> > Are you going to make this work on all platforms? Where's the NT,
> > Windows/98, Mac, etc., ports? Etc.
>
> Refer to my previous comment about caring. I doubt many people doing
> serious Linux development give a damn if there's a Windows 98/NT/2000
> port.
Yes they do. That Windows garbage will generate the revenue to make the
product better and to make other products. All of which the Linux
community gets for free. So it's true they don't care directly, but
somebody has to do the work and if Windows ends up paying to make Linux
get better tools, are you seriously going to suggest that people are
going to be unhappy with that?
> Or did you miss the original subject (you know, that part about a -Linux-
> project management system?), and choose to simply plow on with your
> marketing spiel whilst belittling someone who produces a competing product
> to yours for free?
I'm not belittling anyone. I'm holding Peter accountable, just as he would
hold me accountable if the situation were reversed. If he wants to step
up to the plate with a real solution, why hasn't he? I'm not the guy who
started this thread, nor am I the guy with the market spiel. I was happily
working on code when this started and I'll be hapily working on code when
it is done.
Look, I don't really care who solves problems, I only care that they
get solved. If I thought that Aegis could do the job, I would have
encouraged its use myself. And that is still the case. If Peter could
produce a product which Linus thinks solves the problems, well, hell,
that frees me and my team up to go do something else. Like clusters,
which are a ton sexier than source management. But I'm sick to death
of people saying "Oh, this system could solve the problem" when that
isn't true.
Just out of curiousity, Edward, what's the biggest project you've managed
under Aegis? What are the advantages and disadvantages of the system in
your experience?
--
---
Larry McVoy l...@bitmover.com http://www.bitmover.com/lm
-
Nat Lanza
allowed to even consider systems other than the much-hyped and
not-so-much-seen BitKeeper, so pitching this kind of fit at
Peter Miller is a bit presumptuous.
Yes, you think you have a great product. Good. It'd really be a
problem if you didn't. But you have to accept that not everyone will
be immediately convinced, especially since very few of us have ever
seen it.
Larry McVoy <l...@bitmover.com> writes:
> It is a _fact_ that disks go bad and that file systems can corrupt
> files. It is a _fact_ that billons of dollars of IP are put into
> source control systems. And you think it is OK to just say "fix
> the OS"? Bully for you but hell would freeze over before I'd let
> you near my source.
There's this invention you might be interested in. It's called
a "tape drive". It's pretty new, so I'll understand if you haven't
heard of it. It's pretty cool, though -- it actually lets you make
backup copies of your files in case something bad happens. If it does,
you can restore from tape, and just like magic your files are back!
You can even automate the use of one of these amazing things.
Sorry for the sarcasm, but he's right. Filesystem corruption is NOT
the problem of each individual application. If you stand to lose
billions of dollars worth of data by losing a disk, then perhaps you
should look into frequent backups.
> Any source system which doesn't detect corruption immediately should
> be tossed out the door. My personal feeling is that BK should be
> more paranoid about this, not less.
Sure, having an application detect corruption can be cool. But with a
reasonably stable operating system and a tiny bit of effort put into
backups, it's not even close to the showstopper you make it out to be.
My lab uses CVS for its work. We keep our repository in an AFS volume
that gets backed up every now and then. Sometimes CVS fucks up and
trashes something. When that happens, we send mail to the department's
computer facilities folks, and they restore the volume for us. Over
time, I lose far more data to my own idiocy than to CVS not handling
file corruption.
> BitKeeper is as open as it can be. You get source, you get to wack
> it and redistribute it for free. The license isn't as open as you
> want but that's because we need to make money in order to support BK
> and move on to developing the next generation of tools: bug tracking,
> binary file revisioning, project management, etc. It's wonderful that
> your tool is open source, but I'm a little concerned with your roadmap.
> I'd really like to know when it will solve the same set of problems BK
> can solve (making it a clearly better product), and when you are going
> to integrate bug tracking in, mailing list and web server support, etc.
> And how are you going to pay for all that development?
"clearly better product". Can I download BitKeeper _right now_? If
not, then it's not a clearly better product in any way. It's simply a
prettier cloud of vapor. Boast about how cool it is when it's
released, not before.
Also, at our lab we have a variety of projects. Some, like the NASD
code, end up being released as free software. Some, like some of our
video scheduling work or some of the VIA stuff we're doing, cannot be
released as free software due to various NDAs or patent restrictions
(grumble). Because of that, I'm not comfortable with putting my
changelogs on the web.
From the pricing on the chart at www.bitkeeper.com, it seems that
BitKeeper would cost me $136,000 to use for the roughly 40 grad
students, faculty, staff, and undergrads around here, using the "one
time buyout" plan. By contrast, Perforce would cost $1000/year, since
they have an educational discount. But even if they didn't, it'd be
$23,000 for the first year plus $4800/year of support after that.
$136,000 is a lot of money. Maybe it really is a great product, but
that's still a lot of money. Say I use a system for three years. By my
simple little math, I'd have spent $32,600 on Perforce by the end of
it and $136,000 on BitKeeper. That's over 4 times as much, so pardon
me if I continue to be interested other source control options.
> These are serious questions. If you have a good set of answers, I'd just
> love to hear them. But if you don't, if you aren't going to solve 100%
> of the problem and give people the assurances that they need that Peter,
> Inc. is going to stand behind Aegis for the next 20 years, then why are
> you here? Don't you think it is a little unfair to get people interested
> in infrastructure that you have no intention of supporting at commercial
> levels? What happens when you get interested in something else? I watch
> the Aegis source tree and I'm not seeing this huge wave of development
> coming in from the free software crowd, so that means I need to depend
> on you. Are you going to make this work on all platforms? Where's the
> NT, Windows/98, Mac, etc., ports? Etc.
If I use BitKeeper, what assurance do I have that BitMover will stand
behind it for 20 years? I don't want to be a doomsayer, but we've all
heard the statistics on how many startups make it. If the unfortunate
happens and BitMover tanks, what happens to me and my BitKeeper
license? Am I screwed?
The worst cases you're talking about for Aegis are exactly the same as
the worst cases for BitKeeper. Actually, they're better -- I may not
get much support, but at least I'm not out an eighth of a million
dollars.
So, basically, please take your marketing elsewhere. To take a common
phrase from the free software world, don't talk, show us the code.
--nat
--
nat lanza --------------------- research programmer, parallel data lab, cmu scs
ma...@cs.cmu.edu -------------------------------- http://www.cs.cmu.edu/~magus/
there are no whole truths; all truths are half-truths -- alfred north whitehead
Edward S. Marshall
> Sorry for the sarcasm, but he's right. Filesystem corruption is NOT
> the problem of each individual application. If you stand to lose
> billions of dollars worth of data by losing a disk, then perhaps you
> should look into frequent backups.
I wouldn't even look that far. Backups are for a last resort. If you have
mission-critical storage needs, parity and mirroring are a bare-minimum
requirement. And by your numbers below, it's quite a bit cheaper to have
arrays of mirrored disks and standbys compared to spending $136k on a
software license that still can't completely save you from a hardware
failure.
Application-level support is certainly a bonus, but I'd have a hard time
believing it's a requirement for the project being discussed here.
--
Edward S. Marshall <emar...@logic.net> [ What goes up, must come down. ]
http://www.logic.net/~emarshal/ [ Ask any system administrator. ]
-
Edward S. Marshall
> CVS has had 10 years or so to get it right and they didn't. And you didn't
> help, in spite of your assertions that it is "a very insignificant problem".
Nice wordplay. The insignificant problem I was speaking to was finding
developers to work on a project housing the source for the Linux kernel.
That's it. You'll never hear me state that source management is a simple
problem to solve. You're simply being argumentative.
From the sounds of things, Aegis provides a baseline set of functionality
that will be necessary for management of kernel revisions. So does yours.
So, for the task at hand, what directly-necessary features do you bring to
the table, that make up for your inability to carry on a conversation with
people without becoming insulting?
> Uh huh. Sure.
Is this the "supporting at commercial levels" that we can expect?
> > Refer to my previous comment about caring. I doubt many people doing
> > serious Linux development give a damn if there's a Windows 98/NT/2000
> > port.
>
> Yes they do. That Windows garbage will generate the revenue to make the
> product better and to make other products.
Once again (speaking only for myself), I don't give a damn if you make a
dime off your product. I work on open source projects for my own
amusement. If you happen to find a way to make money doing it, more power
to you. But don't expect me to care about your psuedo-open-source
product's bottom line. Especially because I haven't seen compelling
evidence posted to this forum to convince me that your product is a better
offering -FOR THE TASK AT HAND- than something else, and since your
product is vapor, I can't even check for myself. And your arrogance and
posturing certainly helps one make up their mind.
Considering that you're trying to sell this product to the people here
(even if it's free), maybe you should try a different marketing approach?
> All of which the Linux community gets for free.
Everything has a cost.
> So it's true they don't care directly, but somebody has to do the work
> and if Windows ends up paying to make Linux get better tools, are you
> seriously going to suggest that people are going to be unhappy with
> that?
No, I'm going to suggest that your profit-driven rationale is of little
importance to some people here. Most people just want a system that works;
justify it in terms of what it can do, and people will listen.
Justify it in terms of how bad everyone else's products are, or in terms
of how much money you need to make to support it, and at least a few of us
will find you to simply be petty.
> But I'm sick to death of people saying "Oh, this system could solve
> the problem" when that isn't true.
You'll note that I never once claimed that. I'm simply speaking to your
attack on a competing project, and one which is being held up to have a
similar feature spread to yours, with a license which is more palatable to
some of us.
> Just out of curiousity, Edward, what's the biggest project you've managed
> under Aegis?
None. I've neither used your product, nor Peter's; my source management
experience is via RCS, SCCS, and CVS, along with in-house tools. Again,
I've also never claimed that either system is better, nor have I claimed
expertise with either one. I've simply stated that you've belittled him
for even daring to speak on the subject regarding his offering because he
is not providing commercial support for the product.
A rational discussion is what I was hoping to see when the first posting
was made on the subject. In under 10 postings, it's degraded into a series
of insults, and most of them certainly seemed to eminate from you (Peter
certainly egged you on in a couple of places, but the reaction was far
from civil for this forum).
Nat Lanza
> I wouldn't even look that far. Backups are for a last resort. If you have
> mission-critical storage needs, parity and mirroring are a bare-minimum
> requirement. And by your numbers below, it's quite a bit cheaper to have
> arrays of mirrored disks and standbys compared to spending $136k on a
> software license that still can't completely save you from a hardware
> failure.
Parity and mirroring are nice, but they don't always save you. Our
machine room used to flood every now and then; it's hard to use parity
bits to recover from that.
But yeah, Disk Is Cheap(tm). A bunch of extra disks and a tape drive
will run less than $136k, and I'll sleep a whole lot better at night.
> Application-level support is certainly a bonus, but I'd have a hard time
> believing it's a requirement for the project being discussed here.
Right; there's no off-site backup system quite like mirroring your
code to dozens of servers across the world and sending patches to a
mailing list with thousands of readers.
--nat
--
nat lanza --------------------- research programmer, parallel data lab, cmu scs
ma...@cs.cmu.edu -------------------------------- http://www.cs.cmu.edu/~magus/
there are no whole truths; all truths are half-truths -- alfred north whitehead
-
Larry McVoy
[noise about not seeing BK yet]
There is a good reason for that. It will be released when I believe the
chance of it screwing up your data extremely unlikely. I actually feel
some responsibility - if BK screws up your tree, I fully expect you to
come screaming after me with sharp pointy objects.
> Larry McVoy <l...@bitmover.com> writes:
> > It is a _fact_ that disks go bad and that file systems can corrupt
> > files. It is a _fact_ that billons of dollars of IP are put into
> > source control systems. And you think it is OK to just say "fix
> > the OS"? Bully for you but hell would freeze over before I'd let
> > you near my source.
>
> There's this invention you might be interested in. It's called
> a "tape drive".
In RCS files, the way the file format is layed out, you'll never know
that a revision is bad unless you ask for it and discover that it is
bad. In other words, you can have the middle of a file (like one
block or sector) go bad, and you may not find out until a long time
later. When RCS processes the files, it does nothing to verify that
the old data is valid. So months, years, can go by before you need that
old release to support an important customer. And then those lovely
tapes which you made (you did make them, right?) may well be bad.
And you are suggesting that this is OK? Fine, go to the head of the
line of people I want nowhere near my data.
And another thing: while you and your buddies may not think that this
is a problem to worry about, you aren't exactly representative. If
people put money and/or effort into using a source management system,
don't they have the right to know that the vendor did something to make
sure that things wouldn't break?
> Sorry for the sarcasm, but he's right. Filesystem corruption is NOT
> the problem of each individual application. If you stand to lose
> billions of dollars worth of data by losing a disk, then perhaps you
> should look into frequent backups.
You'd have a point if RCS told you when there is a problem, but it doesn't.
You'd also have a point if this didn't happen in practice, but it does.
So you have no point. He's wrong. If you disagree, you can use a stupid
application to store your data but don't pretend that you speak for the
majority.
> Can I download BitKeeper _right now_? If
> not, then it's not a clearly better product in any way.
Yup, you can. You have to sign a beta agreement because I haven't written
down the license yet, but other than that, yes.
> Also, at our lab we have a variety of projects. Some, like the NASD
> code, end up being released as free software. Some, like some of our
> video scheduling work or some of the VIA stuff we're doing, cannot be
> released as free software due to various NDAs or patent restrictions
> (grumble). Because of that, I'm not comfortable with putting my
> changelogs on the web.
Great. I'm equally uncomfortable with you not paying for it in that case.
> Maybe it really is a great product, but
> that's still a lot of money. Say I use a system for three years. By my
> simple little math, I'd have spent $32,600 on Perforce by the end of
> it and $136,000 on BitKeeper. That's over 4 times as much, so pardon
> me if I continue to be interested other source control options.
Feel free. I have no interest in getting you to use something you don't
need. In fact, when people say "why should I use BK instead of XYZ" I
respond with "Is XYZ working for you? If so, you shouldn't". You'd be
amazed to know that 95% of the people say "No, it isn't working". But if
Perforce or CVS is working for you, why are you in this conversation?
If you're happy, you're happy. If not, and BK costs you less than the
problems cost you, that's a win.
> > These are serious questions. If you have a good set of answers, I'd just
> > love to hear them. But if you don't, if you aren't going to solve 100%
> > of the problem and give people the assurances that they need that Peter,
> > Inc. is going to stand behind Aegis for the next 20 years, then why are
> > you here? Don't you think it is a little unfair to get people interested
> > in infrastructure that you have no intention of supporting at commercial
> > levels? What happens when you get interested in something else? I watch
> > the Aegis source tree and I'm not seeing this huge wave of development
> > coming in from the free software crowd, so that means I need to depend
> > on you. Are you going to make this work on all platforms? Where's the
> > NT, Windows/98, Mac, etc., ports? Etc.
>
> If I use BitKeeper, what assurance do I have that BitMover will stand
> behind it for 20 years? I don't want to be a doomsayer, but we've all
> heard the statistics on how many startups make it. If the unfortunate
> happens and BitMover tanks, what happens to me and my BitKeeper
> license? Am I screwed?
Nope. As I have publicly stated before: if the openlogging.org domain
BK servers go away and stay away for an extended period (3-6 months),
and the company refuses (or is unable because they are gone) to maintain
them, then the software becomes GPLed.
> So, basically, please take your marketing elsewhere.
Hey, Nat, my man. Go back to the start of the thread and notice I didn't
start this, it wasn't my marketing, it was that Aegis guy who got the
ball rolling.
--
---
Larry McVoy l...@bitmover.com http://www.bitmover.com/lm
-
Larry McVoy
> No, I'm going to suggest that your profit-driven rationale is of little
> importance to some people here. Most people just want a system that works;
> justify it in terms of what it can do, and people will listen.
I refer you to my original response on the thread where I asked how Aegis
would solve a set of problems. So where are the answers?
> None. I've neither used your product, nor Peter's; my source management
> experience is via RCS, SCCS, and CVS, along with in-house tools. Again,
> I've also never claimed that either system is better, nor have I claimed
> expertise with either one. I've simply stated that you've belittled him
> for even daring to speak on the subject regarding his offering because he
> is not providing commercial support for the product.
That's simply not true. I took him to task because he doesn't have a
solution which solves the problems nor does he have a business model
which will give him the resources that it is reasonable to expect are
needed to solve the problem.
We had this discussion a year ago with the same conclusion.
Anthony Towns
> Bugzilla has proven itself to be a very adaptable and usable system. The bug
> reporting system should allow bug reporting directly from the console without
> a web browser via an automated program that would gather system information,
> however the bulk of bug reports should be done with a web browser. The bug
> reports should contain all critical information including system components,
> crash information, etc.
The Debian BTS [0] might be worth considering here too. It's used by at
least Debian, Gnome and KDE [1], and has both web (for viewing) and mail
(for manipulating and viewing) interfaces. Every bug gets forwarded via
email to the appropriate maintainer and a couple of mailing lists as
soon as its received, and added to the archives. It's GPLed for what
that's worth. There are a number of tools to make bug reporting and
viewing easier (reportbug.deb and bug.deb come to mind) and it's also
a fairly well tested and reliable system.
It's designed to handle lots of small programs though, rather than one
big project with lots of inter-related modules though.
> The peer review system should be web based. The web provides an efficient
> means of managing something as complex as the Linux kernel. Patches should be
> sorted into individual sets, new patches, accepted patches, and denied
> patches. You should be able to comment on the patches. Patch comments should
> be delivered via email by the author if requested. It should be as tightly
> integrated with Bugzilla as possible as Bugzilla already implements most of
> the features required for the peer review system.
Debian uses "wishlist" bug reports for this, more or less (bug fixes get
normal priority, security fixes get critical priority). For most bugs,
it works pretty well --- you can send a feature request, and add patches
/ comments or whatever. When the maintainer uploads a fixed version,
they close the report. Having the bug reports go to various mailing lists
also helps this.
Too much discussion really doesn't work that well though --- the important
points get lost in the discussion so if you just want to find out the key
points from some month old bug report, it becomes pretty painful. The
archived discussion also isn't threaded, which can be a pain in these
cases.
Cheers,
aj
[0] http://benham.net/debbugs/
[1] http://bugs.debian.org/, http://bugs.gnome.org/, http://bugs.kde.org/
--
Anthony Towns <a...@humbug.org.au> <http://azure.humbug.org.au/~aj/>
I don't speak for anyone save myself. PGP encrypted mail preferred.
``Like the ski resort of girls looking for husbands and husbands looking
for girls, the situation is not as symmetrical as it might seem.''
Steve Underwood
> On Mon, Sep 27, 1999 at 10:04:55PM -0400, Nat Lanza wrote:
>
> [noise about not seeing BK yet]
>
> [earnest comments about Larry's good will to all mankind]
>
> > Larry McVoy <l...@bitmover.com> writes:
> > > It is a _fact_ that disks go bad and that file systems can corrupt
> > > files. It is a _fact_ that billons of dollars of IP are put into
> > > source control systems. And you think it is OK to just say "fix
> > > the OS"? Bully for you but hell would freeze over before I'd let
> > > you near my source.
> >
> > There's this invention you might be interested in. It's called
> > a "tape drive".
>
> In RCS files, the way the file format is layed out, you'll never know
> that a revision is bad unless you ask for it and discover that it is
> bad. In other words, you can have the middle of a file (like one
> block or sector) go bad, and you may not find out until a long time
> later. When RCS processes the files, it does nothing to verify that
> the old data is valid. So months, years, can go by before you need that
> old release to support an important customer. And then those lovely
> tapes which you made (you did make them, right?) may well be bad.
I have mixed feelings about the features of every revision control system I have
seen, but here is a point I can fully concur with. This problem has bitten me with
both SCCS and RCS. Something nasty gets into the tree, and nobody notices. A year
later, when the problem is spotted (probably because the screwed up version is
needed for support purposes) its damned hard to figure out what went wrong. Its
also pretty messy cleaning up the trail of change upon change that will have
followed the original unfortunate incident. I consider this one of the significant
risk factors in using any revision control system.
By the way, aren't most tape backups WOMs?
Steve
Nat Lanza
> In RCS files, the way the file format is layed out, you'll never know
> that a revision is bad unless you ask for it and discover that it is
> bad. In other words, you can have the middle of a file (like one
> block or sector) go bad, and you may not find out until a long time
> later. When RCS processes the files, it does nothing to verify that
> the old data is valid. So months, years, can go by before you need that
> old release to support an important customer. And then those lovely
> tapes which you made (you did make them, right?) may well be bad.
>
> And you are suggesting that this is OK? Fine, go to the head of the
> line of people I want nowhere near my data.
No, I'm not. I'm not suggesting that no compensation for the fact that
disks and filesystems go bad is needed. What I'm saying is that that
compensation should not need to be application-level. If I need to do
a lot of checksumming and manual parity calculation and such in my
application, then either the OS author or whoever put together my
machine seriously fucked up somewhere. If my data gets silently
corrupted without me noticing for months or years, then there are
problems that no SCM can solve.
Also, I'm not claiming that having an SCM do extra stuff like
corruption checking is bad. It's pretty cool, actually. But you made
it sound like Peter should be run out of town and have small children
publically mock him for not building it into his system. That's not
just wrong, it's rude.
> And another thing: while you and your buddies may not think that this
> is a problem to worry about, you aren't exactly representative. If
> people put money and/or effort into using a source management system,
> don't they have the right to know that the vendor did something to make
> sure that things wouldn't break?
"you and your buddies"? "you aren't exactly representative"?
Larry, could you do me a favor? Could you be a little more
condescending? I almost got the impression for a second there that you
might possibly have the tiniest smidgeon of respect for someone who
might dare to disagree with you.
We have over five years worth of research code in our repository. I
think that's more than a little effort, and we're definitely worried
about protecting it. That's why we make sure it's regularly backed up
and we often burn snapshots to CDROM. We may be a little insignificant
research group and not big important Captains Of Industry, but we do
care about our code. Our sponsors, who _are_ Captains Of Industry,
also care about our code and us not losing it, so that's a little
extra motivation right there.
Also, if I buy BitKeeper, store lots of important code in it, and it
fucks up my repository, what can I do? I mean, I can whine about it a
lot in public, which might make me feel better, but can I do anything
that gets me my code back or provides me with actual compensation? Can
I get a refund? Can I sue you? Can I say "He did it! Blame him!" to my
boss or my sponsors? If the system breaks, then fundamentally I really
don't care whose fault it is, I just want my data back.
> You'd have a point if RCS told you when there is a problem, but it doesn't.
RCS shouldn't have to do this. If I need my data uncorrupted, I'd
better use mirrored disks with parity. If I might need data for months
or years, I'd better have offsite backups and burn important bits onto
CD in case a tape dies.
It's nice if my application helps me out there, but if I'm depending
on it doing so, then I deserve to lose my data when it fails. I'm sure
BK's integrity system is nice, but I think I'd prefer to have more
than a single piece of software to fall back on.
> You'd also have a point if this didn't happen in practice, but it does.
> So you have no point. He's wrong. If you disagree, you can use a stupid
> application to store your data but don't pretend that you speak for the
> majority.
Can you explain to me where you got this "Nat uses raw RCS with no
other safeguards" strawman from? I'd really love to know.
In practice, stuff breaks and overheats and catches fire and get
stolen and deleted and lost and water damaged and what have you. Will
BitKeeper protect me from that too? If so, cool, sign me up. If not,
pardon me for wanting to have safeguards other than some piece of
software.
> Great. I'm equally uncomfortable with you not paying for it in that case.
I'm glad to know our research isn't pure enough for you, Larry. Will
you admit that maybe all the people who complain that when you call
BitKeeper "open" you don't actually mean "open" have a point now, please?
> Feel free. I have no interest in getting you to use something you don't
> need. In fact, when people say "why should I use BK instead of XYZ" I
> respond with "Is XYZ working for you? If so, you shouldn't". You'd be
> amazed to know that 95% of the people say "No, it isn't working". But if
> Perforce or CVS is working for you, why are you in this conversation?
> If you're happy, you're happy. If not, and BK costs you less than the
> problems cost you, that's a win.
Funny, a few messages ago you seemed to be saying something totally
different -- it was more like "Why would you be stupid enough to use
Aegis when you could use BitKeeper instead, which will solve all your
problems and make toast for you in the morning?".
Part of the point here is that those of us outside of BitMover and the
beta groups really have no idea whether BK will solve ANY of the
problems that CVS, Perforce, ClearCase, Aegis, or whatever don't. You
say they will, but you haven't produced anything to back it up with.
You attack Peter for having the temerity to think that he's produced a
good system by waving your ridiculously expensive vaporware around,
and then when someone mentions a much cheaper product that a lot of
people are pretty happy with from an actual commercial company with
support, you get huffy. I'm just trying to point out that BitKeeper is
not the only game in town, no matter how much you love it. Yelling at
people will never convince them that BitKeeper is good. Only BitKeeper
itself can do that. Yelling will only make them think you're arrogant
and obnoxious, and wonder what kind of support you'll provide. That's
no help.
> > If I use BitKeeper, what assurance do I have that BitMover will stand
> > behind it for 20 years? I don't want to be a doomsayer, but we've all
> > heard the statistics on how many startups make it. If the unfortunate
> > happens and BitMover tanks, what happens to me and my BitKeeper
> > license? Am I screwed?
>
> Nope. As I have publicly stated before: if the openlogging.org domain
> BK servers go away and stay away for an extended period (3-6 months),
> and the company refuses (or is unable because they are gone) to maintain
> them, then the software becomes GPLed.
Right, so I'm at exactly the same place I would be if I used
Aegis. Since something Linux shows us that users can keep free
software alive and make it great, I'm not screwed. I'm okay. Except,
wait, no, I'm not. I'm also out an eighth of a million dollars.
Fuck, I guess I _am_ screwed.
> Hey, Nat, my man. Go back to the start of the thread and notice I didn't
> start this, it wasn't my marketing, it was that Aegis guy who got the
> ball rolling.
But you jumped into it and harangued him about it. "He started it" is
not and never has been a good excuse for this sort of thing, Larry.
Uh, I mean, "Larry, my man".
(hint: take the marketing hat off. talk about the technical
details. give us reasons other than "because I say so". accept that
others can be more than just mouth-breathing annoyances. we'll like
you for it. really. we might even buy your software.)
(especially if you consider adding an educational discount. education
is the future, y'know.)
--nat
--
nat lanza --------------------- research programmer, parallel data lab, cmu scs
ma...@cs.cmu.edu -------------------------------- http://www.cs.cmu.edu/~magus/
there are no whole truths; all truths are half-truths -- alfred north whitehead
-
Larry McVoy
On Tue, Sep 28, 1999 at 12:23:32AM -0400, Nat Lanza wrote:
> Also, I'm not claiming that having an SCM do extra stuff like
> corruption checking is bad. It's pretty cool, actually. But you made
> it sound like Peter should be run out of town and have small children
> publically mock him for not building it into his system.
If you say so. Regardless of how much you don't like the way it was
said, the point remains: source management systems which don't provide
integrity checks, when it is known that such checks are needed, especially
because without them the users won't know that the data is bad, those
source management systems are bad, bad, bad.
As to your argument that this doesn't need to be in the application,
I happen to think you are wrong. So does Red Hat, for that matter,
run rpm --verify lately? It's got the same sort of thing built in.
Perhaps you should start a crusade to get it removed, it's obviously
not useful according to, in your world nothing ever goes wrong.
> "you and your buddies"? "you aren't exactly representative"?
>
> Larry, could you do me a favor? Could you be a little more condescending?
Sorry about that, I'll try harder. Really, I promise.
> We have over five years worth of research code in our repository. I
> think that's more than a little effort, and we're definitely worried
> about protecting it. That's why we make sure it's regularly backed up
> and we often burn snapshots to CDROM.
Pick one of your backups and go extract every old version and try and
build it. Give us some status on whether they all worked. If you have
a shred of intellectual integrity (see, I'm trying harder), you'll admit
it when you find out you've been backing up corrupted data. And I'll
bet you 40 BK licenses that when you find out that there is bad data in
there, you can't find the right tape to correct it. Because remember,
you've got 5 years of backups and no way of know when in those five
years things got screwed up.
In 10 or more messages you've failed to get the point. Doing backups of
bad data is not very useful. How the hell do you know that you need to
restore from tape if the system never tells you that anything is wrong?
RCS / CVS store the latest version in clear text - every earlier version
could be completely corrupted and you'll cluelessly (through no fault of
your own) be backing up, mirroring, raiding, and CDROM burning COMPLETE
AND UTTER GARBAGE. Which you won't find out until you need to get an
_OLD_ version to fix a bug.
See the problem? Backing up corrupted bits is of no help. Having a
source management system which doesn't tell you when they are corrupted,
and has no means to tell you that, is completely insane and, yes, people
who advocate this should certainly NOT have any part in designing a
source management system. Sorry, I won't budge on that one. No sane
person would. No person with an ounce of responsibility would.
> If the system breaks, then fundamentally I really
> don't care whose fault it is, I just want my data back.
Exactly. So what are you going to do when you realize that it broke
5 years ago and that code, which you swore up and down was worthless,
you'd never need to go back to it, now you need it and you need it bad?
> > You'd have a point if RCS told you when there is a problem, but it doesn't.
>
> RCS shouldn't have to do this. If I need my data uncorrupted, I'd
> better use mirrored disks with parity.
You are at a good university; perhaps they have a copy of an old but
famous paper by Dave Clark on end-to-end error checking. Read it.
If you can't find it, I'll fax you a copy. Then think about how much
use those mirrored disks are going to be if the page cache is busted and
puts the wrong page in the file, or you have an undetected memory error,
or the bus corrupts, or any of the many other errors occur. Once again,
you're mirroring GARBAGE. And the source system, which could have warned
you about this, didn't.
> > You'd also have a point if this didn't happen in practice, but it does.
> > So you have no point. He's wrong. If you disagree, you can use a stupid
> > application to store your data but don't pretend that you speak for the
> > majority.
>
> Can you explain to me where you got this "Nat uses raw RCS with no
> other safeguards" strawman from? I'd really love to know.
You use CVS. You're advocating another system which uses the same broken
file format. In fact, I've just gone over the entire thread and 100%
of the places you mention something specific, it's RCS based. Forgive
me for listening to you, it won't happen again.
> > > If I use BitKeeper, what assurance do I have that BitMover will stand
> > > behind it for 20 years? I don't want to be a doomsayer, but we've all
> > > heard the statistics on how many startups make it. If the unfortunate
> > > happens and BitMover tanks, what happens to me and my BitKeeper
> > > license? Am I screwed?
> >
> > Nope. As I have publicly stated before: if the openlogging.org domain
> > BK servers go away and stay away for an extended period (3-6 months),
> > and the company refuses (or is unable because they are gone) to maintain
> > them, then the software becomes GPLed.
>
> Right, so I'm at exactly the same place I would be if I used
> Aegis. Since something Linux shows us that users can keep free
> software alive and make it great, I'm not screwed. I'm okay. Except,
> wait, no, I'm not. I'm also out an eighth of a million dollars.
Sheesh. If I had read this first, I wouldn't have bothered replying.
So you whine and whine and whine and then when I say that your recourse
is exactly what you have been holding up as the perfect answer, you
whine some more.
Jeeze, I'm sorry Nat. I didn't realize your importance. Let me get
Cindy Crawford on plane in the morning with 10,000 copies of BitKeeper
plus a check for $50,000,000 made out to you personally. Because I want
you to be happy. Really.
--
---
Larry McVoy l...@bitmover.com http://www.bitmover.com/lm
-
Nat Lanza
> Perhaps you should start a crusade to get it removed, it's obviously
> not useful according to, in your world nothing ever goes wrong.
You don't listen, do you? No matter how much I say "application-level
checking is good but you still need other measures", all you hear is
"application-level checking is bad". Look, I give up. Believe whatever
you want, okay? Don't let what I actually said get in the way or
anything.
> In 10 or more messages you've failed to get the point. Doing backups of
> bad data is not very useful. How the hell do you know that you need to
> restore from tape if the system never tells you that anything is wrong?
> RCS / CVS store the latest version in clear text - every earlier version
> could be completely corrupted and you'll cluelessly (through no fault of
> your own) be backing up, mirroring, raiding, and CDROM burning COMPLETE
> AND UTTER GARBAGE. Which you won't find out until you need to get an
> _OLD_ version to fix a bug.
Yeah, it's possible. But now we've moved from "totally screwed" to
"way inconvenienced because I have to play hunt-for-the-tape for a
long time". No, it's not optimal. I'm not claiming it is. I'm simply
claiming that it's possible to avoid being screwed without having your
version control system do lots of fancy error checking.
If I've failed to get your point, then maybe you've also failed to get
mine.
> > RCS shouldn't have to do this. If I need my data uncorrupted, I'd
> > better use mirrored disks with parity.
>
> You are at a good university; perhaps they have a copy of an old but
> famous paper by Dave Clark on end-to-end error checking. Read it.
> If you can't find it, I'll fax you a copy. Then think about how much
> use those mirrored disks are going to be if the page cache is busted and
> puts the wrong page in the file, or you have an undetected memory error,
> or the bus corrupts, or any of the many other errors occur. Once again,
> you're mirroring GARBAGE. And the source system, which could have warned
> you about this, didn't.
Question: Does BitKeeper prevent me from having garbage in my
repository? Completely? Or does it just make it less likely? I believe
it just makes it less likely, which is why I claim that backups and
disk arrays are still more important.
> Sheesh. If I had read this first, I wouldn't have bothered replying.
> So you whine and whine and whine and then when I say that your recourse
> is exactly what you have been holding up as the perfect answer, you
> whine some more.
You miss the point. Totally.
You claimed that Aegis was worse than BK because of support. You
claimed that since Peter didn't have commercial support in place,
users might have to maintain the code themselves if they wanted
support, and that your solution was better.
I pointed out that, in fact, your solution could be a lot worse, since
instead of losing support people can also have ended up throwing away
a lot of money for support that is no longer there.
In response, you whined and got insulting. Hardly a stellar example of
why I or anyone else should trust you for good support.
> Jeeze, I'm sorry Nat. I didn't realize your importance. Let me get
> Cindy Crawford on plane in the morning with 10,000 copies of BitKeeper
> plus a check for $50,000,000 made out to you personally. Because I want
> you to be happy. Really.
I'm glad you want me to be happy. I'd like you to be happy too.
If you'd really like me to be happy, maybe you should try keeping the
women and free product and cash and just start listening to people
without all your damn arrogance for once. I'm sick of it.
It's become clear to me over the past few messages that you really
don't care about what anyone who disagrees with you has to
say. They're all either fools or amateurs or deluded or in some way or
another idiots who should not be listened to. Only you have the
answer, and it is BitKeeper. You made it so the Linux kernel could use
it and love it, and by God we'd better all be grateful. You deserve
more than the insolence of dissenting opinions, you deserve
applause. So:
<clap> ... <clap> ... <clap>. huzzah!
Look, Larry, let's make a deal. I'll stop trying to discuss this issue
and you'll stop belittling me and telling me why anything I think is
inferior to what you think. I'll stop questioning your product, and
you'll stop hyping it, because by now I'm pretty sure I don't want
it. If you can be this arrogant and insulting to me for simply
disagreeing with you on a mailing list, I don't ever want to see what
happens if I've used your product and end up having a real problem
with it. I've tried to restrain myself, be reasonable, and keep the
discussion focussed on the issues at hand. Maybe I failed and went too
far to the side of insult, but I'd like to believe that at least I
tried to be nice about it. I don't believe you've made any such
effort, and I don't think you have any intention of doing so. As such,
I don't think I have anything more to say to you about this.
--nat
--
nat lanza --------------------- research programmer, parallel data lab, cmu scs
ma...@cs.cmu.edu -------------------------------- http://www.cs.cmu.edu/~magus/
there are no whole truths; all truths are half-truths -- alfred north whitehead
-
Nat Lanza
> Look, Larry, let's make a deal. I'll stop trying to discuss this issue
> and you'll stop belittling me and telling me why anything I think is
> inferior to what you think. I'll stop questioning your product, and
> you'll stop hyping it, because by now I'm pretty sure I don't want
> it. If you can be this arrogant and insulting to me for simply
> disagreeing with you on a mailing list, I don't ever want to see what
> happens if I've used your product and end up having a real problem
> with it. I've tried to restrain myself, be reasonable, and keep the
> discussion focussed on the issues at hand. Maybe I failed and went too
> far to the side of insult, but I'd like to believe that at least I
> tried to be nice about it. I don't believe you've made any such
> effort, and I don't think you have any intention of doing so. As such,
> I don't think I have anything more to say to you about this.
In retrospect, this was too harsh. I don't want to suggest that Larry
is in any way unprofessional or dishonorable in his dealings, and I
apologize for the implication that he might allow personal
disagreements to make him so.
Larry McVoy
> You don't listen, do you? No matter how much I say "application-level
> checking is good but you still need other measures", all you hear is
> "application-level checking is bad".
Sure I do and I heard you and Peter the first time. And I've been
in this argument a hundred times in a hundred different disciplines.
If you had read Clark's paper (don't they make people read that in OS
101 anymore?), you'd realize that what you are saying is like this:
if I'm going from point A to point Z through B, C, D, E ...
then if each link is safe, then the whole way is safe. I.e.,
if A<->B doesn't screw it up, and B<->C doesn't screw it up,
etc., then it follows that A<->Z doesn't screw it up.
And I am saying that's wrong (which is point of Dave's paper).
You have to do end to end checking, you can't do intermediate checking,
that can't be guarenteed to be safe. This is networking 101 and it is
also systems architecture 101 - any hardware class should teach you this.
You keep telling me that backups and mirroring etc are needed and what
you are missing is that the things doing the backups and mirroring
have no way of knowing if that which they are storing is good or bad.
> > In 10 or more messages you've failed to get the point. Doing backups of
> > bad data is not very useful. How the hell do you know that you need to
> > restore from tape if the system never tells you that anything is wrong?
> > RCS / CVS store the latest version in clear text - every earlier version
> > could be completely corrupted and you'll cluelessly (through no fault of
> > your own) be backing up, mirroring, raiding, and CDROM burning COMPLETE
> > AND UTTER GARBAGE. Which you won't find out until you need to get an
> > _OLD_ version to fix a bug.
>
> Yeah, it's possible. But now we've moved from "totally screwed" to
> "way inconvenienced because I have to play hunt-for-the-tape for a
> long time". No, it's not optimal. I'm not claiming it is. I'm simply
> claiming that it's possible to avoid being screwed without having your
> version control system do lots of fancy error checking.
How? The only way I know of is to save infinite backups to the
beginning of time and go through them all one at a time. Let's see,
you were complaining about the cost of the product. Let's suppose you
have to do this once in the lifetime of your project and it is under
extreme pressure because the NSA needs a bug fix right now to a 5 year
old version of your code and you can't reproduce that 5 year old version
because your repository was corrupted. Now you have a couple hundred
backup tapes to look through to find this.
At this point, should this have happened, exactly how much would you
pay to get rid of the problem? And how hard would you be cursing the
guy that thought backups were a reasonable answer?
> Question: Does BitKeeper prevent me from having garbage in my
> repository? Completely? Or does it just make it less likely? I believe
> it just makes it less likely, which is why I claim that backups and
> disk arrays are still more important.
BK gives you back what you gave it. Period. BK checksums the file
every time you do any operation on it. You do a checkin, checkout, diff,
whatever, it checksums the entire file. It's a pretty shitty checksum,
but none the less, it checksums it. It also checksums each version of
each file; these checksums are passed in patches and checked when the
revision history is sent.
In addition, BK is inherently distributed with each repository having
copies of the revision history. So when a file goes bad, the likelihood
of you being able to get a good copy from some other repository is
extremely high.
BK can't prevent garbage in. I don't claim it does. What I claim is
that BK tries hard to make sure it can reproduce your world as of any
point in time. If you checked in garbage, you get back THE SAME GARBAGE,
not different garbage.
> You claimed that Aegis was worse than BK because of support.
I did no such thing. I pointed out that Aegis has some problems and
there is no plausible long term remedy to those problems. The point
was that the free software model, for these sorts of systems, has had
years and years to prove that it works and it has failed miserably.
This stuff just isn't sexy enough for people to work on it for free and
really solve all the problems. You can argue all you want but they are
hollow arguments - Aegis has been here for 7 or 8 years, CVS for longer,
and yet people feel the problem still isn't solved.
That's not a support issue, that's a business model issue.
> don't care about what anyone who disagrees with you has to
> say. They're all either fools or amateurs or deluded or in some way or
> another idiots who should not be listened to. Only you have the
> answer, and it is BitKeeper. You made it so the Linux kernel could use
> it and love it, and by God we'd better all be grateful. You deserve
> more than the insolence of dissenting opinions, you deserve
> applause.
Nat, I've shipped a lot of commercial code. Mostly in I/O and
file systems. I've spent an enormous amount of effort trying to make
sure that the system doesn't screw up the customers data. I take data
integrity seriously, perhaps because I've worked at places like SGI where
performance mattered more than integrity (the same line of reasoning
which gets RCS into trouble). I've done tons of customer visits where
I got to listen to frustrated customers scream at me for bad decisions
that other people made, decisions which ended up in corrupt data.
All that left me with a healthy respect for user data. And, yes, I think
people that don't take that seriously are very deluded and have absolutely
no business in the source management area (and I'd like to get them out
of operating systems in general, perhaps NFS would actually work then).
Sometimes I think that I'm not yet worried enough about integrity and
it makes my skin crawl to hear other people blithely dismiss it with
statements like "that's what backups are for".
I'm sorry if you feel unheard, or beat up, or whatever. I'm not attacking
you personally, I don't even know you. I'm attacking what sad experience
has taught me are misguided principles. There's a difference. You are
very likely an extremely sharp guy, most opinionated people are. But you
are also saying some things which I know to be incorrect. I'm attacking
those statements, not you.
--
---
Larry McVoy l...@bitmover.com http://www.bitmover.com/lm
-
texelsoft
for more than just dumpster bait.
-evt
Eric van Tassell
Texel Software, Inc. - System Software Engineering for Windows/NT
277 Cochran Hill Rd. Voice : 603-487-5006
New Boston, NH 03070 USA Fax : 603-487-5166
email : e...@texelsoft.com
http://www.texelsoft.com
> -----Original Message-----
> From: owner-lin...@vger.rutgers.edu
> [mailto:owner-lin...@vger.rutgers.edu]On Behalf Of Larry McVoy
> Sent: Tuesday, September 28, 1999 2:38 AM
> To: linux-...@vger.rutgers.edu
> Subject: Re: [ OFFTOPIC ] Re: The Linux Kernel Project Management System
> (INITIAL PROPOSAL)
>
>
Rik van Riel
> > > Don't you think it is a little unfair to get people interested in
> > > infrastructure that you have no intention of supporting at commercial
> > > levels? What happens when you get interested in something else?
> >
> > Do you think any of us here care? Many of us are here, supporting a
> > non-commercial project, at non-commercial levels. I think you just
> > insulted most of us.
>
> Uh huh. Sure.
I don't usually participate in flamewars, but he's right Larry.
You just insulted most of the Open Source community, even though
you seem to be using Open Source as a bitkeeper selling point...
Besides, Open Source is the _only_ guarantee that the product
will be around in 20 years. If bitmover falls apart or gets
aquired by someone, we have no chance in hell to keep using
bitkeeper -- and that's all because of that weird licensing.
Rik
--
The Internet is not a network of computers. It is a network
of people. That is its real strength.
--
work at: http://www.reseau.nl/
home at: http://www.nl.linux.org/~riel/
Konrad Rosenbaum
discussion? The flame war you're doing here (esp. Larry McVoy) is
definitely non-productive. Remember: we talk about Source Management for
Linux and not for a big archive or commercials.
Jordan Mendelson wrote:
> If a solution to kernel source management is to be found, it must
> incorporate
> all the same features that the current system allows. There are four
> distinct
> pieces to the kernel development process:
>
> § Source Repository
> § Bug Reporting & Tracking System
> § Patch Submission & Peer Review System
> § API Documentation Submission
To add some other points:
*the traditional solution for bugfixes in OpenSource is "take the newest
version from the stable tree" - so the backup of older versions is
definitely _no_ problem. Only few scientists will be interested in
five-year-old Linux versions.
*Especially the Linux sources are mirrored on hundreds of servers all over
the world, so if one server eats the files on it we can restore by
catching any copy out there. If the main server (at Linus' side) crashes
he still can catch a copy from somewhere and then ask the developers to
resend their patches. Crashes are unlikely (under a stable Linux server)
so this case will be very rare and the manual solution is acceptable.
*The mirror servers have quite large disks (remember: diskspace is cheap
these days) - the size of the local image is no problem.
*Many developers have only slow and expensive Modem-lines to transfer and
update theyr database. The questions are:
- how much does the system compress my data (or even: may I plugin newer
and better compression algorithms)
- how much overhead does the transfer need (counted in bytes transferred
through the net, not bytes stored on local/remote harddisk; call it
compression of metadata)
- may I chose in which changes I am interested and which not to transfer
(eg. Alan may decide to transfer everything from Linus and some other
trusted users but he might want to review a short description of the
patch for those patches from everyone else)
*OpenSource is less hierarchical than commercial development - so it is
necessary to get good (easy to understand) views of the various source
trees and patches. And we would need flexible tools to (un)merge and
change patches and to move whole sets of patches from one tree to
another.
*Many developers will decide to hold their own local tree of patchsets. So
we need a tool with a kind of access rights and distributed
administration. The base server for the linus-tree will be another one
than the base server for ac or davem. And the base servers definitely do
not want to mirror my local tree. But I will want to crossreference
patches from the linus-tree into my own.
*Most developers and most mirror sites don't have the money to buy $1000
software (or more expensive), so it should be either cheap enough or
OpenSource. And they don't need commercial support, they need a good
README - they still are experts.
*Bugreports become difficult for Linux, because a) more and more newbies
send bugreports without knowing what is important and b) there are many
experts which just don't know enough about the kernel (because they are
busy with other projects) to send excellent reports. So this tool (or a
connected one) should be able to force the user to enter all important
data (by asking appropriate questions) and then redirect the report to
the right forum/developer.
*Most important: Linux changes in source _and_ in organization (with every
feature added to the kernel a new group evolves and with every
restructuring the responsibilities change). So the system should be very
changeable - what means it should last only few weeks till there is a new
feature in it. I know that any OpenSource project has this ability (if
the maintainer disappeared or refuses to change it - I can ask anyone
else to do that).
We talk about a really _big_ OpenSource project with really _many_ backups
all over the net. So don't talk about hypothetic cases where all the
servers crash or some nifty admin refuses to give me the backup tapes.
What is needed is a way to handle the source management for Linux and
nothing else. My simple (but hard to answer) questions are these ones:
*Which of the above abilities does BK have and which does Aegis (or any
other free tool) have?
*Which do they not have and how long would it take to fix that?
*Now the heaviest: does it become easier and faster to do this work with
that tool? Or does it only help to solve the problems which we wouldn't
have without it?
BTW: Larry, if you can't offer really cheap licenses to the mirror sites
and developers then be quiet from now on, we talk about OpenSource
development not about rich firms.
happy thinking,
Konrad Rosenbaum
akpm
> ...
> We talk about a really _big_ OpenSource project with really _many_ backups
> all over the net. So don't talk about hypothetic cases where all the
> servers crash or some nifty admin refuses to give me the backup tapes.
Yes. The repository corruption issue is valid, but it's irrelevant
for Linux.
Linux is at much greater risk of _deliberate_ corruption (trojans), as
happened with TCP wrappers and <mumble> ftpd. The risk is low, but being
able to demonstrate that a process is in place to avoid it would be
a plus.
Shawn Leas
However Larry put it, and I'm not defending anyone here, he is at least in
one respect correct. No open source product, or even combination of smaller
apps has yet to do what BK does.
Larry... Could you clarify what would happen if you and your company were
bombed by foreign terrorists? How would BK's license fair in allowing
derivative works?
-Shawn
-----Original Message-----
From: Rik van Riel [mailto:ri...@humbolt.geo.uu.nl]
Sent: Tuesday, September 28, 1999 4:48 AM
To: Larry McVoy
Cc: linux-...@vger.rutgers.edu
Subject: Re: The Linux Kernel Project Management System (INITIAL
PROPOSAL)
On Mon, 27 Sep 1999, Larry McVoy wrote:
> > > Don't you think it is a little unfair to get people interested in
> > > infrastructure that you have no intention of supporting at commercial
> > > levels? What happens when you get interested in something else?
> >
> > Do you think any of us here care? Many of us are here, supporting a
> > non-commercial project, at non-commercial levels. I think you just
> > insulted most of us.
>
> Uh huh. Sure.
I don't usually participate in flamewars, but he's right Larry.
You just insulted most of the Open Source community, even though
you seem to be using Open Source as a bitkeeper selling point...
Besides, Open Source is the _only_ guarantee that the product
will be around in 20 years. If bitmover falls apart or gets
aquired by someone, we have no chance in hell to keep using
bitkeeper -- and that's all because of that weird licensing.
Rik
--
The Internet is not a network of computers. It is a network
of people. That is its real strength.
--
work at: http://www.reseau.nl/
home at: http://www.nl.linux.org/~riel/
Charles K Hardin
Why don't some of the people here start looking into other solutions
like ClearCase for some of their project needs instead of bitching and
moaning about the alternatives?
From interactions and reputations of Mr. McVoy, I am fairly confident
he will generate a good product when it is ready... it still amazes me
that software people whine about their not being a product out there,
when they should know that you cann't give a hard deadline that this
software will be written and done on a such and such a date... if you
believe in the architecture, you are going to have to wait until an
implentation that matches the design is ready to go, so take some
valium and explore the alternatives until the product is ready.
Henning P. Schmiedehausen
>Cool. So explain to us how Aegis handles the following:
> . disconnected operation with updates to all repositories
> . undo - including things you don't do but incorporated
> . multiple views, each of which include some subset of the others
> . compressed repositories
> . disk and/or file systems which corrupt the underlying files
> . low bandwidth links
You tell us. I've read "http://www.bitkeeper.com/bk18.html" and while
I am really impressed with the statements (I'm counting the days until
I can finally get my hands on bk), where is the "BitKeeper vs. Aegis"
section?
Kind regards
Henning
--
Dipl.-Inf. (Univ.) Henning P. Schmiedehausen -- h...@tanstaafl.de
TANSTAAFL! Consulting - Unix, Internet, Security
Hutweide 15 Fon.: 09131 / 50654-0 "There ain't no such
D-91054 Buckenhof Fax.: 09131 / 50654-20 thing as a free Linux"
Larry McVoy
> Larry... Could you clarify what would happen if you and your company were
> bombed by foreign terrorists? How would BK's license fair in allowing
> derivative works?
While I haven't written this down in detail, I've been consistent in this
answer for over a year.
BitKeeper uses open logging of your change comments on host[s] in the
openlogging.org domain. When/after we release 1.0, there will be a
web server on bitkeeper.openlogging.org where you can go to browse the
change logs.
The presence of that web server - and it's active updating - is a
requirement for the !GPL license. If that web server/domain/etc go away
for a continuous period of 6 months (with all the restrictions like
it can't be because California fell into the ocean), then the source
becomes GPLed.
In other words: if whatever corporation which happens to own the rights
to BK no longer cares enough to maintain the business model, or if they
have gone out of business, then you get it under the GPL.
Good enough?
--
---
Larry McVoy l...@bitmover.com http://www.bitmover.com/lm
-
Larry McVoy
> l...@bitmover.com (Larry McVoy) writes:
> >Cool. So explain to us how Aegis handles the following:
I dunno how Aegis handles all of these, that's why I asked. I have some
hunches though, but I thought Peter could provide real answers.
I can say how BK does it (briefly):
> > . disconnected operation with updates to all repositories
All repositories have history, the history is a graph, changes are
propogated as subsections of each graph. The graphs are restored
identically in all repositories, so you can run md5 over the revision
history files and compare them.
> > . undo - including things you don't do but incorporated
We're changeset based,which means what you think of as a patch comes in
as a recorded, immutable, chunk of work. You can do all operations on
changesets: send them, receive them, browse, diff, and undo them.
> > . multiple views, each of which include some subset of the others
That's what LOD are - a named branch into which you can pull changes (or
actually type them in :-)
BK (really SCCS) is actually a set oriented system. To get a version of the
file, you determine the elements of the set and ask the file to give you
that. Each delta is an element, and the normal way of getting the set is
to start here and work your way back up to the root of the graph.
But you don't have to do it that way - you can take elements from any and
all branches to make up a delta. You can create a set which has never been
seen before if you like.
An LOD is the recorded history of the sets which have been instantiated
in this branch. Normally, you don't think about that, you just do checkin
and checkout. But when you are putting together a release, you may well
create an LOD and pull bits and pieces of other LODs together to create
the release.
> > . compressed repositories
We gzip the data portion of the file (typically 95% of the file is
data but it can vary so if you typed more comments than data then the
compression won't help). The kernel history - WITH NO COMMENTS -
compressed from 140MB to 38MB. With comments, I'd guess that the
compression would be more like 45-50MB (i.e., less good).
> > . disk and/or file systems which corrupt the underlying files
We have a really bad 16 bit checksum over the entire revision history
and then another checksum over the data in each version of the file.
The checksum is this stupid 16 bit thing because that is what SCCS uses
and we are compatible. If we ever hit a case where it doesn't catch an
error, we'll break compat and do adler32 (what gzip uses). We have a
pretty good mechanism for checking if it is broken, we can check each
delta checksum.
> > . low bandwidth links
When using ssh, we compress. Regardless, we send data which is very close
to the number of bits you added (look at diff -n output, it's like that).
Jamie Lokier
> it still amazes me that software people whine about their not being a
> product out there, when they should know that you cann't give a hard
> deadline that this software will be written and done on a such and
> such a date...
In this group we often release software _before_ it's finished,
precisely so that people can see how it's going. Make suggestions.
Contribute, participate. There is no requirement for hard deadlines.
A lot of software is announced and never released. I think the
complaint is that BitKeeper is in the same category as, say, PC-Tel's
winmodem driver for Linux. I.e. it probably exists but until we see it,
it may as well not exist.
For some the fact that it won't be free in the freedom sense makes it a
liability too.
-- Jamie
Larry McVoy
> Charles K Hardin wrote:
> > it still amazes me that software people whine about their not being a
> > product out there, when they should know that you cann't give a hard
> > deadline that this software will be written and done on a such and
> > such a date...
>
> In this group we often release software _before_ it's finished,
> precisely so that people can see how it's going. Make suggestions.
> Contribute, participate. There is no requirement for hard deadlines.
>
> A lot of software is announced and never released. I think the
> complaint is that BitKeeper is in the same category as, say, PC-Tel's
> winmodem driver for Linux. I.e. it probably exists but until we see it,
> it may as well not exist.
Talk to the Linux/Merced folks, they've been using it for 3-4 months.
So, why, you ask, isn't it generally released yet? Mainly because
we aren't set up to support the thundering herd. We'd really like
to know that we can't break it before it is released.
On top of that, Linus & Co. are likely to try it out when it gets released.
That's sort of a one shot deal, in my opinion. He isn't real likely to
try it, not like it, go back to tarballs, and then try it again. At least,
in my experience, people switch source management systems either never or
once, but very rarely twice. So I'd kind of like this to be as painless
as possible.
For what it is worth, we're expanding our beta to include the Linux/PPC
crowd and if that works out we'll open it up some more.
--
---
Larry McVoy l...@bitmover.com http://www.bitmover.com/lm
-
Rogier Wolff
> > Can I download BitKeeper _right now_? If
> > not, then it's not a clearly better product in any way.
>
> Yup, you can. You have to sign a beta agreement because I haven't written
> down the license yet, but other than that, yes.
Hi Larry,
I have signed and snail-mailed you the beta-agreement about three
months ago. I haven't heard from you since. Any word on when I will
be able to test-drive BK?
Roger.
--
** R.E....@BitWizard.nl ** http://www.BitWizard.nl/ ** +31-15-2137555 **
*-- BitWizard writes Linux device drivers for any device you may have! --*
------ Microsoft SELLS you Windows, Linux GIVES you the whole house ------
Jes Sorensen
Larry> Talk to the Linux/Merced folks, they've been using it for 3-4
Larry> months. So, why, you ask, isn't it generally released yet?
Larry> Mainly because we aren't set up to support the thundering herd.
Larry> We'd really like to know that we can't break it before it is
Larry> released.
Just to clarify this one, some of the other guys may be using it - we
use CVS at our end at CERN and have done so since we started working
on Linux/ia64 back in March.
Jes