Versioned directories
Tobias Weingartner
>
> Nothing else is ever changed in a directory without that change being
> recorded in association with the object within the directory that
> actually changed. Period. Trying to track this information twice would
> be creating duplicate records in a very confusing manner.
This is not true. There are *very* good reasons to version directories.
> What I'm trying to say is that CVS already version controls directories
> by virtue of the fact that it manages the creation of them based on the
> need to put version-controlled files in them. All of the information
> you should need to know about a directory created by CVS can be inferred
> from the revision information associated with each of the files in that
> directory (and in particular the initial checkin and final remove
> revision comments).
Including the fact that some file has been renamed? The fact that some
file has been copied? These are very good reasons to version directories.
> Directories are merely meta-information. The sooner all CVS users
> understand this, the better.
In the current implementation, maybe yes.
> This is meaningless. The only time a version-controlled directory would
> change is if a sub-directory is created, or an old one deleted. These
> events will not normally happen in relation to any other important dates
> w.r.t. the other files in the directory.
Or if a file is moved/renamed. And the names of files do play a role, at
least the last time I looked at most any 'Makefile', and even a good portion
of the '*.c' files as well. All these files can have references to the
names of files controlled by some VC software.
> Use tags for what they are meant for, not some other inappropriate
> mechanism.
With this I agree. Tags should not be used for tracking directory versioning,
nor for other weird things like that. They should be used to "TAG" a version
of a particular file. Something easier to remember than a list of versions on
a number of files.
> > I've already gone over some of the details of an implementation. Here
> > is a summary (with some additions):
> >
> > 1. 2. 3. 4. [DELETED]
>
> Wow. There's a lot of complexity growing here. This'll certainly cause
> performance problems if implemented in this way, if nothing else.
> You'll have to modify CVS to walk back out of the directory tree while
> carrying a whole pile of information with it -- no a pretty prospect!
This is not the way to do this. Much too error prone, not to mention the
fact that it is simply not portable to different platforms. (Not everything
is Unix... as much as I would like it to be...)
> > I want to work on this, and, I will find a way to submit the patch.
>
> I certainly won't/can't stop you from working on something like this,
> but hopefully I can show you (and everyone else) why it's really not
> necessary and certainly not worth the bother.
Again, I don't believe the way outlined in 1-4 above would be the way to
do it. If you wish to version directories, you will ultimately need a new
repository structure. I'm almost convinced of that by now. The other thing
you will have to realize, is the fact that "file in workarea" !=
"file in repo". Once you can take that conceptual jump, versioning
"directories" become fairly simple.
For a fairly "clean" way of doing that, feel free to have a look at some
other VC software, such as the NUCM project from colorado.edu, or J.
MacDonald's MSc thesis stuff.
Cheers,
--Toby.
*----------------------------------------------------------------------------*
| Tobias Weingartner | Email: wein...@cs.ualberta.ca | Nothing here yet |
| Apt B 7707-110 St. |--------------------------------| Nor here yet... |
| Edmonton, AB | Unix Guru? Admin, Systems-Dude | Nor here... |
| T6G 1G3 ---------------------------------------------------------------|
| Canada / %SYSTEM-F-ANARCHISM, The operating system has been overthrown |
*----------------------------------------------------------------------------*
Noel L Yap
>file has been copied? These are very good reasons to version directories.
I wasn't going to address these issues on the first pass. Versioned directories can eventually deal with these issues (eg via a cvs mv command and such).
>> Use tags for what they are meant for, not some other inappropriate
>> mechanism.
>
>With this I agree. Tags should not be used for tracking directory versioning,
>nor for other weird things like that. They should be used to "TAG" a version
>of a particular file. Something easier to remember than a list of versions on
>a number of files.
With the implementation I envision, there would be a file that contains the names of files, subdirectories, and their version numbers. What happens when everything in the directory is tagged? Should the .directory file also be tagged? What happens when that tag is checked out? What state should the .directory file be in?
>This is not the way to do this. Much too error prone, not to mention the
>fact that it is simply not portable to different platforms. (Not everything
>is Unix... as much as I would like it to be...)
I don't see the Unix-specifics. Do you have another design/implementation in mind?
>Again, I don't believe the way outlined in 1-4 above would be the way to
>do it. If you wish to version directories, you will ultimately need a new
>repository structure. I'm almost convinced of that by now. The other thing
>you will have to realize, is the fact that "file in workarea" !=
>"file in repo". Once you can take that conceptual jump, versioning
>"directories" become fairly simple.
I don't see why a new repository structure is necessary (at this point). I also don't see how I've violated the "file-in-workarea != file-in-repo" rule.
Also, if you're saying that my implementation is impossible, I already posted that it's been done (except for the tags -- I'm still not sure if that's the way I want to do it).
Noel
--== Sent via Deja.com http://www.deja.com/ ==--
Share what you know. Learn what you don't.
Greg A. Woods
> Subject: Re: Versioned directories
> Including the fact that some file has been renamed? The fact that some
> file has been copied? These are very good reasons to version directories.
Please, everyone, forget about "renaming" things in CVS. This simply
does not (and should not) ever happen. What happens is a "remove" and
an "add" (in either order). Calling it a "rename" is simply an
optimization but does not change the low level operations that must be
performed.
If you want to make those two operations atomic then you can do that,
but they're still logically two separate operations. The only reason
for making them atomic is if there's some interlock mechanism that
depends on the fact that the target file not exist before the source
file is removed -- this is not likely to be a problem in a version
control system like it is in a filesystem (especially in the latter when
such operations are often used as primitive locking tools). If someone
accidentally tags a release revision with both the old and new files
still in it, then that can easily be fixed in the QA process (which
would hopefully generate a report at the end of each cycle which would
strongly recommend against allowing developers to add/remove/rename
files so close to the end of the release cycle! ;-).
If you want to leave an audit trail to show a "rename" or a "copy"
operation then do that in the version trail of the *two* files
affected. I.e. leave a comment in the 'dead' revision pointing at the
new file, and check in the new file, un-changed, with a comment stating
where it was renamed/copied from. If you want your version control
system to automate these functions, and more importantly the auditing
functions that could depend on this information, then write wrapper
programs around CVS to do this. I'm sure anyone who published such a
wrapper would be widely applauded.
> Again, I don't believe the way outlined in 1-4 above would be the way to
> do it. If you wish to version directories, you will ultimately need a new
> repository structure. I'm almost convinced of that by now. The other thing
> you will have to realize, is the fact that "file in workarea" !=
> "file in repo". Once you can take that conceptual jump, versioning
> "directories" become fairly simple.
Yes, you'd need a new repository structure, to say the least, even if
that repository still contained RCS ',v' files.
However the requirement to do directory versioning is what's at question
here. I'm claiming that it's pure folley. So far nobody has come up
with any requirement that is not better met by the way CVS currently
works. Brad has shown just how complex directory versioning can get if
you try to do it properly.
BTW, there is one reason to do directory versioning, but it does *not*
apply to any concurrent versioning system such as CVS. There's simply
no need to lock a directory when you add a new file. The first
developer to add the file on a given branch wins.
--
Greg A. Woods
+1 416 218-0098 VE3TCP <gwo...@acm.org> <robohack!woods>
Planix, Inc. <wo...@planix.com>; Secrets of the Weird <wo...@weird.com>
Paul D. Smith
>> Including the fact that some file has been renamed? The fact that
>> some file has been copied? These are very good reasons to version
>> directories.
gaw> Please, everyone, forget about "renaming" things in CVS. This
gaw> simply does not (and should not) ever happen. What happens is a
gaw> "remove" and an "add" (in either order). Calling it a "rename"
gaw> is simply an optimization but does not change the low level
gaw> operations that must be performed.
It does not happen now, but it definitely _should_. Until CVS can do
"rename" as a true low-level operation that _isn't_ just a remove/add
pair, with all that implies in terms of preserving history, versioning,
etc., it will never be a total source code control solution. If you've
used a system that _can_ really do this you'd never want to go back.
A remove then add is not the same thing at all.
gaw> However the requirement to do directory versioning is what's at
gaw> question here. I'm claiming that it's pure folley. So far
gaw> nobody has come up with any requirement that is not better met by
gaw> the way CVS currently works.
Go read the CVS manual itself: it shows just how badly CVS implements
these operations and all the problems inherent in the current
workarounds.
A real directory versioning scheme removes _all_ those problems.
gaw> Brad has shown just how complex directory versioning can get if
gaw> you try to do it properly.
It's not complex at all, though. It works just like normal filesystem
operations and is quite straightforward. Things like "you must check
out a directory" are details specific to the ClearCase environment and
have no relevance to CVS.
The thing that makes it all easy is the separation of workspace pathname
and repository pathname. If you do that, directory versioning is
simple. If you don't, it's quite complicated (I'd say, not worth it).
gaw> BTW, there is one reason to do directory versioning, but it does
gaw> *not* apply to any concurrent versioning system such as CVS.
gaw> There's simply no need to lock a directory when you add a new
gaw> file. The first developer to add the file on a given branch
gaw> wins.
Uh... how does this differ in any real way from how it works in a
non-concurrent versioning system? No one would suggest something like
directory versioning merely to avoid this very trivial issue.
PS. We use ClearCase as a concurrent versioning system: all our
checkouts are unreserved. There's no magic or uniqueness to CVS's
abilities in this respect.
--
-------------------------------------------------------------------------------
Paul D. Smith <psm...@baynetworks.com> Network Management Development
"Please remain calm...I may be mad, but I am a professional." --Mad Scientist
-------------------------------------------------------------------------------
These are my opinions---Nortel Networks takes no responsibility for them.
Brad Appleton
> Please, everyone, forget about "renaming" things in CVS. This simply
> does not (and should not) ever happen. What happens is a "remove" and
> an "add" (in either order). Calling it a "rename" is simply an
> optimization but does not change the low level operations that must be
> performed.
I don't agree at all. An add and remove does not preserve version history,
and if you ask for changes between release X and release X+1 then all
those add+removes look like entirely new files and code was added and
deleted when in fact maybe it was only moved or only moved and then
*only* a few lines changes. Furthermore if you want to go back to previous
versions of a file's contents before it was renamed, you can't.
So add+remove isn't even close to the same thing as a true rename,
It completely severs the link between history and content of the two
elements, treating them as separate and distinct when in fact t hey
aren't.
> Brad has shown just how complex directory versioning can get if
> you try to do it properly.
Hmmn - I didn't mean to say it was extraordinarily complex (nor did I
mean to say we did it properly ;-) I think the way ClearCase does it
makes very good sense and is very closely aligned with how similar
things are done in a regular file-system.
But a lot of people are not used to the concept of versioned directories
and don't necessarily understand all the ramifications. I consider those
"gotchas" because people may not have considered them, not because
they're horridly complex, because I don't think they are; I think there
can be a big difference between "complex" and "unfamiliar" and I'd say
this is one such example.
--
Brad Appleton <bra...@enteract.com> http://www.enteract.com/~bradapp/
"And miles to go before I sleep." -- Robert Frost
Brad Appleton
> Would you have any rationales for not including version information
> as a piece of meta-information?
Yeah - I don't think its part of the usual information for a non-versioned
directory for most file systems. Its just information about the contents,
not version info about them. I think they closer your notion is of
the content being version to what's its like in regular file systems the
better off you will be. I can also tell you that in one of the proprietary
systems I worked in we *did* capture the version of each file in a given
version of the directory and came to wish we really had *not* done that
much later.
Things that capture versions of each of multiple elements are really
*configurations* of some aggregate thing. Directories in a file system
aren't configurations of version is of files, its just a listing of
the set of files and some of their attributes. We don't add version
information for each paragraph of procedure in a text-file, and if we did
it makes things a lot more complicated. A text file contains text, and a
given version of that file captures the content of the file at that point
in time. So I'd rather see a given version of a CVS directory capture
only the entries and their static (rather than dynamic) attributes. Use
tags and/or configuration components to group together versions of
multiple things under a single name/id.
> >The tricky parts of directory versioning in this manner are as follows:
> > 1. Dealing with files added in the same directory at the same time
> > using the same name in different versions of the directory.
>
> This sounds like a job for branching.
Its even trickier than that. Lets say I add a new file "foo" and its
a perl script to do s particular task. Someone else at the same time
creates a file names foo in the same directory and its a Tcl script to
do some completely different task (though foo might be an acronym that
still makes sense for both). Trying to handle those as the different
versions of the same archive file wouldn't be a good fit.
Each branch can have its own version of the directory, but you still
have to know to use different revision archives for the same filename
in each branch.
Furthermore, in my experience it is more often the case than not that
people would rather not be easily allowed to create both files in this
case. They'd either want the second creation attempt to be denied, or else
denied with a warning thats still lets you do it with a '-force' option
if you're really sure thats what you want. People add "triggers" to do
this kind of thing all the time with ClearCase. Others frequently go so
far as to disable branching of directories to avoid this, or even to the
extreme of not allowing anyone to create new files/directories unless they
are a particular person, or have approval from that particular person.
> I think ClearCase is too complicated.
Perhaps - but not because of reserved checkouts. Its because clearcase
uses a virtual file system.
> I see no need for CVS to checkout a directory before files can be
> added, moved, or renamed.
Then it can be extremely difficult to handle correctly. Once you see
your first repository add/remove/rename for something in a directory,
CVS can certainly infer that is an implicit "checkout" of the directory,
but at what point is it safe to apply it has been "checked in."? Do
you wait until a commit? Is that to late? Or do you assume it happens
after every such operations? is that too frequent? Or perhaps you simply
provide checkout and checkin for directories, but let add/remove/rename
auto-checkout a directory (perhaps with a warning, or requiring a
'-force' option) but still require explicit checkin changes, with the
only "implicit" directory-checkin being inferred at commit-time? This
all boils down do what is the "right" granularity of "atomicity" for
change-transactions involving a directory.
Cheers!
Noel L Yap
--
On Tue, 24 Aug 1999 18:51:12 Brad Appleton wrote:
>On Tue, Aug 24, 1999 at 08:07:31AM -0700, Noel L Yap wrote:
>> Would you have any rationales for not including version information
>> as a piece of meta-information?
>
>Yeah - I don't think its part of the usual information for a non-versioned
>directory for most file systems. Its just information about the contents,
>not version info about them. I think they closer your notion is of
>the content being version to what's its like in regular file systems the
>better off you will be. I can also tell you that in one of the proprietary
>systems I worked in we *did* capture the version of each file in a given
>version of the directory and came to wish we really had *not* done that
>much later.
>
>Things that capture versions of each of multiple elements are really
>*configurations* of some aggregate thing. Directories in a file system
>aren't configurations of version is of files, its just a listing of
>the set of files and some of their attributes. We don't add version
>information for each paragraph of procedure in a text-file, and if we did
>it makes things a lot more complicated. A text file contains text, and a
>given version of that file captures the content of the file at that point
>in time. So I'd rather see a given version of a CVS directory capture
>only the entries and their static (rather than dynamic) attributes. Use
>tags and/or configuration components to group together versions of
>multiple things under a single name/id.
OK, I'm convinced. In fact, I now think there should be no meta-information contained within the Directory file. Information such as symlinks, hardlinks, whether the file is a directory, permissions, ... should be captured by the archive file of the file it's archiving. Implementing things in this manner would simplify things a great deal more than just removing version info.
>
>> >The tricky parts of directory versioning in this manner are as follows:
>> > 1. Dealing with files added in the same directory at the same time
>> > using the same name in different versions of the directory.
>>
>> This sounds like a job for branching.
>
>Its even trickier than that. Lets say I add a new file "foo" and its
>a perl script to do s particular task. Someone else at the same time
>creates a file names foo in the same directory and its a Tcl script to
>do some completely different task (though foo might be an acronym that
>still makes sense for both). Trying to handle those as the different
>versions of the same archive file wouldn't be a good fit.
>
>Each branch can have its own version of the directory, but you still
>have to know to use different revision archives for the same filename
>in each branch.
In this case, I guess something like inodes would solve the problem. This would be a fairly large change for cvs (eg since each newly added file would have its own unique id, the filename would just be an attribute rather than the unique id it is now).
>> I see no need for CVS to checkout a directory before files can be
>> added, moved, or renamed.
>
>[deleted some stuff] This
>all boils down do what is the "right" granularity of "atomicity" for
>change-transactions involving a directory.
I'd say the correct action would be to do it a commit time just like any other cvs transaction (except the inconsistent case of adding subdirectories).
Stefan Monnier
> Would you have any rationales for not including version information
> as a piece of meta-information?
In such a case of adding info that will require more handling to keep that
information up-to-date, I'd start by asking the opposite: would you
have any rationale for including version information as a piece of
meta-information ?
It seems like a bad idea at first because it requires updating .directory
for every single commit rather than just for add/remove/move.
Then, you start thinking about tagging the .directory file and what happens if
TAG1 gives you a .directory file that says "file1 1.2.3.4" while file1
has a TAG1 which says "1.6" ?
It just doesn't sound like a good idea, so what does it allow to do that's
impossible or difficult without this (apparently redundant) information ?
Stefan
Greg A. Woods
> Subject: Re: Versioned directories
>
> It does not happen now, but it definitely _should_. Until CVS can do
> "rename" as a true low-level operation that _isn't_ just a remove/add
> pair, with all that implies in terms of preserving history, versioning,
> etc., it will never be a total source code control solution. If you've
> used a system that _can_ really do this you'd never want to go back.
Please tell me what requirements you're trying to fill, what information
isn't available, what auditing functions you cannot do, etc., etc., etc.
I think I've already adequately shown (many times in the past, in fact)
that every SCM question related to file manipulation that a developer
wants to answer can still be answered even without this fictional
"rename" operation.
> A remove then add is not the same thing at all.
Ah, but it is -- both in theory, *and* in practice if you make the
implementation convenient enough to use.
In fact there is no such thing as a "rename". Computer Science 101
should teach you this. Fundamentally a "rename" is always distilled
down to an add and a remove. In a software change management system the
need to understand when a rename has happened as opposed to a simple
un-related file add and remove operation can be easily filled by simply
encoding into the audit trail that this particular pair of operations is
related (and of course by providing tools to help automate this
procedure and the procedures to use this audit trail should such
operations become frequent enough to be otherwise painful).
> Go read the CVS manual itself: it shows just how badly CVS implements
> these operations and all the problems inherent in the current
> workarounds.
The CVS manual is just making poor excuses. It is also confused because
people have tried to implement hacks with all the right intentions but
all the wrong ideas. I may even have contributed to some of this
confusion by trying to focus on particular facets of this problem while
ignoring the bigger picture. I particpated in describing and refining
the procedures outlined in the manual. I was one of several people who
almost simultaneously suggested doing the repository copy in order to
make it trivial for a renamed file to maintain its previous revision
history, though I now feel this is in general the wrong thing to do and
will not solve the more general problem very well because it causes
unnecessary and confusing duplication of revision history. Let's please
call a "spade" a "spade" -- it sure won't bother me!
(Remember the current manual was written by a "third" party based on
personal experience and the collected wisdom of the original Usenet CVS
FAQ. It is not, and has never been, an ideal description of the
original design goals of CVS even though it was an excellent manual
right from the day it was published, as it is today. Ancient archives
of this forum, if they still exist, plus Brian Berliner's original paper
and other original documentation, are probably far better expositions of
the original design goals of CVS.)
> A real directory versioning scheme removes _all_ those problems.
So it might, but in a simple system such as CVS all you'd be doing is
re-inventing the wrong wheels for the wrong reasons.
> It's not complex at all, though. It works just like normal filesystem
> operations and is quite straightforward. Things like "you must check
> out a directory" are details specific to the ClearCase environment and
> have no relevance to CVS.
Exactly. Why re-create a complete filesystem implementation when
there's a perfectly adequate one already working quite well in the
operating system CVS runs upon?
(and if this way of doing things is not working well enough then define
exactly what the inadequacies are and try to see if there's an elegant
and simple way to fill those voids and yet keep the spirit and goals of
the original design.)
[[ yes, special purpose filesystems always have advantages over general
purpose filesystems, but they also always have disadvantages too! ]]
> The thing that makes it all easy is the separation of workspace pathname
> and repository pathname. If you do that, directory versioning is
> simple. If you don't, it's quite complicated (I'd say, not worth it).
Ah, um, no, it's the other way around. If you separate workspace and
repository pathames (or simply eliminate the latter in favour of some
entry in a relational database table or whatever) then you *must* have
some way of creating the workspace directory structure -- some way that
can be version controlled along with the files being managed since this
structure is indeed part of the overall "configuration".
If you don't need to have massively radical differences in the
repository and workspace layout then there's nothing gained by having
change mangement on your directories -- only added cost and complexity.
That's why CVS is best as it is because it automatically provides
directory structure management in a way that's sufficient to allow for
change management without requiring all the burden of change controlled
pathname management.
Several times now in this forum we've discussed the merits of using
directory structure to help maintain variant versions. However we
always seem to come to the (natural, IMNSHO) conclusion that this isn't
a requirement in the scenarios where CVS is best used. I.e. CVS fills a
perfect niche where such change controlled pathname management just
isn't necessary and would overly complicate SCM for those scenarios.
On a slightly related note we sometimes hear calls from folks who would
like to have some way of having variants of the modules file which
correspond to releases of the modules it describes, especially when
modules are used hierarchically. However the discussion usually boils
down to the fact that if the build system is made aware of module
releases (just as it should probably be made aware of tool releases)
then it's possible to do this by simply doing simple revision control
over the build-system control files/scripts/etc.
> gaw> BTW, there is one reason to do directory versioning, but it does
> gaw> *not* apply to any concurrent versioning system such as CVS.
> gaw> There's simply no need to lock a directory when you add a new
> gaw> file. The first developer to add the file on a given branch
> gaw> wins.
>
> Uh... how does this differ in any real way from how it works in a
> non-concurrent versioning system? No one would suggest something like
> directory versioning merely to avoid this very trivial issue.
Exactly.
In a non-concurrent versioning system where you need to avoid the
potential of creating a conflict when new files are created (or indeed
when files are removed), especially when you are trying to manage an
entire group of files as one configuration unit, then some form of
reserved checkout at the directory structure layer is one possible way
to mechanise this conflict avoidance.
However this alone may not be worth the effort of trying to implement
version controlled directory structures. If you're building an entire
virtual filesystem then it is of course a simple and obvious feature to
add, but only if.
Further it is not worth the effort to implement a complete virtual
filesystem, or even a minimal directory versioning system, just to
provide some macro rename operation when the same information can be
provided through much less painful and disruptive means.
CVS achieves much of its simplicity and elegance through the very
careful application of design choices which provide a mechanism for
managing change in a group of files that can be placed in a rather
arbitrary (though of course being restricted to being tree-like just as
the underlying operating system filesystem is) directory structure.
Through the careful refinement of the implementation over the recent
years it is now possible to have CVS manage file existance on a
per-branch basis. These capabilities fulfill 99.9% of the needs of
small to medium sized software development projects within the realm of
mapping basic change control procedures onto groups of source files.
That is what CVS was designed to do and as even Brian Berliner noted in
his original paper, it meets its design goals exceedingly well.
In my opinion anyone who wants CVS to meet different, and as it turns
out often opposing, design goals is looking at the wrong tool (even when
there's no other tool available to fit their needs!).
That's not to say that CVS is the final word in source code control
tools for medium to small projects, but it is an existing, free,
solution that can be made to work quite well (as has been shown
empirically by the extremely wide use it has been put to).
Bishop, Murray
requests like "Show me how this collection of files was at 1997-09-15."
or "Show me how this collection of files was for RELEASE-1-23.".
Lets say I had old/file.c up to revision 1.10, and then decided I
wanted to call it new/newfile.c. I move the file to it's new place in
my working directory, do 'cvs rm file.c' and 'cvs add newfile.c'.
Lets say a few weeks later I want to try a build with the contents of
new/newfile.c being the same as revision 1.5 of old/file.c.
I use cvs to get revision 1.5 of old/file.c, copy that over
new/newfile.c and try. If I like the result, I can commit
new/newfile.c.
> -----Original Message-----
> From: Brad Appleton [SMTP:bra...@enteract.com]
> Sent: Wednesday, August 25, 1999 10:14 AM
> To: CVS-II Discussion Mailing List
> Subject: Re: [info-cvs] Re: Versioned directories
>
> On Tue, Aug 24, 1999 at 01:06:42PM -0400, Greg A. Woods wrote:
> > Please, everyone, forget about "renaming" things in CVS. This simply
> > does not (and should not) ever happen. What happens is a "remove" and
> > an "add" (in either order). Calling it a "rename" is simply an
> > optimization but does not change the low level operations that must be
> > performed.
>
> I don't agree at all. An add and remove does not preserve version history,
> and if you ask for changes between release X and release X+1 then all
> those add+removes look like entirely new files and code was added and
> deleted when in fact maybe it was only moved or only moved and then
> *only* a few lines changes. Furthermore if you want to go back to previous
> versions of a file's contents before it was renamed, you can't.
>
> So add+remove isn't even close to the same thing as a true rename,
> It completely severs the link between history and content of the two
> elements, treating them as separate and distinct when in fact t hey
> aren't.
[...snip...]
Laird Nelson
> I think I've already adequately shown (many times in the past, in fact)
> that every SCM question related to file manipulation that a developer
> wants to answer can still be answered even without this fictional
> "rename" operation.
Did you address Brad Appleton's objection that no amount of cvs logging
or cvs statusing on newfile.c will tell you that newfile.c was once
oldfile.c?
I've found that on Java projects in particular renaming happens quite a
bit more than on C projects since the name of the file is more important
in the former language than in the latter. I've also found that CVS
leaves something to be desired in this department: once I've renamed
OldFile.java to NewFile.java (having worked on OldFile.java for a while)
no one else in my department--using just CVS and independent of other
communication channels--can tell that NewFile.java arose as a result of
conceptually renaming OldFile.java. How would you advise me to use CVS
to mitigate this problem?
Cheers,
Laird
Noel L Yap
Rex_J...@notes.ymp.gov
Laird Nelson <laird....@time0.com> on 08/26/99 04:29:03 AM
Please respond to ljne...@unix.amherst.edu
To: CVS-II Discussion Mailing List <info...@gnu.org>
cc: (bcc: Rex Jolliff/YM/RWDOE)
Subject: Re: Versioned directories
no one else in my department--using just CVS and independent of other
communication channels--can tell that NewFile.java arose as a result of
conceptually renaming OldFile.java. How would you advise me to use CVS
to mitigate this problem?
REN OldFile.java NewFile.java
cvs remove OldFile.java
cvs add NewFile.java
cvs commit -m "Renamed from OldFile.java to NewFile.java" OldFile.java
NewFile.java
(sheesh)
Rex.
Brad Appleton
> OK, I'm convinced. In fact, I now think there should be no
> meta-information contained within the Directory file. Information
> such as symlinks, hardlinks, whether the file is a directory,
> permissions, ... should be captured by the archive file of the
> file it's archiving.
Well - you probably need at least a few things. For example, in order
to be able to gracefully handle renaming or concurrent add-files, you
probably need to include a path (either absolute or relative) to the
version archive file to use for that version of the directory (because one
version of the directory might have a script named "foobar" and the other
might have a binary image file named "foobar" and you can't use the same
version archive for each because they're really different entities that
happen to have the same name in different versions of the directory).
You may also want some "attribute" that allows you to reference or
otherwise recognize and lookup a file or file-version that was moved
from that directory to someplace else.
Furthermore, things like hard-links or symlinks typically won't even
have an archive file. They *only* have the directory entry and the
inode/pathname to which they refer (just like a real Unix directory).
Brad Appleton
> "Greg A. Woods" wrote:
> > I think I've already adequately shown (many times in the past, in fact)
> > that every SCM question related to file manipulation that a developer
> > wants to answer can still be answered even without this fictional
> > "rename" operation.
>
> or cvs statusing on newfile.c will tell you that newfile.c was once
> oldfile.c?
Hmmn - I thought that was Paul Smith (not myself) that said that.
I would add that I frequently run into the case where I wasn't to do a
diff between the REL_X.1 tagged version and REL_X.2 tagged version of a
file such as foo.c. And I don't want every developer to have to know that
foo.c was actually named foobar.c back in the days of release REL_X.1
(or that it was in a parent directory rather than a subdirectory). I
want them to be able to give the name of the file that is currently
visible in their sandbox, and give the "from" and "to" versions to CVS
and have the file-comparison listing come out (perhaps even with the
old filename/filepath indicated at the top where it shows the from &&
too versions).
This can't be easily done if you just do an add/remove, and for me it
continues to be extremely convenient when I use ClearCase. Especially
when some upper manager insists on me gathering lines-of-code metrics
for added/deleted/changed lines between two releases; because I do *not*
want the VC tool telling me that all the lines in the file under the old
ame are removed and all the line sunder the new name are added if in fact
only a few lines changed besides the name of the file changing as well.
Stefan Monnier
> Did you address Brad Appleton's objection that no amount of cvs logging
> or cvs statusing on newfile.c will tell you that newfile.c was once
> oldfile.c?
Yes, this has been addressed several times. The basic idea is that
some kind of rename operation is needed, but it does not call for directory
versioning (and directory versioning does not in and of itself provide
rename either):
% cat cvsrename (this is pure fiction)
#!/bin/sh
cp $1 $2
cvs remove $1
cvs commit -m "-- Moved to $2 --" $1
cvs add $2
cvs commit -m "-- Moved from $1 --" $2
then you need to add smarts to all the other operations like `log'
`diff', etc... to look for those magic sentences. Although one can
rely on the user-operator's smarts to do that part of the work ;-)
It's more painful when moving directories, of course.
Stefan
Tim Endres
Rex> no one else in my department--using just CVS and independent of other
Rex> communication channels--can tell that NewFile.java arose as a result of
Rex> conceptually renaming OldFile.java. How would you advise me to use CVS
Rex> to mitigate this problem?
Rex>
Rex> REN OldFile.java NewFile.java
Rex> cvs remove OldFile.java
Rex> cvs add NewFile.java
Rex> cvs commit -m "Renamed from OldFile.java to NewFile.java" OldFile.java
Rex> NewFile.java
One thing people do is use commitinfo/loginfo to email notice on all commits.
Tim Endres - ti...@ice.com
ICE Engineering, Inc. - http://www.ice.com/
"USENET - a slow moving self parody." - Peter Honeyman
Brad Appleton
> Using add and remove is one way to allow CVS to respond truthfully to
> requests like "Show me how this collection of files was at 1997-09-15."
> or "Show me how this collection of files was for RELEASE-1-23.".
And so is what's being discussed here, since the directory versioning
would allow exactly that, even if it was a rename instead of an add
and remove. The directory still records what the name/location of the
file is so you still get truthful answers to these queries (ClearCase
has been doing that for years). And you also get the ability to easily
and truthfully ask for the difference between two versions of the file
foo.c, even if foo.c had a different name/location for the previous
version (which plain add/remove can't do without a lot of extra effort
that is at least as much work as providing a facility to the user tha
conceptually has all the same benefits as a "rename" command).
Tobias Weingartner
>
> REN OldFile.java NewFile.java
> cvs remove OldFile.java
> cvs add NewFile.java
>
> (sheesh)
Hmm, so how would you do "cvs diff -u -rOldFile.java:1.1 -rNewFile.java:1.2"?
--Toby.
Greg A. Woods
> Subject: Re: [info-cvs] Re: Versioned directories
>
> and if you ask for changes between release X and release X+1 then all
> those add+removes look like entirely new files and code was added and
> deleted when in fact maybe it was only moved or only moved and then
> *only* a few lines changes. Furthermore if you want to go back to previous
> versions of a file's contents before it was renamed, you can't.
Show me how it doesn't preserve the revision history. So far as I can
see it's all there and there's a perfectly clear and usable audit trail
to show who/what/why/when/where, etc.
The only thing you can do to make a rename that works the way you seem
to want it to work is to duplicate the history and remove old tags from
the new copy. This is bogus extra work that can easily be done wrong
and it not provides no new information, but it needlessly duplicates
information. Indeed the way most people manually do a "rename by
repository copy" often leads to loss of information!
All we're missing is a simple wrapper on "cvs log" et al to joint it all
into one report with one command invocation. One could also write a
simple wrapper that did the "rename" operationS too just to make sure
everything in the logs is always kept consistent and easily machine
parsable. The "atomicity" of this procedure is easily achieved by doing
the commit on both the add and remove at the same time.
Rex_J...@notes.ymp.gov
"Laird J. Nelson" <laird....@time0.com> on 08/26/99 01:35:07 PM
Please respond to ljne...@unix.amherst.edu
To: Rex Jolliff/YM/RWDOE
cc: ljne...@unix.amherst.edu
Subject: Re: Versioned directories
>
>Next step: now then, having consulted the revision history, how can they get
>a diff of the contents of the file?
cvs co -p -r1.3 OldFile.java > __OldFile_java_1_3
cvs co -p -r1.2 NewFile.java > __NewFile_java_1_2
diff -u __OldFile_java_1_3 __NewFile_java_1_2
>Are you of the opinion that cvs does not need a rename operation? I'm having
>trouble tracking you.
I havent had enough use for a rename operation to justify making
major changes to the structure of CVS to accompish this. However,
as it is open source, people are free to make such changes, and I
would probably roll such a change into my baseline.
Rex.
Brad Appleton
> % cat cvsrename (this is pure fiction)
> #!/bin/sh
> cp $1 $2
> cvs remove $1
> cvs commit -m "-- Moved to $2 --" $1
> cvs add $2
> cvs commit -m "-- Moved from $1 --" $2
>
> then you need to add smarts to all the other operations like `log'
> `diff', etc... to look for those magic sentences. Although one can
> rely on the user-operator's smarts to do that part of the work ;-)
Speaking as someone who has implemented this type of thing before with
a homegrown system a top RCS/SCCS and did it both ways. We found that
having versioned directories was a *lot* easier to get rename right than
it was to "add smarts to all the other operations" via an add/remove
implementation. After we first did it the straightforward add+remove way
and tried to build in these smarts it was just a great big hassle to get
things to go right. Every time yoy thought you had it, there was yet one
more thing you missed or another operation to do it for. Once it occurred
to us to version directory contents using contents not unlike the Unix
filesystem does for directories, it made everything waaaayyyyy easier
(except for the version numbers - which we should have left out of the
directories).
Versioning directory contents may seem a bit more complex at the
high-level or architectural level than add+remove, but we found that
at the code-level, it really and truly made things a whole lot easier
- not harder. Yes - you can do it with add+remove IF you add a whole
lot of smarts on top of it, but for us at least it was just way more
problematic; and versioned directory contents made most of the tough
problems fallout naturally. The main difficulty was getting folks used
to the concept of a versioned directory that could be checked in/out
and have parallel add/removal of its contents, just like any other file.
Now MAYBE CVS is sufficiently different from the RCS/SCCS-based homebrew
that we had developed and none of this would apply to CVS. Maybe
it really would be conceptually easier on the design to have CVS to
add+remove+smarts to do renames rather than directory versioning. I'd be
(perhaps pleasantly) surprised, but I'm willing to believe it could be
possible. I'm just saying my experience trying it both ways resulted to
versioned directories yielding a much simpler design for the homebrew
system I worked on that was a lot of our own code atop SCCS/RCS.
Brad Appleton
> Show me how it doesn't preserve the revision history. So far as I can
> see it's all there and there's a perfectly clear and usable audit trail
> to show who/what/why/when/where, etc.
Paul Smith already covered this. Yes - its all there in the repository.
But the comment isn't enough "smarts" to make the various operations now
about the move. Something under the scenes needs to recognize that this
archive file is really a continuation of these other archive file by
this other name in this other directory *AND* it has to have the smarts
to figure out that rev-numbers like 1.1, or 1.2.1.1 which may occur in
both the removed file and the added file actually correspond to revisions
of what the user feels is conceptually the *same* file.
> The only thing you can do to make a rename that works the way you seem
> to want it to work is to duplicate the history and remove old tags from
> the new copy.
Not at all. If you version directories - it is easy :-) If you do
add+remove, then you still don't have to duplicate everything like
you side. But if you don't, then it is a heckuva *lot* of effort to
figure out the corresponding relationship mappings yourself and code up
the result. And its a *lot* more than just the logs. You have to figure
out the right "mappings" between the revision numbers, the branches,
the sequence numbers (which may be an SCCS only thing) and several other
things too.
We found coding up all that extra smarts to keep track of things
and keep them consistent in the logs and the archives and the rest
of the database was far more effort and much more error prone than
versioning the directories. Then only other solution we thought of was
the copy+append when adding the new file, which as you pointed out adds
a lot of redundancy, and wastes diskspace (which is why we rejected that
alternative :-).
If you really and truly want the user to be able to think that all they
did was rename or relocate an existing file (and perhaps thats not really
what your after, but if it is) then directory versioning may be a bit more
of a conceptual leap, as well as some effort to change the repository a
bit, but we found the coding and maintenance effort involved to be much
less hassle, and much more natural (in our own subjective opinions of
course) by simply versioning the directory contents.
Of course I'm not going to be coding this thing for CVS ;-) So if the
parties concerned there are convinced the opposite of the above is true
in their situation, then perhaps it is (even if it wouldn't be for me ;-)
YMMV of course. I'm just telling you what I know IMHO from having done this
sort of thing before (but not with CVS - which may or may not be a critical
difference).
Sudheer D V
I am new to this CVS world please give me some info
where I can start learning CVS from the basics.
regards,
Sudheer
"Greg A. Woods" wrote:
> [ On Thursday, August 26, 1999 at 17:56:09 (-0500), Brad Appleton wrote: ]
> > Subject: Re: [info-cvs] Re: Versioned directories
> >
> >[[....]]
> >
> > Now MAYBE CVS is sufficiently different from the RCS/SCCS-based homebrew
> > that we had developed and none of this would apply to CVS. Maybe
> > it really would be conceptually easier on the design to have CVS to
> > add+remove+smarts to do renames rather than directory versioning.
>
> Yes, that's exactly the case. Directories are managed automatically by
> CVS -- this is a basic part of the design of CVS in the way it
> automatically recurses into all sub-directories. Some of us think this
> is a very good thing despite the fact that it makes a tiny number of
> rare jobs a little bit more difficult.
>
> Not only is it easier but in real life, at least in projects best suited
> to CVS, the need for understanding the history of a renamed file rapidly
> diminishes over time and indeed the need is primarily with the
> developers who already are well enough aware of how to use CVS (or at
> least should be and even if not should know where to look to learn) that
> they can very easily make use of the human-supplied information to
> follow the audit trail back into the previously
>
> Improvements in the history log and reporting mechanisms that could use
> it instead of raw RCS logs would even further eliminate the need for
> more complex rename tracking.
>
> One place I've seen developers complain about renames is with 'cvs
> rdiff' when they're actually reading the diffs, not just sending them to
> 'patch' or some such. Current behavour is to show renames as file
> deletions and creations, which is of course extremely verbose in "diff"
> format and completely missing in that it effectively hides any other
> changes to those files. It would be cool if 'cvs rdiff' et al could
> look back in the history log, and/or for trigger comments in "head" and
> "base" revisions to try and generate just the "diffs" for renamed files.
> I don't think this would be as hard as it sounds even in face of the
> complexities branches would cause (though the former method does require
> the history log contain every action on every file -- something I
> decried earlier!). It would also have to be a non-default option
> because I don't know of any version of 'patch' which understands "mv
> file-a file-b" yet, though perhaps this too would be a welcome
> improvement.
>
> I know Henry Spencer and others who have manually generated their
> patches over the years have often kept track of renames and inserted
> instructions to entice the "user" to manually move files before applying
> the patch (as I recall he did this in C News patches despite repeated
> complaints from many people who wanted to be able to automatically apply
> the patch without having to trust any shell commands in a prefixed
> script).
>
> In my deeply considered opinion this ('cvs rdiff') is the only CVS
> function for which there is any real need to be able to track file
> renames with ease. (Note that 'cvs rlog' [where "rlog" means
> "repository log", a feature we're still waiting for] doesn't have the
> same problem since it'll generate the relevant log entries for both the
> old and the new files and all will be well.)
>
> I still think having 'cvs rdiff' understand renames is a very rare
> requirement though since I rarely find people generating global diffs
> for an entire module unless they want to simply apply them to some other
> variant with 'patch'. Usually anyone looking to read the actuall diffs
> will be focusing on one or a very small number of files at a time and of
> course they're instantly clued in when they find the file didn't exist
> in the "previous" release, or that it seems to have been completely
> replaced by something entirely different in the "current" release. A
> quick glance at "cvs log" and they'll know instantly to do their 'diff'
> on tempory files containing the revisions of interest.
>
> BTW, the repository copy method of renaming files does not improve the
> output of 'cvs rdiff' either -- it makes no difference what-so-ever.
>
> > I'd be (perhaps pleasantly) surprised, but I'm willing to believe it
> > could be possible. I'm just saying my experience trying it both ways
> > resulted to versioned directories yielding a much simpler design for
> > the homebrew system I worked on that was a lot of our own code atop
> > SCCS/RCS.
>
> I'm not sure if your home-brew CM system was designed to force
> concurrent edits, or not, but I think this is another facet of CVS that
> makes directory versioning much more difficult *and* distasteful.
Brad Appleton
> Yes, but that's not the point Brad. We're talking *explicitly* about
> CVS in this forum
I'm quite aware of that - but thanks for the reminder ;-). Still, it is
at least one of the points being discussed in this thread.
> and CVS not only doesn't need "version controlled directories", but it
> would be extremely disruptive and inelegant to try and add such a
> feature to CVS (at least as a first-class feature).
Whether or not it "needs" them remains to be seen. I've seen a non-trivial
number of non-pointy-haired CVS users describe how the results of doing
so would make the CVS part of their daily lives easier (even if the actual
implementation does something else instead of versioning directories). SO
at the very least I would say there seems to be a legitimate need to
handle renames/relocates (with which I see little disagreement). Versioned
directories is one way to do that. All useful systems need to adapt and
grow sometimes for needs that may not have been part of their original
design. I'm not yet 100% convinced that this is absolutely necessary
in this particular case at present, but I'm definitely convinced its a
bona-fide prospect that deserves serious consideration rather than quick
dismissal.
I may not have said so explicitly, but I am familiar with CVS for the
past nine years (not just my homebrew systems, and yes all the homebrews
I've worked on were intended for concurrent access rather than reserved
locking ;-). I'm sure your CVS experience is longer and more in-depth
than mine, but I don't agree that versioned-directories would be at all
near as disruptive and inelegant as you suggest. I do agree it would
be a non-trivial amount of design and coding and would certainly add
complexity. But no more so than the current "modules" system added.
Modules and directories are both clearly useful hierarchical
decompositions of the systems file-structure. And it makes no less
sense to version both then to version one or the other. But they are
different slices/views of the same set of elements and the consistency
issues clearly would need to be addressed. That doesn't mean any such
solution would _have_ to be ugly or unwieldy. But if its done without
due consideration for the other, it likely would be.
Allowing both to be versioned is not as bad as you make it sound, but I
wholeheartedly agree that for coding/design simplicity, something very
coherent and very "common" (in the sense of being able to reuse the
same concepts and code) needs to be used for both if thats what someone
intends to implement. Otherwise it will indeed be an "impedance mismatch."
> Directories are managed automatically by CVS -- this is a basic part
> of the design of CVS in the way it automatically recurses into all
> sub-directories. Some of us think this is a very good thing despite
> the fact that it makes a tiny number of rare jobs a little bit more
> difficult.
No one is saying the above is not a good thing. And version controlling
directories doesn't make it any less of a good thing, nor does it make
it significantly harder to do (except for those "tiny number of rare
jobs" ;-). But the desire for such a thing stems from more than just
mere reporting needs capturing auditing/logging info. It involves more
than that, and those needs don't diminish so rapidly over time.
> BTW, the repository copy method of renaming files does not improve the
> output of 'cvs rdiff' either -- it makes no difference what-so-ever.
I don't recall saying that it did.
PLEASE NOTE THAT IM NOT SAYING VERSIONED DIRECTORIES IS THE ONLY WAY
TO ACHIEVE THE DESIRED RESULTS! I do think there are other ways, even
ones that probably make more sense for CVS (particularly if it manages
to unify with the "modules" concept and implementation). But I don't
think add+remove is one of them, based on my own experience with
stuff that did indeed force concurrent edits and have a "modules"-ish
concept and lots of other things not too terribly different from CVS (but
certainly not exactly the same either).
So while I'm not saying versioned directories is *the* way to go, I am
claiming its less of a mess than add+remove+smarts. There probably is a
better way for CVS than either of them, but I don't agree that the latter
is more elegant than the former.
Cheers!
Tor Arntsen
>One place I've seen developers complain about renames is with 'cvs
>rdiff' when they're actually reading the diffs, not just sending them to
>'patch' or some such. Current behavour is to show renames as file
>deletions and creations, which is of course extremely verbose in "diff"
>format and completely missing in that it effectively hides any other
This is the exact problem I have with renames (aka removals/adds) in CVS
(I don't have any other problems with it). I use cvs rdiff a *lot* (and cvs
diff even more), there's not a single day that I don't spend at least 1/4 of
the time looking at diffs. Diffs between versions, diffs between branches,
diffs between releases etc. I have put a lot of things under CVS for the
single purpose of being able to diff versions. Looking at (unified) diffs
is IMO a very powerful tool for finding problems and understanding changes.
> It would be cool if 'cvs rdiff' et al could
>look back in the history log, and/or for trigger comments in "head" and
>"base" revisions to try and generate just the "diffs" for renamed files.
A diff -u now looks something like this:
Index: software_version_no_file.conf
===================================================================
RCS file: /rehost/REPOSITORY/lrdpf/src/software_version_no_file.conf,v
retrieving revision 1.52
retrieving revision 1.55.4.1
diff -u -r1.52 -r1.55.4.1
--- software_version_no_file.conf 1998/07/08 15:34:00 1.52
+++ software_version_no_file.conf 1999/07/04 13:35:18 1.55.4.1
@@ -1,11 +1,11 @@
!
! @(#) RH-SW-SPT-400-20 $Id$
-! @(#) Copyright (c) Spacetec A.S 1993-1998. All Rights Reserved
+! @(#) Copyright (c) Spacetec A.S 1993-1999. All Rights Reserved
If the file was renamed to sw_version.conf between those two versions
I would be happy if there was a way of "hooking" the new one to the old
one so I could see something more or less like this:
Index: sw_version.conf
===================================================================
RCS file: /rehost/REPOSITORY/lrdpf/src/sw_version.conf,v
retrieving revision 1.52
retrieving revision <some other number>
diff -u -r1.52 -r<some other number>
--- software_version_no_file.conf 1998/07/08 15:34:00 1.52
+++ sw_version.conf 1999/07/04 13:35:18 <some other number>
@@ -1,11 +1,11 @@
!
! @(#) RH-SW-SPT-400-20 $Id$
-! @(#) Copyright (c) Spacetec A.S 1993-1998. All Rights Reserved
+! @(#) Copyright (c) Spacetec A.S 1993-1999. All Rights Reserved
instead of the usual two diffs with all-lines-removed of file 1 and
all-lines-new of file 2.
This output would be maybe tricky for 'patch' to understand, but I really
don't use cvs diff as input for 'patch' very much. I basically *look* at
them, then I let CVS merge the diffs. Of course for CVS merges to work
as before the internal merge actions must not be changed. Maybe an extra
option to cvs [r]diff could trigger it to look for the "hook". And then
the external 'patch' could be used just as before too.
[...]
>In my deeply considered opinion this ('cvs rdiff') is the only CVS
>function for which there is any real need to be able to track file
>renames with ease. [...]
It would be equally useful for 'cvs diff'.
[...]
>I still think having 'cvs rdiff' understand renames is a very rare
>requirement though since I rarely find people generating global diffs
>for an entire module unless they want to simply apply them to some other
>variant with 'patch'.
Hm, I do this all the time.. I'm used to read huge diffs (I read *all*
Linux kernel diffs from Linus and have done so since 1992). And as in
my job I'm in charge of merging as well as keeping track of many
development branches with busy developers I just *have* to read diffs.
> Usually anyone looking to read the actuall diffs
>will be focusing on one or a very small number of files at a time and of
>course they're instantly clued in when they find the file didn't exist
>in the "previous" release, or that it seems to have been completely
>replaced by something entirely different in the "current" release. A
>quick glance at "cvs log" and they'll know instantly to do their 'diff'
>on tempory files containing the revisions of interest.
This is correct for the daily little work, but far too much work in a big
project when it comes to merge time. Even with just the occasional rename
(they don't come too often, naturally) it'll stop my Thursday Evening
preparations for the big Friday release. I do a lot of these things over
a slow modem (9600 bps GSM, international call rates) and running into these
renames with the two huge null-info diffs is like a flat tyre when cruising
down the road. Even when working locally. A single rename can easily
double the time I spend for the whole merge.
As I said earlier, in general I don't have any problems with doing remove/add
for renames, the problem is that it messes up my diffs. If there was a way
to get smooth diffs then I for one would be completely happy.
-Tor
Noel L Yap
--
On Thu, 26 Aug 1999 16:54:48 Greg A. Woods wrote:
>The only thing you can do to make a rename that works the way you seem
>to want it to work is to duplicate the history and remove old tags from
>the new copy. This is bogus extra work that can easily be done wrong
>and it not provides no new information, but it needlessly duplicates
>information. Indeed the way most people manually do a "rename by
>repository copy" often leads to loss of information!
This isn't true. I can do anything with an additional indirection. If cvs kept filenames as attributes rather than as unique identifiers, it can version filenames and, therefore allow renames while keeping extremely good track of version history. For example, within the repo, a file 0x1234abcd5678ef90,v can exist and represent any file it wanted to. The stuff between the 0x and ,v is similar to an inode.
I'm not saying we should do this right away. I think we should at least get other types of directory entries working moderately consistently first. At that point, we'll know enough to see how to proceed with this type of indirection, if we desire.
>
>All we're missing is a simple wrapper on "cvs log" et al to joint it all
>into one report with one command invocation. One could also write a
>simple wrapper that did the "rename" operationS too just to make sure
>everything in the logs is always kept consistent and easily machine
>parsable. The "atomicity" of this procedure is easily achieved by doing
>the commit on both the add and remove at the same time.
How will this wrapper script know when you do a move rather than a separate and valid add and remove?
Noel L Yap
--
On Thu, 26 Aug 1999 23:00:17 Greg A. Woods wrote:
>(And the primary reason that these tools are probably still missing is
>likely that only the pointy-haired types really want such reports -- the
>actual CVS users generally know how to find such things out on the spot
>and don't really need to separately document their own actions! CVS
>isn't a tool for pointy-haired types -- it's a tool to get the job done.)
I've already said that such processing is complex and slow.
Noel
PS
I'm not a pointy-haired type.
Aldo Longhi
I too am new to CVS (about a week) and have had to teach myself everything
(I wouldn't have it any other way!). I can offer you my list of bookmarks, in
hopes that the information contained in the various sites will be helpful. I
started by pretty much reading through the "manual" by Per Cederqvist, which
you will find in various forms (PS, TEX, HTML, etc.) in various places.
Some of these links may not apply to what you want to do, but there is
information scattered in there about CVS, tkCVS, jCVS, WinCVS,
CVSweb, jCVSlet, Texi2HTML, Perl, etc.
Happy hunting!
- Aldo
http://www.cyclic.com/cvs/info-ref.html (CVS Quick Reference)
http://www.cs.utah.edu/csinfo/texinfo/cvs/FAQ.txt (CVS FAQ)
http://www.egroups.com/group/info-cvs/ (CVS Mailing List Archive)
http://www.loria.fr/~molli/cvs-index.html (CVS BUBBLES)
http://www.cyclic.com:80/ (Cyclic Software)
http://www.cyclic.com/tkcvs/index.html (Cyclic tkCVS page)
http://www.freebsd.org/~fenner/cvsweb/ (CVS Web)
http://linux.fh-heilbronn.de/~zeller/cgi/cvsweb.cgi/ (Hen's cvsweb CVS Repository)
http://sourceware.cygnus.com/cgi-bin/cvsweb.cgi/ (Sourceware CVS Repositories)
http://www.sslug.dk/cvs2html/ (cvs2html (Perl script))
http://www.cs.utah.edu/csinfo/texinfo/cvs/cvs_toc.html (CVS Online Manual)
http://durak.org:81/cvswebsites/doc/ (CVS Online Manual (another one))
http://durak.org:81/cvswebsites/howto-cvs/ (CVS How-To)
http://www.fido.de/~kama/cvs-en.html (CVS Help Files)
http://www.wincvs.org/ (WinCvs.org)
http://www.ice.com/java/jcvs/ (jCVS - A Java Based CVS Client)
http://www.jcvs.org/ (jCVS Home Page)
http://www.mathematik.uni-kl.de/~obachman/Texi2html/ (Texi2html's current Homepage)
http://wwwinfo.cern.ch/dis/texi2html/ (texi2html - a Texinfo to HTML Translator)
http://hpux.cae.wisc.edu/hppd/hpux/Languages/perl-5.004_04/ (perl-5.004_04 (for
HP-UX))
Sudheer D V wrote:
> Hello ,
>
> I am new to this CVS world please give me some info
> where I can start learning CVS from the basics.
>
> regards,
> Sudheer
--
Aldo Longhi
Associate Implementation Consultant
Open Systems Services
Parametric Technology Corporation
Phone: 781-398-6191
Email: alo...@ptc.com
Jack Christensen
toss out my $.02 worth. I have used and administered systems which
used versioned directories (CCase) and systems which did not (Perforce
& CVS). So here's my experience:
1. In many ways, versioned directories would appear to be an
elegant solution. However, from a user's point of view (the
guys we're supporting) it's very confusing, especially if
you start branching the directories also. I have seen a bunch
of engineering time lost due to versioned and branched
directories.
Versioned directories are expensive in terms of wasted
engineering time.
2. The main problem w. not versioning directories is renaming
files and maintaining history. CVS is very close to having
this ability, all that is necessary to to develop an "official"
command to perform the operation safely and easily.
cp <old_file>,v <new_file>,v
cvs remove <old_file>
cvs commit <old_file>
cvs tag -d <non-branch-tags> <new_file>
The only issue is how to add some info to <new_file> to
indicate that it used to be <old_file>. I'm sure people
with much more knowledge about the internals of CVS could
come up with a solution for this problem. To my mind, a
simple hack would suffice.
IMHO, trying to solve problems w. versioned directories is like
trying to kill a fly with a sledge hammer. Even if you kill
the fly, you will do so much peripheral damage that you will wish
you didn't.
thanx,
jack c.
Greg A. Woods
> Subject: Re: [info-cvs] Re: Versioned directories
>
> If cvs kept filenames as attributes rather than as unique identifiers,
> it can version filenames and, therefore allow renames while keeping
> extremely good track of version history. For example, within the
> repo, a file 0x1234abcd5678ef90,v can exist and represent any file it
> wanted to. The stuff between the 0x and ,v is similar to an inode.
CVS can't do that. Some other tool could, but CVS can't do that and
still be the same "cvs" that has become useful and popular amongst its
current user base. You'd have to strip CVS right down to its
user-interface requirements and re-implement it from the ground up and
you'd have to toss out a few features in favour of their counterparts.
The result would be a much more forcibly structured tool, one that would
no doubt be more appealing to those who like to have iron-fisted control
over how the development process works, and which would probably
alienate much of the current user base (assuming you could even entice
them with promises of "bigger, better, faster, more features" in the
first place).
> How will this wrapper script know when you do a move rather than a
> separate and valid add and remove?
Well, one could simply rely on the fact that if both comment messages
are identical (i.e. the "base" of the old file and the "head" of the new
file). Of course you'd have to first detect that some files don't have
both requested revisions in them before you try and sort out which
belongs with which, if any. You'd also have to restrict such "diff" and
"log" operations to using tags -- there are too many dark alleys
otherwise, but this would be a very *GOOD* thing to do anyway (you
shouldn't be trying to compare things of unknown stability).
More structure is of course possible -- you could force only comments of
the form ``moved "oldfile" "newfile"'' to be matched and then you'd be
off to the races without having to pair up the comment messages.
Stefan Monnier
> diff even more), there's not a single day that I don't spend at least 1/4 of
> the time looking at diffs. Diffs between versions, diffs between branches,
That reminds me that I should probably announce the availability
of my Emacs mode for diffs. I used to distribute a much more
rudimentary version of it as part of PCL-CVS, but it is now a
separate package. Provides highlighting, navigation, editing
and conversion functions as well as a few other goodies.
Try it:
ftp://rum.cs.yale.edu/pub/monnier/misc/diff-mode.el
The version 1.1 has no real changes apart from the fact that it should
now work on Emacsen earlier than 20.4 as well as on XEmacs.
Stefan
Brad Appleton
> 1. In many ways, versioned directories would appear to be an
> elegant solution. However, from a user's point of view (the
> guys we're supporting) it's very confusing, especially if
> you start branching the directories also. I have seen a bunch
> of engineering time lost due to versioned and branched
> directories.
>
> Versioned directories are expensive in terms of wasted
> engineering time.
One could say exactly the same thing about concurrent editing of anything
(including plain text files). Its confusing *at* *first*, especially when
you add branching into the picture. I have seen a bunch of engineering
time lost due to versioned and branched text-files.
So its certainly a surprise that takes getting used to if you weren't
expecting it and hadn't seen it done before (or hadn't seen it done well).
Thats no more reason to eliminate versions directories than it is to
eliminate branching of text-files. Concurrent add-file certainly is an
extra gotcha, and I know lots of folks who use ClearCase and do lots of
branching of files but forbid branching of directories.
remove+add or even copy does not easily solve the renaming problem,
renaming + ref/linkage probably can however, or even vis-a-vis (with or
without versioned directories), so if that really and truly is *all* you
want, you can still get a nice simple solution without going as far with
version dirs or as little as copy or remove/add.
> IMHO, trying to solve problems w. versioned directories is like
> trying to kill a fly with a sledge hammer. Even if you kill
> the fly, you will do so much peripheral damage that you will wish
> you didn't.
Again - I see hoards of people assert the same equally overstated
assertion about file branching. And yet those of us who have learned
to use it effectively know the truth ;-)
. . . . . . . . . . . . . . . . . .
And could folks *please* keep in mind that I have *not* been telling
anyone that they should use versioned directories!!! There's a whole lot
of heat being generated, lots of it because some people seem to think
I'm saying this; but I haven't been saying that at all!
I saw folks talking about versioned directories. I chimed in with my
experience on several of the implementation issues to address. I did not
say you should use versioned directories. Others expressed their opinion
that remove+add was simpler & better and I chimed in with my experience
to the contrary, including some examples of things that add+remove didn't
do as nicely as a real conceptual rename. I still did *not* tell anyone
to use versioned directories. More recently I even mentioned an alternative
to add+remove *and* to versioned directories that should do what is
desired. But I didn't even tell people to do that either.
So could folks *puh-leeezz* lighten up just a bit. I'm not trying to tell
you what to do (honest). I'm telling you what I know from experience,
which sometimes involves adding some discussion of issues, and other
times involves clearing up some mistatements. But I've been doing that
*without* telling you what you should do. I am not the enemy!!! So can we
please remember that before some of us start getting so confrontational?
I'm just sharing real-life experience with these things without any strings
attached about what I want you to do. Do what you want. Just do it with
the full information.
Silly me - I though I might be providing useful information without any
philosophy attached. Perhaps I should just go back to lurking on this list
like I have been for the past couple years. I'll have to go ponder that.
Greg A. Woods
> Subject: Re: Versioned directories
>
> So while I'm not saying versioned directories is *the* way to go, I am
> claiming its less of a mess than add+remove+smarts. There probably is a
> better way for CVS than either of them, but I don't agree that the latter
> is more elegant than the former.
OK, on a strictly theoretical level we certainly agree.
I'm quite certain that CVS is better off handling directories
automatically though (and I don't believe it's alone in doing this).
(I question the need for smarter rename handling at a more basic level,
but enough people seem to want it that I'm unlikely to convince them
otherwise. I think the added effort of always having to do revision
control on your directories, even if it's all hidden away and automated
behind the scenes, is far more "expensive" than simply going to the
additional effort of even manually doing diffs on renamed files in those
instances when you really really must see just the bare changes.)
Greg A. Woods
> Subject: Re: Versioned directories
>
> Silly me - I though I might be providing useful information without any
> philosophy attached. Perhaps I should just go back to lurking on this list
> like I have been for the past couple years. I'll have to go ponder that.
Hey except for the odd WinCVS or pserver question it's often all
philosophy around here!
You've certainly more than adequately made up for any unintended (on
your part) confusion I may have had about your preferences regarding
version controlling directories.
I'm usually tightly focused on CVS when I'm reading and writing in this
particular forum, though I'm always trying to keep more general SCM
issues and ideas in mind. I do apologise for saying that version
controlled directories are a bad idea without explicitly qualifying that
claim as applying specifically to CVS.
Greg A. Woods
> Subject: Re: Versioned directories
>
> 2. The main problem w. not versioning directories is renaming
> files and maintaining history. CVS is very close to having
> this ability, all that is necessary to to develop an "official"
> command to perform the operation safely and easily.
>
> cp <old_file>,v <new_file>,v
> cvs remove <old_file>
> cvs commit <old_file>
> cvs tag -d <non-branch-tags> <new_file>
>
> The only issue is how to add some info to <new_file> to
> indicate that it used to be <old_file>. I'm sure people
> with much more knowledge about the internals of CVS could
> come up with a solution for this problem. To my mind, a
> simple hack would suffice.
That's the "repository copy" method (also called "Another tricky,
alternative way" in the manual) that I've been trying to say is the
wrong approach.
I now believe (through bad experiences with the failure modes of the
other methods, including the above one) that the only correct procedure
to rename a file in CVS is:
# make sure there are no un-committed changes in <old_file>
mv <old_file> <new_file>
cvs remove <old_file>
cvs add <new_file>
cvs commit -m 'Renamed <old_file> to <new_file>' <old_file> <new_file>
This has the least chance of causing other breakage:
1. it always leaves a complete audit trail of what really happened
2. it never causes unnecessary duplication of information (you could
be copying zillions of old revisions, and if someone wants to
change an old log message they'll not have to change it in two
places, and if you rename the file again the dupliation doesn't
happen yet again with even more data)
3. it is also easy to do today without having to have direct access to
the repository machine (though of course CVS could be modified to
do the copy for you via an enhanced client/server protocol) and of
course it works equally well even if you do have access to the
repository machine
4. it won't fail if you already had an old <new_file> that had been
previously removed and is now being resurrected (i.e. a
<new_file>,v already exists in the repository)
5. it has almost always been documented as the "Normal way to Rename"
files (sans the critical "avoid moving un-committed changes" step)
Of course this procedure could (and maybe should) be coded directly into
CVS so that the initial pre-conditions can be properly checked for and
so that it would be easier to move multiple files from one directory to
another (and so that people would stop repeating this FAQ: "How do I
rename files in CVS?" despite it's prominent documentation in the
manual!) and so that the log comment message can be hard-coded such that
it can be depended upon by wrapper scripts.
At the very least this procedure should promoted in the CVS manual to
being the one proper and "supported" method of renaming files and
mention of the other methods should be removed from the manual.
I.e. the current section "The Normal way to Rename" should be popped up
a level to be the sole content of the "Moving files" section and it
should be re-written to reflect the fact that it's the only documented
way to do such an operation. (I'd personally remove all mention of
"Assigning revisions" from the manual too, but that's another story.)
The wrapper script I've discussed for "cvs rdiff" and/or "cvs rlog"
could look in the base and/or head revisions of a file if the requested
tag is not available and use this standardised log message to look in
the right file to find the missing information. Indeed such features
could eventually be built into CVS to make them more efficient once
their mechanics are well understood.
(BTW, the repository copy method can retain the history of the file
across the rename if the 'cvs rm' is done before the copy and then the
file is also moved in the working directory and is then 'cvs add'ed
under its new name. I.e. do the "normal rename", but also copy the ,v
file at a strategic time. Of course you still have to remove the old
tags and it'll still break if you're actually trying to resurrect a file
under the new name, and it still duplicates info unnecessarily.)
Kevin Brannen
opinion surely won't hurt. :-)
As I've been watching this, I think I can see both sides of the
arguement, i.e. why each want to do/not-do it. But Greg finally made an
incredible point, that I can't believe hasn't be trumpeted more!
[quoted below]
As quoted below, CVS had a goal, small & simple CM. I'd say that most
of us are wanting more than small & simple CM. As Brad (I think)
correctly points out, most systems evolve & grow over time. In the same
way, our understanding/processes/needs grow over time too. Maybe some
of us are outgrowing CVS. :-)
It's at v1.10. Maybe it needs a complete re-write with a some re-design
and goto v2.0; or maybe some of us need to just "give it up" and go on
to another tool. If this discussion has done nothing else for me, it's
made me see a few more of CVS design philosophies (besides what's in the
docs), good & bad.
For me, I see more limitations than I do a tool that fit my needs as my
project grows. I think it may be time to leave CVS behind...
My $.02 worth,
Kevin
"Greg_A._Woods" wrote:
>
> CVS achieves much of its simplicity and elegance through the very
> careful application of design choices which provide a mechanism for
> managing change in a group of files that can be placed in a rather
> arbitrary (though of course being restricted to being tree-like just as
> the underlying operating system filesystem is) directory structure.
> Through the careful refinement of the implementation over the recent
> years it is now possible to have CVS manage file existance on a
> per-branch basis. These capabilities fulfill 99.9% of the needs of
> small to medium sized software development projects within the realm of
> mapping basic change control procedures onto groups of source files.
> That is what CVS was designed to do and as even Brian Berliner noted in
> his original paper, it meets its design goals exceedingly well.
>
> In my opinion anyone who wants CVS to meet different, and as it turns
> out often opposing, design goals is looking at the wrong tool (even when
> there's no other tool available to fit their needs!).
>
> That's not to say that CVS is the final word in source code control
> tools for medium to small projects, but it is an existing, free,
> solution that can be made to work quite well (as has been shown
> empirically by the extremely wide use it has been put to).
>
> --
Tor Arntsen
>This wouldn't only be tricky for 'patch' to understand -- it would be
>down right impossible. Patch *expects* the file names to be different.
Patch can take a filename as input (patch file < diff) and then it
doesn't care about names in the diff, but that's irrelevant because
I said I wanted a new option to trig that output format. It would
be for *visual* inspection, not as input to 'patch'.
>> It would be equally useful for 'cvs diff'.
>
>How often do you do a 'cvs diff' on more than just a very few files at a
>time? The only time I do this is when I'm looking at un-committed
>changes in a working directory, otherwise I always use 'cvs rdiff'.
I don't use rdiff that much, there's almost nothing that cvs diff can't
do (or at least cvs diff -Nd) that rdiff can do any better. So there's
not much need to invoke rdiff when you're somewhere inside the tree
anyway, and of course there's nothing with 'rdiff' that makes it fare
any better than 'diff' when it comes to diff many files in one go.
>If the rename isn't committed yet then the addded and removed files
>should be not only blatantly obvious, but which was renamed to what
>should be still fresh in your memory! ;-)
Sure, but that's not the point. The point is not to have to mess up
things by making temporary files and diff them separately for single files
when otherwise one simple command ('cvs diff -u [...] | more') can do
it all in one go.
>I don't understand. I've not had any trouble with merging since 1.10 or
>so, and I've been merging *all* of both NetBSD and FreeBSD at times,
>including both the main sources, as well as X11 and the ports/pkgs trees
>almost 500MB each! So long as I've been careful to keep track of my own
>local renames then I've only got a few "removed" and "added" files to
>commit with each merge, and since I use PCL-CVS this is extremely
>trivial to do with only a few keystrokes. If you have to do the join
>itself over a slow modem then just start it before you go to bed or do
>some other longer term task.
I'm talking about the visual 'cvs diff', not the merge. Besides, you *don't*
leave a GSM phone connected when you don't watch it. If something makes it
hang connected an hour extra it costs hundreds of dollars. My GSM phonebills
are high enough that I could literally have bought a *car* instead. Giving
the financial manager here breathing problems now and then..
But again, I'm not talking about the 'cvs join' command, it is of course
happy with the way CVS does the job today, and I never wanted that changed.
What I would have liked is that different format for my *visual* 'cvs diff'
between branches or whatever before (and after) the merge, or for whatever
other reason I would like to have a look at what's changed. I don't see that
many renamed files, but it's *extremely* ugly when they do appear.
>If you're doing vendor merges then you just have to be extremely careful
>*not* to use the recommended merge command but instead one that looks
>more like:
>
> cvs update -j prev-vendor-tag -j new-vendor-tag .
I know, I do use the one above. But this hasn't got anything to do with
the problem in question :-)
>(even the trailing '.' seems to be important)
I never used that '.', but I do add '-d'. This always works fine. The
only problem I've seen with this is when other people are merging *partial*
imports with the command above, that messes things up almost beyond repair
(and that's when I change to another language when I comment it so not to
burn people's ears off, it's enough that they see my body language :-)
-Tor
Brad Appleton
> OK, on a strictly theoretical level we certainly agree.
>
> I'm quite certain that CVS is better off handling directories
> automatically though (and I don't believe it's alone in doing this).
Versioning directories need not preclude handling them automatically
(though the former certainly adds more wrinkles to the latter). One has
to make the right implementation choices for how/when/if to check them
in/out in an "automatic" fashion. That certainly adds more "gotchas"!
Still - I think we agree that versioned directories isn't required to
solve the renaming problem that is being discussed.
> otherwise. I think the added effort of always having to do revision
> control on your directories, even if it's all hidden away and automated
> behind the scenes, is far more "expensive" than simply going to the
> additional effort of even manually doing diffs on renamed files in those
> instances when you really really must see just the bare changes.)
I guess that depends on whether the manual effort involved goes the full
nine yards to make the illusion complete. If it does, IMHO it quickly
starts approaching the hardship of directory versioning, and can even
surpass it surprisingly fast. However if you've got a suitable compromise at
providing less than the "full" illusion, then this need not be the case.
BUT I am getting more and more convinced that versioned directories is
likely more solution than is needed here. There are some other things
that versioned directories give you. A few of them have been mentioned,
but only a select few of those seem to have been really pleaded for. And
some of the other benefits haven't been mentioned at all (and aren't real
easy to describe in 25 words or less either :-).
So since the full nine yards isn't (yet) being asked for here by most
folks, I would probably recommend holding off on versioning them for CVS
just yet. I'd be more inclined to lean toward a rename + some kind of xref
(either as its own kind of version archive or other special file, or else
in a centralized and readily accessible file in the repository where
it won't be too expensive to search for this info).
Whether the archive file is renamed and the xref is the old name, or
vice versa (or whether its starts off as one but then goes to the other
after a commit) is an important implementation decision (*if* you use an
xref-"file" rather than an xref-entry within some other file).
> I'm usually tightly focused on CVS when I'm reading and writing in this
> particular forum, though I'm always trying to keep more general SCM
> issues and ideas in mind. I do apologise for saying that version
> controlled directories are a bad idea without explicitly qualifying that
> claim as applying specifically to CVS.
Well - I did understand your claim was applying specifically to CVS.
I just thought you were still overstating it a just tad ;-)
Cheers!
Greg A. Woods
> Subject: Re: Versioned directories
>
> Versioning directories need not preclude handling them automatically
> (though the former certainly adds more wrinkles to the latter). One has
> to make the right implementation choices for how/when/if to check them
> in/out in an "automatic" fashion. That certainly adds more "gotchas"!
Let's put it this way -- I don't believe it's possible to implement
version controlled directories to any degree where they will do any good
without effectively building a complete virtual filesystem structure.
The infamous "rename database" that was once proposed for CVS suffered
from problems of complexity and was already 90% of the way to being a
completely virutal layout system anyway. Trying to hack a part-time
naming system into place in a system such as CVS just leaves a whole lot
of sharp corners, blunt objects, and massive coils of rope lying around,
especially if one tries to make it work as automatically as possible.
CVS has already addressed some of the sharpest of these corners in a
very elegant and simple way (the "dead" state), and it doesn't belay too
much rope out. The only remaining hard part is that you can't have a
file with the same name as a now-defunct directory (and probably vice
versa) and that you cannot avoid without building complete virtual
filesystem management.
> Still - I think we agree that versioned directories isn't required to
> solve the renaming problem that is being discussed.
One way or another, no, they certainly are not! ;-)
> I guess that depends on whether the manual effort involved goes the full
> nine yards to make the illusion complete. If it does, IMHO it quickly
> starts approaching the hardship of directory versioning, and can even
> surpass it surprisingly fast. However if you've got a suitable compromise at
> providing less than the "full" illusion, then this need not be the case.
Well, let's just put it this way -- in the half dozen years since I've
been using CVS not only is this issue relatively rare in my experience,
but it also seems to have been relatively rare in discussion in public
forums. Sure there's lots of hand waving and logs being thrown on the
fire, but in general it seems everyone either finds they can live with
the current mechanisms, or they go off and put their money into some
commercial solution that purports to offer such features. Oddly enough
nobody has actually done anything about "fixing" this problem in CVS
itself. Either it's too hard to fix without "breaking" the rest of CVS,
or people are just doing their usual blue-sky dreaming and not actually
worrying about changing anything. In this realm of "free" software
pleading for new features doesn't usually get you very far -- doing is
what's required.
In fact in general I'd have to say that necessity is the mother of
invention and since nobody's actually provided any other solution in ~6
years there mustn't be much of a need for it!
> BUT I am getting more and more convinced that versioned directories is
> likely more solution than is needed here. There are some other things
> that versioned directories give you. A few of them have been mentioned,
> but only a select few of those seem to have been really pleaded for. And
> some of the other benefits haven't been mentioned at all (and aren't real
> easy to describe in 25 words or less either :-).
I still don't see any way to get CVS to implement directory versioning
and yet keep it looking anything like the CVS we all know and love, not
to mention but that such a change would almost certainly require changes
to the repository structure, and CVS users seem even more reluctant to
change their overall repository structure than Unix users wanting not to
change their filesystem structure. One ever-repeating fact of life is
that CVS continues to attract RCS users because they know they can go
back to using bare RCS with their own home-grown processes should CVS
prove ineffective for them.
*I* really don't see what could possibly be hard about automating the
tasks of searching back through an audit trail to put together a
concrete picture of how things have progressed in a module from one
point in time to another. In my experience of doing it by hand on the
rare occasions where I've found such a picture necessary it doesn't
require any real intelligence at all -- it's just a step-by-step process
and it should be trivial to mechanise. After all that's what CVS has
done for us in general right from day one (i.e. CVS simply automates
rote procedures that we all might have done manually without it to
manage groups of files.) In fact I'd have volunteered to automate this
audit trail tracing for renames long ago but I don't think it's going to
make me famous if I do, and it certainly isn't going to save *me* any
time either. I personally have no practical or pressing need for such a
feature (at least not since I've used CVS) so anyone who wants me to
implement it will have to pay me by the hour to get it done! ;-)
Meanwhile advice is always worth what you pay for it! ;-)
[[ Wow! CVS-II is over six years old! Brian's paper is dated
1992-01-30. No wonder my freeware repository is bursting at the seams
with all kinds of ancient crud I've almost totally forgotten about! ]]
Paul Sander
>I've thought about this a bit more. I think a general solution would be to
>have a ,v file for each entity within a directory. The ,v file would have a
>per-revision field that describes the type of the entity (ie file, directory,
>symlink, hardlink, ...). Directories will also have a ,v file associated
>with them (probably located in the CVS subdirectory in the repo). The
>Directory file will contain only the names of the enitities located within
>that directory. The only complication is that the contents of an entity can
>be archived within that entity's ,v file or within that entity's Directory,v
>file, however, the decision as to which one to use is clear from the entity
>type field within the entity's ,v file.
Please consider having a ,v file represent each directory. Each version of
that file would list, for each artifact contained in that directory: Its
name, its type, the location of its RCS file (this is very important), plus
other meta-data as needed.
Upon checkout or update, the relevant version of this file is copied into
the CVS directory, and the local copy is modified according to local
operations, and committed as with any other change. Merges during update
can expose conflicts, which must be resolved much the same way as update
does currently with concurrent adds and removes.
Repository operations like rlog and rtag must check out the proper version
of the directory's ,v file to drive their actions, because they record the
locations of the affected RCS files.
I noted above that the location of an artifact's RCS file is important to
remember. This is necessary to track changes to any file (directories
included) when the affected directory is deep in a directory tree and a
directory rename was made on a distant ancestor. It's better to compute
directory names from RCS file paths than the other way around because in
certain borderline cases it appears that directory histories can become
disjoint. It also allows replacement of any artifact with a different
artifact (of any type) with the same name.
Note that this method has problems with regard to keyword expansion. In
the cases where a file is renamed or is replaced by a different artifact
having the same name, the RCS file may not have any relation to the source
file name. (In the case of a replacement, the RCS file name will probably
be generated at random.) In such cases, RCS will expand keywords in a way
consistent with the name of the RCS file, not the source file.
--
Paul M. Sander | Every day is a weekend when you work for a startup
pa...@wakawaka.com | company.
Paul Sander
>>>>>> "Laird" == Laird Nelson <laird....@time0.com> writes:
>> Did you address Brad Appleton's objection that no amount of cvs logging
>> or cvs statusing on newfile.c will tell you that newfile.c was once
>> oldfile.c?
>Yes, this has been addressed several times. The basic idea is that
>some kind of rename operation is needed, but it does not call for directory
>versioning (and directory versioning does not in and of itself provide
>rename either):
> % cat cvsrename (this is pure fiction)
> #!/bin/sh
> cp $1 $2
> cvs remove $1
> cvs commit -m "-- Moved to $2 --" $1
> cvs add $2
> cvs commit -m "-- Moved from $1 --" $2
>then you need to add smarts to all the other operations like `log'
>`diff', etc... to look for those magic sentences. Although one can
>rely on the user-operator's smarts to do that part of the work ;-)
>It's more painful when moving directories, of course.
What about "cvs update -j -j"? To merge branches using this method
requires generating patches and applying them by hand. That's too
clumsy a workaround to be useful even if just a single directory is
renamed.
There's also no assurance that a user won't write such special sequences
in his commit message. (Suppose the comment refers to the file's content
being added to a different file.)
Expecting special strings in the commit comments is asking for trouble
because you can't control the content. A workaround is to preprocess it in
some way to distinguish the magic strings you maintain from the stuff the user
provides.
Andy Glew
> from problems of complexity and was already 90% of the way to being a
> completely virutal layout system anyway.
Any pointers to where / how / when this infamous rename databae was discussed?
Greg A. Woods
> Subject: Re: Versioned directories
>
> > % cat cvsrename (this is pure fiction)
> > #!/bin/sh
> > cp $1 $2
> > cvs remove $1
> > cvs commit -m "-- Moved to $2 --" $1
> > cvs add $2
> > cvs commit -m "-- Moved from $1 --" $2
BTW, the above method isn't quite right.
It should be more like:
if [ -n "`cvs -nq update $1`" ] ; then
echo "Rename aborted, $1 not up-to-date!" 2>&1
exit 1
fi
if [ -f $2 ] ; then
echo "Rename aborted, $2 exists!" 2>&1
exit 1
fi
if mv $1 $2 ; then
echo "Rename aborted, mv $1 $2 failed!" 2>&1
exit 1
fi
cvs remove $1
cvs add $2
cvs commit -m "Renamed $1 to $2 (do not edit this message!)" $1 $2
# in theory the commit will fail if the remove or add failed so
# I don't think we need to check for errors there...
If the above were coded carefully into CVS itself then we'd be one step
further along....
> What about "cvs update -j -j"? To merge branches using this method
> requires generating patches and applying them by hand. That's too
> clumsy a workaround to be useful even if just a single directory is
> renamed.
This isn't as big an issue as it might seem to be unless you have
programmers who rename things without thinking, and provided that you're
not trying to use branches as a full-scale lines-of-development feature
for variant versions. If you do have a lot of inter-release renames
happening in your module then branches are not the best solution for
variant versions -- compile-time or even run-time options are much
better. Branches work OK in face of renames if you're just using them
for maintaining patch releases and you've got regular release cycles
that do eventually retire old branches. You can of course "merge the
rename" across the branches too and thus go back to automatic merges.
> There's also no assurance that a user won't write such special sequences
> in his commit message. (Suppose the comment refers to the file's content
> being added to a different file.)
If the documented procedure strongly advises that only a single commit
command be used then they cannot enter a different comment for each
file. That's why this really should be mechanised though....
If this scheme were coded into CVS there could still be a way to add an
optional user-supplied message to the comment to further explain the
change -- the tools using these comments only need to be taught to
ignore the optional part.
> Expecting special strings in the commit comments is asking for trouble
> because you can't control the content. A workaround is to preprocess it in
> some way to distinguish the magic strings you maintain from the stuff the user
> provides.
So long as the comments are identical and have the same timestamp and
author it really doesn't matter too much what they say. If an MD5 hash of
the concatenation of these values compares identically (and the other
pre-conditions are right, i.e. the head of $1 is 'dead' and the base
of $2 matches) then the action was a rename.
Mechanising this operation does make it easier though since then the
comment string can be relied upon (esp. if it itself warns that it
cannot be changed without breaking something! ;-).
Noel L Yap
>Silly me - I though I might be providing useful information without any
>philosophy attached. Perhaps I should just go back to lurking on this list
>like I have been for the past couple years. I'll have to go ponder that.
I for one have appreciated everyone's input on this thread. I have taken your info as simply that, info. Please don't be discouraged by those who don't.
Thanks,
Noel
Noel L Yap
>(I think the added effort of always having to do revision
>control on your directories, even if it's all hidden away and automated
>behind the scenes, is far more "expensive" than simply going to the
>additional effort of even manually doing diffs on renamed files in those
>instances when you really really must see just the bare changes.)
Perhaps, but this should be decided by the team, not the tool.
Noel L Yap
My first (and probably only) implementation of this would be the implementations of the commitinfo and loginfo scripts that'll build and checkin the files .directory and .module. The contents will contain only the directory's and module's entry names -- it will not contain the version info.
Note that tagging the aggregate files (ie .directory and .module) could still pose a disintegrity problem regardless of whether version info is included. The problem will exist if developers add/remove files and other developers haven't updated, then tag. I have not written anything to prevent this (as of yet).
Noel
--
>--
>Paul M. Sander | Every day is a weekend when you work for a startup
>pa...@wakawaka.com | company.
>
>
Paul Sander
Another worrisome case is when a file is replaced twice, which is the
result (potentially on a large scale) when a directory is renamed and a
second directory assumes the first directory's old name. This method
fails because unrelated files now share version history. Branch management
also leads to failures when there are collisions on branch names before
and after the rename.
>If the above were coded carefully into CVS itself then we'd be one step
>further along....
>> What about "cvs update -j -j"? To merge branches using this method
>> requires generating patches and applying them by hand. That's too
>> clumsy a workaround to be useful even if just a single directory is
>> renamed.
>This isn't as big an issue as it might seem to be unless you have
>programmers who rename things without thinking, and provided that you're
>not trying to use branches as a full-scale lines-of-development feature
>for variant versions. If you do have a lot of inter-release renames
>happening in your module then branches are not the best solution for
>variant versions -- compile-time or even run-time options are much
>better. Branches work OK in face of renames if you're just using them
>for maintaining patch releases and you've got regular release cycles
>that do eventually retire old branches. You can of course "merge the
>rename" across the branches too and thus go back to automatic merges.
Sometimes they do. It's not uncommon to rename an artifact, commit the
change, and later decide to reverse the change. And many shops use
branches as a porting mechanism. Yes, there are better ways to handle
porting, but using branches is common when releases are staggered and the
porting teams are separate from initial development.
>> There's also no assurance that a user won't write such special sequences
>> in his commit message. (Suppose the comment refers to the file's content
>> being added to a different file.)
>If the documented procedure strongly advises that only a single commit
>command be used then they cannot enter a different comment for each
>file. That's why this really should be mechanised though....
Sure they can, if the commit spans multiple directories. CVS will even
bring up a new edit session, thus encouraging the user to enter a different
comment.
>If this scheme were coded into CVS there could still be a way to add an
>optional user-supplied message to the comment to further explain the
>change -- the tools using these comments only need to be taught to
>ignore the optional part.
Hence, my suggestion to preprocess commit comments...
>> Expecting special strings in the commit comments is asking for trouble
>> because you can't control the content. A workaround is to preprocess it in
>> some way to distinguish the magic strings you maintain from the stuff the user
>> provides.
>So long as the comments are identical and have the same timestamp and
>author it really doesn't matter too much what they say. If an MD5 hash of
>the concatenation of these values compares identically (and the other
>pre-conditions are right, i.e. the head of $1 is 'dead' and the base
>of $2 matches) then the action was a rename.
Murphy requires the timestamps to differ: One RCS file can be modified
one second, the second updated a second later. This opens up the door
for users to mimic renames using other CVS operations, be it by accident
or by design.
>Mechanising this operation does make it easier though since then the
>comment string can be relied upon (esp. if it itself warns that it
>cannot be changed without breaking something! ;-).
You can warn the user all you want, doesn't mean they'll read it, let alone
follow instructions. I've seen developers perform really damaging actions
despite very verbose warnings and confirmations. Too often they think they
know what they're doing, and later discover that they don't, usually by
strong reprimand.
But taking away the option of screwing up is definitely the way to go.
Noel L Yap
>Mechanising this operation does make it easier though since then the
>comment string can be relied upon (esp. if it itself warns that it
>cannot be changed without breaking something! ;-).
I don't think the move info should be placed in the log. Problems will occur if the user decides to mimic the automatic log that CVS places.
Noel
Stefan Monnier
> I don't think the move info should be placed in the log. Problems will occur
> if the user decides to mimic the automatic log that CVS places.
That's the least of the problems that will show up with such a trick.
That's merely a name clash problem, just like the use of a `CVS'
subdirectory and a `!' in cvsignore and ...
And then if a user really puts
<< Moved file2 to file2 (do not change this line) >>
somewhere in his log, you might argue that cvs *should* interpret it
in the same way as if it put it in itself.
Stefan
Greg A. Woods
> Subject: Re: Versioned directories
>
> I don't think the move info should be placed in the log. Problems
> will occur if the user decides to mimic the automatic log that CVS
> places.
Unless I've missed some magical trick one can do with RCS files there is
no other place to put such information without changing the repostiory
structure.
Greg A. Woods
> Subject: Re: Versioned directories
>
> Another worrisome case is when a file is replaced twice, which is the
> result (potentially on a large scale) when a directory is renamed and a
> second directory assumes the first directory's old name. This method
> fails because unrelated files now share version history. Branch management
> also leads to failures when there are collisions on branch names before
> and after the rename.
There'll always be a 'dead' revision to let you know to stop reading.
Indeed the default behaviour of 'cvs log' might be changed to ignore all
entries from the dead revision back (in which case a wrapper script
that looks back across a file rename would need to specify an option to
allow reading back from a specified dead revision until the next dead
revision appears).
> Sometimes they do. It's not uncommon to rename an artifact, commit the
> change, and later decide to reverse the change. And many shops use
> branches as a porting mechanism. Yes, there are better ways to handle
> porting, but using branches is common when releases are staggered and the
> porting teams are separate from initial development.
OK, then if you must use branches for variant versions then do not allow
renames except during full code freeze periods. You can't have your
cake and eat it too.
> Murphy requires the timestamps to differ: One RCS file can be modified
> one second, the second updated a second later. This opens up the door
> for users to mimic renames using other CVS operations, be it by accident
> or by design.
OK, then a fudge factor for the timestamp.....
> You can warn the user all you want, doesn't mean they'll read it, let alone
> follow instructions. I've seen developers perform really damaging actions
> despite very verbose warnings and confirmations. Too often they think they
> know what they're doing, and later discover that they don't, usually by
> strong reprimand.
"Doctor! Doctor! It hurts really bad when I screw up automatically
generated messages!"
I just don't have a sympathetic shoulder for such fools to lean on.
Remember I'm a Unix programmer. When you remove a file, it's gone.
When you kill a process, it's dead. There's lots of rope hanging from
convenient places. The system does not hold our hands when we cross the
street. Nothing asks you twice about anything unless you've already
given it a hint that you want to know about particularly undoable
things that might be done.
Noel L Yap
Anyway, here they are.
The line in commitinfo should be something like:
ALL $CVSROOT/CVSROOT/commitinfo.pl
and the line in loginfo should be something like:
ALL $CVSROOT?CVSROOT/loginfo.pl %{sv}
Note that the inefficiencies in the scripts are mostly due to workarounds for difficiencies in CVS (ie not allowing specification of checkin time upon commit, self-locks, ...). These issues would go away if the implementation were put directly into CVS.
I welcome any comments and improvements.
Paul D. Smith
Whew! I was gone for a long weekend and I see the argument has moved
on.
I'll just add some comments and call it good.
gaw> [ On , August 24, 1999 at 14:34:44 (-0400), Paul D. Smith wrote: ]
>> It does not happen now, but it definitely _should_. Until CVS can do
>> "rename" as a true low-level operation that _isn't_ just a remove/add
>> pair, with all that implies in terms of preserving history, versioning,
>> etc., it will never be a total source code control solution. If you've
>> used a system that _can_ really do this you'd never want to go back.
gaw> Please tell me what requirements you're trying to fill, what
gaw> information isn't available, what auditing functions you cannot
gaw> do, etc., etc., etc.
I will give you some examples of things I've had to do recently, that
were trivial with ClearCase due to directory versioning:
1) One of the main portions of one of our products is a "network data
model". Previously we had only one model, now we are introducing
more than one. This means that one entire section of directory
structure must be modified to allow for this cleanly.
Since ClearCase has directory versioning I can make a branch and
do _all_ the moving I need to do, without disturbing anyone else on
any other branch. When I'm done, it's simple to merge these changes
into the mainline... and still anyone (like the sustaining
organization) working on older versions won't be able to tell the
difference. Also, any merging back and forth between branches
(which we do all the time, since we have at least 3 mainstream
development efforts, in different phases of completion, happening at
once, plus many sustaining versions, etc.) is quite trivial, since
the SCM tool knows that these differently-named files are in fact
the same element and can take that into account when performing the
merge.
As has been discussed here, tools like textual history viewers don't
need any special code, since the element is unchanged. Performing
diffs between version A and version B, even if they have wildly
different names or live in completely different places in the tree,
don't require any special knowledge that the filenames are
different, since there's only one element.
Further, since I'm just changing the directory structure but _not_
the files, I can move an entire subtree down a level, say, by merely
using:
$ ct mkdir ndm
$ ct mv foo ndm/bar
Not only have I moved the directory, I also renamed it at the same
time. The key thing to note here is that _all_ the file and
directory elements in the foo subtree are not changed at all: all
changes made to those files by others can be seen by me just as if
I'd not moved the directory (e.g., they're not new elements).
Thus, at the end of my work my merge operation is trivial... all I'm
merging is the change in the directory structure and _NOT_ the
changes to the files, etc. I don't have to continually take changes
to another element and merge them into my elements... because
they're the same, they just have different names.
Also, if I decide that I've made these changes incorrectly or they
need to be withdrawn or even changed again I can easily undo them
all, with a single command. At the same time, they do exist in the
SCM repository and not just in my sandbox, so there can be many
people working on my view of the directory structure at the same
time.
2) On a simpler level, we have an internal compiler tool that depends
heavily on filenames. It happens fairly regularly that someone
needs to change the name of a file in this case. And often the
reason they need to change it is that some completely different
implementation needs to use that name. For example, perhaps we are
inserting a virtual class at that level and pushing down the real
class implementation to a lower level, to implement some inheritance
feature.
Renaming files and creating new, different files with previously
existing names, even within a directory, is an important function in
many environments.
Perhaps I can be more concrete. I feel that all these commands should
be completely immune to any glitch whatsoever over the course of the
conceptual lifetime of an element in the repository, even if that
element is renamed, moved to another directory, or any directory in
its path is renamed or moved:
cvs annotate
cvs checkout
cvs diff
cvs history
cvs log
cvs rdiff
cvs rtag
cvs tag
cvs update
In short, any operation that accepts a -r option, most especially if the
argument is a tag, should work seemlessly across the conceptual lifetime
of the element.
The inclusion of update here implies, for example, that if I execute a
"rename" (add/remove/whatever) operation then perform an update, any
changes made by others to the element under the old name should be
correctly applied to my version of the element under the new name.
A lesser (to me), but still very important, consideration is the degree
of difficulty involved with making these changes in the first place (in
CVS right now simply performing the trivial step I showed above, in two
commands, of pushing down a directory structure one level, is a
_massive_ undertaking).
gaw> In my opinion anyone who wants CVS to meet different, and as it
gaw> turns out often opposing, design goals is looking at the wrong
gaw> tool (even when there's no other tool available to fit their
gaw> needs!).
I'm not sure I agree that the addition of directory versioning, even
though it would require some fundamental changes in the implementation,
would represent a move away from the design goals of CVS.
But if you are simply content to concede my original point, that until
and unless CVS can provide these features it will never be considered a
complete SCM solution, and that's OK and if you need one you should look
elsewhere, then I'm content that I have made it.
--
-------------------------------------------------------------------------------
Paul D. Smith <psm...@baynetworks.com> Network Management Development
"Please remain calm...I may be mad, but I am a professional." --Mad Scientist
-------------------------------------------------------------------------------
These are my opinions---Nortel Networks takes no responsibility for them.
Tobias Weingartner
>
> Remember I'm a Unix programmer. When you remove a file, it's gone.
> When you kill a process, it's dead. There's lots of rope hanging from
> convenient places. The system does not hold our hands when we cross the
> street. Nothing asks you twice about anything unless you've already
> given it a hint that you want to know about particularly undoable
> things that might be done.
Unfortunately, you seem to also be standing on the ledge label'd:
Unix == Restricted functionality. I've always been in the camp of:
Give them the tools that do the job, let them figure out if it is
a good thing to do. A vendor does not setup a system without "mv",
right? You "could" do the same thing with "cp" and "rm"... :-)
Anyhow, it's pointless to continue arguing my point. You guys, I'm
sure, will know my view on it by now. :-)
--Toby.
Stefan Monnier
> I'm not sure I agree that the addition of directory versioning, even
> though it would require some fundamental changes in the implementation,
> would represent a move away from the design goals of CVS.
How about if someone takes CVS, adds *proper* directory versioning
and calls it DCVS or something ?
Stefan
Paul D. Smith
>>>>>> "Paul" == Paul D Smith <psm...@baynetworks.com> writes:
>> I'm not sure I agree that the addition of directory versioning, even
>> though it would require some fundamental changes in the implementation,
>> would represent a move away from the design goals of CVS.
sm> How about if someone takes CVS, adds *proper* directory versioning
sm> and calls it DCVS or something ?
Sounds cool. And perhaps, as happened with GCC/EGCS, the result might
end up becoming CVS again if (as I suspect ;) it turns out that DCVS is
not much more complex (from a user's perspective, at least), and at the
same time is much more useful.
Paul D. Smith
jc> 1. In many ways, versioned directories would appear to be an
jc> elegant solution. However, from a user's point of view (the
jc> guys we're supporting) it's very confusing, especially if
jc> you start branching the directories also. I have seen a bunch
jc> of engineering time lost due to versioned and branched
jc> directories.
jc> Versioned directories are expensive in terms of wasted
jc> engineering time.
I've seen this stated over and over, but in every case I've investigated
it's really been a lack of any introduction to the concept. People seem
comfortable with files and extending that into a "z-axis" of versioning,
but they're not familiar with how doing the same with directories would
work.
I've never had to spend more than a half-hour explaining it to anyone
before they were comfortable with the concept. Those who have an
understanding of how the UNIX filesystem works get it almost
immediately, since it's completely analogous.
In other words, I don't believe it is inherently more confusing than
versioning files, people just need a brief introduction.
Also, note that merging directories is almost always trivial, since a
directory merge consists merely of adding and removing files; no chance
for a conflict.
There is only one stumbling block in directory versioning in ClearCase:
the chance that two people will create two new elements on different
branches of the same directory, with the same name. This is trivially
handled by installing a trigger.
jc> 2. The main problem w. not versioning directories is renaming
jc> files and maintaining history.
Maintaining the textual history log is, IMO, the least important aspect
of renaming or moving files.
_Much_ more important is the ability to perform diffs and merges between
versions that "span" renames, using appropriate tags, etc.
Admittedly ClearCase's ability to track merges between versions via
merge hyperlinks makes the need for a single container throughout the
lifetime of the element much greater than in CVS, which can't do that.
However, the need is still there even in CVS.
See my other post, yesterday, with a list of CVS commands I think would
need to be enhanced to grok renames/moves in order to provide a complete
solution.