Feature request - SVN command to clean a working copy of all unversioned and ignored files and directories
Simon Dean
Are there any plans to add a command to SVN that cleans a working copy or path of all unversioned and/or ignored files and directories?
This is a very common need for automated Continuous Integration builds where a working copy is reused for multiple runs of the same build. Currently there is no simple and fast way to restore a working copy to a prestine state. Often users have to choose between i) completely deleting the working copy for every build and then doing a fresh checkout from scratch or ii) living with lots of unversioned and ignored files and directories building up with each successive build.
The only option at the moment is to write a shell/batch script to provide this feature which is messy and there's common way to do this. A new SVN command or enhanced exiting command that provided this functionality would be incredibly useful.
As an example, here is the DOS batch script that I use at the moment:
@echo off
:: revert any uncommitted changes
svn revert . --recursive
:: remove all unversioned and all ignored files and directories
for /f "usebackq tokens=1*" %%i in (`svn status --depth infinity --no-ignore ^| findstr /r "^[\?I]"`) do (
if not %%j == %~nx0 (
if exist "%%j\*" (
echo deleting unversioned directory "%%j"
attrib -h "%%j" /d /s
rmdir /s /q "%%j"
) else (
echo deleting unversioned file "%%j"
attrib -h "%%j"
del /f "%%j"
)
)
)
A possible command line syntax might look something like this:
svn revert . --ignored --unversioned --recursive
Many thanks
Simon Dean
-----------------------------------------------------------------------------------------------------------------------------------------
The information contained in this message may be CONFIDENTIAL and is intended for the addressee only. Any unauthorised use, dissemination of the information, or copying of this message is prohibited. If you are not the addressee, please notify the sender immediately by return e-mail and delete this message. Although this e-mail and any attachments are believed to be free of any virus, or other defect which might affect any computer or system into which they are received and opened, it is the responsibility of the recipient to ensure that they are virus free and no responsibility is accepted by Moneysupermarket.com Financial Group Limited for any loss or damage from receipt or use thereof.
The views expressed are of the individual, and do not necessarily reflect the views of Moneysupermarket.com Financial Group Limited.
Moneysupermarket.com Limited is an appointed representative of Moneysupermarket.com Financial Group Limited, which is authorised and regulated by the Financial Services Authority (FSA FRN 303190).
Moneysupermarket.com Financial Group Limited, registered in England No. 3157344.
Registered Office: Moneysupermarket House, St. David’s Park, Ewloe, CH5 3UZ. Telephone 01244 665700.
Giulio Troccoli
Sorry, but to me this has got nothing to do with Subversion. Your CI
tool is should clean up itself.
Having said that, if someone wants to implement such feature I don't
think I would have anything against it. But I doubt it will (be implemented)
Giulio
Bob Archer
FYI: If you are on windows the TortoiseSVN client's "Clean up" function allows you to delete unversioned and ignored files if you want. Also, it allows you to revert all changes when you clean up as well.
BOb
Simon Dean
>
> Sorry, but to me this has got nothing to do with Subversion. Your CI
> tool is should clean up itself.
>
> Having said that, if someone wants to implement such feature I don't
> think I would have anything against it. But I doubt it will (be
> implemented)
>
> Giulio
Thanks for the reply. There are 5 or 6 popular CI tools out there (and there's probably more of them than that). If implemented in the CI tool, each tool would have to implement it for themselves.
Plus a CI tool would have to implement a separate solution for each VCS it supports (e.g. git, perforce, mercurial etc).
As this is a feature that requires an SVN specific implementation and isn't a feature specific to CI builds - it's just a feature to restore a working copy to pristine state - SVN seems like a good place for it?
I suspect developers would also benefit from such a feature on their development PCs too and not just on CI servers?
Kind regards
Simon
Simon Dean
>
> FYI: If you are on windows the TortoiseSVN client's "Clean up" function
> allows you to delete unversioned and ignored files if you want. Also, it allows
> you to revert all changes when you clean up as well.
>
> BOb
>
TortoiseSVN feature is very good. Unfortunately it only seems available through its GUI and not through it's command line.
Here's a quick sample of various people trying to accomplish this feature for themselves through various scripts etc.
http://stackoverflow.com/questions/9082706/using-powershell-and-svn-to-delete-unversioned-files
http://www.guyrutenberg.com/2008/01/18/delete-unversioned-files-under-svn/
http://stackoverflow.com/questions/2803823/how-can-i-delete-all-unversioned-ignored-files-folders-in-my-working-copy
http://patforna.blogspot.com/2009/10/remove-unversioned-files-in-subversion.html
Simon
Giulio Troccoli
On 09/03/12 14:35, Simon Dean wrote:
>> From: Giulio Troccoli [mailto:giulio....@mediatelgroup.co.uk]
>>
>> Sorry, but to me this has got nothing to do with Subversion. Your CI tool is
>> should clean up itself.
>>
>> Having said that, if someone wants to implement such feature I don't think I
>> would have anything against it. But I doubt it will (be implemented)
>>
>> Giulio
> Thanks for the reply. There are 5 or 6 popular CI tools out there (and there's probably more of them than that). If implemented in the CI tool, each tool would have to implement it for themselves.
>
> Plus a CI tool would have to implement a separate solution for each VCS it supports (e.g. git, perforce, mercurial etc).
>
> As this is a feature that requires an SVN specific implementation and isn't a feature specific to CI builds - it's just a feature to restore a working copy to pristine state - SVN seems like a good place for it?
>
> I suspect developers would also benefit from such a feature on their development PCs too and not just on CI servers?
>
[CC to the list, remember to reply-to-all]
Why would the CI implement a different solution for each VCS? Those, I
understand, are files created during the build process, they have got
nothing to do with SVN or any other VCS. And it's not a SVN specific
implementation as, again, the files were not created by SVN so they've
got nothing to do with it.
From Subversion's point of view the the WC is absolutely fine. The
unversioned files are ignore and there are no missing files (and if
there are a simple svn up will restore them)
But maybe I'm missing something?
Simon Dean
> understand, are files created during the build process, they have got nothing
> to do with SVN or any other VCS. And it's not a SVN specific implementation
> as, again, the files were not created by SVN so they've got nothing to do with
> it.
>
> From Subversion's point of view the the WC is absolutely fine. The
> unversioned files are ignore and there are no missing files (and if there are a
> simple svn up will restore them)
>
> But maybe I'm missing something?
A CI implementation would have to implement it specifically each VCS as it would have to call the VCS to found out what files/directories are unversioned and ignored.
Interestingly TortoiseSVN also implements the same feature (as Bob Archer highlights). This is a good example of the same feature being used for non-CI purposes. Unfortunately the TortoiseSVN implementation is both Windows specific and not available from the command line.
It is helpful though to see how TortoiseSVN describes the feature (quoted from http://tortoisesvn.net/docs/nightly/TortoiseSVN_en/tsvn-dug-cleanup.html)
Delete unversioned files and folders, Delete ignored files and folders
======================================================================
This is a fast and easy way to remove all generated files in your working copy. All files and folders that are not versioned are moved to the trash bin.
Note: you can also do the same from the TortoiseSVN → Revert dialog. There you also get a list of all the unversioned files and folders to select for removal.
Giulio Troccoli
On 09/03/12 15:03, Simon Dean wrote:
>> From: Giulio Troccoli [mailto:giulio....@mediatelgroup.co.uk]
>>
>> Why would the CI implement a different solution for each VCS? Those, I
>> understand, are files created during the build process, they have got nothing
>> to do with SVN or any other VCS. And it's not a SVN specific implementation
>> as, again, the files were not created by SVN so they've got nothing to do with
>> it.
>>
>> From Subversion's point of view the the WC is absolutely fine. The
>> unversioned files are ignore and there are no missing files (and if there are a
>> simple svn up will restore them)
>>
>> But maybe I'm missing something?
> A CI implementation would have to implement it specifically each VCS as it would have to call the VCS to found out what files/directories are unversioned and ignored.
So the CI would rely on another piece of software, SVN in this case, to
know what it has created in terms of files. Well, it doesn't seem right
to me.
Simon Dean
With TortoiseSVN providing this functionality through a GUI to end-users, I guess the discussion of whether such a feature should be implemented in a CI tool or in the SVN client is a moot point.
Either way, I think we'll have to agree to disagree.
Kind regards
Simon
Andreas Krey
...
> Sorry, but to me this has got nothing to do with Subversion.
'course it does. It knows which files are to be ignored, and thus
can be savely thrown away, and it does know which files are not
under version control, and thus should be removed for a clean
working copy.
Likewise, svn forces me to use the \( -name .svn -prune \) clause
in all my find-greps, even though the presense of the .svn folders
is clearly svns business.
> Your CI tool is should clean up itself.
That would be not the CI tool but the entire build system, potentially
including editors used etc. pp. (CI being just an example of the usefulness
of an hypothetical 'svn cleanup'.) We go the way of keeping a clean
sandbox and making a copy of it for each build.
And it's not as if svn itself always cleans up after itself.
Andreas
--
"Totally trivial. Famous last words."
From: Linus Torvalds <torvalds@*.org>
Date: Fri, 22 Jan 2010 07:29:21 -0800
Stefan Sperling
> Hi
>
> Are there any plans to add a command to SVN that cleans a working copy or path of all unversioned and/or ignored files and directories?
There is a related open feature request in our issue tracker:
http://subversion.tigris.org/issues/show_bug.cgi?id=3549
As far as I know, nobody is currently working on this.
You could add yourself to the Cc list there to be informed once the
status of this issue changes.
Or, instead of waiting passively, you could try to compose a detailed
design spec for this feature and send it for discussion to the dev@
list. Once this discussion reaches consensus on the proposed design
you could implement it and send a patch.
This is one of those features that shouldn't involve too much work,
and where new contributors could get their feet wet with design and
implementation work. And it is quite a popular feature request so anyone
who designs and implements this will earn a good stash of community
karma points.
Geoff Hoffman
On Fri, Mar 09, 2012 at 01:56:34PM +0000, Simon Dean wrote:
> Hi
>
> Are there any plans to add a command to SVN that cleans a working copy or path of all unversioned and/or ignored files and directories?
There is a related open feature request in our issue tracker:
http://subversion.tigris.org/issues/show_bug.cgi?id=3549
As far as I know, nobody is currently working on this.
You could add yourself to the Cc list there to be informed once the
status of this issue changes.
Or, instead of waiting passively, you could try to compose a detailed
design spec for this feature and send it for discussion to the dev@
list. Once this discussion reaches consensus on the proposed design
you could implement it and send a patch.
This is one of those features that shouldn't involve too much work,
and where new contributors could get their feet wet with design and
implementation work. And it is quite a popular feature request so anyone
who designs and implements this will earn a good stash of community
karma points.
This email, including any attachments, is for the sole use of the intended recipient and may contain confidential information. If you are not the intended recipient, please immediately notify us by reply email or by telephone, delete this email and destroy any copies. Thank you.
Les Mikesell
>
>>> But maybe I'm missing something?
>> A CI implementation would have to implement it specifically each VCS as it would have to call the VCS to found out what files/directories are unversioned and ignored.
>
> So the CI would rely on another piece of software, SVN in this case, to
> know what it has created in terms of files. Well, it doesn't seem right
> to me.
So how would you propose doing this across different VCS? I don't see
how adding a new command to subversion that would do the equivalent of
deleting everything except svn metadata followed by a revert and maybe
an update helps with VCS independence. Your CI already has to know
how to use each VCS independently anyway.
--
Les Mikesell
lesmi...@gmail.com
Stefan Sperling
> A couple of things I find interesting... the bug tracker you linked to is 3
> years old and still on tigris.org. Assuming that is still the valid place
> for tracking subversion features & bugs?
The issue tracker has not been migrated to apache.org yet.
See http://subversion.apache.org/reporting-issues.html#queries
The bug is 3 years old because someone had the same feature
idea 3 years ago, filed an issue, and nothing has happened since.
> Hypothetically speaking, how would svn revert --recursive *--force* PATH be
> different/better than svn cleanup *--remove-unversioned-files* PATH (a bit
> verbose IMO)? I guess it does make more sense for the proposed feature to
> be a switch on svn cleanup.
svn revert deletes some unversioned files in certain circumstances.
This was recently discussed here:
http://thread.gmane.org/gmane.comp.version-control.subversion.devel/134154
In my opinion removing unversioned data is not the job of 'svn revert'.
It is supposed to roll back the working copy into the state that it was in
after the last checkout or update. In other words, it destroys changes
made to *versioned* files and directories.
Anyway, which subcommand this feature belongs in is a question that should
be answered by a concrete design proposal. It doesn't really matter all
that much.
> This is such great advice. If as many people would give to open source as
> take from it, imagine how much richer everything would be?
Yes, and the world would be full of flying pigs, too!
> I went over to http://subversion.apache.org/download/ and
> was surprised there doesn't seem to be an svn checkout option, even read
> only?
See http://subversion.apache.org/source-code.html (linked from the
very bottom of the download page -- perhaps not as easy to find as it
should be).
Geoff Hoffman
Bob Archer
I expect this issue hasn't been actioned because it is so easy to mitigate by the developer...
1. Ensure your build artifacts are kept separate or group in a folder.
2. Create a clean batch or shell file that removes all the build artifacts.
Personally, I have a clean.bat file in our project root that deletes all the bin and obj folders (Visual Studio inherently keeps them in their own folders) recursively. It took about 5 minutes to create and commit.
That said, I don't object to a --include-unversioned --include-ignored switches to svn CLI.
BOb
Kuno Meyer
clean-tree command. With the bzr-svn plugin installed, you should be able to
directly operate on SVN working trees:
bzr clean-tree --ignored --unknown --detritus --force
Johan Corveleyn
<Simon...@moneysupermarket.com> wrote:
> Hi
>
> Are there any plans to add a command to SVN that cleans a working copy or path of all unversioned and/or ignored files and directories?
>
> This is a very common need for automated Continuous Integration builds where a working copy is reused for multiple runs of the same build. Currently there is no simple and fast way to restore a working copy to a prestine state. Often users have to choose between i) completely deleting the working copy for every build and then doing a fresh checkout from scratch or ii) living with lots of unversioned and ignored files and directories building up with each successive build.
>
> The only option at the moment is to write a shell/batch script to provide this feature which is messy and there's common way to do this. A new SVN command or enhanced exiting command that provided this functionality would be incredibly useful.
There is also an existing perl script here:
http://svn.apache.org/repos/asf/subversion/trunk/contrib/client-side/svn-clean
but I don't know its status (I've never used it, I just know it exists).
--
Johan
Andreas Krey
...
> > So the CI would rely on another piece of software, SVN in this case, to
> > know what it has created in terms of files. Well, it doesn't seem right
> > to me.
>
> So how would you propose doing this across different VCS? I don't see
> how adding a new command to subversion that would do the equivalent of
> deleting everything except svn metadata followed by a revert and maybe
> an update helps with VCS independence. Your CI already has to know
> how to use each VCS independently anyway.
Yes. But knowing to invoke 'svn update' and 'svn cleanup -fdX' is somewhat
different from knowing to "svn status --no-ignore | awk '/^I / {print $2}'
| xargs rm -rf" (or similar) with all the caveats that come with strange
characters in file names.
If you argue that a CI/XY tool should find out for itself what files are
not under svn control then one could argue analogously that it should
as well bypass svn for doing updates. :-)
But then, svn currently has more important issues than implementing
'svn cleanup'.
Les Mikesell
>
> If you argue that a CI/XY tool should find out for itself what files are
> not under svn control then one could argue analogously that it should
> as well bypass svn for doing updates. :-)
I'd argue that tools have no business removing any files they didn't
create unless you name them explicitly. And that complicated things
that you want a CI to automate should be scripted with the script
managed in your VCS anyway so it works the same when you do it
yourself as when automated.
--
Les Mikesell
lesmi...@gmail.com
Andreas Krey
...
> create unless you name them explicitly. And that complicated things
> that you want a CI to automate should be scripted with the script
> managed in your VCS anyway so it works the same when you do it
> yourself as when automated.
Except for the part where not everyone should be forced to reinvent
the wheel of 'put the sandbox in a pristine state' as in 'cd ..;
rm -r $sandboxname; svn checkout -r $rev $url $sandboxname', but more
efficiently and without hitting the network.
While this is handy in a CI environment (since then we don't need to
deal with the specifics of the build procedure[1]) it is in no way
restricted to that case.
Andreas
[1] Like "'make clean' doesn't fully clean up because someone forgot to
commit the 'clean' rule for the new source's object file".
Or simply "oops, we forgot to do 'make clean' before switching branches".
David Chapman
> On Sat, 10 Mar 2012 10:47:55 +0000, Les Mikesell wrote:
> ...
>> I'd argue that tools have no business removing any files they didn't
>> create unless you name them explicitly. And that complicated things
>> that you want a CI to automate should be scripted with the script
>> managed in your VCS anyway so it works the same when you do it
>> yourself as when automated.
> Except for the part where not everyone should be forced to reinvent
> the wheel of 'put the sandbox in a pristine state' as in 'cd ..;
> rm -r $sandboxname; svn checkout -r $rev $url $sandboxname', but more
> efficiently and without hitting the network.
>
And except for the part where a cross-platform developer like me has to
reinvent the wheel for every platform I work on. I try to ensure that
"make clean" is very robust, but the makefiles aren't always updated
consistently. Every now and then debris is left behind on one platform
or another. And that occurs even without CI.
If I were rich or bored, I'd be tempted to try to implement this feature
myself. The pieces should already be in the code base.
--
David Chapman dcch...@acm.org
Chapman Consulting -- San Jose, CA
Software Development Done Right.
www.chapman-consulting-sj.com
Les Mikesell
>>
> Except for the part where not everyone should be forced to reinvent
> the wheel of 'put the sandbox in a pristine state' as in 'cd ..;
> rm -r $sandboxname; svn checkout -r $rev $url $sandboxname', but more
> efficiently and without hitting the network.
Our VCS/CI and build slaves sit in the same lab on a fast network
(which seems like it would be the common arrangement...) so I've never
been particularly concerned about hitting the network - that's what it
is designed for. And the CI knows how to do a clean checkout itself.
--
Les Mikesell
lesmi...@gmail.com
Konstantin Kolinko
> Hi
>
> Are there any plans to add a command to SVN that cleans a working copy or path of all unversioned and/or ignored files and directories?
>
> This is a very common need for automated Continuous Integration builds where a working copy is reused for multiple runs of the same build. Currently there is no simple and fast way to restore a working copy to a prestine state. Often users have to choose between i) completely deleting the working copy for every build and then doing a fresh checkout from scratch or ii) living with lots of unversioned and ignored files and directories building up with each successive build.
>
> The only option at the moment is to write a shell/batch script to provide this feature which is messy and there's common way to do this. A new SVN command or enhanced exiting command that provided this functionality would be incredibly useful.
>
> As an example, here is the DOS batch script that I use at the moment:
>
> @echo off
> :: revert any uncommitted changes
> svn revert . --recursive
If you use svn 1.7 (that is you have only ".svn" directory) it is easy to
1. just delete everything
2. do your "svn revert".
The revert will restore missing files and folders from their pristine copies.
It would not work if you have externals there, though. Those will be
restored if you do "svn up".
Best regards,
Konstantin Kolinko
Nico Kadel-Garcia
Les Mikesell
>>
>> > Except for the part where not everyone should be forced to reinvent
>> > the wheel of 'put the sandbox in a pristine state' as in 'cd ..;
>> > rm -r $sandboxname; svn checkout -r $rev $url $sandboxname', but more
>> > efficiently and without hitting the network.
>>
>> Our VCS/CI and build slaves sit in the same lab on a fast network
>> (which seems like it would be the common arrangement...) so I've never
>> been particularly concerned about hitting the network - that's what it
>> is designed for. And the CI knows how to do a clean checkout itself.
>
>
> Some of us have to work through VPN's, and have relatively large checkouts.
> A 100 Meg checkout is certainly not unusual in Java environments,
> where bulky .war files are common, and that can be really slow over a home
> DSL setup or
That seems wrong or at least unnecessarily inconvenient for a CI
setup. And if you are doing it by hand, why not just delete
everything but your .svn directory and revert?
--
Les Mikesell
lesmi...@gmail.com
Andreas Krey
...
> setup.
You're a bit hung up on the 'CI' token. That isn't the only situation
where a 'svn cleanup' can be useful.
> And if you are doing it by hand, why not just delete
> everything but your .svn directory and revert?
Typical VCS operations should not only be possible but also easy.
(And even the 'everything but .svn' part is tricky.)
Andreas
Les Mikesell
> ...
>> That seems wrong or at least unnecessarily inconvenient for a CI
>> setup.
>
> You're a bit hung up on the 'CI' token. That isn't the only situation
> where a 'svn cleanup' can be useful.
Subversion probably isn't the best VCS to use if you can't arrange
reasonable connectivity to the repository to make clean checkouts
feasible.
>> And if you are doing it by hand, why not just delete
>> everything but your .svn directory and revert?
>
> Typical VCS operations should not only be possible but also easy.
> (And even the 'everything but .svn' part is tricky.)
With post-1.7 versions, it shouldn't be hard at all - at least on
systems where filenames starting with '.' don't expand in wildcards.
But, if I know I'm going to want to return to a particular
working-copy state, I just copy the whole thing locally before making
the changes I may want to discard. I'd be more likely to do that to
preserve a set of local changes that I wasn't sure about committing or
to hold a checkpoint offline like you might in a distributed VCS, but
it should work as well to have an 'already reverted' copy sitting
locally when bandwidth is a concern. This isn't particularly optimal
with subversion either since you end up with two full copies of
everything in each instance of the working copies, but at least it
gives you a choice of disk or network resources.
--
Les Mikesell
lesmi...@gmail.com
Daniel Shahaf
> If you use svn 1.7 (that is you have only ".svn" directory) it is easy to
> 1. just delete everything
> 2. do your "svn revert".
>
> The revert will restore missing files and folders from their pristine copies.
>
>
> It would not work if you have externals there, though. Those will be
> restored if you do "svn up".
For completeness, 'svn up -r BASE' will probably be useful. (And
assuming the externals definitions are pegged.)
Nico Kadel-Garcia
On Sun, Mar 11, 2012 at 1:10 PM, Andreas Krey <a.k...@gmx.de> wrote:
> ...
>> That seems wrong or at least unnecessarily inconvenient for a CI
>> setup.
>
> You're a bit hung up on the 'CI' token. That isn't the only situation
> where a 'svn cleanup' can be useful.Subversion probably isn't the best VCS to use if you can't arrange
reasonable connectivity to the repository to make clean checkouts
feasible.
>> And if you are doing it by hand, why not just delete
>> everything but your .svn directory and revert?
>
> Typical VCS operations should not only be possible but also easy.
> (And even the 'everything but .svn' part is tricky.)
With post-1.7 versions, it shouldn't be hard at all - at least on
systems where filenames starting with '.' don't expand in wildcards.
But, if I know I'm going to want to return to a particular
working-copy state, I just copy the whole thing locally before making
the changes I may want to discard.
I'd be more likely to do that to
preserve a set of local changes that I wasn't sure about committing or
to hold a checkpoint offline like you might in a distributed VCS, but
it should work as well to have an 'already reverted' copy sitting
locally when bandwidth is a concern.
This isn't particularly optimal
with subversion either since you end up with two full copies of
everything in each instance of the working copies, but at least it
gives you a choice of disk or network resources.
Les Mikesell
>
>> Subversion probably isn't the best VCS to use if you can't arrange
>> reasonable connectivity to the repository to make clean checkouts
>> feasible.
>
>
> Been there, done that, got my wrist slapped for questioning the mandated
> standard, no matter what it was. In this sort of case.
Mandating the use of a tool should go hand in hand with providing the
necessary resources. And with subversion, one of those resources is
reasonable network connectivity to the one-and-only repository (or the
harder task of replicated repositories)...
> The lack of a
> "restore this working copy to the pristine state" feature does not seem
> fundamental to any particular Subversion approach, does it? The "CVS done
> right" approach, the "centralized source control with cheap branching", or
> any of the other features?
Not deleting random unrelated files could be considered a feature.
> Right: it can be written locally, and it can be written *wrong* in a lot of
> ways locally.
Yes, but imagine the ways a command that removes files not under
version control could go wrong.
An option to 'status' that only shows the unversioned filenames so you
don't have to parse them out of the larger list if you want to remove
them might be nice. Doing it blindly seems a little heavy-handed.
--
Les Mikesell
lesmi...@gmail.com
Nico Kadel-Garcia
On Sun, Mar 11, 2012 at 5:28 PM, Nico Kadel-Garcia <nka...@gmail.com> wrote:
>
>> Subversion probably isn't the best VCS to use if you can't arrange
>> reasonable connectivity to the repository to make clean checkouts
>> feasible.
>
>
> Been there, done that, got my wrist slapped for questioning the mandated
> standard, no matter what it was. In this sort of case.
Mandating the use of a tool should go hand in hand with providing the
necessary resources. And with subversion, one of those resources is
reasonable network connectivity to the one-and-only repository (or the
harder task of replicated repositories)...
> The lack of a
> "restore this working copy to the pristine state" feature does not seem
> fundamental to any particular Subversion approach, does it? The "CVS done
> right" approach, the "centralized source control with cheap branching", or
> any of the other features?
Not deleting random unrelated files could be considered a feature.
> Right: it can be written locally, and it can be written *wrong* in a lot of
> ways locally.
Yes, but imagine the ways a command that removes files not under
version control could go wrong.
An option to 'status' that only shows the unversioned filenames so you
don't have to parse them out of the larger list if you want to remove
them might be nice. Doing it blindly seems a little heavy-handed.
David Weintraub
* This is a Subversion client function and not a function of
Subversion itself. Subversion thoughtfully publishes a Subversion API
that developers can use to create their own Subversion clients. Notice
that TortoiseSVN does not require a Subversion command line client
unlike TortoiseCVS that requires a CVS command line client. Notice
that Eclipse and AnkhSvn don't require a Subversion command line
client. Anyone can create a Subversion client with the API, and if
they choose, can create this type of functionality. Even if you put
this functionality in the Subversion command line client, Subversion
clients that use the API might not have it.
* Many continuous integration systems (like Jenkins) already have an
option to remove non-versioned files before doing a build. The OP
mentioned this as an issue.
* It is traditional when you design your build system to make sure
your built objects are stored in a temporary directory, so they don't
pollute your source area. This way, it's easy to remove all of your
built objects by simply deleting the temporary folder. In Java, I tell
developers to create a folder called "target" under their project
root, and store all built files and logs over there. This also ensures
that you don't accidentally add built files into your repository.
* It is traditional as part of your build system to have a "clean"
target that does this.
* It is easy enough to create a script to do this job for you. In
Unix, if you don't have spaces in your file names, the one liner "svn
status | awk '/^\?/ {print $2}' | xargs rm -rf" will do the job.
* There are more serious features that Subversion is missing. The best
known is a true "obliterate" command to remove obsolete revisions of
files. For example, if someone commits a file that contains customer
proprietary information, there's no easy way to completely remove that
revision from Subversion. You have to take down the repository, and do
a dump, filter, and load. I'd rather the Subversion team work on this
issue, which involves the way the Subversion server acts, rather than
this issue.
This maybe why this has never even been considered as a feature. It
really doesn't affect Subversion itself. This is something that the
build system should be handling. In many open source projects, the
source and build files are tarred up and distributed. What happens
when the source is distributed, and you can't depend upon the version
control system to do this for you?
Nico Kadel-Garcia commented that this is important because his build
contains many war files. You really should never check in built
objects into Subversion. Instead, these objects should be stored in a
release system where they can be downloaded as needed. Maven does
this, but you can easily do this with Ant and Ivy too. Storing wars
and jars in version control is an easy way to start off a project, but
will cause major issues later on. Most of the time, you lose track of
what version of the war or jar you're build is depending upon. Plus,
no matter what version control system you're using, checking in and
out binary based files is slow and takes up a lot of space.
It gets to the point where your project is fighting a constant battle
with stability. We have a project with five copies of the same jar
file. Even worse, there are three completely separate versions of this
jar in our project with two different ones in the same ear file. Every
time something changes, something goes wrong. It'll take us months to
clean up this mess.
--
David Weintraub
qaz...@gmail.com
Simon Dean
Not many third-party SVN clients (any?) implement the SVN client-side implementation themselves. They use SVN libraries for that - something like SVNKit (http://svnkit.com/)
In fact, the Subverson project discourages other clients from directly manipulating the contents of the .svn directory. To quote http://subversion.apache.org/docs/release-notes/1.7.html:
"Subversion 1.7 working copies have just one .svn directory...This directory includes (among other things) an SQLite-backed database which contains all of the metadata Subversion needs for that working copy...We highly discourage external tools from modifying the data held in this database, as such modification is likely to result in working copy corruption."
Other people have commented on the fragility of the "clean" task of a build script. If you use things like NuGet and Bundler in codebases, they result in multiple directories that need "cleaning" - e.g. .\vendor\bundle, .\packages etc. You'd be surprised how many unversioned files creep into a CI build when all you're relying on is the build script's "clean" task
> -----Original Message-----
> From: David Weintraub [mailto:qaz...@gmail.com]
> Sent: 12 March 2012 16:33
> To: Nico Kadel-Garcia
> Cc: Les Mikesell; Andreas Krey; Giulio Troccoli; Simon Dean;
> us...@subversion.apache.org
> Subject: Re: Feature request - SVN command to clean a working copy of all
> unversioned and ignored files and directories
>
-----------------------------------------------------------------------------------------------------------------------------------------
Simon Dean
Is it the Subversion API that provides the bulk of their various clients' functionality? What keeps the command arguments etc. of the various clients the same?
Thanks
Simon
Stefan Sperling
> Not having contributed to SVN before, I don't really know how the SVN client(s) are implemented. How much of the code in the various Windows and Linux SVN command line clients (e.g. CollabNet's Windows command line client, SlikSVN's Windows command line client, TortoiseSVN's svn.exe etc) comes from the codebase that Apache now manage and how much is unique to each client?
>
> Is it the Subversion API that provides the bulk of their various clients' functionality? What keeps the command arguments etc. of the various clients the same?
>
The bulk of code comes from Apache Subversion and is shared among client
implementations.
Clients mostly provide an interface built around the client library
provided by Apache Subversion. The command line client is a very
minimal wrapper around this library.
The GUI clients offer more features, and some ship extra functionality
(e.g. TortoiseSVN's revision graph) which is usually implemented on
top of the Subversion APIs.
SlikSVN and CollabNet's binaries are just a compiled builds of Apache
Subversion. I believe they are not even patched in any significant way,
if at all.
The odd one out is SVNKit, which is a separate implementation of the
Subversion client library in JAVA. It is fully compatible, but is not
using any Apache Subversion code directly. Clients that use SVNKit are
not using the Apache Subversion code base.
The JavaHL bindings provide a Java interface to the Apache Subversion
code base. JavaHL and SVNKit developers work together to keep their
APIs in sync, so that Java-based GUIs like Subclipse or SmartSVN can
use either implementation.
Nico Kadel-Garcia
Here's why I don't think this is a feature for the Subversion project:
* This is a Subversion client function and not a function of
Subversion itself. Subversion thoughtfully publishes a Subversion API
* Many continuous integration systems (like Jenkins) already have an
option to remove non-versioned files before doing a build. The OP
mentioned this as an issue.
* It is traditional when you design your build system to make sure
your built objects are stored in a temporary directory, so they don't
pollute your source area. This way, it's easy to remove all of your
built objects by simply deleting the temporary folder. In Java, I tell
developers to create a folder called "target" under their project
root, and store all built files and logs over there. This also ensures
that you don't accidentally add built files into your repository.
* It is traditional as part of your build system to have a "clean"
target that does this.
* It is easy enough to create a script to do this job for you. In
Unix, if you don't have spaces in your file names, the one liner "svn
status | awk '/^\?/ {print $2}' | xargs rm -rf" will do the job.
* There are more serious features that Subversion is missing. The best
known is a true "obliterate" command to remove obsolete revisions of
files. For example, if someone commits a file that contains customer
proprietary information, there's no easy way to completely remove that
revision from Subversion. You have to take down the repository, and do
a dump, filter, and load. I'd rather the Subversion team work on this
issue, which involves the way the Subversion server acts, rather than
this issue.
This maybe why this has never even been considered as a feature. It
really doesn't affect Subversion itself. This is something that the
build system should be handling. In many open source projects, the
source and build files are tarred up and distributed. What happens
when the source is distributed, and you can't depend upon the version
control system to do this for you?
Nico Kadel-Garcia commented that this is important because his build
contains many war files. You really should never check in built
objects into Subversion. Instead, these objects should be stored in
release system where they can be downloaded as needed. Maven does
this, but you can easily do this with Ant and Ivy too. Storing wars
and jars in version control is an easy way to start off a project, but
will cause major issues later on. Most of the time, you lose track of
what version of the war or jar you're build is depending upon. Plus,
no matter what version control system you're using, checking in and
out binary based files is slow and takes up a lot of space.
It gets to the point where your project is fighting a constant battle
with stability. We have a project with five copies of the same jar
file. Even worse, there are three completely separate versions of this
jar in our project with two different ones in the same ear file. Every
time something changes, something goes wrong. It'll take us months to
clean up this mess.
David Weintraub
<Simon...@moneysupermarket.com> wrote:
> I suspect TortoiseSVN uses the official Subversion client code under the hood. There's no way they'd
> re-implement a whole SVN client from scratch.
I don't know if Tortoise uses any Subversion command line client code,
but TortoiseSVN does use the official Subversion API libraries. That's
what made Subversion so different from CVS. With CVS, you either had
to reimplement the client yourself, or use the offical CVS client as a
backend. In Subversion you write your own client, and just use the
API.
SVNKit is very different. SVNKit is a Java third party
reimplementation of the Subversion API, and it's not the complete API
either. On the other hand, the JavaHL API is a front end to the
official Subversion API.
So, it's possible for someone to write a Subversion client that does
do a "clean up". In fact, the Jenkins Continuous build system has the
option of doing a thorough clean before doing an update.
> Other people have commented on the fragility of the "clean" task of a build script. If you use things like NuGet
> and Bundler in codebases, they result in multiple directories that need "cleaning" - e.g. .\vendor\bundle,
> .\packages etc. You'd be surprised how many unversioned files creep into a CI build when all you're relying
> on is the build script's "clean" task
For some reason, the .NET/C# world is behind in this concept when
compared to the Java world. By default, there's a "clean" target in
VisualStudio builds, but it doesn't do a very good job of cleaning.
However, it is still the developer's responsibility to make sure that
their "clean" target cleans everything. There's a special
"BeforeClean" target that's called when the "clean" target is called.
You can add your own code here to clean up things that the default
"clean" target doesn't do.
--
David Weintraub
qaz...@gmail.com
Andreas Krey
...
> So, it's possible for someone to write a Subversion client that does
> do a "clean up".
So, what you're saying is that, because it is possible to implement
'svn cleanup' on top of the svn client libs, the official svn client
won't get such a feature? (This immediately begs the question why
there is an svn CLI client at all, and not just the library. ;-)
...
> However, it is still the developer's responsibility to make sure that
> their "clean" target cleans everything.
No actually; 'make clean' is responsible to clean up everything that
make produced, not 'everything', like editor backup files (or the
remains of aborted merges).
A 'throw out everything unversioned' VCS command comes in quite
handy when
- you switched branches before calling 'make clean' because now the
.o file of some source existing only in one branch will be left over,
- you're simply trying to get 'make clean' to actually work and don't
want to iterate through the 'svn status' output each time.
And 'remove; make new checkout' isn't always a valid workaround because
you may want to keep local modifications of versioned files.
Summary: There are a lot of reason not to implement a 'scrub the WC
of unversioned files', but in my opinion "it's not our job" isn't one
of them.
Bert Huijben
> From: David Weintraub [mailto:qaz...@gmail.com]
> Sent: woensdag 14 maart 2012 2:14
> To: Simon Dean
> Cc: Nico Kadel-Garcia; Les Mikesell; Andreas Krey; Giulio Troccoli;
> us...@subversion.apache.org
> Subject: Re: Feature request - SVN command to clean a working copy of all
> unversioned and ignored files and directories
>
The standard files produced by Visual Studio should clean up everything, but
if you customize your build you should also customize the cleanup. (This
assumes that you pass the same custom variables to the build as the cleanup
step of MSBuild. If not it cleans up a different build)
And Visual Studio makes adding custom build tasks very easy, but doesn't
provide an easy way to add your own cleanup tasks. (I see too many scripts
that just call things like xcopy and assume this magically cleans up)
When running from Visual Studio it appears that there are leftovers, but
this is mostly because the Intellisense uses the same infrastructure to get
its information.
If you do find places where the common tasks leave files behind please
report them as Microsoft is still taking bug reports for Visual Studio 11.
And if these are Subversion and/or AnkhSVN specific problems please also
send me a mail so I might be able to reach out via a few other channels.
Bert
Simon Dean
I use Rake and Gradle (migrated to Gradle from Maven). Rake is used for .NET codebases and Gradle for Java. It's very easy for files to slip through a "clean" task. Problem is, a "clean" task doesn't "fail fast" (an agile principle which is also a more general principle: http://en.wikipedia.org/wiki/Fail-fast)
Andreas Krey
...
> I use Rake and Gradle (migrated to Gradle from Maven). Rake is used for .NET codebases and Gradle for Java. It's very easy for files to slip through a "clean" task.
Actually the whole notion of a 'clean task' is misleading. Any build
task should automatically contribute to a list of files/directories to
be deleted on 'clean'. After all, e.g. a javac task incarnation knows
what directories it would create, so it can put them on the clean
list. (Speaking for ant here, other build tools may be smarter there.)
> Problem is, a "clean" task doesn't "fail fast"
It can't. Failure there is an omission to do something; which could at best
be noted after the fact by the output of 'svn clean -n' being nonempty.
Nico Kadel-Garcia
What if you deliver your source code *outside* of your revision control system? This is quite common for open source projects. If I can't trust the build process to do a clean, I'm in trouble.
Les Mikesell
So that's what, 500 bytes total if you had just deleted the whole
thing and checked it out clean?
--
Les Mikesell
lesmi...@gmail.com