size of .svn/pristine directory (svn 1.7.x)

[Vincent Lefevre]

I noticed that the size of the .svn/pristine directory can get huge
over time (several times the expected size). A "svn cleanup" solves
the problem, but
1. this isn't documented (I'm wondering how many users know that);
2. this isn't automatic.

About (2), svn could warn the user when a cleanup could be needed.
I don't know what is the best solution.

I've reported a bug on the Debian BTS about that:

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=731769

--
Vincent Lef�vre <vin...@vinc17.net> - Web: <http://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

[Stefan Sperling]

On Mon, Dec 09, 2013 at 05:30:21PM +0100, Vincent Lefevre wrote:
> I noticed that the size of the .svn/pristine directory can get huge
> over time (several times the expected size). A "svn cleanup" solves
> the problem, but
> 1. this isn't documented (I'm wondering how many users know that);
> 2. this isn't automatic.
>
> About (2), svn could warn the user when a cleanup could be needed.
> I don't know what is the best solution.
>
> I've reported a bug on the Debian BTS about that:
>
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=731769

Where is the point in filing bugs with Debian about that?
Are they going to do anything else than tracking related
bug entries upstream?

There already is http://subversion.tigris.org/issues/show_bug.cgi?id=4071

Would you like to submit a documentation patch, for example to
the svnbook? See http://svnbook.red-bean.com/

[Daniel Shahaf]

Vincent Lefevre wrote on Mon, Dec 09, 2013 at 17:30:21 +0100:
> I noticed that the size of the .svn/pristine directory can get huge
> over time (several times the expected size). A "svn cleanup" solves
> the problem, but
> 1. this isn't documented (I'm wondering how many users know that);

It's documented in the 1.7 release notes.

http://subversion.apache.org/docs/release-notes/1.7#wc-pristines

> 2. this isn't automatic.

Yes, that's a known bug that we need to fix someday.

[Ivan Zhakov]

Btw it's not a bug for some use cases: Subversion doesn't download
files if user switches to branch and then back to trunk. Which is very
useful in some scenarios.


--
Ivan Zhakov

[Daniel Shahaf]

Agreed, that'd be an argument for not deleting pristines *as soon as*
they're not necessary, but less often then that. You should mention it
on the aforelinked issue ticket if it's not already there :)

[Ivan Zhakov]

Done. Thanks for suggestion!


--
Ivan Zhakov
CTO | VisualSVN | http://www.visualsvn.com

[Vincent Lefevre]

On 2013-12-09 19:55:54 +0200, Daniel Shahaf wrote:
> Vincent Lefevre wrote on Mon, Dec 09, 2013 at 17:30:21 +0100:
> > I noticed that the size of the .svn/pristine directory can get huge
> > over time (several times the expected size). A "svn cleanup" solves
> > the problem, but
> > 1. this isn't documented (I'm wondering how many users know that);
>
> It's documented in the 1.7 release notes.
>
> http://subversion.apache.org/docs/release-notes/1.7#wc-pristines

Unfortunately there are lots of things in the release notes, and it
is difficult to know what is important or not. "svn cleanup" doesn't
say much about what it does.

There are also Debian specific problems: in a huge upgrade, the user
doesn't necessary notice that a new major version (with significant
changes and regressions) of some software is installed, unless this
is announced in NEWS.Debian (it usually is, but it wasn't for 1.7).

[Vincent Lefevre]

On 2013-12-09 17:37:28 +0100, Stefan Sperling wrote:
> On Mon, Dec 09, 2013 at 05:30:21PM +0100, Vincent Lefevre wrote:
> > I noticed that the size of the .svn/pristine directory can get huge
> > over time (several times the expected size). A "svn cleanup" solves
> > the problem, but
> > 1. this isn't documented (I'm wondering how many users know that);
> > 2. this isn't automatic.
> >
> > About (2), svn could warn the user when a cleanup could be needed.
> > I don't know what is the best solution.
> >
> > I've reported a bug on the Debian BTS about that:
> >
> > http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=731769
>
> Where is the point in filing bugs with Debian about that?
> Are they going to do anything else than tracking related
> bug entries upstream?

Yes, sometimes they do. For instance, the change (which is some kind
of regression) could be announced in NEWS.Debian to make the user
aware of it (for new features, pointing to the release notes should
be sufficient). An optional cron script to check the working copies
could be useful too.

> There already is http://subversion.tigris.org/issues/show_bug.cgi?id=4071
>
> Would you like to submit a documentation patch, for example to
> the svnbook? See http://svnbook.red-bean.com/

First, "svn help cleanup" currently says:

cleanup: Recursively clean up the working copy, removing locks, resuming
unfinished operations, etc.

I suggest to change it to something like:

cleanup: Recursively clean up the working copy, removing locks, resuming
unfinished operations, removing unreferenced pristines, etc.
In 1.7, we recommend to run "svn cleanup" periodically to reduce the size
of the working copy.

I've attached a patch for the svnbook (not tested).

[Vincent Lefevre]

On machines with sufficient disk space, I agree that this can be
useful.

I suppose it is also the case when doing "svn up -r<some_rev>" if the
pristines of <some_rev> have been kept.

[Stefan Sperling]

On Tue, Dec 10, 2013 at 01:28:52AM +0100, Vincent Lefevre wrote:
> First, "svn help cleanup" currently says:
>
> cleanup: Recursively clean up the working copy, removing locks, resuming
> unfinished operations, etc.
>
> I suggest to change it to something like:
>
> cleanup: Recursively clean up the working copy, removing locks, resuming
> unfinished operations, removing unreferenced pristines, etc.
> In 1.7, we recommend to run "svn cleanup" periodically to reduce the size
> of the working copy.

Perhaps saying "to remove unferenced pristine copies of files" or
something similar is clearer than "reduce the size of the wording copy"?

> I've attached a patch for the svnbook (not tested).

> Index: ch02-basic-usage.xml
> ===================================================================
> --- ch02-basic-usage.xml (revision 4563)
> +++ ch02-basic-usage.xml (working copy)
> @@ -2892,6 +2892,21 @@
>
> </sect2>
>
> + <!-- =============================================================== -->
> + <sect2 id="svn.tour.cleanup.size">
> + <title>Reducing the Size of a Working Copy</title>
> +
> + <para>Subversion 1.7 does not automatically remove pristines
> + that are no longer referenced. As a consequence, a working
> + copy can grow up to several times the size that is really
> + needed. We recommend to run <command>svn cleanup</command>
> + periodically in order to claim back the disk space of
> + unreferenced pristines<footnote><para>See <ulink
> + url="http://subversion.tigris.org/issues/show_bug.cgi?id=4071"
> + /> for details.</para></footnote>.</para>
> +
> + </sect2>
> +
> </sect1>
>
> <!-- ================================================================= -->

Thanks!
This list is not the right list for svnbook patches.
I've forwarded your patch to the right list, and put you in Cc.

[Vincent Lefevre]

On 2013-12-10 12:12:06 +0100, Stefan Sperling wrote:
> On Tue, Dec 10, 2013 at 01:28:52AM +0100, Vincent Lefevre wrote:
> > First, "svn help cleanup" currently says:
> >
> > cleanup: Recursively clean up the working copy, removing locks, resuming
> > unfinished operations, etc.
> >
> > I suggest to change it to something like:
> >
> > cleanup: Recursively clean up the working copy, removing locks, resuming
> > unfinished operations, removing unreferenced pristines, etc.
> > In 1.7, we recommend to run "svn cleanup" periodically to reduce the size
> > of the working copy.
>
> Perhaps saying "to remove unferenced pristine copies of files" or
> something similar is clearer than "reduce the size of the wording copy"?

Or "to claim back the disk space of unreferenced pristines" like in
the release notes?

Though "remove unreferenced pristine copies of files" implies more
disk space, saying it explicitly makes it clear that the goal of
this operation is to free some disk space (in addition to rare
security reasons, in case a file with sensitive information was
added by mistake in the past).

[Branko Čibej]

On 10.12.2013 14:15, Vincent Lefevre wrote:

On 2013-12-10 12:12:06 +0100, Stefan Sperling wrote:

On Tue, Dec 10, 2013 at 01:28:52AM +0100, Vincent Lefevre wrote:

First, "svn help cleanup" currently says:

cleanup: Recursively clean up the working copy, removing locks, resuming
unfinished operations, etc.

I suggest to change it to something like:

cleanup: Recursively clean up the working copy, removing locks, resuming
unfinished operations, removing unreferenced pristines, etc.
In 1.7, we recommend to run "svn cleanup" periodically to reduce the size
of the working copy.

Perhaps saying "to remove unferenced pristine copies of files" or
something similar is clearer than "reduce the size of the wording copy"?

Or "to claim back the disk space of unreferenced pristines" like in
the release notes?

The correct English term is "reclaim". We should review our release notes more diligently. :)

"Reclaim disk space used by unreferenced pristine files" sounds about right.

-- Brane

--
Branko Čibej | Director of Subversion
WANdisco // Non-Stop Data
e. br...@wandisco.com

[Ben Reser]

On 12/10/13 5:28 AM, Branko Čibej wrote:
> The correct English term is "reclaim". We should review our release notes more
> diligently. :)

Agreed. Fixed in r1549892.