Commit is extremly slow on network shares with many unversioned files
Rainer Schmidt
- TortoiseSVN 1.7.2, Build 22327 - 32 Bit / Windows XP SP3
- working copy on a Windows network share (13.610.790.912 Bytes /
252.410 files / 34.157 directories)
- ~1000 files within this network share are under version control with
subversion
- just a simple checkout from one repository with no externals
If i try to commit this folder, i have to wait 1-2 hours until i see
the changed files withing the commit dialog. The old TortoiseSVN needs
only a few seconds for the same action (v1.6.x). It seems that the
actual TortoiseSVN checks every file, versioned or not, within this
folder. Commits on network shares are impossible for me at this time.
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2892466
To unsubscribe from this discussion, e-mail: [users-un...@tortoisesvn.tigris.org].
Felix Saphir
> Situation:
>
> - TortoiseSVN 1.7.2, Build 22327 - 32 Bit / Windows XP SP3
> - working copy on a Windows network share (13.610.790.912 Bytes /
> 252.410 files / 34.157 directories)
> - ~1000 files within this network share are under version control with
> subversion
> - just a simple checkout from one repository with no externals
>
> If i try to commit this folder, i have to wait 1-2 hours until i see
> the changed files withing the commit dialog. The old TortoiseSVN needs
> only a few seconds for the same action (v1.6.x). It seems that the
> actual TortoiseSVN checks every file, versioned or not, within this
> folder. Commits on network shares are impossible for me at this time.
Canonical answer: Move your working copy from the network share to a
local drive. That's highly recommended anyway.
Felix
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2892470
Lübbe Onken
Canonical answer: Move your working copy from the network share to a
local drive. That's highly recommended anyway.
- Lübbe
--
Please help me get more space on Dropbox :)
https://www.dropbox.com/referrals/NTIwMzcxNjI5
___
oo // \\ "De Chelonian Mobile"
(_,\/ \_/ \ TortoiseSVN
\ \_/_\_/> The coolest Interface to (Sub)Version Control
/_/ \_\ http://tortoisesvn.net PGP Key ID 0x23F511AB
Andy Levy
<luebbe.to...@googlemail.com> wrote:
>> Canonical answer: Move your working copy from the network share to a
>> local drive. That's highly recommended anyway.
>
>
> A cleanup of the working copy might help too. It is possible that due to
> timestamp differences, Subversion scans all the files in the folder for
> changes.
>
> But anyway: Move yor WC to a local disk. That's recommended best practice.
>
There was some discussion on the SVN Users list a few weeks ago around
the performance of SQLite on a network share (NFS mount in that case).
That was dealing with how svn rm scales but may apply here as well.
Start at http://svn.haxx.se/users/archive-2011-10/0891.shtml
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2892491
Rainer Schmidt
is absolutly nessesary to use a network share. A local working copy is
no option in our environment.
I think the problem is not the database or the network share.
TortoiseSVN seems to read/analyse every file in the working copy on
the network share. But the question is why? In my opinion only the
~1000 versioned files needs to be checked and not the whole 250.000
files, because most of them do not stay under version control.
Cleanup or anything else do not help...
On 12 Dez., 13:02, Andy Levy <andy.l...@gmail.com> wrote:
> On Mon, Dec 12, 2011 at 05:15, Lübbe Onken
>
> <luebbe.tortoise...@googlemail.com> wrote:
> >> Canonical answer: Move your working copy from the network share to a
> >> local drive. That's highly recommended anyway.
>
> > A cleanup of the working copy might help too. It is possible that due to
> > timestamp differences, Subversion scans all the files in the folder for
> > changes.
>
> > But anyway: Move yor WC to a local disk. That's recommended best practice.
>
> There was some discussion on the SVN Users list a few weeks ago around
> the performance of SQLite on a network share (NFS mount in that case).
> That was dealing with how svn rm scales but may apply here as well.
>
> Start athttp://svn.haxx.se/users/archive-2011-10/0891.shtml
>
> ------------------------------------------------------http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMess...
>
> To unsubscribe from this discussion, e-mail: [users-unsubscr...@tortoisesvn.tigris.org].
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2892537
Simon Large
> We use subversion in a corporate network with different locations. It
> is absolutly nessesary to use a network share. A local working copy is
> no option in our environment.
> I think the problem is not the database or the network share.
> TortoiseSVN seems to read/analyse every file in the working copy on
> the network share. But the question is why? In my opinion only the
> ~1000 versioned files needs to be checked and not the whole 250.000
> files, because most of them do not stay under version control.
TortoiseSVN also notifies you about unversioned files, so it has to at
least see what is there.
> Cleanup or anything else do not help...
Have you tried 'svn st -v /path/to/wc' for the command line client? If
that is equally slow then the issue lies within the subversion
library, not TortoiseSVN.
Simon
--
: ___
: oo // \\ "De Chelonian Mobile"
: (_,\/ \_/ \ TortoiseSVN
: \ \_/_\_/> The coolest Interface to (Sub)Version Control
: /_/ \_\ http://tortoisesvn.net
> On 12 Dez., 13:02, Andy Levy <andy.l...@gmail.com> wrote:
>> On Mon, Dec 12, 2011 at 05:15, Lübbe Onken
>>
>> <luebbe.tortoise...@googlemail.com> wrote:
>> >> Canonical answer: Move your working copy from the network share to a
>> >> local drive. That's highly recommended anyway.
>>
>> > A cleanup of the working copy might help too. It is possible that due to
>> > timestamp differences, Subversion scans all the files in the folder for
>> > changes.
>>
>> > But anyway: Move yor WC to a local disk. That's recommended best practice.
>>
>> There was some discussion on the SVN Users list a few weeks ago around
>> the performance of SQLite on a network share (NFS mount in that case).
>> That was dealing with how svn rm scales but may apply here as well.
>>
>> Start athttp://svn.haxx.se/users/archive-2011-10/0891.shtml
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2892538
Andy Levy
<mumpit...@googlemail.com> wrote:
> We use subversion in a corporate network with different locations. It
> is absolutly nessesary to use a network share. A local working copy is
> no option in our environment.
> I think the problem is not the database or the network share.
> TortoiseSVN seems to read/analyse every file in the working copy on
> the network share. But the question is why? In my opinion only the
> ~1000 versioned files needs to be checked and not the whole 250.000
> files, because most of them do not stay under version control.
>
> Cleanup or anything else do not help...
Rather than "I think", you need to do some more analysis to determine
what's *really* happening. and where it's bogging down.
At a minimum, Tortoise needs to scan all the files to find out what's
there so that it can then check against what it knows to be versioned.
Why is the network share "absolutely necessary"? The way you're
choosing to keep your working copies is contrary to accepted best
practices for using Subversion - surely there are ways to remove this
requirement. Subversion is designed for use with a *local* working
copy that is dedicated to a single user - most network shares are
meant for multiple people to use, which is again contrary to how
Subversion is intended to be used.
> On 12 Dez., 13:02, Andy Levy <andy.l...@gmail.com> wrote:
>> On Mon, Dec 12, 2011 at 05:15, Lübbe Onken
>>
>> <luebbe.tortoise...@googlemail.com> wrote:
>> >> Canonical answer: Move your working copy from the network share to a
>> >> local drive. That's highly recommended anyway.
>>
>> > A cleanup of the working copy might help too. It is possible that due to
>> > timestamp differences, Subversion scans all the files in the folder for
>> > changes.
>>
>> > But anyway: Move yor WC to a local disk. That's recommended best practice.
>>
>> There was some discussion on the SVN Users list a few weeks ago around
>> the performance of SQLite on a network share (NFS mount in that case).
>> That was dealing with how svn rm scales but may apply here as well.
>>
>> Start athttp://svn.haxx.se/users/archive-2011-10/0891.shtml
>>
>> ------------------------------------------------------http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMess...
>>
>> To unsubscribe from this discussion, e-mail: [users-unsubscr...@tortoisesvn.tigris.org].
>
> ------------------------------------------------------
> http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2892537
>
> To unsubscribe from this discussion, e-mail: [users-un...@tortoisesvn.tigris.org].
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2892539
Rainer Schmidt
TortoiseSVN issue.
And i think i found the problem. If we have the following working copy
structure:
repository/directoryA (with some versioned files)
/directoryB (with NO versioned files (directory is
unversioned too))
Now i can activate my hd monitor and can see that TortoiseProc looks
into directory A and directoryB and makes a lot of database stuff for
both directories like (same for every subdirectory of directoryA and
directoryB):
17:02:07 TortoiseProc.ex:3584 READ ...\.svn\wc.db SUCCESS Offset: 0
Length: 100
I do not understand why this is needed because the checkbox "show
unversioned files" is unchecked (within the commit dialog).
TortoiseSVN could ignore unversioned directories if this checkbox is
set or it could make this option global. This would result in a huge
performance increase in my opinion. The look for everything option is
only needed if i want to see the unversioned files in my commit
dialog?!
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2892545
Ulrich Eckhardt
> Why is the network share "absolutely necessary"?
There are companies where people don't have a designated desktop
machine. BTW: An alternative to local checkouts is running a remote
session on a dedicated machine where the working copy is on local
storage. You can still keep on editing using the share, but restrict SVN
operations to the remote system. With the size given, I could even
imagine the data on a thumb drive for portability, though the sysadmin
cringes at this idea...
> The way you're choosing to keep your working copies is contrary to
> accepted best practices for using Subversion - surely there are ways
> to remove this requirement. Subversion is designed for use with a
> *local* working copy that is dedicated to a single user - most
> network shares are meant for multiple people to use, which is again
> contrary to how Subversion is intended to be used.
I'd like to throw in an explanation: SVN is designed to use little
network bandwidth at the expense of additional harddisk usage. The
assumption is that network is expensive, while harddisk storage is
cheap. Now, in your scenario, "harddisk" access actually requires
network bandwidth, creating a case that SVN wasn't designed for.
Cheers!
Uli
**************************************************************************************
Domino Laser GmbH, Fangdieckstra�e 75a, 22547 Hamburg, Deutschland
Gesch�ftsf�hrer: Thorsten F�cking, Amtsgericht Hamburg HR B62 932
**************************************************************************************
Visit our website at http://www.dominolaser.com
**************************************************************************************
Diese E-Mail einschlie�lich s�mtlicher Anh�nge ist nur f�r den Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empf�nger sein sollten. Die E-Mail ist in diesem Fall zu l�schen und darf weder gelesen, weitergeleitet, ver�ffentlicht oder anderweitig benutzt werden.
E-Mails k�nnen durch Dritte gelesen werden und Viren sowie nichtautorisierte �nderungen enthalten. Domino Laser GmbH ist f�r diese Folgen nicht verantwortlich.
**************************************************************************************
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2892558
Simon Large
> 'svn st -v /path/to/wc' is done in a few seconds. So it should be a
> TortoiseSVN issue.
>
> And i think i found the problem. If we have the following working copy
> structure:
>
> repository/directoryA (with some versioned files)
> /directoryB (with NO versioned files (directory is
> unversioned too))
>
> Now i can activate my hd monitor and can see that TortoiseProc looks
> into directory A and directoryB and makes a lot of database stuff for
> both directories like (same for every subdirectory of directoryA and
> directoryB):
>
> 17:02:07 TortoiseProc.ex:3584 READ ...\.svn\wc.db SUCCESS Offset: 0
> Length: 100
>
> I do not understand why this is needed because the checkbox "show
> unversioned files" is unchecked (within the commit dialog).
> TortoiseSVN could ignore unversioned directories if this checkbox is
> set or it could make this option global. This would result in a huge
> performance increase in my opinion. The look for everything option is
> only needed if i want to see the unversioned files in my commit
> dialog?!
Have you added those non-versioned folders to the ignore list? That
may prevent TortoiseSVN from digging down to find unversioned content.
Simon
--
: ___
: oo // \\ "De Chelonian Mobile"
: (_,\/ \_/ \ TortoiseSVN
: \ \_/_\_/> The coolest Interface to (Sub)Version Control
: /_/ \_\ http://tortoisesvn.net
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2892576
Rainer Schmidt
>Status->"Recurse into unversioned folders" all works like before with
TortoiseSVN v1.6.x.
But i think the performance could be improved in both cases: local
working copy and network share. Most of the time a access to the
sqlite database within .svn could be exclusive and there is no need to
lock and unlock the database that much. I think it would be possible
to lock the database on the harddisk and load a temporary copy into
memory to enhance the speed. Unlock the database on harddisk after
finishing e.g. the commit. This could improve the general performance
a lot in my opinion. Other users have to wait if they want to access
the same working copy at the same time. Such a waiting time is
acceptable in most cases.
It would be nice if some performance settings could be set for local
working copies and network shares independantly. Examples would be the
"Recurse into unversioned folders" and the status cache.
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2893160