SVN checkout slow on windows compare to linux
Lenney Yip
I'm trying to figure out why my checkout times on a Windows box a lot slower than a Mac or Linux.
I'm using the latest version of Tortoise on windows xp. I have the AV off. Even though that helped improve my time, but not close to the checkout times of a Mac or Redhat.
For example. The check out time for win xp is 30secs compare to 8secs on Redhat. The repository is about 11mb.
I'm wondering if there are some tweaks in the application that might help. I read that ntfs have a problem handling many files, so I tried fat32, but it was still slow.
Any suggestions would be great.
Thank you,
Lenney
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2649309
To unsubscribe from this discussion, e-mail: [users-un...@tortoisesvn.tigris.org].
Lenney
Thank you,
Lenney
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2651223
Chris Feldhacker
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2651227
Andy Levy
> Bump. This is kind of urgent. If there isn't a solution at this time please let me know.
If it's truly urgent, you should be paying for support.
You have omitted a lot of details needed to provide any help. What
access method are you using? System specs (especially HD speed) on the
workstations? Network configuration, including/especially any
firewalls? What versions of the software involved?
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2651228
Lenney
I'm the system specs are Core 2 Duo T7200, 2.0GB mem, 1gb network. 7200PRM HD.
I don't know what you mean by what access method? I'm on wired and I'm accessing the repository by right clicking on a directory and choose repo browser. Then I find the folder and check it out.
I've also tried to use a command line based svn client like slik, but the results aren't much better.
A redhat workstation can download the repository in 8secs while my computer does the same job in 30secs.
Here is the link to the repository:
http://svn.apache.org/repos/asf/spamassassin/trunk
Maybe you can try it and post your times.
THank you,
Lenney
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2651236
Lenney
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2651239
Phil Sayers
Have you tried the svn client? Not a 3rd party one like tortoise or slik,
the one that comes from collabnet.
Use the same version of the svn client from the same publisher on both win &
linux.. then we are comparing apples to apples.
Between both win/linux.. any write cache enabled for the disks?
Is the O/S on the same physical disk on both win & linux?
Same disk setup? Raid1/raid0/non raid... identical raid controllers on both
systems.
What else is installed on Win that may not be on the linux box? Antivirus?
DiskKeeper?
To point the blame specifically at tsvn on windows you must show that
everything else is either [a]identical or [b]very close to identical.
FYI.
Your access method is http over the public internet.
Thanks for your reply.
http://svn.apache.org/repos/asf/spamassassin/trunk
THank you,
Lenney
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2
651236
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2651245
Felix Saphir
>
> A redhat workstation can download the repository in 8secs while my
> computer does the same job in 30secs.
Why do 22 seconds impose an urgent problem?
Felix
P.S. Please quote the message you're replying to.
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2651246
Felix Saphir
> What are the specs of the linux box compared to the win box?
One OS is running NTFS and FAT, the other one has a real filesystem.
NTFS isn't know for it's high performance, neither is FAT.
Felix
P.S. Please don't top-post
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2651249
Chris Feldhacker
Try editing the registry under:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters
Add a DWORD key named "TcpWindowSize" with a value of something like "500000" (decimal).
Then reboot, try again.
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2651265
Lenney
>
> Try editing the registry under:
> HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters
>
> Add a DWORD key named "TcpWindowSize" with a value of something like "500000" (decimal).
>
> Then reboot, try again.
Chris,
I tried that but it didn't work. Any other suggestions?
Thank you,
Lenney
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2651278
Jean-Marc van Leerdam
On 25 August 2010 16:47, Lenney <l...@rice.edu> wrote:
> Thanks for your reply.
>
> I'm the system specs are Core 2 Duo T7200, 2.0GB mem, 1gb network. 7200PRM HD.
>
> I don't know what you mean by what access method? I'm on wired and I'm accessing the repository by right clicking on a directory and choose repo browser. Then I find the folder and check it out.
>
Do you mean you are using the file:// access to the repository? What
is the exact URL you checkout from?
File access could be much slower than accessing a real svnserve or
apache based svn server.
> I've also tried to use a command line based svn client like slik, but the results aren't much better.
That seems to indicate that it is either a windows issue, a local
issue (antivirus/firewall) or something embedded in the svn libraries.
Neither of these can be influenced much by TSVN.
--
Regards,
Jean-Marc
--
. ___
. @@ // \\ "De Chelonian Mobile"
. (_,\/ \_/ \ TortoiseSVN
. \ \_/_\_/> The coolest Interface to (Sub)Version Control
. /_/ \_\ http://tortoisesvn.net
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2651294
Bob Archer
> time please let me know.
>
> Thank you,
>
> Lenney
Perhaps to you it is, but to those that donate their time on this mail list to help others, it really isn't. It seems to me that waiting an extra 22 seconds solves your problem. I'm not sure how much quicker it could be solved.
That said, several people have done benchmarks on Linux vs Windows on the SAME exact hardware and shown that Linux is just faster at most client side operations that do a lot of disk access. So, there really is no solution for this except improving your hard ware. Perhaps an SSD. Assuming you have disabled any Antivirus scan and indexing in windows on the folder you are doing your svn work in... there's not really much more to do.
Supposedly svn 1.7's new working copy storage is supposed to bring windows and linux perf into parity, improving perf on both OSes. But, that is probably still 6+ months out from what I see following the svn dev mail list.
BOb
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2651298
Stefan Küng
> Supposedly svn 1.7's new working copy storage is supposed to bring
> windows and linux perf into parity, improving perf on both OSes. But,
> that is probably still 6+ months out from what I see following the
> svn dev mail list.
Actually, plans are to finish 1.7 by the end of October.
Stefan
--
___
oo // \\ "De Chelonian Mobile"
(_,\/ \_/ \ TortoiseSVN
\ \_/_\_/> The coolest Interface to (Sub)Version Control
/_/ \_\ http://tortoisesvn.net
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2651301
Bob Archer
>
> > Supposedly svn 1.7's new working copy storage is supposed to
> bring
> > windows and linux perf into parity, improving perf on both OSes.
> But,
> > that is probably still 6+ months out from what I see following
> the
> > svn dev mail list.
>
> Actually, plans are to finish 1.7 by the end of October.
>
> Stefan
Do you mean done and released... or done (code complete) and in alpha testing?
yes, I also recall one of the devs expecting that 1.7 would be out in the summer (after 1.6 came out).... was it 2009.
It's open source, it will be done when it is done. I would be very happy to see it released/done that soon.
BOb
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2651305
Lenney
>
> That said, several people have done benchmarks on Linux vs Windows on the SAME exact hardware and shown that Linux is just faster at most client side operations that do a lot of disk access. So, there really is no solution for this except improving your hard ware. Perhaps an SSD. Assuming you have disabled any Antivirus scan and indexing in windows on the folder you are doing your svn work in... there's not really much more to do.
>
> Supposedly svn 1.7's new working copy storage is supposed to bring windows and linux perf into parity, improving perf on both OSes. But, that is probably still 6+ months out from what I see following the svn dev mail list.
>
> BOb
Bob,
It seems like everyone is on my case about it being urgent. I was just being anxious because I haven't heard a response in days. I apologize to everyone if me being anxious offended anyone. I do appreciate everyone's time here.
Both of the systems are similar. They have single drives. The Win machines were clean installed with no AV or firewall enable/on.
22secs difference for a 11mb download might not be that big of a difference. But if you have download 1gb that would be a different story.
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2651375
Andy Levy
> Perhaps to you it is, but to those that donate their time on this mail list to help others, it really isn't. It seems to me that waiting an extra 22 seconds solves your problem. I'm not sure how much quicker it could be solved.
>>
>> That said, several people have done benchmarks on Linux vs Windows on the SAME exact hardware and shown that Linux is just faster at most client side operations that do a lot of disk access. So, there really is no solution for this except improving your hard ware. Perhaps an SSD. Assuming you have disabled any Antivirus scan and indexing in windows on the folder you are doing your svn work in... there's not really much more to do.
>>
>> Supposedly svn 1.7's new working copy storage is supposed to bring windows and linux perf into parity, improving perf on both OSes. But, that is probably still 6+ months out from what I see following the svn dev mail list.
>>
>> BOb
>
> Bob,
>
> It seems like everyone is on my case about it being urgent. I was just being anxious because I haven't heard a response in days. I apologize to everyone if me being anxious offended anyone. I do appreciate everyone's time here.
>
> Both of the systems are similar. They have single drives. The Win machines were clean installed with no AV or firewall enable/on.
>
> 22secs difference for a 11mb download might not be that big of a difference. But if you have download 1gb that would be a different story.
You're making an assumption that the timing gap between Linux &
Windows is linear as checkout size increases. It will greatly depend
upon a number of factors and in all likelihood it will not be linear.
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2651382
Lenney
Bob,
I tried to search for some of the benchmarks you mention regarding Linux vs windows on the same hardware, but did not have any success. Do you have those figures handy? So you think svn 1.7 could improve the window's SVN performance.
Thanks,
Lenney
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2651766
Liam Friel
the SAME exact hardware and shown that Linux is just faster
>at most client side operations that do a lot of disk access.
Agree with Bob 100%. This is absolutely the case. I have tested this on
a dual-boot machine under as near identical conditions as possible (as
part of an internal activity to get our corporate anti-virus solution
improved).
NTFS filesystem + anti-virus software turned on, SVN 1.6.x, creating a
large WC from scratch (~5GB) with ~35K files, Windows XP SP2 took 5
times longer than Linux Fedora 11 (ext3 FS I think) to create the WC.
By tweaking the anti-virus stuff and disabling indexing and such, it was
possible to improve this on Windows a bit. But still, Linux was 3 times
faster on the same hardware than Windows and there is nothing to be done
about it until svn 1.7 hopefully improves things.
It is not a TSVN issue.
/Liam
The information contained in this e-mail and in any attachments is confidential and is designated solely for the attention of the intended recipient(s). If you are not an intended recipient, you must not use, disclose, copy, distribute or retain this e-mail or any part thereof. If you have received this e-mail in error, please notify the sender by return e-mail and delete all copies of this e-mail from your computer system(s).
Please direct any additional queries to: communi...@s3group.com.
Thank You.
Silicon and Software Systems Limited (S3 Group). Registered in Ireland no. 378073.
Registered Office: South County Business Park, Leopardstown, Dublin 18
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2651779
David Balažic
>
> >That said, several people have done benchmarks on Linux vs Windows on
> the SAME exact hardware and shown that Linux is just faster
> >at most client side operations that do a lot of disk access.
>
> Agree with Bob 100%. This is absolutely the case. I have
> tested this on
> a dual-boot machine under as near identical conditions as possible (as
> part of an internal activity to get our corporate anti-virus solution
> improved).
>
> NTFS filesystem + anti-virus software turned on, SVN 1.6.x, creating a
> large WC from scratch (~5GB) with ~35K files, Windows XP SP2 took 5
> times longer than Linux Fedora 11 (ext3 FS I think) to create the WC.
>
> By tweaking the anti-virus stuff and disabling indexing and
> such, it was
> possible to improve this on Windows a bit. But still, Linux
> was 3 times
> faster on the same hardware than Windows and there is nothing
> to be done
> about it until svn 1.7 hopefully improves things.
What times did you get with not AV on Windows?
Regards,
David
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2651983
web...@tigris.org
While checking out on Windows (Windows 7 x64, svn 1.6.12, TortoiseSVN 1.6.10, NTFS), TortoiseProc.exe eats 50% CPU (that is, one of the two cores). When a large file is being downloaded, the load drops. It clearly showed me that there is a per-file problem (instead of throughput, bandwidth limit, some kind of conversion [we made even test to check if the CR -> CR/LF conversion takes too much time], etc.)
Now I've fired up ProcMon from SysInternals. Here are some bottlenecks I've found:
1. Anytime "entries" is read, I see the following sequence: Open, Read 80 bytes, Close, Open, Read 80 bytes, Close, Open, Read whole file, Close. What is the reason behind this?
2. I've also found that the same "entries" file is being read several times (in the above way) consecutively, without any writes to that file, without any other operations between the two queries. So O, R80, C, O, R80, C, O, Rall, C, O, R80, C, O, R80, C, O, Rall, C, etc.
3. In some directories I see a loop. Svn tries to create a file "tempfile.tmp" and gets NAME COLLISION result. "tempfile.2.tmp" is tried then with the same result. And so on. Sometimes going up to even "tempfile.340.tmp". Seems some DeleteFile is missing. But why not use the GetTempFileName function anyway?
4. When a large file is being checked out, I see the following sequence:
Write 4k from offs 0,
Write 4k from offs 4k,
Write 4k from offs 8k,
Read 16k from offs 16k, <- Why?
Write 4k from offs 12k,
Write 4k from offs 16k,
etc.
It also shows that either the TCP packet size is set to 4096 bytes, or the file buffer size is set to this silly small value in svn.
5. It seems that the "entries" in the whole directory tree is checked for each repository file. Say file "root/dirA/dirB/fileC" is processed, and both dirA and dirB is already created. svn checks "root/entries", "root/dirA/entries", "root/dirA/dirB/entries", deals with the file, the checks (reads!) "root/dirA/dirB/entries" again, then "root/dirA/entries" and "root/entries".
All in all: Most of the lost time is spent with the "entries" files.
I'm willing to check any tests or test versions sent to me.
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2663800
Stefan Küng
Your barking up the wrong tree.
This is an issue with the SVN library, not TSVN itself.
Stefan
--
___
oo // \\ "De Chelonian Mobile"
(_,\/ \_/ \ TortoiseSVN
\ \_/_\_/> The coolest Interface to (Sub)Version Control
/_/ \_\ http://tortoisesvn.net
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2663801
Yogurt
1. I've found the original issue here (not closed) so I've posted my foundings here.
2. Since it was TortoiseProc.exe (clearly part of TSVN) that drives the CPU nuts, how should I know that a given function belongs to the SVN library or not?
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2663808
Geoff Rowell
> Hi Stefan,
>
> 1. I've found the original issue here (not closed) so I've posted my foundings here.
>
> 2. Since it was TortoiseProc.exe (clearly part of TSVN) that drives the CPU nuts, how should I know that a given function belongs to the SVN library or not?
>
> ------------------------------------------------------
> http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2663808
I have a directory structure with only about 3 or 4 directory levels,
but a few of the low-level directories contain 10K+ small (3-4 KB)
files. I attempted to check it out onto a Windows XP box. Bad idea. It
ended up taking 3 or 4 days. You could see the SVN (v1.6 command line)
output progressively slowing down.
Did the same thing under Ubuntu Linux. It was done within 10 minutes.
--
Geoff Rowell
geoff....@gmail.com
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2663812
Stefan Küng
> Hi Stefan,
>
> 1. I've found the original issue here (not closed) so I've posted my
> foundings here.
>
> 2. Since it was TortoiseProc.exe (clearly part of TSVN) that drives
> the CPU nuts, how should I know that a given function belongs to the
> SVN library or not?
Because in this very thread, it was already mentioned that this is an
issue of the svn library.
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2651294
Stefan
--
___
oo // \\ "De Chelonian Mobile"
(_,\/ \_/ \ TortoiseSVN
\ \_/_\_/> The coolest Interface to (Sub)Version Control
/_/ \_\ http://tortoisesvn.net
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2663816
Stefan Küng
> On Thu, Sep 23, 2010 at 3:56 PM, Yogurt<jog...@impulzus.com> wrote:
>> Hi Stefan,
>>
>> 1. I've found the original issue here (not closed) so I've posted my foundings here.
>>
>> 2. Since it was TortoiseProc.exe (clearly part of TSVN) that drives the CPU nuts, how should I know that a given function belongs to the SVN library or not?
>>
>> ------------------------------------------------------
>> http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2663808
>
> I have a directory structure with only about 3 or 4 directory levels,
> but a few of the low-level directories contain 10K+ small (3-4 KB)
> files. I attempted to check it out onto a Windows XP box. Bad idea. It
> ended up taking 3 or 4 days. You could see the SVN (v1.6 command line)
> output progressively slowing down.
>
> Did the same thing under Ubuntu Linux. It was done within 10 minutes.
3-4 days? Seriously?
this only leaves two options:
* you're a git fanboy, trying to discredit SVN
* your Windows setup is completely screwed up
Seriously: I often check out projects with 35K files within an hour or
so, and that's with one of the worst AV installed (Kaspersky).
If it takes you that long, something is wrong, and it definitely is
*not* svn or TSVN.
Stefan
--
___
oo // \\ "De Chelonian Mobile"
(_,\/ \_/ \ TortoiseSVN
\ \_/_\_/> The coolest Interface to (Sub)Version Control
/_/ \_\ http://tortoisesvn.net
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2663817
Geoff Rowell
> On 23.09.2010 22:10, Geoff Rowell wrote:
>> On Thu, Sep 23, 2010 at 3:56 PM, Yogurt<jog...@impulzus.com> wrote:
>>> Hi Stefan,
>>>
>>> 1. I've found the original issue here (not closed) so I've posted my foundings here.
>>>
>>> 2. Since it was TortoiseProc.exe (clearly part of TSVN) that drives the CPU nuts, how should I know that a given function belongs to the SVN library or not?
>>>
>>> ------------------------------------------------------
>>> http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2663808
>>
>> I have a directory structure with only about 3 or 4 directory levels,
>> but a few of the low-level directories contain 10K+ small (3-4 KB)
>> files. I attempted to check it out onto a Windows XP box. Bad idea. It
>> ended up taking 3 or 4 days. You could see the SVN (v1.6 command line)
>> output progressively slowing down.
>>
>> Did the same thing under Ubuntu Linux. It was done within 10 minutes.
>
> 3-4 days? Seriously?
>
> this only leaves two options:
> * you're a git fanboy, trying to discredit SVN
> * your Windows setup is completely screwed up
>
> Seriously: I often check out projects with 35K files within an hour or
> so, and that's with one of the worst AV installed (Kaspersky).
> If it takes you that long, something is wrong, and it definitely is
> *not* svn or TSVN.
* I could count on one hand the number of times I've used git. Nice try, but no.
* Install Windows. Install Subversion. svn checkout. How is that
screwed up? Again, nice try, but no.
--
Geoff Rowell
geoff....@gmail.com
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2663823
Wayne Johnson
> levels,
> >> but a few of the low-level directories contain 10K+ small (3-4 KB)
> >> files. I attempted to check it out onto a Windows XP box. Bad idea.
> It
> >> ended up taking 3 or 4 days. You could see the SVN (v1.6 command
> line)
> >> output progressively slowing down.
> >>
> >> Did the same thing under Ubuntu Linux. It was done within 10
> minutes.
> >
> > 3-4 days? Seriously?
> >
> > this only leaves two options:
> > * you're a git fanboy, trying to discredit SVN
> > * your Windows setup is completely screwed up
> >
> > Seriously: I often check out projects with 35K files within an hour
> or
> > so, and that's with one of the worst AV installed (Kaspersky).
> > If it takes you that long, something is wrong, and it definitely is
> > *not* svn or TSVN.
>
> try, but no.
>
> * Install Windows. Install Subversion. svn checkout. How is that
> screwed up? Again, nice try, but no.
>
I found this interesting article with the Google.
http://support.microsoft.com/kb/130694
In this article it points out that with long file names directory operations on NTFS are an O(n^2).
It is also possible for NTFS to become fragmented.
The Google was not as helpful in providing any benchmark comparisons be NTFS on Windows vs EXT3/4 on Linux.
Still I don't think this explains 3-4 days to create a working copy. How many files are in the working copy total? How big is the working copy when you get done checking it out?
There has to be some sort of configuration problem. 10 mins. vs 4320 mins. is quite a large difference.
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2663837
Leo Davidson
> 2. Since it was TortoiseProc.exe (clearly part of TSVN) that drives the CPU nuts,
> how should I know that a given function belongs to the SVN library or not?
Do the same thing using the command-line SVN client and if similar
problems exist then you know it's probably an issue with the
underlying SVN library and not TortoiseSVN.
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2664014