CVS slow and TCP Compression
Scott Willy
which we would like to share.
We have been using cvs on a Solaris server for 18 months or so and decided
to get a Cube and dedicate it to cvs. After installing cvs on the cube we
did some tests and all seems to be working fine. We took repositories with
1000's of files of source code and do checkouts no problem (and very fast).
Then...
We did a test with repositories containing libraries of big (100KB-2MB)
jar/dll/exe files (and not C++ or java source code). After a few files, the
checkout would slow to a craw. If we left it long it enough it would finish
the checkout, but it was too slow to be usable.
We found a workaround by using the -z option to set the TCP compression to
something other than zero. Anything between 1-9 seems to work just fine. The
checkouts of the large files is fast.
Bizarre...
Another observacation we had, was that when doing a checkout we would get
tons of Ethernet collisions on the LAN. This was true for both the Solaris
server we have in production and the Cube we were testing. This was true
even if we only had cvs server and one client on the LAN.
We still had tons of collisions using our workaround (-z), so they might not
be part of our problem, but...
We would be most interested if anybody else has seen something like this or
has any comments.
_______________________________________________
Info-cvs mailing list
Info...@gnu.org
http://mail.gnu.org/mailman/listinfo/info-cvs
Daniel Beckham
anything to do with cvs, but I would think it's a simple matter of network
performance. It would make sense that using compression will speed up the
checkout process, but what kind of speed problems are we talking about? Do
you mean, a 2MB file take 30 minutes to download or 1 minute? And how much
faster does a 2MB file checkout with compression on? What sort of network
are you running? When you say tons of collisions, what do you mean?
Collisions on an Ethernet segment is a normal thing. If I remember
correctly, under 3% is ok, but more than that could indicate a network
issue.
Daniel
Larry Jones
>
> Another observacation we had, was that when doing a checkout we would get
> tons of Ethernet collisions on the LAN. This was true for both the Solaris
> server we have in production and the Cube we were testing. This was true
> even if we only had cvs server and one client on the LAN.
That is almost certainly the root of your problem. My guess would be
that you have one or more devices on your ethernet that are erroneously
running in full duplex mode when they should be running in half duplex
instead.
-Larry Jones
Another casualty of applied metaphysics. -- Hobbes
Scott Willy
Some more info...
1) Not sure its a duplex problem as FTP does not have this speed problem.
I just downloaded 8MB in 37 seconds. One would think a duplex problem
would affect all protocols.
[8647568 bytes sent in 37.5 secs (2.3e+02 Kbytes/sec)]
2) By slow, I mean 2MB takes 30 minutes with the -z option and 10 seconds
with the -z option.
3) We have the same behavior when we made a private LAN (i.e. a hub, the
cvs server and one client). There was no other traffic. Nothing else
connected.
4) By tons of collisions, I mean that the little blinky lite on the hub goes
from not blinking to full-on when I start a cvs checkout.
csw
Larry Jones
>
> 1) Not sure its a duplex problem as FTP does not have this speed problem.
> I just downloaded 8MB in 37 seconds. One would think a duplex problem
> would affect all protocols.
> [8647568 bytes sent in 37.5 secs (2.3e+02 Kbytes/sec)]
Duplex problems frequently affect one direction much more than the
other. Try uploading with ftp rather than downloading. And make sure
you're using exactly the same two machines that you're using for CVS.
> 4) By tons of collisions, I mean that the little blinky lite on the hub goes
> from not blinking to full-on when I start a cvs checkout.
That definitely indicates a serious network problem.
-Larry Jones
Do you think God lets you plea bargain? -- Calvin
Gerhard Sittig
>
> Some more info...
>
> 1) Not sure its a duplex problem as FTP does not have this speed problem.
> I just downloaded 8MB in 37 seconds. One would think a duplex problem
> would affect all protocols.
> [8647568 bytes sent in 37.5 secs (2.3e+02 Kbytes/sec)]
That's *download* which is not the same as transferring data in
_both_ directions. In FTP you only have small(!) handshake
packets going both ways, while the main stuff flows in one
direction only.
> 4) By tons of collisions, I mean that the little blinky lite on the hub goes
> from not blinking to full-on when I start a cvs checkout.
If this is not a sign for a duplex mismatch, what else is?
Honestly: Check your network setup! If autonegotiation takes
place, double check the results. If autonegotiation won't work,
hardcode things. But do it consistently and do it everywhere!
How about starting a log analyzer to see how traffic flows and
what the timing looks like?
virtually yours 82D1 9B9C 01DC 4FB4 D7B4 61BE 3F49 4F77 72DE DA76
Gerhard Sittig true | mail -s "get gpg key" Gerhard...@gmx.net
--
If you don't understand or are scared by any of the above
ask your parents or an adult to help you.
Scott Willy
the PC (the cvs client) and to/from the cobalt cube (cvs server). The speed
was fine over 100KBytes/sec. Still could be a duplex problem, but the
evidence is not clear.
We are still trying to figure out how to configure the cube's Ethernet
driver to force it into half-duplex mode (it is easy to do so on the PC).
Any case, we can leave it here. I think the most important thing to note is
that by changing the TCP compression level (e.g. -z3 )some network problems
might be solved (or at least worked around).
csw
> -----Original Message-----
> From: info-cv...@gnu.org [mailto:info-cv...@gnu.org]On Behalf Of
> Gerhard Sittig
> Sent: 02 July 2001 20:47
> To: info...@gnu.org
> Subject: Re: CVS slow and TCP Compression
>
>
Steven Rosenstein
This is just a shot in the dark (I am fairly new to CVS).
I believe you mentioned in an earlier email that you are moving a large number
of binary files. Is it possible that CVS is somehow examining the contents of
these files (say, to determine end of line?). In a binary file this would be a
very time-consuming task, and might generate a significant amount of traffic
between the client and the server. When you turn on compression, regardless of
the level of compression, CVS "knows" that you are moving a binary file, and
turns off all file internal checks?
If this is the case, I believe you can use the -kb commandline modifier to tell
CVS the file you are committing is binary.
I hope this helps
--- Steve
"Scott Willy" <csw...@iname.com> on 07/03/2001 06:31:12 AM
To: info...@gnu.org
cc: (bcc: Steven Rosenstein/FTCI)
Subject: RE: CVS slow and TCP Compression
We have re-done the FTP tests. Uploads and downloads work fine. Both to/from
the PC (the cvs client) and to/from the cobalt cube (cvs server). The speed
was fine over 100KBytes/sec. Still could be a duplex problem, but the
evidence is not clear.
We are still trying to figure out how to configure the cube's Ethernet
driver to force it into half-duplex mode (it is easy to do so on the PC).
Any case, we can leave it here. I think the most important thing to note is
that by changing the TCP compression level (e.g. -z3 )some network problems
might be solved (or at least worked around).
csw
_______________________________________________