Why is TCP/FTP so slow on high speed connections ?
Skybuck Flying
during the transfer the speed is around 120 kilobyte / sec.
My connection is a cable modem ( 700 kb / sec download, 16 kb/sec upload )
The other person's connection is a high speed connection like 10/100 megabit
(campus of university)
The ping time between us is about 30 milliseconds.
With my own tool I tested the speed from him to me which is about 700
kilobyte/sec with UDP.
From me to him it's about 16 kilobyte / sec with UDP.
But during the TCP/FTP transfer the transfer rate from him to me is only 120
kilobyte / sec ???
Which seems to be only a few procent of the total connection speed.
So my question is why is TCP/FTP so slow ? Why can't it do better ?
I could think up so many reasons which could have something to do with:
TCP itself is slow, or FTP is slow, or overhead is too much, or FTP server
software is slow or FTP client software is slow.
Maybe writing the file to HD or reading it is to slow...
Maybe the stack is slow ? maybe the stack options in my operating system
like window size or MTU is not optimized...
Maybe TCP is just using only a few procent to allow other programs to access
the internet...
Maybe TCP is just inefficient and drops in speed when a packet is lost ?
Maybe routers on the network reduce the TCP speed or something ?
Anyone have any clue why TCP is so slow in this case ?
Angel
An obvious question here. Could you possibly be mixing your bits and bytes?
120 KiloBytes per second suggests a 1 Megabit per second cable
connection. 700 KiloBytes per second suggests something like a 6
Megabit per second cable connection. If you have this as a residential
cable installation I'm incredibly envious ;)
angel
Dan Swartzendruber
enough@spam_already says...
> An obvious question here. Could you possibly be mixing your bits and bytes?
>
> 120 KiloBytes per second suggests a 1 Megabit per second cable
> connection. 700 KiloBytes per second suggests something like a 6
> Megabit per second cable connection. If you have this as a residential
> cable installation I'm incredibly envious ;)
he should check RWIN on his PC. if it's an older windows (e.g. win95)
or was set up for dialup, it might have only 8192 bytes for RWIN, which
makes TCP perform suboptimally. if this is the case, search for and
download DrTCP which lets you tweak RWIN.
Skybuck Flying
bytes?
Nope... it's kilobyte 100% sure.
> 120 KiloBytes per second suggests a 1 Megabit per second cable
> connection. 700 KiloBytes per second suggests something like a 6
> Megabit per second cable connection. If you have this as a residential
> cable installation I'm incredibly envious ;)
Well don't be. The 700 kilobyte / sec download it's nice... but the upload
limit of 16 kilobyte / sec is still very low.
I am envious at the dude that can upload at 700 kilobyte/sec !!! that's
awesome for games :) and all sorts of other stuff ;)
But to do that we would probably need new software... which I am developing
my self really slowly.
With download accelerator I have downloaded files at 300 to 500 kb/sec...
but I dont use it anymore... sometimes it might fail or
it takes long to build the output.
Also internet explorer sometimes downloads to temporarely files and then has
to copy it after download which takes quite some time for 600 meg... like a
few minutes... damn
Skybuck Flying
"Dan Swartzendruber" <dsw...@druber.com> wrote in message
news:MPG.1815df3e4...@news.supernews.net...
My operating system is windows XP.
Speed was the same on windows 98 by the way.
Bob
<nos...@hotmail.com> wrote:
>> An obvious question here. Could you possibly be mixing your bits and
>bytes?
>
>Nope... it's kilobyte 100% sure.
>
>> 120 KiloBytes per second suggests a 1 Megabit per second cable
>> connection. 700 KiloBytes per second suggests something like a 6
>> Megabit per second cable connection. If you have this as a residential
>> cable installation I'm incredibly envious ;)
>
>Well don't be. The 700 kilobyte / sec download it's nice... but the upload
>limit of 16 kilobyte / sec is still very low.
I also think you have your bits and bytes confused. In your original
message you stated you had a 700mb download connection and your friend
had a 10/100mb network.
Now you are saying that you have 700megabyte download. That is 4 times
the speed of a full T1 which is 1.54mbs. I doubt you have that at
home.
Amn
"Skybuck Flying" <nos...@hotmail.com> wrote in message
news:DKZq9.85356$48.93...@zwoll1.home.nl...
helo ;)
it runs a very respectable and objective online test of your line along with
your settings on MTU, RWIN, ping, packet loss etc.
It will tell you if your connection is optimised.
Good Luck !
Fester Rancid
"120 kilobyte / sec" (works out to be ~1Mbps, ignoring parity, etc.),
downloads ?
You obviously can't count!
FR
"Skybuck Flying" <nos...@hotmail.com> wrote in message
news:wJZq9.85353$48.93...@zwoll1.home.nl...
bobb
<nos...@hotmail.com> wrote:
>The other person's connection is a high speed connection like 10/100 megabit
>(campus of university)
Don't forget in the university, there are "others" sharing the
bandwidth. Plus no network engineer worth his salt will let a single
FTP server take up ALL of the available bandwidth.
-bobb
Craig Partridge
>But during the TCP/FTP transfer the transfer rate from him to me is only 120
>kilobyte / sec ???
How much data are you FTPing -- the TCP has to exchange several windows of
data to start sending at full data rate.
Craig
Skybuck Flying
"Bob" <removebc...@dns2go.com> wrote in message
news:eksoqu8rilo3rdk3r...@4ax.com...
> On Tue, 15 Oct 2002 21:20:21 +0200, "Skybuck Flying"
> <nos...@hotmail.com> wrote:
>
> >> An obvious question here. Could you possibly be mixing your bits and
> >bytes?
> >
> >Nope... it's kilobyte 100% sure.
> >
> >> 120 KiloBytes per second suggests a 1 Megabit per second cable
> >> connection. 700 KiloBytes per second suggests something like a 6
> >> Megabit per second cable connection. If you have this as a residential
> >> cable installation I'm incredibly envious ;)
> >
> >Well don't be. The 700 kilobyte / sec download it's nice... but the
upload
> >limit of 16 kilobyte / sec is still very low.
>
> I also think you have your bits and bytes confused. In your original
> message you stated you had a 700mb download connection
No, apperently you mis-interpreted the text, what I ment was:
I download a file from him that was 600 megabytes at a speed of 120
kilobytes / sec which is not optimal.
I also did some testing with him with my own tool UDP Speed Test
(www.mycgiserver.com/~skybuck) which showed a speed of 700 kilobytes / sec
from him to me, and 16 kilobyte /sec from me to him.
> and your friend had a 10/100mb network.
Correct, he lives at a campus of a university. He can use the universities
network at home to surf the internet.
>
> Now you are saying that you have 700megabyte download.
No the file is about 600 megabytes.
Skybuck Flying
connections. ( pc's work with bytes... who cares about how many bits are on
your harddisk or in memory... people rather talk about how much bytes they
have in memory and on disk etc )
I said 700 kb/sec which you rightfully thought ment 700 kilobit/sec
But I really ment 700 kilobytes / sec (KB).
"Fester Rancid" <Fester...@nospam.com> wrote in message
news:3dac8761$0$1292$cc9e...@news.dial.pipex.com...
Skybuck Flying
"bobb" <No...@NoWhere.com> wrote in message
news:3db29125....@news.rcn.com...
How can a network engineer prevent that ?
Skybuck Flying
"Craig Partridge" <cra...@world.std.com> wrote in message
news:H41tF0...@world.std.com...
The file is 600 megabytes. So we are FTP-ing 600 meg.
Angel
It's quite easy to impose bandwidth limiting or traffic shaping on a per
subnet or per IP basis.
I do it on a much smaller scale at home to make sure nobody using my
publicliy available wireless lan can flood my DSL. For the first 2MB of
a single download they can use as much bandwidth as possible after the
first 2MB it gets throttle back to no more than 50% of the bandwidth.
Meaning surfing and smaller downloads still happen very quickly while at
same time preventing the downloading of larger files swamping the link.
angel
Angel
> I think it's stupid that people talk about bits when it comes to
> connections. ( pc's work with bytes... who cares about how many bits are on
> your harddisk or in memory... people rather talk about how much bytes they
> have in memory and on disk etc )
>
> I said 700 kb/sec which you rightfully thought ment 700 kilobit/sec
>
> But I really ment 700 kilobytes / sec (KB).
>
>
People may prefer to talk about bytes but when you are discussing data
rates it's better to talk in bits or at least be clear and specific if
talking in both. The reason for this is tranfer rates are based on bits
as that is what you are chucking around.
Anyhow, could you tell us exactly what kind of cable pipe you are using
please. The rated speed as it was sold to you, etc. With that info we
can better begin to try and provide some useful comments on the transfer
rates.
I know you stated some transfer rates that you obtained via your own app
based on UDP transmissions but as we aren't familiar with that app or
exactly how it works we cannot say for sure that it is reporting
correctly and not making any misinterpretations. I'm not taking
anything away from you coding skills or your app but as we don't have
any knowledge of it we simply can't assume that whatever it reports is
correct. In this instance it's better to know exactly what kind of
Internet pipe you are using and it's speed ratings.
angel
Skybuck Flying
"Angel" <enough@spam_already> wrote in message
news:3dad2bff$0$33003$bed6...@pubnews.gradwell.net...
It's an @home cable modem
And trust me the program I mentioned works perfectly at measuring rates.
I know what I am doing :)
Skybuck Flying
"Angel" <enough@spam_already> wrote in message
What software do you use for this ? :)
Craig Partridge
First, we'll try some basic limits:
1. Window size. We don't know your receive window size. If it is 8192 bytes
then the fastest your data transfer can go is:
8192 bytes/30ms = 273 KB/s
The rate scales linearly with window size, so if you running a 32KB
window size (largest you can do without PAWS), the max rate is
1.1 MB/s.
2. ACK limits. TCP sends a 40 byte ack for every two data segments. The
return like is 16 KB/s so the maximum ACK rate is:
16 KB/s / 40 bytes = 410 ACKs per second
Inverting this, if data segments are 512 bytes each, and we ack
two per ack, the maximum data rate is:
420 KB/s
Side note -- this observation says DO NOT enable TCP large windows --
you will reduce the number of ACKs received per second (because the
ACKs are larger) and get *worse* throughput.
OK, now both these results (esp. ACK limits) say you're rate limited
to somewhere around 200KB/s, but that's still not 120 KB/s. So on to
the harder calculations.
3. TCP performance, sending side. Assuming a perfect ACK stream
(i.e., no limits on ACKs), TCP is going to do one slow slow start
and then send as fast as possible.
If we assume a 8192KB window and a 512 byte starting segment size,
we'll:
send 512 bytes, wait for delayed ack (230ms) then
send 1024 bytes, wait for ack (30ms) then
send 2048 bytes, wait for ack (30ms) then
send 4096 bytes, wait for ack (30ms) then
send 8196 bytes, wait for ack (30ms) then
send full rate.
These startup round trip times are negligible against the total
number of round-trips, so there's probably NO slow start effect.
4. Let's look at the ACK stream more carefully.
On the sending side, we can transmit 700KB/s / 512 byte segments
per second -- that's 1400 segments. But we can only ack 820 of
them. That's a problem as it means we'll overload queues and
drop ACKs, causing TCP to restart the slow start process.
It isn't quite this bad. My guess is your window size is likely
16KB (not the 8KB used above), which means the maximum data
rate we can achieve is 546 KB/s. We can only ack 420KB/s.
That suggests that about once a second or so, we'll overflow
the ACK queue. So let's do a picture until that ACK queue gets
to be a few packets deep:
0.000 sec: send 512 bytes, wait for delayed ack (230ms) then
0.230 sec: send 1024 bytes, wait for ack (30ms) then
0.260 sec: send 2048 bytes, wait for ack (30ms) then
0.290 sec: send 4096 bytes, wait for ack (30ms) then
0.320 sec: send 8196 bytes, wait for ack (30ms) start cong control
0.350 sec: send 8704 bytes, wait for ack (30ms) then
0.380 sec: send 9216 bytes, wait for ack (30ms) then
0.410 sec: send 9728 bytes, wait for ack (30ms) then
0.440 sec: send 10240 bytes, wait for ack (30ms) then
0.470 sec: send 10752 bytes, wait for ack (30ms) then
0.500 sec: send 11264 bytes, wait for ack (30ms) then
0.530 sec: send 11776 bytes, wait for ack (30ms) then
0.560 sec: send 12288 bytes, wait for ack (30ms) then
0.690 sec: send 12800 bytes, wait for ack (30ms) then ** too fast to ack **
0.620 sec: send 13312 bytes, wait for ack (30ms) then ** too fast to ack **
0.650 sec: send 13824 bytes, wait for ack (30ms) then ** too fast to ack **
0.680 sec: send 14336 bytes, wait for ack (30ms) then ** too fast to ack **
0.710 sec: send 14848 bytes, wait for ack (30ms) then ** too fast to ack **
0.740 sec: send 15360 bytes, wait for ack (30ms) then ** too fast to ack **
OK, so we've spent about three quarters of a second to get the point that
the ACK queue is overloading.... a packet drop will soon ensue (exactly
how soon depends on how packets are queued... could be seconds, could
be fractions of a second).
Suppose this is all we get to send before a segment gets lost. When
the segment gets lost, the ACK queue will be overloaded, and the
ACK will likely get lost. So we have to rely on the retransmission
timer going off. That will take some hundreds of milliseconds to one
second. Suppose it takes 1 sec to retransmit and restart the sequence
above.
Then we'll send 184,324 bytes about every 1.74 seconds and our data rate
is 106 KB/s. Distressingly close to your data rate.
So my hypothesis is that you're suffering from a congestion on the ACK
direction.
One way to test this would be to substantially up the MTU size from the
server you're downloading from. If you double the MTU and your data rate
grows sharply (doubling or more), you know where the problem is...
Craig
Angel
linux with QoS. Was just using rudimentary traffic shapping with squid
as a transparent proxy beforehand. ;)
angel
Fernando Gont
<nos...@hotmail.com> wrote:
>I have downloaded some big files like 600 meg with FTP/TCP software and
>during the transfer the speed is around 120 kilobyte / sec.
>My connection is a cable modem ( 700 kb / sec download, 16 kb/sec upload )
>The other person's connection is a high speed connection like 10/100 megabit
>(campus of university)
>The ping time between us is about 30 milliseconds.
To get the best performance, the sender's socket send buffer and the
receiver's socket receive buffer should be at least (each one)
bandwith*RTT bytes.
If any of them is less than that, the performance you'll get will be
less than expected.
Note than in "Long Fat Pipe" situations (a large bandwith-delay
product), you'll need the "TCP extensions for High Speed" (to use
windows larger than 65535 bytes), which I don't know whether Windows
and the FTP server you're using implement or not.
--
Fernando Gont
e-mail: fern...@ANTISPAM.gont.com.ar
[To send a personal reply, please remove the ANTISPAM tag]
Petr Swedock
;; "Bob" <removebc...@dns2go.com> wrote in message
;; news:eksoqu8rilo3rdk3r...@4ax.com...
;;
;; No, apperently you mis-interpreted the text, what I ment was:
Your spelling leaves much room for various interpretations. If you
expect an effort, make an effort.
;; I download a file from him that was 600 megabytes at a speed of 120
;; kilobytes / sec which is not optimal.
Apparently, the rest of us missed the memo telling us to cease and
desist all activity when you're online...
The internet is 'best effort' and overcrowded. There are no guarantees.
Specifically, there are no guarantees:
0) that your NIC is configured optimally;
1) that your home network is optimal;
2) that your cable LAN isn't oversubscribed;
3) that your ISP doesn't route you through Singapore to get to NYC from
Boston;
4) that the college/university network to-and-from which you are FTP'n
isn't straight out of the 80's, AND/OR is being rate-limited by
their ISP;
5) or that the box on the other end isn't a Sparc IPX running ancient
software and being a file-sharing whore for the rest of the music-
sharing Internet.
All this over an asynchronous line and I'd say you've got it good
at that speed.
Peace,
Petr
Skybuck Flying
news:86r8eqy...@blade-runner.mit.edu...
> "Skybuck Flying" <nos...@hotmail.com> writes:
>
> ;; "Bob" <removebc...@dns2go.com> wrote in message
> ;; news:eksoqu8rilo3rdk3r...@4ax.com...
> ;;
> ;; No, apperently you mis-interpreted the text, what I ment was:
>
> Your spelling leaves much room for various interpretations. If you
> expect an effort, make an effort.
>
> ;; I download a file from him that was 600 megabytes at a speed of 120
> ;; kilobytes / sec which is not optimal.
>
> Apparently, the rest of us missed the memo telling us to cease and
> desist all activity when you're online...
>
> The internet is 'best effort' and overcrowded. There are no guarantees.
>
> Specifically, there are no guarantees:
>
> 0) that your NIC is configured optimally;
I assume TCP and UDP would be both affected the same ammount if it was not
optimal.
> 1) that your home network is optimal;
Same here.
> 2) that your cable LAN isn't oversubscribed;
Same here.
> 3) that your ISP doesn't route you through Singapore to get to NYC from
> Boston;
Same here.
> 4) that the college/university network to-and-from which you are FTP'n
> isn't straight out of the 80's, AND/OR is being rate-limited by
> their ISP;
I know nothing about rate limitations.
Is it normally done per protocol or per connection/interface etc.
I can imagine someone limiting a certain connection.
In that case TCP and UDP would both be limited ?
In the other case only TCP would be limited ?
> 5) or that the box on the other end isn't a Sparc IPX running ancient
> software and being a file-sharing whore for the rest of the music-
> sharing Internet.
Same here.
>
> All this over an asynchronous line and I'd say you've got it good
> at that speed.
120 kilobyte / sec is not bad. What frustrate me is that I know it can do
much better namely 700 kilobyte / sec and the transfer
could have completed a few hours earlier. 700 KB/sec is ofcourse in best
case scenerio... I think it should at least achieve 500 KB/sec given some
overhead and packet loss etc...
What I don't know is why TCP does not do better. Does anyone know ?
Bob
<nos...@hotmail.com> wrote:
>I think it's stupid that people talk about bits when it comes to
>connections. ( pc's work with bytes... who cares about how many bits are on
>your harddisk or in memory... people rather talk about how much bytes they
>have in memory and on disk etc )
>
>I said 700 kb/sec which you rightfully thought ment 700 kilobit/sec
>
>But I really ment 700 kilobytes / sec (KB).
Alot of posters have tried to explain this to you but for some reason
you are not listening. please read this definition.
How kilobits per second stack up to Kilobytes per second.
The Internet measures data-transfer rates one way -- bits per second
-- but the most popular Web browsers show file download performance in
bytes per second. How do you figure this out? By popular request, I am
re-publishing the information given in last week's Broadband Report
newsletter in an easier to read table format. An explanation of the
Data Transfer Rate Conversion Table follows the table.
Broadband Data Transfer Rates
Service Level kilobits per second (kbps) Kilobytes per second (KB/sec)
Estimated Ideal Throughput
"28.8K" Modem 28.8-kbps 3.6-KB/sec 2.8-KB/sec
"33.6K" Modem 33.6-kbps 4.2-KB/sec 3.3-KB/sec
"56K" Modem 53.3-kbps 6.6-KB/sec 5.2-KB/sec
One-channel ISDN 56-kbps 7-KB/sec 5.4-KB/sec
One-channel ISDN 64-kbps 8-KB/sec 6.2-KB/sec
Two-channel ISDN 115.2-kbps 14.4-KB/sec 11.2-KB/sec
Two-channel ISDN / IDSL 128-kbps 16-KB/sec 12.5-KB/sec
Fractional T-1 256-kbps 32-KB/sec 25-KB/sec
"384K" DSL 384-kbps 48-KB/sec 37.5-KB/sec
Satellite 400-kbps 50-KB/sec 39-KB/sec
Fract. T-1 / cable modem 512-kbps 64-KB/sec 50-KB/sec
DSL / Fract. T-1 / cable 768-kbps 96-KB/sec 75-KB/sec
1-Mbps / DSL / cable 1,000-kbps 125-KB/sec 98-KB/sec
T-1 (1.544-Mbps) / DSL 1,544-kbps 193-KB/sec 151-KB/sec
Transfer Rates and Throughput
Bits and bytes. That's where this starts. Bit is short for BInary
Digit, and it's the smallest unit of binary measure, usually either a
0 or a 1. A byte consists of eight bits (in mainstream computing). Two
of many common areas where the terms bits and bytes are applied are
data storage and network data-transfer rates. It's that second one
being described here, and the definitions that follow don't in every
case apply to the data storage definitions of the same terms.
Data-transfer rate is the speed at which data transmits from one
device to another. Among the best known data-transfer rates is the one
for conventional analog modems, whose top speed is currently "56K."
Actually, its fastest download data-transfer rate is governed at
53.3-kbps. 53.3-kbps literally means 53,300 bits per second. There are
1,000 bits in a kilobit. Measuring bits per second is the traditional
way to convey network data- transfer rates. Although technically
speaking, the term kilobit should have a lowercase initial letter,
most published reports capitalize it in abbreviation, resulting in
"56-Kbps," or even the really confusing "56K."
What complicates matters greatly is the fact that most Web surfers see
data-transfer rates measured on a different scale on their browser
download dialog boxes. The newer versions of Internet Explorer and
Netscape Navigator both display data-transfer rates measured in
Kilobytes-per-second (KB/sec). A Kilobyte is 1,024 bytes, and it is
more properly used to measure data storage, not network data-transfer
rates. The numbers in the table show pure mathematical data-transfer
rates. I used a shareware program called Master Converter from Savard
Software to double check my math.
A lot of my newsletter readers have written to me insistently that the
numbers in the table above are wrong. And in a way they're right, and
in a way they're wrong. There's a difference between a data-transfer
rate and throughput. When you're making a kilobit-per-second
conversion to a Kilobyte-per-second conversion, the question you have
to ask is what bits and bytes are you counting.
Even though there are 8 bits in every byte, the nature of asynchronous
transmission (used by most connection methods for accessing the
Internet) requires the receiving of two "dead" bits for every 8 bits
of useful data transferred: the start and stop bits. Bottom line, it
takes 10 bits of transmission horsepower to deliver 8 bits of useful
information. The question this begs is, how much data actually moves
from one place to another in a given amount of time. And that's the
definition of throughput.
Other factors also have an effect on throughput, things like packet
loss, data compression, line quality, and server load. The very
essence of asynchronous data transfer is that it is very flexible,
able to deal with variations in performance. That's why throughput is
an inexact science, and why the table shows guesstimates only, which
use the "divide by 10" method. Some people say it's more proper to
divide by 9, but in my experience, something more like 12 might come
closer to approximating what people actually see on an ideal basis.
Still, I've chosen to divide by 10 because that's the number most
people consider the best one. In the end, you'll have to make your own
judgment.
When you're measuring data-transfer rates, however, that is precise.
Whether you're sending data as kilobits per second or as Kilobytes per
second, many of the throughput-affecting factors affect both rates
equally. So, while a received Kilobyte may not actually equally a full
Kilobyte of useful information, the same is true if you measure it as
kilobits pers second. What I don't know, though, and will endeavor to
find out, is just exactly how the Web browsers are figuring their
Kilobyte per second download rates. Are they counting 10-bits to a
byte, or are they counting 8-bits to a byte? Or are they figuring it
another way entirely? If I learn something that changes any of the
information on this page, I will come back and update it. And point it
out to you in Win98 Insider.
The best way to test the throughput of your Internet connection is to
download a large file from your ISP's server. MediaOne and MSN both
offer this sort of download to their subscribers. Ask your ISP. While
you can download large files from other sources, such a download is
much more subject to the vagaries of the Internet -- although
downloading a 10MB or 15MB file from a variety of servers around the
Internet, and doing so repeatedly over several days, and then
averaging the times would give a good approximation of real-world
throughput. Those are the only numbers that really matter when you get
right down to it.
Rune Svendsen
if a package gets lost nothing is done about it. With TCP it checks for
errors and lost packages and sends them again if they get lost. So TCP
probably uses some bandwidth for error checking etc.
-Rune Svendsen
"Skybuck Flying" <nos...@hotmail.com> wrote in message
news:aojufe$slu$1...@news1.tilbu1.nb.home.nl...
Skybuck Flying
> OK, let's try some computations to try to explain the 120KB/s data rate.
That would be cool...
> First, we'll try some basic limits:
>
> 1. Window size. We don't know your receive window size. If it is 8192
bytes
> then the fastest your data transfer can go is:
>
> 8192 bytes/30ms = 273 KB/s
>
> The rate scales linearly with window size, so if you running a 32KB
> window size (largest you can do without PAWS), the max rate is
> 1.1 MB/s.
I'm on windows XP, speed was the same on windows 98.
I could try to increase the window size on my side and his and see if that
makes a difference... I don't really like the idea of messing with the
registry... especially if I am not sure what the effect will be. Maybe I'll
try this out if the other dude wants to try it out.
This sounds like a nice new socket option for winsock :) being able to set
these kinds of things on a per socket basis :)
instead of messing with the registry :)
Maybe winsock already has a function to do this... but the software just
doesn't have the option... :)
I do know about a socket option to set the socket buffer size... but I don't
think that's the same.
> 2. ACK limits. TCP sends a 40 byte ack for every two data segments. The
40 bytes for IP+TCP+FTP header /data ?
> return like is 16 KB/s so the maximum ACK rate is:
>
> 16 KB/s / 40 bytes = 410 ACKs per second
>
> Inverting this, if data segments are 512 bytes each, and we ack
> two per ack, the maximum data rate is:
>
> 420 KB/s
The segments are probably 1500. hmmm I could sniff this with ethereal.
>
> Side note -- this observation says DO NOT enable TCP large windows --
> you will reduce the number of ACKs received per second (because the
> ACKs are larger) and get *worse* throughput.
>
> OK, now both these results (esp. ACK limits) say you're rate limited
> to somewhere around 200KB/s, but that's still not 120 KB/s. So on to
> the harder calculations.
>
> 3. TCP performance, sending side. Assuming a perfect ACK stream
> (i.e., no limits on ACKs), TCP is going to do one slow slow start
> and then send as fast as possible.
>
> If we assume a 8192KB window and a 512 byte starting segment size,
> we'll:
These values are probably different.
> send 512 bytes, wait for delayed ack (230ms) then
> send 1024 bytes, wait for ack (30ms) then
> send 2048 bytes, wait for ack (30ms) then
> send 4096 bytes, wait for ack (30ms) then
> send 8196 bytes, wait for ack (30ms) then
> send full rate.
>
> These startup round trip times are negligible against the total
> number of round-trips, so there's probably NO slow start effect.
>
> 4. Let's look at the ACK stream more carefully.
>
> On the sending side, we can transmit 700KB/s / 512 byte segments
> per second -- that's 1400 segments. But we can only ack 820 of
> them. That's a problem as it means we'll overload queues and
> drop ACKs, causing TCP to restart the slow start process.
Question is which queues are overloaded ?
On the sending / On the receiving side ?
You mean the sending part or the receiving part ?
There are two things I am thinking of:
1. TCP sends only 1 ack per packet... which means the overhead is huge.
2. I am not sure but I think TCP does not use up all the 16 KB/sec
bandwidth...
Right now I am chatting with the other dude and we are trying to get some
more data on the subject...
>
> One way to test this would be to substantially up the MTU size from the
> server you're downloading from. If you double the MTU and your data rate
> grows sharply (doubling or more), you know where the problem is...
The MTU of the server is probably 1500 (ethernet)
Upping the MTU is I think not possible.
Getting more interested in this I quickly find some stuff which is more or
less about this post...
http://www.extremetech.com/article2/0,3973,671,00.asp
Skybuck.
Dragan Cvetkovic
> Data-transfer rate is the speed at which data transmits from one
> device to another. Among the best known data-transfer rates is the one
> for conventional analog modems, whose top speed is currently "56K."
> Actually, its fastest download data-transfer rate is governed at
> 53.3-kbps. 53.3-kbps literally means 53,300 bits per second. There are
> 1,000 bits in a kilobit. Measuring bits per second is the traditional
> way to convey network data- transfer rates. Although technically
> speaking, the term kilobit should have a lowercase initial letter,
> most published reports capitalize it in abbreviation, resulting in
> "56-Kbps," or even the really confusing "56K."
>
Not to mention the standard (marketing) confusion, mostly prevailed with
hard disk manufacturer to use 1KB for 1000 bytes, 1MB for 1000000 bytes
etc. That's one of the reason International Electrotechnical Commission
(IEC) tries to introduce Ki, Mi prefixes etc for 2^10, 2^20 etc (see
http://physics.nist.gov/cuu/Units/binary.html for details) so that
1KiB = 1024 bytes, 1KB = 1000 bytes
or
disk of 20GB is actually 18.62645 GiB
etc.
Bye, Dragan
--
Dragan Cvetkovic,
To be or not to be is true. G. Boole No it isn't. L. E. J. Brouwer
Skybuck Flying
Some things about my stuff:
It's in practical to talk about bits per second.
But if you want it here it is:
700 KB * 8 = 5600 kb / sec
Wow ! now that's a big number ! it almost looks unbelievable.
Also my tool measures how much real data it can transfer... all bytes wasted
on overhead, error correction all that stuff is not in the equation. So when
my tool reports 712.123 bytes / sec... then it really is 712.123 bytes per
second on usuable data.
The speed as your text states will depend on overhead etc... that's why you
can change the data size at real time in the tool to see how it impacts the
usuable data rate... as you like.
"Bob" <removebc...@dns2go.com> wrote in message
news:302rqu4l87gtp3vcr...@4ax.com...
Angel
> I am well aware of how these things work.
>
> Some things about my stuff:
>
> It's in practical to talk about bits per second.
>
> But if you want it here it is:
>
> 700 KB * 8 = 5600 kb / sec
>
> Wow ! now that's a big number ! it almost looks unbelievable.
As I said initially. Judging by your figures you have a cable
connection providing ~6Mbps downstream and ~128kbps upstream. I
wouldn't mind having one of those. I could live with the ~128kbps
upstream quite happily. Unfortunately, to my knowledge, such a thing
doesn't exist where I am.
angel
Skybuck Flying
news:aohu2n$1iae$1...@news.dataguard.no...
>
> "Skybuck Flying" <nos...@hotmail.com> wrote in message
> news:DKZq9.85356$48.93...@zwoll1.home.nl...
> >
> > "Dan Swartzendruber" <dsw...@druber.com> wrote in message
> > news:MPG.1815df3e4...@news.supernews.net...
> > > In article <3dac1107$0$33006$bed6...@pubnews.gradwell.net>,
> > > enough@spam_already says...
> > > > An obvious question here. Could you possibly be mixing your bits
and
> > bytes?
> > > >
> > > > 120 KiloBytes per second suggests a 1 Megabit per second cable
> > > > connection. 700 KiloBytes per second suggests something like a 6
> > > > Megabit per second cable connection. If you have this as a
> residential
> > > > cable installation I'm incredibly envious ;)
> >
> > > he should check RWIN on his PC. if it's an older windows (e.g. win95)
> > > or was set up for dialup, it might have only 8192 bytes for RWIN,
which
> > > makes TCP perform suboptimally. if this is the case, search for and
> > > download DrTCP which lets you tweak RWIN.
> >
> > My operating system is windows XP.
> >
> > Speed was the same on windows 98 by the way.
> >
> >
>
> helo ;)
>
> www.dslreports.com/tweaks
I did the test and the site reported this:
Receive Window (RWIN): 17520
Window Scaling: off
Path MTU Discovery: ON
RFC1323 Window Scaling: OFF
RFC1323 Time Stamping: OFF
Selective Acks: ON
MSS requested: 1460
TTL: unknown
TTL remaining: 111
Example 146000 byte download
Actual data bytes sent: 146000
Actual data packets: 100
Max packet sent (MTU): 1500
Max packet recd (MTU): 1500
Retransmitted data packets: 0
sacks you sent: 0
pushed data pkts: 15
data transmit time: 1.154 secs
our max idletime: 121.1 ms
transfer rate: 84312 bytes/sec
transfer rate: 674 kbits/sec
This is not a speed test!
transfer efficiency: 100%
Ofcourse a file of 146000 bytes is way to small to test the connection
speed.
My connection is able to download 700 KB/sec which is much greater than this
small file... but ok... I'll play along with the test...
The test also mentions a little note: "This is not a speed test!"
That note says it all... pretty fucking worthless.
But ok, I believe the settings in windows XP are the same as these results.
How does this site know what is optimal ?
Is it based on pure theory... or is it based on practical tests.
It's probably based on some theory that these settings would be optimal.
I just can't believe that TCP is only cable of transferring 115 KB/sec
between me and that other person. I am giving TCP the advantage of doubt. I
am trying to figure out why TCP is limited to 115 KB/sec in my mentioned
situation. I refuse to believe these settings are optimal because that would
mean TCP is shitslow.
Maybe these settings are optimal for local area network or something...
Or maybe it's optimal for connection which are symetric... upload and
download speed are the same.
But in my situation the upload speed is limited and the download speed seems
almost unlimited...
These settings are probably not optimal in my situation... that's the only
explanation I'm willing to except...
Excepting that TCP is so crappy at this situation is fricking irritating if
not unacceptable hehe.
>
> it runs a very respectable and objective online test of your line along
with
Respectable hardly... Objective ?
> your settings on MTU, RWIN, ping, packet loss etc.
Yes... this is what it does ok, except packet loss haha... 147.00 bytes I
don't think so.
> It will tell you if your connection is optimised.
Simple said: bull...
>
> Good Luck !
>
I need it :)
Skybuck Flying
news:3dad895e$0$79645$edfa...@dspool01.news.tele.dk...
> I'm not sure, but if you send UDP packets, there are no error checking and
> if a package gets lost nothing is done about it. With TCP it checks for
> errors and lost packages and sends them again if they get lost. So TCP
> probably uses some bandwidth for error checking etc.
:D
You are 100% right... the thing is... it still does not explain the big
difference between 700 KB/s and 115 KB/s.
700-115 = 585 KB/s overhead + error correction + lost packets.
No way.
Skybuck Flying
"Angel" <enough@spam_already> wrote in message
I would be a bit more happy to... if:
1. ftp software doesn't crash,hang and shit like that.
2. tcp or tcp+ftp is able to achieve 500 to 600 KB/sec instead of just 115
KB/sec.
I would be a shitload more happy if:
3. My upstream limit was 6 Mbps as well.
Sky :D
Skybuck Flying
"Fernando Gont" <arie...@softhome.net> wrote in message
news:3dad3c9c...@News.CIS.DFN.DE...
Craig Partridge
>"Craig Partridge" <cra...@world.std.com> wrote in message
>> 2. ACK limits. TCP sends a 40 byte ack for every two data segments. The
>40 bytes for IP+TCP+FTP header /data ?
20 bytes of IP header and 20 bytes of TCP header -- no data, this is just
an ACK.
>> 4. Let's look at the ACK stream more carefully.
>>
>> On the sending side, we can transmit 700KB/s / 512 byte segments
>> per second -- that's 1400 segments. But we can only ack 820 of
>> them. That's a problem as it means we'll overload queues and
>> drop ACKs, causing TCP to restart the slow start process.
>Question is which queues are overloaded ?
>On the sending / On the receiving side ?
The queue at the receiving side immediately closest to the 16KB link is
the one that will overload.
>There are two things I am thinking of:
>1. TCP sends only 1 ack per packet... which means the overhead is huge.
Sends 1 ack every other packet.
>2. I am not sure but I think TCP does not use up all the 16 KB/sec
>bandwidth...
It won't if you are congested on the return (16KB link). You'll see it
periodically completely full, followed by about half a second when it
is lightly to moderately loaded. That's the beauty/pain of TCP's
congestion behavior.
Craig
Dana
It is very easily done.
>
>
>
John S. Giltner, Jr.
per second on 10 Mb Ethernet and 16 Mb Token Ring. Heck, I average
150-170 KB on a T1. However on the LAN connections I had to
increase my received window size to 24KB before I could not get any
better througput.
If you are truly getting 700 KB (kilobytes) this means that the
slowest down stream connection point between you and the ftp server
is most likley a 6 Mb (megabits). Not to bad, I think most cable
providers in the USA limit home users to 1 Mb.
If youf friend is on a collage campus, they are most likley using
traffic limiting software either in transparent proxy servers or
with options configured in routers, to limit bandwidth used.
Your UDP test may show 700 KB because of what they may be doing to
limit the bandwidth. Most routers limit bandwidth based on the port
used and TCP vs. UDP. So their routers may limit traffic usint TCP
on port 20 to a specific bandwidth, but other ports may not be
limited. Other programs limit bandwidth based on the throuput for a
single connection, as UDP has no connection, it would not be limited.
Have you tried using passive FTP? Some simple routers can not be
configured to detect passive FTP and their for can not limit the
bandwidth, however firewalls or routers could be setup to block
passive FTP.
In one other post you said that your TCP Window size was about 17
KB. I have had to use a Window size of 24 KB to drive 10 MB full
duplex Ethernet links. As the up stream connection is much slower
than your down stream the ACK's could be causing the problem. I
would have to work out the math, but the FTP server could send out
17 KB of data before the first ACK got back, so the FTP server would
then need to wait until it got the ACK.
I would suggest that you try to either enable TCP Window scaling, or
increase your window size to 32KB or so. If you do not have one you
may want to get a sniffer program and trace the traffic from your
side to see if that could help determine where the problem is.
Skybuck Flying wrote:
> I have downloaded some big files like 600 meg with FTP/TCP software and
> during the transfer the speed is around 120 kilobyte / sec.
>
> My connection is a cable modem ( 700 kb / sec download, 16 kb/sec upload )
>
> The other person's connection is a high speed connection like 10/100 megabit
> (campus of university)
>
> The ping time between us is about 30 milliseconds.
>
> With my own tool I tested the speed from him to me which is about 700
> kilobyte/sec with UDP.
>
> From me to him it's about 16 kilobyte / sec with UDP.
>
> But during the TCP/FTP transfer the transfer rate from him to me is only 120
> kilobyte / sec ???
>
> Which seems to be only a few procent of the total connection speed.
>
> So my question is why is TCP/FTP so slow ? Why can't it do better ?
>
> I could think up so many reasons which could have something to do with:
>
> TCP itself is slow, or FTP is slow, or overhead is too much, or FTP server
> software is slow or FTP client software is slow.
>
> Maybe writing the file to HD or reading it is to slow...
>
> Maybe the stack is slow ? maybe the stack options in my operating system
> like window size or MTU is not optimized...
>
> Maybe TCP is just using only a few procent to allow other programs to access
> the internet...
>
> Maybe TCP is just inefficient and drops in speed when a packet is lost ?
>
> Maybe routers on the network reduce the TCP speed or something ?
>
> Anyone have any clue why TCP is so slow in this case ?
>
>
Steve Watt
Skybuck Flying <nos...@hotmail.com> wrote:
>I have downloaded some big files like 600 meg with FTP/TCP software and
>during the transfer the speed is around 120 kilobyte / sec.
>
>My connection is a cable modem ( 700 kb / sec download, 16 kb/sec upload )
And that is almost certainly your problem. TCP is designed for
networks where the transfer rates are not wildly assymetrical. For
each two segments that your system transmits, you will receive one
ACKnowledgement. A TCP ACK takes 40 bytes (or a little bit more,
depending on options in use on the connection).
Thus, if your connection segment size is relatively small (which it
is by default on Windows platforms), you are only able to receive a
relatively small amount of data (by your downstream transfer rate's
definition) before you run out of bandwidth on your upstream transfer
rate trying to send ACKnowledgemets.
Because some ACKs will be dropped by your cable modem (they're exceeding
your allocated upstream bandwidth), the TCP at the far end will back
off, assuming that it's overrunning the receiver.
The moral of this story is that TCP can't saturate a link one way if the
link the other way is less than about one tenth the size. So a 700kB
downlink should have a 70kB uplink. But cable modem companies know
this, and that is part of how they can advertise such wonderous rates --
it's not guaranteed service (bandwidth varies a lot because it's shared
by a lot of people), and they always severely restrict the upstream
rates.
About the only thing you can do about it is to ensure that you advertise
as large an RWIN as possible. There are tools out there (check
dslrpeorts.com) that allow you to tweak those settings in a pretty
fashion. Bigger RWIN is almost always better, unless you have a
really lossy network.
--
Steve Watt KD6GGD PP-ASEL-IA ICBM: 121W 56' 57.8" / 37N 20' 14.9"
Internet: steve @ Watt.COM Whois: SW32
Free time? There's no such thing. It just comes in varying prices...
Barry Margolin
>In article <GjTq9.84650$48.91...@zwoll1.home.nl>,
>Skybuck Flying <nos...@hotmail.com> wrote:
>>I have downloaded some big files like 600 meg with FTP/TCP software and
>>during the transfer the speed is around 120 kilobyte / sec.
>>
>>My connection is a cable modem ( 700 kb / sec download, 16 kb/sec upload )
>
>And that is almost certainly your problem. TCP is designed for
>networks where the transfer rates are not wildly assymetrical. For
>each two segments that your system transmits, you will receive one
>ACKnowledgement. A TCP ACK takes 40 bytes (or a little bit more,
>depending on options in use on the connection).
>
>Thus, if your connection segment size is relatively small (which it
Do you mean receive window size?
>is by default on Windows platforms), you are only able to receive a
>relatively small amount of data (by your downstream transfer rate's
>definition) before you run out of bandwidth on your upstream transfer
>rate trying to send ACKnowledgemets.
>
>Because some ACKs will be dropped by your cable modem (they're exceeding
>your allocated upstream bandwidth), the TCP at the far end will back
>off, assuming that it's overrunning the receiver.
Although TCP normally sends an ACK for every 2 segments received, it's not
critical that they all get through, as long as the window size is large
enough. This is because an ACK subsumes all previous ACKs. If every other
ACK is lost, it's effectively as if you were ACKing every 4th segment
instead of every 2nd segment.
>The moral of this story is that TCP can't saturate a link one way if the
>link the other way is less than about one tenth the size. So a 700kB
>downlink should have a 70kB uplink. But cable modem companies know
>this, and that is part of how they can advertise such wonderous rates --
>it's not guaranteed service (bandwidth varies a lot because it's shared
>by a lot of people), and they always severely restrict the upstream
>rates.
His asymmetry is pretty extreme. My cable modem service (AT&T Broadband)
is 1.5Mbps downstream and 256Kbps upstream (I think that's right -- they
changed the speeds a few months ago to reconcile the former AT&T RoadRunner
and @Home services that were subsumed).
--
Barry Margolin, bar...@genuity.net
Genuity, Woburn, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
Steve Watt
Barry Margolin <bar...@genuity.net> wrote:
>In article <H455w...@Watt.COM>, Steve Watt <st...@nospam.Watt.COM> wrote:
>>In article <GjTq9.84650$48.91...@zwoll1.home.nl>,
>>Skybuck Flying <nos...@hotmail.com> wrote:
>>>I have downloaded some big files like 600 meg with FTP/TCP software and
>>>during the transfer the speed is around 120 kilobyte / sec.
>>>
>>>My connection is a cable modem ( 700 kb / sec download, 16 kb/sec upload )
>>
>>And that is almost certainly your problem. TCP is designed for
>>networks where the transfer rates are not wildly assymetrical. For
>>each two segments that your system transmits, you will receive one
>>ACKnowledgement. A TCP ACK takes 40 bytes (or a little bit more,
>>depending on options in use on the connection).
>>
>>Thus, if your connection segment size is relatively small (which it
>
>Do you mean receive window size?
No, I meant the receive MSS, because ACKs are based on segments,
not windows. Up too late last night.
>>is by default on Windows platforms), you are only able to receive a
>>relatively small amount of data (by your downstream transfer rate's
>>definition) before you run out of bandwidth on your upstream transfer
>>rate trying to send ACKnowledgemets.
>>
>>Because some ACKs will be dropped by your cable modem (they're exceeding
>>your allocated upstream bandwidth), the TCP at the far end will back
>>off, assuming that it's overrunning the receiver.
>
>Although TCP normally sends an ACK for every 2 segments received, it's not
>critical that they all get through, as long as the window size is large
>enough. This is because an ACK subsumes all previous ACKs. If every other
>ACK is lost, it's effectively as if you were ACKing every 4th segment
>instead of every 2nd segment.
Except that in most cases, it'll be tail-drop unless there's an
unusually clever router in the path. So the last pile of ACKs will
drop, and a retransmit will happen to get things restarted.
>>The moral of this story is that TCP can't saturate a link one way if the
>>link the other way is less than about one tenth the size. So a 700kB
>>downlink should have a 70kB uplink. But cable modem companies know
>>this, and that is part of how they can advertise such wonderous rates --
>>it's not guaranteed service (bandwidth varies a lot because it's shared
>>by a lot of people), and they always severely restrict the upstream
>>rates.
>
>His asymmetry is pretty extreme. My cable modem service (AT&T Broadband)
>is 1.5Mbps downstream and 256Kbps upstream (I think that's right -- they
>changed the speeds a few months ago to reconcile the former AT&T RoadRunner
>and @Home services that were subsumed).
Yup. I've seen similar problems on links that were satellite one
way and modem the other. Don't ask why it was that way -- it was
a *really* weird system, but worked OK for where and when it was.
Skybuck Flying
news:H455w...@Watt.COM...
> In article <GjTq9.84650$48.91...@zwoll1.home.nl>,
> Skybuck Flying <nos...@hotmail.com> wrote:
> >I have downloaded some big files like 600 meg with FTP/TCP software and
> >during the transfer the speed is around 120 kilobyte / sec.
> >
> >My connection is a cable modem ( 700 kb / sec download, 16 kb/sec
upload )
>
> And that is almost certainly your problem. TCP is designed for
> networks where the transfer rates are not wildly assymetrical. For
> each two segments that your system transmits, you will receive one
> ACKnowledgement.
Let me start by saying that this is not the probably.
Thank god I made a logfile with ethereal the last time I transferred a file
from him to me, so I was able to analyze what was going in and out, so I am
slowly starting to learn to make /gather more professional test results, I
need those bad :)
What I did was I looked at the logfile and I counted all the TCP/ACKS.
There are about 60 to 70 acks sent from my system per second. Each ack is 40
bytes (ip length)
70 * 40 bytes = 2.800 bytes per second.
So clearly this is not the problem.
My system/connection can easily send 16 KB/sec.
Also when I download from certain sites in my country I am able to download
with 200 KB/sec sometimes even 300 KB/sec.
( Simply via internet explorer, maybe internet explorer 6 is using multiple
TCP connection just like download accelerator, that
could explain why internet explorer sometimes makes a temporarely file... to
build the output later on )
> A TCP ACK takes 40 bytes (or a little bit more,
> depending on options in use on the connection).
>
> Thus, if your connection segment size is relatively small (which it
What exactly is a segment ?
> is by default on Windows platforms), you are only able to receive a
> relatively small amount of data (by your downstream transfer rate's
> definition) before you run out of bandwidth on your upstream transfer
> rate trying to send ACKnowledgemets.
>
> Because some ACKs will be dropped by your cable modem (they're exceeding
> your allocated upstream bandwidth), the TCP at the far end will back
> off, assuming that it's overrunning the receiver.
>
> The moral of this story is that TCP can't saturate a link one way if the
> link the other way is less than about one tenth the size. So a 700kB
> downlink should have a 70kB uplink. But cable modem companies know
> this, and that is part of how they can advertise such wonderous rates --
> it's not guaranteed service (bandwidth varies a lot because it's shared
> by a lot of people), and they always severely restrict the upstream
> rates.
>
> About the only thing you can do about it is to ensure that you advertise
> as large an RWIN as possible. There are tools out there (check
> dslrpeorts.com) that allow you to tweak those settings in a pretty
> fashion. Bigger RWIN is almost always better, unless you have a
> really lossy network.
Like I said upstrairs... only 2KB/sec is used so sometimes else is causing
the 'problem'.
Skybuck.
Rick Jones
> Sends 1 ack every other packet.
Or less depending on the stack and circumstances.
rick jones
--
Wisdom Teeth are impacted, people are affected by the effects of events.
these opinions are mine, all mine; HP might not want them anyway... :)
feel free to post, OR email to raj in cup.hp.com but NOT BOTH...
Fernando Gont
<nos...@hotmail.com> wrote:
>Don't you think windows xp will set the buffers to max ?
It doesn't matter what I *think* XP should do. It matters what it
actually does.
I don't run any Windows XP host, so I cannot test it.
If you run one, put a packet sniffer on the wire, and see what it
actually happens.
John S. Giltner, Jr.
> There are about 60 to 70 acks sent from my system per second. Each ack is 40
> bytes (ip length)
>
> 70 * 40 bytes = 2.800 bytes per second.
>
> So clearly this is not the problem.
>
> My system/connection can easily send 16 KB/sec.
>
> Also when I download from certain sites in my country I am able to download
> with 200 KB/sec sometimes even 300 KB/sec.
> ( Simply via internet explorer, maybe internet explorer 6 is using multiple
> TCP connection just like download accelerator, that
> could explain why internet explorer sometimes makes a temporarely file... to
> build the output later on )
If you are sending out 60-70 acks per second, then assuming 1 ack
for every two inbound packets, then you are receiving 120-140
packets a second. Assuming 1500 byte packets, this is 180-210 KB
per/second (or 175 - 205 KB if using 1024 = 1 K). Although this is
still slower that what your side should be able to receive, this is
faster that what you stated you were getting. In the ethereal trace
are you actually receiving 120-140 1500 byte packets per second?
Internet Explorer does NOT use multiple TCP connections. Interent
Explorer uses the temporary file as a way to perform restart processing.
Are you using Explorer to download from your friends site, or the
FTP client? I do not know about XP, but Windows NT's ftp client
used a 4K buffer for sending and receiving data. What you may want
to try using the command ftp -w:32768 ftp.server.name this will
tell the ftp client to use a 32K buffer. Under NT this speeds up
the ftp client a lot.
What FTP server is your friend running? I beleive that the
non-Server (the one with the personal Web server for NT Workstation)
Windows FTP server also uses a 4K buffer size, which slows it down
considerabilty.
Skybuck Flying
"John S. Giltner, Jr." <gil...@earthlink.net> wrote in message
news:wh3s9.1724$U97.1...@newsread2.prod.itd.earthlink.net...
> Skybuck Flying wrote:
> > There are about 60 to 70 acks sent from my system per second. Each ack
is 40
> > bytes (ip length)
> >
> > 70 * 40 bytes = 2.800 bytes per second.
> >
> > So clearly this is not the problem.
> >
> > My system/connection can easily send 16 KB/sec.
> >
> > Also when I download from certain sites in my country I am able to
download
> > with 200 KB/sec sometimes even 300 KB/sec.
> > ( Simply via internet explorer, maybe internet explorer 6 is using
multiple
> > TCP connection just like download accelerator, that
> > could explain why internet explorer sometimes makes a temporarely
file... to
> > build the output later on )
>
> If you are sending out 60-70 acks per second, then assuming 1 ack
> for every two inbound packets, then you are receiving 120-140
> packets a second. Assuming 1500 byte packets, this is 180-210 KB
> per/second (or 175 - 205 KB if using 1024 = 1 K). Although this is
> still slower that what your side should be able to receive, this is
> faster that what you stated you were getting. In the ethereal trace
> are you actually receiving 120-140 1500 byte packets per second?
I counted them it was something like:
95 ftp data 1460 bytes.
5 ftp data 324 bytes.
> Internet Explorer does NOT use multiple TCP connections. Interent
> Explorer uses the temporary file as a way to perform restart processing.
What is that 'restart-processing' ?
>
> Are you using Explorer to download from your friends site, or the
> FTP client? I do not know about XP, but Windows NT's ftp client
> used a 4K buffer for sending and receiving data. What you may want
> to try using the command ftp -w:32768 ftp.server.name this will
> tell the ftp client to use a 32K buffer. Under NT this speeds up
> the ftp client a lot.
The ftp client software I am using is WS_FTP le.
>
> What FTP server is your friend running? I beleive that the
> non-Server (the one with the personal Web server for NT Workstation)
> Windows FTP server also uses a 4K buffer size, which slows it down
> considerabilty.
The ftp server software he uses is 'war ftp'.
Skybuck Flying
Windows XP build in ftp client is 3x times as fast as WS_FTP le.
What a big difference !
And how stupid !
This is what I don't like about FTP etc...
There is still no explanation why WS_FTP is 3x times slower than Windows XP
ftp.
And even then windows xp ftp is only getting 300 KB/sec.
My own TCP tool is getting between 200 and 400 KB/sec.
And my UDP speed test tool indicates that my connection is able of 700
KB/sec.
The receive buffer size has nothing to do with.
Windows XP build in ftp client is 3x times as fast as WS_FTP le.
What a big difference !
And how stupid !
This is what I don't like about FTP etc...
There is still no explanation why WS_FTP is 3x times slower than Windows XP
ftp.
And even then windows xp ftp is only getting 300 KB/sec.
My own TCP tool is getting between 200 and 400 KB/sec.
And my UDP speed test tool indicates that my connection is able of 700
KB/sec.
Fernando Gont
Partridge) wrote:
>1. Window size. We don't know your receive window size. If it is 8192 bytes
> then the fastest your data transfer can go is:
> 8192 bytes/30ms = 273 KB/s
> The rate scales linearly with window size, so if you running a 32KB
> window size (largest you can do without PAWS), the max rate is
> 1.1 MB/s.
Why do you say 32KB is "the largest you can do without PAWS"?
>2. ACK limits. TCP sends a 40 byte ack for every two data segments. The
> return like is 16 KB/s so the maximum ACK rate is:
> 16 KB/s / 40 bytes = 410 ACKs per second
> Inverting this, if data segments are 512 bytes each, and we ack
> two per ack, the maximum data rate is:
> 420 KB/s
> Side note -- this observation says DO NOT enable TCP large windows --
> you will reduce the number of ACKs received per second (because the
> ACKs are larger) and get *worse* throughput.
>OK, now both these results (esp. ACK limits) say you're rate limited
According to your calculations, the limit the ACK-rate imposes is 420
KB/s, while the limit the window size imposes is 273 KB/s.
So that the limit is the window size, and not the rate at which you
can send ACKs.
.
>3. TCP performance, sending side. Assuming a perfect ACK stream
> (i.e., no limits on ACKs), TCP is going to do one slow slow start
> and then send as fast as possible.
> If we assume a 8192KB window and a 512 byte starting segment size,
> we'll:
> send 512 bytes, wait for delayed ack (230ms) then
> send 1024 bytes, wait for ack (30ms) then
> send 2048 bytes, wait for ack (30ms) then
> send 4096 bytes, wait for ack (30ms) then
> send 8196 bytes, wait for ack (30ms) then
> send full rate.
If we suppose the receiver ACKs every other segment, shouldn't this
be:
send 512 bytes (1 segment), wait for delayed ack (230ms) then
send 1024 bytes (2 segments), wait for ack (30ms) then
send 1536 bytes (3 segments)
.
.
.
?
I mean, the increase of cwnd does not happen as fast.
Or is it that you increase cwnd according to the *number of bytes* the
ACKs acknowledge, and not according the *number of ACKs* you receive?
> These startup round trip times are negligible against the total
> number of round-trips, so there's probably NO slow start effect.
>4. Let's look at the ACK stream more carefully.
> On the sending side, we can transmit 700KB/s / 512 byte segments
If you suppose a socket send buffer size of 8192 bytes, then you won't
reach 700 KB/s.
Fernando Gont
<nos...@hotmail.com> wrote:
>> About the only thing you can do about it is to ensure that you advertise
>> as large an RWIN as possible. There are tools out there (check
>> dslrpeorts.com) that allow you to tweak those settings in a pretty
>> fashion. Bigger RWIN is almost always better, unless you have a
>> really lossy network.
>Like I said upstrairs... only 2KB/sec is used so sometimes else is causing
>the 'problem'.
The socket send buffer size on the FTP server, perhaps.
John S. Giltner, Jr.
> The receive buffer size has nothing to do with.
>
> Windows XP build in ftp client is 3x times as fast as WS_FTP le.
>
> What a big difference !
>
> And how stupid !
>
> This is what I don't like about FTP etc...
>
> There is still no explanation why WS_FTP is 3x times slower than Windows XP
> ftp.
>
> And even then windows xp ftp is only getting 300 KB/sec.
>
> My own TCP tool is getting between 200 and 400 KB/sec.
>
> And my UDP speed test tool indicates that my connection is able of 700
> KB/sec.
>
> The receive buffer size has nothing to do with.
>
> Windows XP build in ftp client is 3x times as fast as WS_FTP le.
>
> What a big difference !
>
> And how stupid !
>
> This is what I don't like about FTP etc...
>
First, what is in the IP stack is only the default used if the
application does not over ride it. WS-FTP may be over riding the
default receive buffer size. I do not beleive that there is anyway
for you to see what it is using.
As for your TCP tool vs. FTP. As I do not know what your tool does,
I can not comment on it. However, FTP must not only recieve the
data it must also write the data to DISK. Although DISK's are fast,
it does take overhead to write, a few ms here and there can add up.
As for the restart in my earlier post, FTP does support restart of
failed transfers. Some ftp clients do support, what they will do is
store the file under a temporary name until the transfer is
complete. Once the transfer is complete they put the file under the
real name where you wanted it. If the transfer fails, you can
restart it and it will look at the number of bytes it has the the
temporary file and tell the server to start sendin from byte "x".
Now the FTP server must suppor this, and the restart will only
occure if the time/date stamps on the source file and the tempoary
file match.
Fernando Gont
<gil...@earthlink.net> wrote:
>> This is what I don't like about FTP etc...
>The receive buffer size still may have something to do with it.
>First, what is in the IP stack is only the default used if the
>application does not over ride it. WS-FTP may be over riding the
>default receive buffer size. I do not beleive that there is anyway
>for you to see what it is using.
Use a sniffer, watch the advertised window size before the receiver
gets any data (in the segment that contains the receiver's SYN, for
example). It should be equal to the receiver's socket receive buffer
size.
Skybuck Flying
> Skybuck Flying wrote:
> > The receive buffer size has nothing to do with.
> >
> > Windows XP build in ftp client is 3x times as fast as WS_FTP le.
> >
> > What a big difference !
> >
> > And how stupid !
> >
> > This is what I don't like about FTP etc...
> >
> > There is still no explanation why WS_FTP is 3x times slower than Windows
XP
> > ftp.
> >
> > And even then windows xp ftp is only getting 300 KB/sec.
> >
> > My own TCP tool is getting between 200 and 400 KB/sec.
> >
> > And my UDP speed test tool indicates that my connection is able of 700
> > KB/sec.
> >
> > The receive buffer size has nothing to do with.
> >
> > Windows XP build in ftp client is 3x times as fast as WS_FTP le.
> >
> > What a big difference !
> >
> > And how stupid !
> >
> > This is what I don't like about FTP etc...
> >
> The receive buffer size still may have something to do with it.
> First, what is in the IP stack is only the default used if the
> application does not over ride it. WS-FTP may be over riding the
> default receive buffer size. I do not beleive that there is anyway
> for you to see what it is using.
ws_ftp has a option called receive buffer size which can be set...
> As for your TCP tool vs. FTP. As I do not know what your tool does,
> I can not comment on it. However, FTP must not only recieve the
> data it must also write the data to DISK. Although DISK's are fast,
> it does take overhead to write, a few ms here and there can add up.
True... the disk activity seems low though.
>
> As for the restart in my earlier post, FTP does support restart of
> failed transfers. Some ftp clients do support, what they will do is
> store the file under a temporary name until the transfer is
> complete. Once the transfer is complete they put the file under the
> real name where you wanted it. If the transfer fails, you can
> restart it and it will look at the number of bytes it has the the
> temporary file and tell the server to start sendin from byte "x".
> Now the FTP server must suppor this, and the restart will only
> occure if the time/date stamps on the source file and the tempoary
> file match.
That's weird... I would put the temporarely file into the destination folder
and then when it's complete just rename it... now internet explorer has to
copy the whole damn file... :( which takes long for a 600 meg file.
John S. Giltner, Jr.
>
> ws_ftp has a option called receive buffer size which can be set...
You may want to play with this option. At a minumum I would double
the default or just start at 32K and see what happens.
>
>
> True... the disk activity seems low though.
Even with low activity, if it takes 10 ms to write and verify the
data that is 10 ms that most likely no IP activity is taking place.
I do not know this for a fact, just a possibility.
>
>
>
> That's weird... I would put the temporarely file into the destination folder
> and then when it's complete just rename it... now internet explorer has to
> copy the whole damn file... :( which takes long for a 600 meg file.
>
MS is one of the greatest marketing machines in the world today, but
not necessary one of the most technically advanced. How they handle
this is a pain. Especially if the drive that that destination
folder is on has 600 MB free, but the drive that the temporary
folder is not does not have 600 MB free.
Alex Fraser
> Skybuck Flying wrote:
>
> > True... the disk activity seems low though.
>
> Even with low activity, if it takes 10 ms to write and verify the
> data that is 10 ms that most likely no IP activity is taking place.
> I do not know this for a fact, just a possibility.
I doubt it's an issue. But I don't know that for a fact either :).
> > That's weird... I would put the temporarely file into the destination
folder
> > and then when it's complete just rename it... now internet explorer has
to
> > copy the whole damn file... :( which takes long for a 600 meg file.
If the destination directory is on the same drive/partition, it will just be
a rename operation. If it's on a different drive, it shouldn't be too bad.
The killer is if it's a different partition on the same drive.
Alex
Skybuck Flying
"Alex Fraser" <m...@privacy.net> wrote in message
news:apdmev$k72s$1...@ID-149533.news.dfncis.de...
> "John S. Giltner, Jr." <gil...@earthlink.net> wrote in message
> news:Avnu9.5183$6F4.4...@newsread2.prod.itd.earthlink.net...
> > Skybuck Flying wrote:
> >
> > > True... the disk activity seems low though.
> >
> > Even with low activity, if it takes 10 ms to write and verify the
> > data that is 10 ms that most likely no IP activity is taking place.
> > I do not know this for a fact, just a possibility.
>
> I doubt it's an issue. But I don't know that for a fact either :).
TCP is streaming protocol so the packets would arrive in order that way
the harddisk would only have to do a track by track seek but only if the
free space
is not fragmented... also harddisks have a cache...
Also I tried a 32 KB buffer with WS_FTP...
Let's just assume the WS_FTP software has some flaw somewhere which seems
the most logical assumption :)
Windows XP ftp client is 3 times faster no matter what receive buffer size.
>
> > > That's weird... I would put the temporarely file into the destination
> folder
> > > and then when it's complete just rename it... now internet explorer
has
> to
> > > copy the whole damn file... :( which takes long for a 600 meg file.
>
> If the destination directory is on the same drive/partition, it will just
be
> a rename operation.
I wish it was.. yet it's not... I have seen internet explorer copy such a
file...
I only have 1 drive and 1 partition.
It say something like moving temporarely file to destination folder or
something.
Alun Jones
Giltner, Jr." <gil...@earthlink.net> wrote:
>Skybuck Flying wrote:
>> That's weird... I would put the temporarely file into the destination folder
>> and then when it's complete just rename it... now internet explorer has to
>> copy the whole damn file... :( which takes long for a 600 meg file.
>
>MS is one of the greatest marketing machines in the world today, but
>not necessary one of the most technically advanced. How they handle
>this is a pain. Especially if the drive that that destination
>folder is on has 600 MB free, but the drive that the temporary
>folder is not does not have 600 MB free.
Sometimes MS actually makes sound reasoning that seems bizarre to thee and me
- remember that they have a different kind of user, many of whom search for
shoes with velcro or zips because they haven't quite figured out the whole
"shoelace tying" concept yet. Such users would scream blue-murder about
"unreliable, unsafe" software, if it ever downloaded a partial file into a
place they expected the whole file to go into. People posting in this
newsfroup probably have a good enough pool of gunge between our ears that we'd
know to go back and get the rest - Microsoft's target user does not.
Alun.
~~~~
[Please don't email posters, if a Usenet response is appropriate.]
--
Texas Imperial Software | Try WFTPD, the Windows FTP Server. Find us at
1602 Harvest Moon Place | http://www.wftpd.com or email al...@texis.com
Cedar Park TX 78613-1419 | VISA/MC accepted. NT-based sites, be sure to
Fax/Voice +1(512)258-9858 | read details of WFTPD Pro for XP/2000/NT.
Skybuck Flying
"Alun Jones" <al...@texis.com> wrote in message
news:D7Wu9.1430$fa2.59...@newssvr30.news.prodigy.com...
That depends on what internet explorer does.
If internet explorer deletes the temporarely file after a cancelled download
then it could just as well save it in the destination folder.
John S. Giltner, Jr.
> In article <Avnu9.5183$6F4.4...@newsread2.prod.itd.earthlink.net>, "John S.
> Giltner, Jr." <gil...@earthlink.net> wrote:
>
>>Skybuck Flying wrote:
>>
>>>That's weird... I would put the temporarely file into the destination folder
>>>and then when it's complete just rename it... now internet explorer has to
>>>copy the whole damn file... :( which takes long for a 600 meg file.
>>
>>MS is one of the greatest marketing machines in the world today, but
>>not necessary one of the most technically advanced. How they handle
>>this is a pain. Especially if the drive that that destination
>>folder is on has 600 MB free, but the drive that the temporary
>>folder is not does not have 600 MB free.
>
>
> Sometimes MS actually makes sound reasoning that seems bizarre to thee and me
> - remember that they have a different kind of user, many of whom search for
> shoes with velcro or zips because they haven't quite figured out the whole
> "shoelace tying" concept yet. Such users would scream blue-murder about
> "unreliable, unsafe" software, if it ever downloaded a partial file into a
> place they expected the whole file to go into. People posting in this
> newsfroup probably have a good enough pool of gunge between our ears that we'd
> know to go back and get the rest - Microsoft's target user does not.
>
> Alun.
> ~~~~
>
> [Please don't email posters, if a Usenet response is appropriate.]
How so true.
John S. Giltner, Jr.
> "Alex Fraser" <m...@privacy.net> wrote in message
> news:apdmev$k72s$1...@ID-149533.news.dfncis.de...
>
>>"John S. Giltner, Jr." <gil...@earthlink.net> wrote in message
>>news:Avnu9.5183$6F4.4...@newsread2.prod.itd.earthlink.net...
>>
>>>Skybuck Flying wrote:
>>>
>>>
>>>>True... the disk activity seems low though.
>>>
>>>Even with low activity, if it takes 10 ms to write and verify the
>>>data that is 10 ms that most likely no IP activity is taking place.
>>> I do not know this for a fact, just a possibility.
>>
>>I doubt it's an issue. But I don't know that for a fact either :).
>
>
> TCP is streaming protocol so the packets would arrive in order that way
> the harddisk would only have to do a track by track seek but only if the
> free space
> is not fragmented... also harddisks have a cache...
TCP is streaming, however you are not guarteeed how much data will
be in each IP packet. You can only receive up to the MTU in a
single IP packet. You do not know how the software is writing.
example, say you have 4K clusters, I would assume (big assumption)
that the FTP software would not write data until it either gets a
EOF or gets a full cluster's worth of data. This means that
assuming 1460 data bytes max per IP packet, that you have to receive
3 packets before you write. This gives you 4K plus some left over.
Harddrives to have cache, but if the writes are done synchrounsly,
there is still wait time to complete the write
>
> Also I tried a 32 KB buffer with WS_FTP...
Could be an issue with WS_FTP. What you would need to do is sniff
the session, you should see some delays on the ACK.
>
> Let's just assume the WS_FTP software has some flaw somewhere which seems
> the most logical assumption :)
Could also be that MS has code that is part of the OS they are
using, thus they have less overhead or are running at a higher
priority. They could also be doing asynch writes to the disk.
Could be a lot of things.
Skybuck Flying
So that don't matter.. that will even increase harddisk speed.
4 kb is about a per buffer to write to disk.
>
> >
> > Also I tried a 32 KB buffer with WS_FTP...
>
> Could be an issue with WS_FTP. What you would need to do is sniff
> the session, you should see some delays on the ACK.
Already did that see other posts... :)
> >
> > Let's just assume the WS_FTP software has some flaw somewhere which
seems
> > the most logical assumption :)
>
> Could also be that MS has code that is part of the OS they are
> using, thus they have less overhead or are running at a higher
> priority. They could also be doing asynch writes to the disk.
> Could be a lot of things.
Neh... windos ftp is 3x times as fast... that can not be explained by some
missing overhead.
