Network Performance 1GB ports
Personne
I have a Cisco 2960 with 2 x 1 GB ports. I have plugged a computer on
each port. and run a netperf (http://www.osnms.com/2008/07/network-
performance-monitor-using-netperf/) between my 2 computers, and I'm
very surprise and disappointed by the performance I get
Info:
Nothing else is connected to the switch (expect the power :-) )
The 2 x 1GB ports are configured to 1000 Full Duplex
The 2 computers NIC are configured to 1000 Full Duplex
The 2 cables I use are Cat5e
The speed I get is only 60Mbytes/sec (or 480 Mbits/s) compare to the
theoretical speed of a 1 GB port 125MBytes/s
Why is this ?
Mark Huizer
> Hi,
>
> I have a Cisco 2960 with 2 x 1 GB ports. I have plugged a computer on
> each port. and run a netperf (http://www.osnms.com/2008/07/network-
> performance-monitor-using-netperf/) between my 2 computers, and I'm
> very surprise and disappointed by the performance I get
>
> theoretical speed of a 1 GB port 125MBytes/s
>
> Why is this ?
What performance do you get when you connect the two PC's directly with
a cross cable? What network cards do you have? Did you try with jumbo
frames?
The performance limit doesn't have to be in the Cisco, and theoretical
speeds are sometimes just that: theoretical.
I would expect that you need decent PC's and decent network cards to
fill 1Gb. And for real usage (where the PC has to use disk etc) I wonder
if you can get anywhere near that.
Mark
Personne
I did already this test before your post, and you are correct it
doesn't come from the CIsco Switch. With a crosscable (5e) between my
2 computers (not servers) I get the same speed rate (around 50 and 60
MBytes/s)
I know TCP/IP has some overhead but 52% is really a lot.
- Could be a driver issue
- I haven't tried yet Jumbo Frame (I'm not even sure I have the option
in my Windows drivers)
- Netperf doesn't use disk, it is a RAM-RAM test, so no disk overhead.
On Nov 13, 10:04 am, Mark Huizer <xaa
Mark Huizer
>
> I did already this test before your post, and you are correct it
> doesn't come from the CIsco Switch. With a crosscable (5e) between my
> 2 computers (not servers) I get the same speed rate (around 50 and 60
> MBytes/s)
> I know TCP/IP has some overhead but 52% is really a lot.
> - Could be a driver issue
> - I haven't tried yet Jumbo Frame (I'm not even sure I have the option
> in my Windows drivers)
> - Netperf doesn't use disk, it is a RAM-RAM test, so no disk overhead.
>
I know it's a RAM to RAM test, that's why I said that a 'real life case'
probably would be even worse.
TCP/IP doesn't have to be a problem, and you can account for that in
your test. Not sure what you are testing precisely with netperf, but the
parameters you are using can have enough impact on your measurements. I
normally use iperf, I don't know netperf, but it might have the same
options. You'll see different measurements for e.g. udp vs tcp, since
udp is connectionless so no problems with window sizes. If you are using
tcp, play around with the window size/buffer size, and see the
difference.
You might play with smaller/bigger sizes, you'll see a big difference.
And I'm not entirely sure where PC hardware is nowadays. Can't tell you
the max bandwidth of memory, PCI busses, etc. And you will see a big
difference between using an el cheapo NIC and a topline NIC that does
all kinds of offloading, because it will do stuff in hardware that your
CPU doesn't have to worry about anymore.
And to stress that again: don't expect to get anywhere near Gbit speed
in real life usage with standard pc hardware. It's just not realistic.
Mark
Personne
"And to stress that again: don't expect to get anywhere near Gbit
speed in real life usage with standard pc hardware. It's just not
realistic."
I'm not expecting to get a real 1GB connection, but I'm surprise to
get only 50% of the theoretical number on a 1GB connection.
When I use a 100M connection (with cross cable or with a switch) I get
88% of the 100MB bandwidth.
Thanks,
Personne
On Nov 13, 11:53 am, Mark Huizer <xaa
Rob
> Mark,
> "And to stress that again: don't expect to get anywhere near Gbit
> speed in real life usage with standard pc hardware. It's just not
> realistic."
>
> I'm not expecting to get a real 1GB connection, but I'm surprise to
> get only 50% of the theoretical number on a 1GB connection.
> When I use a 100M connection (with cross cable or with a switch) I get
> 88% of the 100MB bandwidth.
That is because the limit is not some percentage of the full bandwidth,
but an absolute number. You are not hitting the limit of bus speed,
cpu speed, memory speed etc. with your 100 Mbit network.
I have done similar tests between Linux systems and got a little higher
rates (600 Mbit if I remember correctly) but also not 1 Gbit.
But I wasn't as surprised as you seem to be.
(I was testing the troughput of a 3com Layer-3 switch when compared to a
cisco router)
Thrill5
should know why servers cost so much, all parts of the architecture are
optimized for throughput.
The reason you only get 88% on a 100MB/s connection is because you can't
actually use 100% of the bandwidth. The counters don't count the ethernet
frame, plus there is a a required inter-frame gap between ethernet frames.
"Personne" <cpdi...@gmail.com> wrote in message
news:c1b85f55-3a0e-4f1a...@e4g2000prn.googlegroups.com...
Peter
> I'm not expecting to get a real 1GB connection, but I'm surprise to
> get only 50% of the theoretical number on a 1GB connection.
> When I use a 100M connection (with cross cable or with a switch) I get
> 88% of the 100MB bandwidth.
If you have a 1Gbit interface and it negotiates a 1 Gbit link speed,
then you really do have 1 Gbit CAPABILITY on that connection.
HOWEVER... I work in Networking with lots of Cisco H/W and 100's of
top end Servers (Dell, IBM, HP, Sun) using many different OS's and ALL
machines have 1GB interfaces, and NONE of them are able to REGULARLY
push more than around 800Mbit through that interface without issues.
In fact most Servers rarely exceed around 700Mbit though a 1Gbit
interface. This is a limitation with the design of the H/W and S/W
being used in the Server, not with the Cisco H/W, as we really do see
aggregate Link speeds of 9.98Mbit throughput on EACH 1Gbit interface
of a MULTIPLE Gbit interfaces between Cisco Switches. The easiest way
to see this is to create Channel groups that consist of Multiple Gbit
interfaces.
I hope this helps............pk.
--
Peter from Auckland.
Tosh
>get only 50% of the theoretical number on a 1GB >connection.
>When I use a 100M connection (with cross cable or >with a switch) I get
>88% of the 100MB bandwidth.
All the times I did the same test I saw one of the server's cpu hitting 100%
usage, that was the limit.
Providing that you have multiple cpus on both servers, maybe if you run
iperf twice you can reach the adapter speed limit with the aggregate
throughput.
Please let me know.
Bye,
Tosh.
6hopsaway
For those of you looking to push the limits of your Ethernet
connections you will need to turn on a few tcp options in windows.
Check out RFC 1323 and Microsoft TcpOpts1323
It is recommended to turn these features on with any link generating
more than 16 mb/s operations. Cisco Routers enable this option by
default. Routers only need tuned outside of the default range when
using jumbo frames across them. Cisco gig links natively support up
to 9k frames where fast Ethernet typically does not. Side note...
Vista/ Windows 7 and 2008 Server all come with RFC1323 enabled. Linux
and Unix have had these options set natively for years. Have fun
pegging your links.
Personne
alexd
tried and tested strategy of:
> All the times I did the same test I saw one of the server's cpu hitting 100%
> usage, that was the limit.
> Providing that you have multiple cpus on both servers, maybe if you run
> iperf twice you can reach the adapter speed limit with the aggregate
> throughput.
Try 'iperf -c ... -P n', where 'n' is the number of threads to run. The man page
does say "The threading implementation is rather heinous", so who knows. A quick
loopback test [ie client and server] on this quad core shows -P 3 to give the
highest 'throughput' [19.2Gbps], which I suppose makes sense.
--
<http://ale.cx/> (AIM:troffasky) (UnSoEs...@ale.cx)
19:09:13 up 31 days, 23:04, 4 users, load average: 0.13, 0.18, 0.22
"Stupid is a condition. Ignorance is a choice" -- Wiley Miller
Tosh
> man page
> does say "The threading implementation is rather heinous", so who knows. A
> quick
> loopback test [ie client and server] on this quad core shows -P 3 to give
> the
> highest 'throughput' [19.2Gbps], which I suppose makes sense.
>
Did you try that in a real envoironment, I mean from host to host?
There is something that I don't understand in that, in all my previous tests
i've seen well clocked cpus jumping at 100% utilization with approx 400mbps
( megabit x sec) of unidirectional iperf traffic, 20 gig is too far from
those results to make sense to me, don't you agree?
Bye,
Tosh.
Jasper Janssen
It says he's using client and server on the same PC, which means that
probably the data isn't leaving the CPU, certainly not RAM, and should be
limited by the throughput of memory bus or better.
In host-to-host tests, it's usually the running of the actual network card
that costs the cpu utilization, not the generating of the data or
(apparently, according to this test) even the running of the IP stack.
Jasper
alexd
tried and tested strategy of:
> There is something that I don't understand in that, in all my previous tests
> i've seen well clocked cpus jumping at 100% utilization with approx 400mbps
> ( megabit x sec) of unidirectional iperf traffic, 20 gig is too far from
> those results to make sense to me, don't you agree?
Depends what is using the CPU. Could be iperf, could be the NIC driver, could be
the IP stack on the OS.
--
<http://ale.cx/> (AIM:troffasky) (UnSoEs...@ale.cx)
21:46:33 up 33 days, 2:05, 4 users, load average: 0.29, 0.23, 0.19
Plant food is a made up drug
Tosh
> could be
> the IP stack on the OS.
>
Iperf is always there, the ip stack should be involved in a loopback
operation, maybe the nic drivers are the cpu eaters?
But thats not the point, I'm curious to know if in a real environment using
more iperf threads can help to saturate the adapter bandwidth.
One day I'll give it a try.
Bye,
Tosh.