Using ethtool to check for dropped packets
lance...@yahoo.com
I'm trying to use ethtool to check for dropped packets by my Myricom
card.
From ethtool -S, I see quite a few statistics that appear to be
related to dropped packets, i.e. dropped_link_overflow
dropped_link_error_or_filtered
dropped_pause
dropped_bad_phy
dropped_bad_crc32
dropped_unicast_filtered
dropped_multicast_filtered
dropped_runt
dropped_overrun
dropped_no_small_buffer
dropped_no_big_buffer
Is there somewhere where I can find explanations of what each of these
statistics mean?
If I do happen to find dropped packets, what are some of the things I
can do to improve the situation? I know of changing network buffer
sizes, i.e. net.core.rmem_max, net.core.wmem_max, net.ipv4.tcp_rmem,
net.ipv4.tcp_wmem and net.core.netdev_max_backlog. Are there any other
things I can try?
Thank you.
Regards,
Rayne
Rick Jones
> I'm trying to use ethtool to check for dropped packets by my Myricom
> card.
> From ethtool -S, I see quite a few statistics that appear to be
> related to dropped packets, i.e. dropped_link_overflow
> dropped_link_error_or_filtered
> dropped_pause
> dropped_bad_phy
> dropped_bad_crc32
> dropped_unicast_filtered
> dropped_multicast_filtered
> dropped_runt
> dropped_overrun
> dropped_no_small_buffer
> dropped_no_big_buffer
> Is there somewhere where I can find explanations of what each of
> these statistics mean?
Silly question, but have you tried a web search and/or poked-around on
www.myricom.com?
> If I do happen to find dropped packets, what are some of the things
> I can do to improve the situation? I know of changing network buffer
> sizes, i.e. net.core.rmem_max, net.core.wmem_max, net.ipv4.tcp_rmem,
> net.ipv4.tcp_wmem and net.core.netdev_max_backlog. Are there any
> other things I can try?
Those are for things several layers up from the driver and will
(should) have no effect on link-level drops, with the only possible
exception being their ability to prevent the system from sending fast
enough to cause drops on outbound, or, if applied to all the systems
sending data to yours, keeping them from causing inbound drops at the
driver/NIC level by keeping the performance low.
rick jones
--
The computing industry isn't as much a game of "Follow The Leader" as
it is one of "Ring Around the Rosy" or perhaps "Duck Duck Goose."
- Rick Jones
these opinions are mine, all mine; HP might not want them anyway... :)
feel free to post, OR email to rick.jones2 in hp.com but NOT BOTH...
lance...@yahoo.com
>
> Silly question, but have you tried a web search and/or poked-around onwww.myricom.com?
>
pretty good explanation of the statistics.
I have another question regarding packet drops.
I am running a test to determine when packet drops occur. I'm using a
Spirent TestCenter through a switch (necessary to aggregate Ethernet
traffic from 5 ports to one optical link) to a server using a Myricom
card.
While running my test, if the input rate is below a certain value,
ethtool does not report any drop (except dropped_multicast_filtered
which is incrementing at a very slow rate). However, tcpdump reports X
number of packets "dropped by kernel". Then if I increase the input
rate, ethtool reports drops but "ifconfig eth2" does not. In fact,
ifconfig doesn't seem to report any packet drops at all. Do they all
measure packet drops at different "levels", i.e. ethtool at the NIC
level, tcpdump at the kernel level etc?
And am I right to say that in the journey of an incoming packet, the
NIC level is the "so-called" first level, then the kernel, then the
user application? So any packet drop is likely to happen first at the
NIC, then the kernel, then the user application? So if there is no
packet drop at the NIC, but packet drop at the kernel, then the
bottleneck is not at the NIC?
Thank you.
Regards,
Rayne
Rick Jones
> I did google for it, but couldn't find much. But myri.com does have a
> pretty good explanation of the statistics.
Cool - can you post the URL?-)
> I have another question regarding packet drops.
> I am running a test to determine when packet drops occur. I'm using
> a Spirent TestCenter through a switch (necessary to aggregate
> Ethernet traffic from 5 ports to one optical link) to a server using
> a Myricom card.
So you have 5 1GbE's feeding into one 10GbE right?
> While running my test, if the input rate is below a certain value,
> ethtool does not report any drop (except dropped_multicast_filtered
> which is incrementing at a very slow rate). However, tcpdump reports
> X number of packets "dropped by kernel".
That would be by the kernel promiscuous mode stuff - it is an
indication of how poorly your tcpdump session is doing keeping-up with
the packet capture. If you aren't already, I would suggest writing
the data to a binary file with the -w option to tcptump and doing the
pretty printing in post-processing.
> Then if I increase the input rate, ethtool reports drops but
> "ifconfig eth2" does not. In fact, ifconfig doesn't seem to report
> any packet drops at all. Do they all measure packet drops at
> different "levels", i.e. ethtool at the NIC level, tcpdump at the
> kernel level etc?
Certainly what you get out of tcpdump and what you get out of ethtool
are different levels/layers. Drifting, speaking of the "nine-layer model"
http://www.isc.org/store/logoware-clothing/isc-9-layer-osi-model-cotton-t-shirt
> And am I right to say that in the journey of an incoming packet, the
> NIC level is the "so-called" first level, then the kernel, then the
> user application?
Yes.
> So any packet drop is likely to happen first at the NIC, then the
> kernel, then the user application?
Not necessarily. For example, for an application using a UDP socket,
it could be that the SO_RCVBUF fills before packets start getting
dropped at the NIC - interrupts from the NIC will usually trump the
application and so get more time. I would consider a drop from a full
SO_RCVBUF as being a drop at/by the application, even though the
SO_RCVBUF is in the kernel. Others might consider that a drop in the
kernel. It would appear in netstat -s output.
> So if there is no packet drop at the NIC, but packet drop at the
> kernel, then the bottleneck is not at the NIC?
Correct.
happy benchmarking,
rick jones
--
denial, anger, bargaining, depression, acceptance, rebirth...
where do you want to be today?
lance...@yahoo.com
> lancer6...@yahoo.com <lancer6...@yahoo.com> wrote:
> > I did google for it, but couldn't find much. But myri.com does have a
> > pretty good explanation of the statistics.
>
> Cool - can you post the URL?-)
Sure.
http://www.myri.com/serve/cache/531.html
> > I have another question regarding packet drops.
> > I am running a test to determine when packet drops occur. I'm using
> > a Spirent TestCenter through a switch (necessary to aggregate
> > Ethernet traffic from 5 ports to one optical link) to a server using
> > a Myricom card.
>
> So you have 5 1GbE's feeding into one 10GbE right?
Yes
> > Then if I increase the input rate, ethtool reports drops but
> > "ifconfig eth2" does not. In fact, ifconfig doesn't seem to report
> > any packet drops at all. Do they all measure packet drops at
> > different "levels", i.e. ethtool at the NIC level, tcpdump at the
> > kernel level etc?
>
> Certainly what you get out of tcpdump and what you get out of ethtool
> are different levels/layers.
So ethtool measures at the NIC level, and tcpdump measures at the
kernel level? tshark should be the same as tcpdump, at the kernel
level, right?
What about ifconfig? The "dropped packets" counter never seem to be
more than 0.
Thanks for your explanations!
Regards,
Rayne
Rick Jones
> On May 21, 2:01 am, Rick Jones <rick.jon...@hp.com> wrote:
> > lancer6...@yahoo.com <lancer6...@yahoo.com> wrote:
> > > I did google for it, but couldn't find much. But myri.com does have a
> > > pretty good explanation of the statistics.
> >
> > Cool - can you post the URL?-)
> Sure.
> http://www.myri.com/serve/cache/531.html
Thanks - a web search didn't seem to find that page, just a PDF file
about the software interface to their card.
> So ethtool measures at the NIC level, and tcpdump measures at the
> kernel level? tshark should be the same as tcpdump, at the kernel
> level, right?
Both are using libpcap (iirc). And while the drops reported by
tcpdump (which gets them from libpcap) did happen in the kernel, they
are only in the packet sniffing path, not elsewhere, so I'm reluctant
to call them "kernel level" drops. To me that implies a drop along
some regular path.
> What about ifconfig? The "dropped packets" counter never seem to be
> more than 0.
I've never had to look it up, so someone else will have to comment.
rick jones
--
the road to hell is paved with business decisions...