tg3 v3.123 in 100Mbps Full-Duplex mode with autoneg off

[Marcin Miotk]

Hi all,

While upgrading some gentoo servers from 3.0.9-gentoo to 3.4.11-gentoo
on HP Proliant DL320p machines I spotted an issue with tg3 driver.
I had port on a switch set to 100FD mode (autoneg off) and NIC set to
'speed 100 duplex full autoneg off' during bootup with ethtool - it's
been working this way for a very long time with no issues, however
after upgrading kernel I noticed that there is no connectivity at all
on the interface - the interface is up, configured (manually), but I'm
not able to ping any host - ethtool result below:


Settings for eth1:
Supported ports: [ TP ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
Supported pause frame use: No
Supports auto-negotiation: Yes
Advertised link modes: Not reported
Advertised pause frame use: Symmetric
Advertised auto-negotiation: No
Speed: Unknown!
Duplex: Unknown! (255)
Port: Twisted Pair
PHYAD: 1
Transceiver: internal
Auto-negotiation: off
MDI-X: Unknown
Supports Wake-on: g
Wake-on: g
Current message level: 0x000000ff (255)
drv probe link timer ifdown ifup rx_err tx_err
Link detected: no


The NIC is working properly when I set the switchport to Autoneg +
100FD advertised and NIC is set to 'autoneg on' - on that config both
NIC and switch shows link as 100FD and there is no issues at all.
I compiled few kernel versions (also from vanilla sources) and the
source of the issue seems to be tg3 driver v3.123 - kernels with tg3
v3.122 works properly.
While browsing the tg3 changelog I noticed that there were some
changes made to autoneg in v3.123 + some code has been removed from
it.
Is this a known issue?

Regards,
Marcin Miotk
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majo...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

[Marcin Miotk]

Hi,

any conclusions re this?

[Michael Chan]

Please tell us what tg3 device you're using. You can provide lspci
output or tg3 probing dmesg output. Thanks.

On Wed, 2013-01-09 at 11:20 +0800, 王金浦 wrote:
> For this kind of driver related question, I suggest you send mail to
> tg3 driver developers, who I already cced.
>
>
> I think they should know what's going on, right?
>
>
> Jack
>
>
> 2013/1/8 Marcin Miotk <marcin...@gmail.com>

[Marcin Miotk]

lspci output:

00:00.0 Host bridge: Intel Corporation 3200/3210 Chipset DRAM
Controller (rev 01)
00:01.0 PCI bridge: Intel Corporation 3200/3210 Chipset Host-Primary
PCI Express Bridge (rev 01)
00:06.0 PCI bridge: Intel Corporation 3210 Chipset Host-Secondary PCI
Express Bridge (rev 01)
00:1c.0 PCI bridge: Intel Corporation 82801I (ICH9 Family) PCI Express
Port 1 (rev 02)
00:1c.2 PCI bridge: Intel Corporation 82801I (ICH9 Family) PCI Express
Port 3 (rev 02)
00:1c.3 PCI bridge: Intel Corporation 82801I (ICH9 Family) PCI Express
Port 4 (rev 02)
00:1c.4 PCI bridge: Intel Corporation 82801I (ICH9 Family) PCI Express
Port 5 (rev 02)
00:1c.5 PCI bridge: Intel Corporation 82801I (ICH9 Family) PCI Express
Port 6 (rev 02)
00:1d.0 USB controller: Intel Corporation 82801I (ICH9 Family) USB
UHCI Controller #1 (rev 02)
00:1d.1 USB controller: Intel Corporation 82801I (ICH9 Family) USB
UHCI Controller #2 (rev 02)
00:1d.2 USB controller: Intel Corporation 82801I (ICH9 Family) USB
UHCI Controller #3 (rev 02)
00:1d.3 USB controller: Intel Corporation 82801I (ICH9 Family) USB
UHCI Controller #6 (rev 02)
00:1d.7 USB controller: Intel Corporation 82801I (ICH9 Family) USB2
EHCI Controller #1 (rev 02)
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 92)
00:1f.0 ISA bridge: Intel Corporation 82801IR (ICH9R) LPC Interface
Controller (rev 02)
00:1f.2 IDE interface: Intel Corporation 82801IR/IO/IH (ICH9R/DO/DH) 4
port SATA Controller [IDE mode] (rev 02)
00:1f.5 IDE interface: Intel Corporation 82801I (ICH9 Family) 2 port
SATA Controller [IDE mode] (rev 02)
01:02.0 VGA compatible controller: Advanced Micro Devices [AMD] nee
ATI ES1000 (rev 02)
01:04.0 System peripheral: Compaq Computer Corporation Integrated
Lights Out Controller (rev 03)
01:04.2 System peripheral: Compaq Computer Corporation Integrated
Lights Out Processor (rev 03)
01:04.4 USB controller: Hewlett-Packard Company Integrated Lights-Out
Standard Virtual USB Controller
01:04.6 IPMI SMIC interface: Hewlett-Packard Company Integrated
Lights-Out Standard KCS Interface
02:00.0 PCI bridge: Broadcom EPB PCI-Express to PCI-X Bridge (rev b5)
03:04.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5715
Gigabit Ethernet (rev a3)
03:04.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5715
Gigabit Ethernet (rev a3)
15:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection

Regards,
Marcin

[Michael Chan]

On Wed, 2013-01-09 at 08:03 +0100, Marcin Miotk wrote:
> 03:04.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5715
> Gigabit Ethernet (rev a3)
> 03:04.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5715
> Gigabit Ethernet (rev a3)

I tested on 3.7 kernel and it seemed to work. Bear in mind that you'll
lose auto MDI-X when both sides don't autoneg. Please provide the
following:

1. ethtool eth0 with a working kernel. I'd like to see the MDI-X
setting when it is working.

2. mii-tool -vvv eth0 with the non-working kernel.

Thanks.

[Marcin Miotk]

Hi Michael,

thx for your reply, here is the info you requested:

> 1. ethtool eth0 with a working kernel. I'd like to see the MDI-X
> setting when it is working.

root@XXXXXX ~ # cat ethtool_working_3.0.9

Settings for eth1:
Supported ports: [ TP ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
Supported pause frame use: No
Supports auto-negotiation: Yes
Advertised link modes: Not reported

Advertised pause frame use: No
Advertised auto-negotiation: No
Speed: 100Mb/s
Duplex: Full

Port: Twisted Pair
PHYAD: 1
Transceiver: internal
Auto-negotiation: off
MDI-X: Unknown
Supports Wake-on: g
Wake-on: g
Current message level: 0x000000ff (255)
drv probe link timer ifdown ifup rx_err tx_err

Link detected: yes

> 2. mii-tool -vvv eth0 with the non-working kernel.

root@XXXXXX ~ # cat miitool_non_working_3.4.11
Using SIOCGMIIPHY=0x8947
eth1: 100 Mbit, full duplex, no link
registers for MII PHY 1:
2100 7949 0020 6340 0041 0000 0004 2001
0000 0000 0000 0000 0000 0000 0000 3000
0001 2000 0000 0000 0000 0000 0000 4000
7277 1000 0000 ffff 2801 0000 0000 0000
product info: vendor 00:08:18, model 52 rev 0
basic mode: 100 Mbit, full duplex
basic status: no link
capabilities: 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD
advertising: 10baseT-FD


What looks suspicious here is the last line of mii-tool output:
'advertising: 10baseT-FD', while only 100baseTx-FD is set on a switch.
'mii-tool -vvv eth1' on working 3.0.9 gives output like below:

root@XXX ~ # cat miitool_working_3.0.9
Using SIOCGMIIPHY=0x8947
eth1: 100 Mbit, full duplex, link ok
registers for MII PHY 1:
2100 794d 0020 6340 0501 0000 0004 2001
0000 0000 0000 0000 0000 0000 0000 3000
0001 0301 0000 0000 0000 0000 0000 4000
7277 0504 0000 ffff 2801 0000 0000 0000
product info: vendor 00:08:18, model 52 rev 0
basic mode: 100 Mbit, full duplex
basic status: link ok
capabilities: 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD
advertising: 100baseTx-FD flow-control

Any ideas?

Regards,
Marcin

[Marcin Miotk]

Re kernel 3.7 - it seems that this kernel have tg3 v3.125, so perhaps
something has been changed here since v3.123, I haven't tried that
kernel here though.

[Michael Chan]

On Thu, 2013-01-17 at 17:59 +0100, Marcin Miotk wrote:
> root@XXXXXX ~ # cat miitool_non_working_3.4.11
> Using SIOCGMIIPHY=0x8947
> eth1: 100 Mbit, full duplex, no link
> registers for MII PHY 1:
> 2100 7949 0020 6340 0041 0000 0004 2001
> 0000 0000 0000 0000 0000 0000 0000 3000
> 0001 2000 0000 0000 0000 0000 0000 4000
> 7277 1000 0000 ffff 2801 0000 0000 0000
> product info: vendor 00:08:18, model 52 rev 0
> basic mode: 100 Mbit, full duplex
> basic status: no link
> capabilities: 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD
> advertising: 10baseT-FD

Autoneg is disabled, so the advertising register that is advertising
only 10Mbps Full Duplex should not matter. What matters is that
register 0 has set it to 100Mbps Full Duplex.

>
>
> What looks suspicious here is the last line of mii-tool output:
> 'advertising: 10baseT-FD', while only 100baseTx-FD is set on a switch.
> 'mii-tool -vvv eth1' on working 3.0.9 gives output like below:
>
> root@XXX ~ # cat miitool_working_3.0.9
> Using SIOCGMIIPHY=0x8947
> eth1: 100 Mbit, full duplex, link ok
> registers for MII PHY 1:
> 2100 794d 0020 6340 0501 0000 0004 2001
> 0000 0000 0000 0000 0000 0000 0000 3000
> 0001 0301 0000 0000 0000 0000 0000 4000
> 7277 0504 0000 ffff 2801 0000 0000 0000
> product info: vendor 00:08:18, model 52 rev 0
> basic mode: 100 Mbit, full duplex
> basic status: link ok
> capabilities: 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD
> advertising: 100baseTx-FD flow-control
>

In the working case, register 0 where it matters is the same (autoneg
disabled and 100Mbps Full Duplex). Register 4 happens to advertise the
same 100 Mbps Full duplex forced speed, and apparently your link partner
cares about that, which is very strange because autoneg is disabled.

There apparently has been a change in behavior in the tg3 code, but
again it should not matter. The 3.7 kernel that I tested on behaved the
same as your non-working case but it worked for me. I guess I can try
to restore the original behavior. I'll need time to look through all
the changes.

[Marcin Miotk]

> There apparently has been a change in behavior in the tg3 code, but
> again it should not matter. The 3.7 kernel that I tested on behaved the
> same as your non-working case but it worked for me. I guess I can try
> to restore the original behavior. I'll need time to look through all
> the changes.

Thanks a lot for that, I will look forward for the results of your
investigation.

Regards,
Marcin

[Marcin Miotk]

Hi Michael,

did you have time to look into it?