TX driver issue detected, PF reset issued

[Tad Wang]

Hi,

My new servers always get Nic Down problem, and I think it's driver's problem. Can CoreOS Upgrade the firmware OR let me know how to upgrade by myself and fix it.

Only the new model servers have such message: 

dmesg:

[26531.332930] i40e 0000:03:00.0 enp3s0f0: NIC Link is Down

[26597.002838] i40e 0000:03:00.0 enp3s0f0: NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None

[26600.376443] i40e 0000:03:00.0 enp3s0f0: NIC Link is Down

[26600.638532] i40e 0000:03:00.0 enp3s0f0: NIC Link is Up 10 Gbps Full Duplex, Flow Control: None

[60150.554603] i40e 0000:03:00.0: TX driver issue detected, PF reset issued

[60151.542604] i40e 00.0 device is FCoE capable

[60151.571723] i40e 0000:03:00.0: i40e_ptp_init: added PHC on enp3s0f0

[60151.599953] i40e 0000:03:00.0: enabling bridge mode: VEPA

[60151.706707] i40e 0000:03:00.0 enp3s0f0: NIC Link is Up 10 Gbps Full Duplex, Flow Control: None

[60151.719669] i40e 0000:03:00.0 enp3s0f0: NIC Link is Down

[60152.082937] i40e 0000:03:00.0 enp3s0f0: NIC Link is Up 10 Gbps Full Duplex, Flow Control: None

[60152.299308] i40e 0000:03:00.0: TX driver issue detected, PF reset issued

[60152.541311] i40e 00.0 device is FCoE capable

[60152.570352] i40e 0000:03:00.0: i40e_ptp_init: added PHC on enp3s0f0

[60152.598533] i40e 0000:03:00.0: enabling bridge mode: VEPA

[60152.704709] i40e 0000:03:00.0 enp3s0f0: NIC Link is Up 10 Gbps Full Duplex, Flow Control: None

[62333.476868] i40e 0000:03:00.0: TX driver issue detected, PF reset issued

[62333.918947] i40e 00.0 device is FCoE capable

[62333.948119] i40e 0000:03:00.0: i40e_ptp_init: added PHC on enp3s0f0

[62333.976249] i40e 0000:03:00.0: enabling bridge mode: VEPA

[62334.082936] i40e 0000:03:00.0 enp3s0f0: NIC Link is Up 10 Gbps Full Duplex, Flow Control: None

[62334.095016] i40e 0000:03:00.0 enp3s0f0: NIC Link is Down

[62334.465269] i40e 0000:03:00.0 enp3s0f0: NIC Link is Up 10 Gbps Full Duplex, Flow Control: None

[62334.578785] i40e 0000:03:00.0: TX driver issue detected, PF reset issued

[62334.916101] i40e 00.0 device is FCoE capable

[62334.945156] i40e 0000:03:00.0: i40e_ptp_init: added PHC on enp3s0f0

[62334.973631] i40e 0000:03:00.0: enabling bridge mode: VEPA

[62335.080514] i40e 0000:03:00.0 enp3s0f0: NIC Link is Up 10 Gbps Full Duplex, Flow Control: None

lspci |grep 10GbE

03:00.0 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE backplane (rev 01)

03:00.1 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE backplane (rev 01)

cat /etc/os-release

NAME=CoreOS

ID=coreos

VERSION=899.17.0

VERSION_ID=899.17.0

BUILD_ID=2016-05-03-2151

PRETTY_NAME="CoreOS 899.17.0"

ANSI_COLOR="1;32"

HOME_URL="https://coreos.com/"

BUG_REPORT_URL="https://github.com/coreos/bugs/issues"

Not only the latest stable version. I have 835.10.0, 835.11.0, 835.13.0, 991.1.0 (Coeur Rouge) and etc.

Somebody said can disable TSO, I tried but no effect on it.

Any reply for me is helpful. 

Thanks!

Tad

[Tad Wang]

ethtool -i enp3s0f0

driver: i40e

version: 1.3.4-k

firmware-version: f4.22.27454 a1.1 n4.26 e14a6

bus-info: 0000:03:00.0

supports-statistics: yes

supports-test: yes

supports-eeprom-access: yes

supports-register-dump: yes

supports-priv-flags: yes

在 2016年5月18日星期三 UTC+8下午1:00:04，Tad Wang写道：

[Ivan]

Hi Tad,

There are two options you can attempt:

1. Update firmware for your Intel X710 Ethernet Controller [1]
   
2. Update CoreOS. We recommend using the latest CoreOS stable release 1010.5.0 [2] with kernel 4.5.0 that has a lot of fixes for the driver i40e [3].
   

Links:
1. https://downloadcenter.intel.com/download/25791/NVM-Update-Utility-for-Intel-Ethernet-Adapters-Linux-?product=82947
2. https://coreos.com/releases/#1010.5.0
3. https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/log/?id=refs%2Ftags%2Fv4.5.6&qt=grep&q=i40e

All the best,
Ivan

[Tad Wang]

Hi, Ivan

Thanks for your reply!

I have upgraded the latest version: 1010.5.0, but it seems no difference and show the same dmesg

[ 3944.590298] i40e 0000:03:00.0: TX driver issue detected, PF reset issued

[ 3945.302740] i40e 00.0 device is FCoE capable

[ 3945.465225] i40e 0000:03:00.0: TX driver issue detected, PF reset issued

[ 3945.475220] i40e 0000:03:00.0 enp3s0f0: NIC Link is Down

[ 3945.794254] i40e 00.0 device is FCoE capable

[ 3945.955124] i40e 0000:03:00.0 enp3s0f0: NIC Link is Up 10 Gbps Full Duplex, Flow Control: None

[ 4840.300994] i40e 0000:03:00.0: TX driver issue detected, PF reset issued

[ 4840.455735] i40e 00.0 device is FCoE capable

[ 4840.623751] i40e 0000:03:00.0 enp3s0f0: NIC Link is Down

[ 4840.632737] i40e 0000:03:00.0: TX driver issue detected, PF reset issued

[ 4841.039760] i40e 00.0 device is FCoE capable

[ 4841.203238] i40e 0000:03:00.0 enp3s0f0: NIC Link is Up 10 Gbps Full Duplex, Flow Control: None

Thanks.

Tad

在 2016年6月2日星期四 UTC+8上午5:56:34，Ivan写道：