Had problems when connected U200 to ixia tester.

[Meng Potter]

Hello, thank you for your wonderful work.

I programed 100G design on U200, and installed the driver successfully.I connected directly two ports on U200 with a cable, tcpreplayed some packets to one port, the other port received packets successfully. So I want to test U200 with ixia tester. But when I tried to connect one port of U200 to one port of ixia tester, the link seemed not established, the link light on U200 is also off. It is the same phenomenon when connecting two ports of U200 to the two ports of ixia tester respectively. So I want to ask do you have any idea about the reason why the link is not established?  I am using 100G QSFP28 DAC cable. Thank you very much.

Here are some pictures from ixia. 

[Alex Forencich]

Are you trying to run as 4x 25G or 1x 100G?  It's possible that the Ixia unit wants 25G links to use RS-FEC, which the open-source MAC used in Corundum by default currently does not support.  You might need to figure out how to turn off RS-FEC on the Ixia unit.  Most Mellanox NICs that I have tried are smart enough to figure that out, but I had to make a manual configuration change on my Arista 7060 switch to get a 25G link working.  Alternatively, you should be able to swap out the open-source MAC for a Xilinx 25G Ethernet subsystem module, which should support RS-FEC.  This may require a paid license from Xilinx. 

Alex Forencich

--
You received this message because you are subscribed to the Google Groups "corundum-nic" group.
To unsubscribe from this group and stop receiving emails from it, send an email to corundum-nic...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/corundum-nic/29b86152-8087-48bf-a872-38fcbb7b4325n%40googlegroups.com.

[Meng Potter]

Sorry to disturb you again.

I am using corundum 100G design for U200, running on 4x25G mode. The design uses Xilinx ultrascale+ 100g ethernet subsystem IP core. I found RS-FEC was enabled by default in corundum, while ixia can not establish link. So I disabled RS-FEC in the IP setting, compiled the project and programed the device, but the link is still down. Is there any other reason why the link can not established? Please don't be hesitate to enlighten me.

[Alex Forencich]

Well, the 100G design using the CMAC core brings up a single 100G link.  I think maybe you have the ixia box configured incorrectly to assume four independent 25G links instead of a single 100G link (that's composed of four lane-bonded 25G links).  If you set the ixia unit for 4x25G, then try it with the 25G variant of corundum. 

Alex Forencich

To view this discussion on the web visit https://groups.google.com/d/msgid/corundum-nic/d239e7b7-2f4e-4bcd-8f4a-2358cdf81730n%40googlegroups.com.

[Meng Potter]

Thanks, Your suggestion is right. I set the ixia unit for 1x100G, the link is established. BTW, where can I find 25G variant of corundum for U200? I can only find fpga_100g and fpga_10g design  on corundum github page.

[Alex Forencich]

Ah, I might need to make one.  It is straightforward to convert a 10G design to a 25G design, you can pick another board and diff the two to see what needs to change.  Mainly it's change the GTY line rate from 10.3125 to 25.78125, disable PTP (it currently doesn't close timing at 25G), and make a couple of changes to the PHY module configuration, namely enabling an additional pipeline register. 

Eventually, the 25G designs will be runtime switchable between 10G and 25G, but I have some additional work to do to make this work correctly and reliably, so I'm holding off making 25G designs for all boards until I have that sorted out so I have less not-quite-duplicate code to maintain. 

Alex Forencich

To view this discussion on the web visit https://groups.google.com/d/msgid/corundum-nic/9d600021-d405-43e7-89e9-fe9e985de3a6n%40googlegroups.com.