VxWorks 6.6 - Ethernet interface driver support
ulrik
I am currently working on the porting of a board system from VxWorks
5.5.1 to VxWorks 6.6. One of the issues is to get DSS Ethernet
interface PMC cards working in conjunction with system/node boards
from CCT, Concurrent Technologies.
So far I have been successful only with the electrical DSS PMC
Ethernet interface cards, deploying the "Intel PRO/1000 VxBus Enhanced
Network Driver" in conjunction with the "Generic PHY driver", both
standard VxWorks 6.6 components.
However, I have the impression that it should be possible to
successfully drive also the optical DSS PMC Ethernet interface cards
with a suitable set of correct configured VxWorks 6.6 standard
components, not having to integrate any driver from DSS.
In the VxWorks 6.6 Workbench help, searching for the keyword TBI, the
following is said:
Generic TBI Driver Support
Many fiber optic controllers use a ten bit interface (TBI) as their
MAC/PHY media connection. The TBI management interface is similar to
that of an ordinary 10/100 copper PHY. However, a TBI PHY supports
only 1000 Mb/s. A MAC driver can be written such that the routines in
the MII bus library (see the reference documentation for miiBus) can
discover the TBI management interface and manage the link just like
that of an ordinary PHY. Most devices that implement TBI use the same
management interface, therefore a genericTbiPhy driver is also
provided to handle these cases.
Unlike the genericPhy driver, the genericTbiPhy driver only attaches
to a MAC driver that reports the correct vendor and device ID values.
The genericTbiPhy.h header defines two values, TBI_ID1 and TBI_ID2. If
a MAC driver's {miiBusRead}( ) method returns these values when a
caller reads the ID registers, the genericTbiPhy driver successfully
attaches to the TBI PHY instance.
The genericTbiPhy driver is included using the INCLUDE_GENERICTBIPHY
component.
I activated the "Generic TBI driver" and/or the "Intel PRO/1000 TBI/
SERDES pseudo driver" components, but the resulting VxWorks image
doesn’t boot with any combination of these components.
Both the electrical and optical PMC card versions from DSS are
equipped with the Intel device 82546EB (PCI device id 0x1010).
However, a distinction between the two card versions can be made by
checking the content of the serial EEPROM (external to the 82546
device), which differs.
Hopefully anyone have an idea what can be wrong!
Many thanks in advance and best regards
Ulrik
nois...@gmail.com
The GEI VxBus driver in 6.6 uses a special driver for fiber optic
support. In order to use both copper and fiber optic PRO/1000 cards in
the same image, you need the following components:
INCLUDE_GEI825XX_VXB_END
INCLUDE_MII_BUS
INCLUDE_GENERICPHY
INCLUDE_GEITBIPHY
You do *NOT* need INCLUDE_GENERICTBIPHY.
The GEITBIPHY driver is a pseudo driver designed to attach to the GEI
driver specifically. The PRO/1000 doesn't provide a generic set of TBI
management registers, so using the generic TBI PHY driver would be
cumbersome. Instead the GEI driver exports access to its custom
registers, and there's a custom geiTbiPhy driver designed exclusively
for use with it. The geiTbiPhy driver isn't generally included by
default since most people tend to use copper PRO/1000 cards (scaling
out the fiber optic support saves a little bit of code space).
The GEI driver detects the presence of a fiber optic NIC by checking
to see if the TBIMODE bit is set in the status register. The state of
this bit is controlled by the contents of the EEPROM. For copper
cards, the bit is always clear.
Unfortunately, you have not described your actual problem in detail.
All you said was "the resulting VxWorks image doesn’t boot with any
combination of these components." What do you mean by "doesn't boot?"
Be more specific. Show us an example. You could mean "once I download
the image into the target, it just freezes and I never see any output"
or you could mean "I'm expecting some network connectivity as part of
the boot process, but I'm not getting any." But we can't know for sure
what you mean until you tell us.
If the image does start up, you really ought to do the following:
- Make sure you have INCLUDE_VXBUS_SHOW included in your configuration
- At the shell prompt, do:
-> vxBusShow (1)
- Show us the output. (*ALL* of the output.)
You should see a geiTbi instance for each fiber optic network port. If
you don't, your configuration is wrong.
You should also look at the link indicator LEDs on the ports
themselves. Assuming the ports have live fiber connections, you should
see the link LED light up, and I think it should flash briefly (this
indicates the transceiver was reset during initialization).
I have personally tested the GEI driver with a couple of different
fiber optic PRO/1000 cards, but sadly there are plenty of different
ones out there that I can't get my hands on for testing. I have
successfully used an 82545EM fiber optic card, which is similar enough
to the 82546 in my opinion that it ought to work.
-Bill
ulrik
Hello Bill
Very many thanks for your detailed and swift response!
The reboot problem turned out to be caused by some log statement that
I had added in the file "/cc/vobs/buildtools/WindRiver/vxworks-6.6/
target/src/hwif/mii/geiTbiPhy.c" (file now reverted to the original
version). I would like to excuse for not telling in the first place
that I had done some code change and through that, as it turned out,
actually caused the reboot problem myself.
The situation that I am now facing is that the system boots and the
optical interfaces are up but not running (checked with command
"ifconfig"), which means that I can't send or receive any traffic over
these interfaces (e.g. by pinging these interfaces).
If you (or someone else) have any further ideas, I would be most
thankful!
Best regards
Ulrik
nois...@gmail.com
In my previous message, I said for you to add the INCLUDE_VXBUS_SHOW
component (along with the shell, which you already have) and do
"vxBusShow(1)" so that we could see the target setup. This is
important, because it would confirm that the right drivers are
actually present, and that the GEI driver has actually been able to
figure out that the two fiber ports are actually fiber and attached
the geiTbiPhy driver correctly (instead of genericPhy). Please, do
this like I asked, and show us the results.
Similarly, you should show us the output of the "ifconfig" command so
that we can see for ourselves what it reports. In particular, it would
show just how many interfaces are in the system.
Assuming you do have your image configured correctly, I think the
RUNNING flag may only show up in ifconfig once a good link is
established on the interface. The driver usually makes a callback to
the TCP/IP stack when there's a link up or link down event. If the
flag isn't set on your interface, then maybe there's no link
established. (You can confirm this by unplugging and replugging the
cable on one of the copper ports and using ifconfig to examine the
RUNNING flag in each case.)
You said "the interfaces are up." Does this mean you actually looked
at the link LEDs and saw that they were lit? Also, what kind of device
do you have the ports cabled to? Is it a switch of some kind? Are you
sure the fiber connection is good? Have you tried unplugging and
replugging the fiber? With my 82545EM fiber card, there is only one
LED and it stays dark until the driver loads. Then it flashes to
indicate traffic activity. (The same is true with my 82572 card, which
is a PCIe fiber optic NIC.)
-Bill
nois...@gmail.com
A followup to my own post:
I managed to find an Intel PRO/1000 MF server adapter, a dual port
fiber optic NIC which uses the 82546EB. I did a quick test with it,
and at least with my setup, it does work with the supplied GEI driver
and the geiTbiPhy driver.
I'm assuming that the cards you have look like this:
http://www.dssnetworks.com/v3/gigabit_pmc_5262.asp
The one I have looks like this:
http://www.intel.com/network/connectivity/products/pro1000mf_dual_server_adapter.htm
These are effectively the same card, except for the form factor.
Note: in your original posting, you mentioned that the PCI ID for your
cards was 0x1010. This should only be true for the copper card. My PRO/
1000 MF has a PCI ID of 0x1012, so I would expect your fiber PMC card
has that ID too.
Also, assuming you have the VxWorks image set up correctly, one other
thing you can do as a simple test is to find a fiber patch cable and
connect the two ports on the card together. The link indicators on
both ports should light up, and the RUNNING flag should appear on both
interfaces when you run ifconfig. If this happens, then the driver
ought to be working correctly.
-Bill
ulrik
>
> These are effectively the same card, except for the form factor.
>
> Note: in your original posting, you mentioned that the PCI ID for your
> cards was 0x1010. This should only be true for the copper card. My PRO/
> 1000 MF has a PCI ID of 0x1012, so I would expect your fiber PMC card
> has that ID too.
>
> Also, assuming you have the VxWorks image set up correctly, one other
> thing you can do as a simple test is to find a fiber patch cable and
> connect the two ports on the card together. The link indicators on
> both ports should light up, and the RUNNING flag should appear on both
> interfaces when you run ifconfig. If this happens, then the driver
> ought to be working correctly.
>
> -Bill
Hello Bill
Many thanks for your earlier postings and excuse for taking so long to
respond!
The “vxBusShow(1)” and “ifconfig” listings can be found below.
When I said that the interfaces are up, I meant that the “ifconfig” is
indicating UP for the optical interfaces “gei3” and “gei4” but not
RUNNING.
I have two “node boards”, PP410 from Concurrent Technologies, which
have there onboard electrical Ethernet interfaces but only two are
actually used, “gei0” and “gei1”.
Each of these node boards have currently a DSS PMC card plugged in the
upper of the two available PMC slots.
The PMC cards have two Ethernet optical interfaces each, controlled by
the Intel device: "0x8086","0x1010","Intel
Corporation","82546EB","Dual Port Gigabit Ethernet Controller
(Copper)". The distinction between copper and fiber is made by
checking the TBIMODE bit, which is set in the fiber case, in the
STATUS register.
The two PMC cards are directly connected with two fiber optic cables
and as it seems they are of type GigMAC PMC Model 5262-LC (PN: 152620-
B2): http://www.dssnetworks.com/v3/gigabit_pmc_5262.asp.
These PMC cards have four LEDs: TX, RX, GIG and FD.
I can be sure that the PMC cards and the fiber optic cables are fine
since I tried them running VxWorks 5.5.1. Under VxWorks 5.5.1 I am
using an END device driver from DSS but now under 6.6 my intention is
to deploy the VxBus (standard) drivers, where the MAC and PHY layers
are separated, which seems to be a requirements for multi processor
deployment.
When comparing the behavior of the DSS cards under VxWorks 5.5.1 and
under VxWorks 6.6, I have noted the following:
VxWorks 5.5.1:
When I boot the boards with the cables plugged, all LEDs showing
activity during the boot process and after the boards have booted, the
GIG and FD LEDs stay lit. When unplugging and re-plugging the cables,
the GIG and FD LEDs switches off and on but the RX and TX LEDs stay
dark.
What I have seen is that the interrupt handling function
“inLiProcessMgmtInts” is called upon plugging/unplugging cables.
However, I am not sure what would be the counter part to this function
under 6.6.
VxWorks 6.6:
When I boot the boards with the cables plugged, the GIG LEDs are
blinking during booting but after the boards have booted all LEDs stay
dark. When unplugging and re-plugging the cables all four LEDs
continue to stay dark.
I can see a steady activity in the function “geiFiberPhyRead” in which
I added some code that logs the content of the STATUS, TXCW and RXCW
registers (for deferred printout).
What I can see is that for both interfaces on both boards, the link is
not up (STATUS register bit 1, LU bit, is zero). However, one board
flags for hardware auto negotiation complete for both interfaces but
the other not (RXCW register bit 32, ANC bit, is 1 respective 0).
geiFiberPhyRead: busID=3 ,pName=gei, unitNumber=3, versionCounter=0
regAddr=1 dataVal=100 GEI_STS=dba0 GEI_TXCW=800001e0 GEI_RXCW=4000000
LEDCTL=7068309
geiFiberPhyRead: busID=3 ,pName=gei, unitNumber=4, versionCounter=0
regAddr=1 dataVal=100 GEI_STS=dba4 GEI_TXCW=800001e0 GEI_RXCW=4000000
LEDCTL=7068309
geiFiberPhyRead: busID=3 ,pName=gei, unitNumber=3, versionCounter=0
regAddr=1 da
taVal=100 GEI_STS=dba0 GEI_TXCW=800001e0 GEI_RXCW=c000000
LEDCTL=7068309
geiFiberPhyRead: busID=3 ,pName=gei, unitNumber=4, versionCounter=0
regAddr=1 da
taVal=100 GEI_STS=dba4 GEI_TXCW=800001e0 GEI_RXCW=c000000
LEDCTL=7068309
I can see that at boot time the ANE bit of the TXCW goes through a 0b
to 1b transition for all interfaces due to the fact that the
“geiTbiPhyModeSet” function is called. However, as already mentioned
auto negotiation is either not triggered correctly or doesn’t finish
successfully.
There is a function called “geiTbiPhyProbe” that returns TRUE if the
vendor id and device id match same preconfigured, currently dummy,
values. I am not sure about the purpose of this function, but anyway
if I change these dummy ids to my actual ones, there is no change/
improvement. I also tried to retrigger the auto negotiation (through
an ANE 0b to 1b bit transition), but without any effect.
If you have some more ideas about possible problem sources to
investiagte on, that would be great.
Many thanks in advance and best regards,
Ulrik
-> vxBusShow(1)
Registered Bus Types:
MII_Bus @ 0x005073cc
PCI_Bus @ 0x005031c8
Local_Bus @ 0x005032a4
Registered Device Drivers:
pentiumPci at 0x00506420 on bus Local_Bus, funcs @ 0x00506400
VxBus version 3
mpApic at 0x00506920 on bus Local_Bus, funcs @ 0x005068a0
VxBus version 3
mc146818Rtc at 0x00507740 on bus Local_Bus, funcs @ 0x00507720
VxBus version 2
loApicTimer at 0x005078c0 on bus Local_Bus, funcs @ 0x005078a0
VxBus version 2
loApicIntr at 0x00506a80 on bus Local_Bus, funcs @ 0x00506a00
VxBus version 3
ioApicIntr at 0x005069c0 on bus Local_Bus, funcs @ 0x00506960
VxBus version 3
iaTimestamp at 0x00507840 on bus Local_Bus, funcs @ 0x00507820
VxBus version 3
ns16550 at 0x005076a0 on bus Local_Bus, funcs @ 0x00507608
VxBus version 3
ns16550 at 0x00507640 on bus PCI_Bus, funcs @ 0x00507608
VxBus version 3
m6845Vga at 0x00506560 on bus Local_Bus, funcs @ 0x005065c8
VxBus version 3
i8042Kbd at 0x005064c0 on bus Local_Bus, funcs @ 0x00506540
VxBus version 3
lxt972Phy at 0x00507420 on bus MII_Bus, funcs @ 0x00507460
VxBus version 3
geiTbiPhy at 0x00507520 on bus MII_Bus, funcs @ 0x00507560
VxBus version 3
genericPhy at 0x005074a0 on bus MII_Bus, funcs @ 0x005074e0
VxBus version 3
miiBus at 0x00507380 on bus PCI_Bus, funcs @ 0x005073e8
VxBus version 3
miiBus at 0x00507340 on bus Local_Bus, funcs @ 0x005073e8
VxBus version 3
gei at 0x00506620 on bus PCI_Bus, funcs @ 0x005066b8
VxBus version 3
plbCtlr at 0x005032c0 on bus Local_Bus, funcs @ 0x005328f8
VxBus version 3
Busses and Devices Present:
Local_Bus @ 0x0350c358 with bridge @ 0x00503300
Device Instances:
mpApic unit 0 on Local_Bus @ 0x0350d318 with busInfo
0x00000000
BAR0 @ 0xffffffff (IO space)
pDrvCtrl @ 0x0350d418
loApicIntr unit 0 on Local_Bus @ 0x0350d618 with busInfo
0x00000000
BAR0 @ 0xfee00000 (IO space)
pDrvCtrl @ 0x0350e418
ioApicIntr unit 0 on Local_Bus @ 0x0350d718 with busInfo
0x00000000
BAR0 @ 0xfec00000 (IO space)
pDrvCtrl @ 0x0350e518
ioApicIntr unit 1 on Local_Bus @ 0x0350d818 with busInfo
0x00000000
BAR0 @ 0xfec10000 (IO space)
pDrvCtrl @ 0x0350e698
ns16550 unit 0 on Local_Bus @ 0x0350d918 with busInfo
0x00000000
BAR0 @ 0x000003f8 (IO space)
pDrvCtrl @ 0x0350da18
ns16550 unit 1 on Local_Bus @ 0x0350db18 with busInfo
0x00000000
BAR0 @ 0x000002f8 (IO space)
pDrvCtrl @ 0x0350dc18
pentiumPci unit 0 on Local_Bus @ 0x0350dd18 with busInfo
0x0350c6d8
pDrvCtrl @ 0x0350de18
i8042Kbd unit 0 on Local_Bus @ 0x03513a18 with busInfo
0x00000000
BAR0 @ 0x00000060 (IO space)
pDrvCtrl @ 0x0b3cc904
m6845Vga unit 0 on Local_Bus @ 0x03513c18 with busInfo
0x00000000
BAR0 @ 0x000003d4 (IO space)
pDrvCtrl @ 0x0b3cc944
loApicTimer unit 0 on Local_Bus @ 0x03513d18 with busInfo
0x00000000
BAR0 @ 0xfee00000 (IO space)
pDrvCtrl @ 0x03513e18
iaTimestamp unit 0 on Local_Bus @ 0x03513f18 with busInfo
0x00000000
pDrvCtrl @ 0x03514018
mc146818Rtc unit 0 on Local_Bus @ 0x03514118 with busInfo
0x00000000
BAR0 @ 0x00000070 (IO space)
pDrvCtrl @ 0x03514218
Orphan Devices:
i8042Mse unit 0 on Local_Bus @ 0x03513b18 with busInfo
0x00000000
BAR0 @ 0x00000060 (IO space)
pDrvCtrl @ 0x00000000
PCI_Bus @ 0x0350c6d8 with bridge @ 0x0350dd18
Device Instances:
gei unit 0 on PCI_Bus @ 0x03511818 with busInfo 0x00000000
BAR0 @ 0xdc000000 (memory mapped)
BAR2 @ 0x00002000 (IO space)
pDrvCtrl @ 0x0b3ce010
gei unit 1 on PCI_Bus @ 0x03511a18 with busInfo 0x00000000
BAR0 @ 0xdc200000 (memory mapped)
BAR2 @ 0x00003000 (IO space)
pDrvCtrl @ 0x0b3d2010
gei unit 2 on PCI_Bus @ 0x03511c18 with busInfo 0x00000000
BAR0 @ 0xdc300000 (memory mapped)
BAR2 @ 0x00004000 (IO space)
pDrvCtrl @ 0x0b3d8010
gei unit 3 on PCI_Bus @ 0x03511e18 with busInfo 0x00000000
BAR0 @ 0xdc400000 (memory mapped)
BAR4 @ 0x00005000 (IO space)
pDrvCtrl @ 0x0b3de010
gei unit 4 on PCI_Bus @ 0x03511f18 with busInfo 0x00000000
BAR0 @ 0xdc420000 (memory mapped)
BAR4 @ 0x00005040 (IO space)
pDrvCtrl @ 0x0b3e4010
miiBus unit 0 on PCI_Bus @ 0x03516318 with busInfo 0x03515518
pDrvCtrl @ 0x03516418
miiBus unit 1 on PCI_Bus @ 0x03516718 with busInfo 0x035155d8
pDrvCtrl @ 0x03516818
miiBus unit 2 on PCI_Bus @ 0x03516b18 with busInfo 0x03515698
pDrvCtrl @ 0x03516c18
miiBus unit 3 on PCI_Bus @ 0x03516f18 with busInfo 0x03515758
pDrvCtrl @ 0x03517018
miiBus unit 4 on PCI_Bus @ 0x03517318 with busInfo 0x03515818
pDrvCtrl @ 0x03517418
Orphan Devices:
(null) unit 0 on PCI_Bus @ 0x0350e018 with busInfo 0x00000000
pDrvCtrl @ 0x00000000
(null) unit 0 on PCI_Bus @ 0x0350e118 with busInfo 0x00000000
BAR0 @ 0xdc501000 (memory mapped)
pDrvCtrl @ 0x00000000
(null) unit 0 on PCI_Bus @ 0x0350e218 with busInfo 0x00000000
BAR2 @ 0x00010100 (memory mapped)
BAR3 @ 0x200000f0 (memory mapped)
BAR4 @ 0x0000fff0 (memory mapped)
BAR5 @ 0x0001fff0 (IO space)
pDrvCtrl @ 0x00000000
(null) unit 0 on PCI_Bus @ 0x03511318 with busInfo 0x00000000
BAR2 @ 0x00020200 (memory mapped)
BAR3 @ 0x200000f0 (memory mapped)
BAR4 @ 0x0000fff0 (memory mapped)
BAR5 @ 0x0001fff0 (IO space)
pDrvCtrl @ 0x00000000
(null) unit 0 on PCI_Bus @ 0x03511418 with busInfo 0x00000000
BAR2 @ 0x00040300 (memory mapped)
BAR3 @ 0x200000f0 (memory mapped)
BAR4 @ 0xdc10dc10 (memory mapped)
BAR5 @ 0x0001fff0 (IO space)
pDrvCtrl @ 0x00000000
(null) unit 0 on PCI_Bus @ 0x03511518 with busInfo 0x00000000
BAR0 @ 0xdc100000 (memory mapped)
BAR2 @ 0x24040400 (IO space)
BAR3 @ 0x24a001f0 (IO space)
BAR4 @ 0x0000fff0 (memory mapped)
BAR5 @ 0x0001fff0 (IO space)
pDrvCtrl @ 0x00000000
(null) unit 0 on PCI_Bus @ 0x03511618 with busInfo 0x00000000
pDrvCtrl @ 0x00000000
(null) unit 0 on PCI_Bus @ 0x03511718 with busInfo 0x00000000
BAR2 @ 0x00050500 (memory mapped)
BAR3 @ 0x00002020 (memory mapped)
BAR4 @ 0xdc00dc00 (memory mapped)
BAR5 @ 0x0001fff0 (IO space)
pDrvCtrl @ 0x00000000
(null) unit 0 on PCI_Bus @ 0x03511918 with busInfo 0x00000000
BAR2 @ 0x00060600 (memory mapped)
BAR3 @ 0x00003030 (memory mapped)
BAR4 @ 0xdc20dc20 (memory mapped)
BAR5 @ 0x0001fff0 (IO space)
pDrvCtrl @ 0x00000000
(null) unit 0 on PCI_Bus @ 0x03511b18 with busInfo 0x00000000
BAR2 @ 0x00070700 (memory mapped)
BAR3 @ 0x00004040 (memory mapped)
BAR4 @ 0xdc30dc30 (memory mapped)
BAR5 @ 0x0001fff0 (IO space)
pDrvCtrl @ 0x00000000
(null) unit 0 on PCI_Bus @ 0x03511d18 with busInfo 0x00000000
BAR2 @ 0x30080800 (memory mapped)
BAR3 @ 0x22a05050 (memory mapped)
BAR4 @ 0xdc40dc40 (memory mapped)
BAR5 @ 0x0001fff0 (IO space)
pDrvCtrl @ 0x00000000
(null) unit 0 on PCI_Bus @ 0x03512018 with busInfo 0x00000000
BAR4 @ 0x00001400 (IO space)
pDrvCtrl @ 0x00000000
(null) unit 0 on PCI_Bus @ 0x03512118 with busInfo 0x00000000
BAR4 @ 0x00001420 (IO space)
pDrvCtrl @ 0x00000000
(null) unit 0 on PCI_Bus @ 0x03512218 with busInfo 0x00000000
BAR0 @ 0xdc702000 (memory mapped)
pDrvCtrl @ 0x00000000
(null) unit 0 on PCI_Bus @ 0x03513318 with busInfo 0x00000000
pDrvCtrl @ 0x00000000
(null) unit 0 on PCI_Bus @ 0x03513418 with busInfo 0x00000000
BAR0 @ 0xdc702400 (memory mapped)
pDrvCtrl @ 0x00000000
(null) unit 0 on PCI_Bus @ 0x03513518 with busInfo 0x00000000
BAR2 @ 0x20090900 (memory mapped)
BAR3 @ 0x228000f0 (memory mapped)
BAR4 @ 0xdbf0d800 (memory mapped)
BAR5 @ 0x0000fff0 (memory mapped)
pDrvCtrl @ 0x00000000
(null) unit 0 on PCI_Bus @ 0x03513618 with busInfo 0x00000000
BAR0 @ 0xd8000000 (memory mapped)
pDrvCtrl @ 0x00000000
(null) unit 0 on PCI_Bus @ 0x03513718 with busInfo 0x00000000
pDrvCtrl @ 0x00000000
(null) unit 0 on PCI_Bus @ 0x03513818 with busInfo 0x00000000
BAR0 @ 0x00000000 (IO space)
BAR1 @ 0x00000000 (IO space)
BAR2 @ 0x00000000 (IO space)
BAR3 @ 0x00000000 (IO space)
BAR4 @ 0x00001460 (IO space)
pDrvCtrl @ 0x00000000
(null) unit 0 on PCI_Bus @ 0x03513918 with busInfo 0x00000000
BAR4 @ 0x00001440 (IO space)
pDrvCtrl @ 0x00000000
MII_Bus @ 0x03515518 with bridge @ 0x03516318
Device Instances:
genericPhy unit 0 on MII_Bus @ 0x03516518 with busInfo
0x00000000
BAR0 @ 0x00000001 (special)
pDrvCtrl @ 0x03516618
Orphan Devices:
MII_Bus @ 0x035155d8 with bridge @ 0x03516718
Device Instances:
genericPhy unit 1 on MII_Bus @ 0x03516918 with busInfo
0x00000000
BAR0 @ 0x00000001 (special)
pDrvCtrl @ 0x03516a18
Orphan Devices:
MII_Bus @ 0x03515698 with bridge @ 0x03516b18
Device Instances:
genericPhy unit 2 on MII_Bus @ 0x03516d18 with busInfo
0x00000000
BAR0 @ 0x00000001 (special)
pDrvCtrl @ 0x03516e18
Orphan Devices:
MII_Bus @ 0x03515758 with bridge @ 0x03516f18
Device Instances:
geiTbiPhy unit 0 on MII_Bus @ 0x03517118 with busInfo
0x00000000
BAR0 @ 0x00000001 (special)
pDrvCtrl @ 0x03517218
Orphan Devices:
MII_Bus @ 0x03515818 with bridge @ 0x03517318
Device Instances:
geiTbiPhy unit 1 on MII_Bus @ 0x03517518 with busInfo
0x00000000
BAR0 @ 0x00000001 (special)
pDrvCtrl @ 0x03517618
Orphan Devices:
value = 0 = 0x0
-> ifconfig
lo0 Link type:Local loopback Queue:none
inet 127.0.0.1 mask 255.255.255.255
inet 224.0.0.1 mask 240.0.0.0
UP RUNNING LOOPBACK MULTICAST
MTU:1500 metric:1 VR:0 ifindex:1
RX packets:195 mcast:0 errors:0 dropped:2
TX packets:197 mcast:0 errors:0
collisions:0 unsupported proto:0
RX bytes:18k TX bytes:18k
gei0 Link type:Ethernet HWaddr 00:40:9e:01:44:93 Queue:none
capabilities: TXCSUM TX6CSUM
inet 96.0.0.11 mask 255.255.255.0 broadcast 96.0.0.255
inet 224.0.0.1 mask 240.0.0.0
UP SIMPLEX BROADCAST MULTICAST
MTU:1500 metric:1 VR:0 ifindex:3
RX packets:0 mcast:0 errors:0 dropped:0
TX packets:0 mcast:0 errors:0
collisions:0 unsupported proto:0
RX bytes:0 TX bytes:0
gei1 Link type:Ethernet HWaddr 00:40:9e:01:44:92 Queue:none
capabilities: TXCSUM TX6CSUM
inet 192.168.1.101 mask 255.255.0.0 broadcast
192.168.255.255
inet 224.0.0.1 mask 240.0.0.0
UP RUNNING SIMPLEX BROADCAST MULTICAST
MTU:1500 metric:1 VR:0 ifindex:4
RX packets:122 mcast:0 errors:0 dropped:0
TX packets:93 mcast:0 errors:0
collisions:0 unsupported proto:0
RX bytes:7399 TX bytes:19k
gei3 Link type:Ethernet HWaddr 00:50:c2:0c:9c:2c Queue:none
capabilities: TXCSUM TX6CSUM
inet 96.0.3.11 mask 255.255.255.0 broadcast 96.0.3.255
inet 224.0.0.1 mask 240.0.0.0
UP SIMPLEX BROADCAST MULTICAST
MTU:1500 metric:1 VR:0 ifindex:5
RX packets:0 mcast:0 errors:0 dropped:0
TX packets:0 mcast:0 errors:0
collisions:0 unsupported proto:0
RX bytes:0 TX bytes:0
gei4 Link type:Ethernet HWaddr 00:50:c2:0c:9c:2d Queue:none
capabilities: TXCSUM TX6CSUM
inet 96.0.4.11 mask 255.255.255.0 broadcast 96.0.4.255
inet 224.0.0.1 mask 240.0.0.0
UP SIMPLEX BROADCAST MULTICAST
MTU:1500 metric:1 VR:0 ifindex:6
RX packets:0 mcast:0 errors:0 dropped:0
TX packets:0 mcast:0 errors:0
collisions:0 unsupported proto:0
RX bytes:0 TX bytes:0
value = 0 = 0x0
nois...@gmail.com
Just to be clear: you're certain that both the copper and fiber DSS
cards report device ID 0x1010? I would have expected the fiber one to
be 0x1012. (It doesn't matter since the silicon is the same in both
cases, but it strikes me as a little strange.)
> The two PMC cards are directly connected with two fiber optic cables
> and as it seems they are of type GigMAC PMC Model 5262-LC (PN: 152620-
> B2):http://www.dssnetworks.com/v3/gigabit_pmc_5262.asp.
> These PMC cards have four LEDs: TX, RX, GIG and FD.
>
> I can be sure that the PMC cards and the fiber optic cables are fine
> since I tried them running VxWorks 5.5.1. Under VxWorks 5.5.1 I am
> using an END device driver from DSS but now under 6.6 my intention is
> to deploy the VxBus (standard) drivers, where the MAC and PHY layers
> are separated, which seems to be a requirements for multi processor
> deployment.
Ah, ok. So the 5.5.1 configuration is using the DSS-supplied driver
code. That may be part of the issue (see below).
> When comparing the behavior of the DSS cards under VxWorks 5.5.1 and
> under VxWorks 6.6, I have noted the following:
>
> VxWorks 5.5.1:
>
> When I boot the boards with the cables plugged, all LEDs showing
> activity during the boot process and after the boards have booted, the
> GIG and FD LEDs stay lit. When unplugging and re-plugging the cables,
> the GIG and FD LEDs switches off and on but the RX and TX LEDs stay
> dark.
>
> What I have seen is that the interrupt handling function
> “inLiProcessMgmtInts” is called upon plugging/unplugging cables.
> However, I am not sure what would be the counter part to this function
> under 6.6.
The VxWorks driver detects the link status change interrupt in
geiEndIntHandle() and will call geiEndLinkUpdate() when it occurs. The
link update routine is periodically called by the miiBus code anyway
(at 2 second intervals) but handling the adapter's link change
interrupt allows the driver to respond more quickly to cable unplug/
replug events. (Not all adapters are nice enough to provide an
instantaneous link change indication like this, so in some cases you
have to wait up to 2 seconds before a link change event is processed.)
There's only one possibility I can think of, which is that while the
EEPROM on the DSS cards seems to correctly program the status register
such that the TBIMODE bit is set, it might not be properly programming
the LINK_MODE registers in the extended control register. Instead,
they may be relying on their own driver software to set the bits
manually, based on the TBIMODE setting. The VxWorks driver doesn't do
this since it expects the default settings loaded from the EEPROM to
be correct. (And indeed, with the Intel-branded PRO/1000 MF card, this
is the case.)
Bits 8 and 9 in the CTRLEXT register (offset 0x18) are what control
the actual operation of the NIC. The values are:
#define GEI_CTRLEXT 0x0018 /* Extended device control */
[...]
#define GEI_CTRLX_LINKMODE 0x00C00000 /* Link mode */
[...]
#define GEI_LINKMODE_GMII 0x00000000 /* Internal copper PHY */
#define GEI_LINKMODE_RSVD 0x00400000 /* reserved */
#define GEI_LINKMODE_SERDES 0x00800000 /* Internal SERDES */
#define GEI_LINKMODE_TBI 0x00C00000 /* External TBI */
Note: the VxWorks GEI driver identifies 0x00C00000 as SERDES, but this
is actually only correct for the PCIe devices. The above values are
correct for the PCI-X class devices. For an 82546 operating in
internal SERDES mode, only bit 9 should be set (0x00800000). Looking
at the specs for the DSS adapter that you have and the Intel PRO/1000
MF that I have, they should both be operating in internal SERDES mode.
(There doesn't appear to be an extra TBI PHY chip mounted on the
board.)
What I would do is insert some code somewhere in the VxWorks gei
driver to dump the contents of the extended control register and check
the settings of the link mode bits. I suspect you will find that
they're not set correctly. If this is the case, then you need to add a
bit of code to geiReset() to force the bits to the right state, e.g:
if (CSR_READ_4(pDev, GEI_STS) & GEI_STS_TBIMODE)
{
CSR_CLRBIT_4(pDev, GEI_CTRLEXT, GEI_CTRLX_LINKMODE);
CSR_SETBIT_4(pDev, GEI_CTRLEXT, 0x00800000);
}
Please let me know if this does the trick. For additional information,
the programming manual for the 82546 and other PCI-X PRO/1000 devices
can be found here:
http://download.intel.com/design/network/manuals/8254x_GBe_SDM.pdf
Note: it sounds like you intend to use the Concurrent Technologies
board in SMP mode (since it is a Core Duo). Assuming you get this
working, I would test it very carefully. That is, do some network
stress tests and verify that the board doesn't lock up under load. SMP
is very tricky to get right, and VxWorks 6.6 is the first release to
support it.
-Bill
ulrik
Hello Bill
Many thanks for your suggestions!
I checked the content of the Extended Device Control Register but the
LINK_MODE is already correctly set (10b = Direct Fiber interface using
internal SerDes).
A thought that I had was, that it may not be possible to combine
electrical Ethernet interfaces (CCT onboard) with optical Ethernet
interfaces (DSS PMC card), so I wanted to disable the Generic PHY
driver component and check the behavior. However, since excluding that
driver (not quick exclude) would result in removing also the Intel PRO/
1000 VxBus Enhanced Network Driver component, hints on that these two
components always need to be deployed in combination.
Another thing I tried was to add the following code in the function
"geiReset" (corresponding to how auto negotiation is restarted by the
DSS driver under VxWorks 5.5.1):
if (CSR_READ_4(pDev, GEI_STS) & GEI_STS_TBIMODE)
{
CSR_CLRBIT_4(pDev, GEI_CTRL, GEI_CTRL_FD);
CSR_CLRBIT_4(pDev, GEI_CTRL, GEI_CTRL_SLU);
tmp = CSR_READ_4(pDev, GEI_TXCW);
tmp &= ~0x1f;
tmp |= (0x60|0x80000000);
CSR_WRITE_4(pDev, GEI_IMC, tmp);
CSR_SETBIT_4(pDev, GEI_CTRL, GEI_CTRL_LRST);
CSR_CLRBIT_4(pDev, GEI_CTRL, GEI_CTRL_LRST);
}
The result is some additional GIG LED flickering (on all interfaces)
but that is all, links don't come up.
Best regards
Ulrik
nois...@gmail.com
[...]
> Hello Bill
>
> Many thanks for your suggestions!
>
> I checked the content of the Extended Device Control Register but the
> LINK_MODE is already correctly set (10b = Direct Fiber interface using
> internal SerDes).
Hm... well then I'm kind of stumped. It sounds like it should be
working right. I don't know why it isn't. Sadly, this is something I
could probably diagnose myself if only I had access to the hardware,
but I don't. :(
> A thought that I had was, that it may not be possible to combine
> electrical Ethernet interfaces (CCT onboard) with optical Ethernet
> interfaces (DSS PMC card), so I wanted to disable the Generic PHY
> driver component and check the behavior. However, since excluding that
> driver (not quick exclude) would result in removing also the Intel PRO/
> 1000 VxBus Enhanced Network Driver component, hints on that these two
> components always need to be deployed in combination.
No, you should be able to use all interfaces at the same time. The GEI
driver depends in the genericPhy driver by default, since it's always
needed for copper operation. This is enforced by a .cdf file that's
used by Workbench. The interfaces with TBIMODE set should never select
the genericPhy driver, only the geiTbiPhy driver.
> Another thing I tried was to add the following code in the function
> "geiReset" (corresponding to how auto negotiation is restarted by the
> DSS driver under VxWorks 5.5.1):
Uhm... the code below has a bug:
> if (CSR_READ_4(pDev, GEI_STS) & GEI_STS_TBIMODE)
> {
> CSR_CLRBIT_4(pDev, GEI_CTRL, GEI_CTRL_FD);
> CSR_CLRBIT_4(pDev, GEI_CTRL, GEI_CTRL_SLU);
You read from the TXCW register here:
> tmp = CSR_READ_4(pDev, GEI_TXCW);
> tmp &= ~0x1f;
> tmp |= (0x60|0x80000000);
But you write to the IMC register here:
> CSR_WRITE_4(pDev, GEI_IMC, tmp);
That should have been: CSR_WRITE_4(pDev, GEI_TXCW, tmp);
> CSR_SETBIT_4(pDev, GEI_CTRL, GEI_CTRL_LRST);
> CSR_CLRBIT_4(pDev, GEI_CTRL, GEI_CTRL_LRST);
> }
>
> The result is some additional GIG LED flickering (on all interfaces)
> but that is all, links don't come up.
Technically this shouldn't be necessary since this is what the
geiTbiPhy driver already does (see geiTbiPhyModeSet() for the IFM_AUTO
case). It's basically advertising 1000FD and 1000HD capabilities and
enabling autoneg.
Unfortunately, since I don't have one of these cards myself, and since
I can't reproduce the problem with the card that I have, I'm kind of
at a loss for more ideas. It should be working. There may be something
hidden in DSS's driver code. You might look to see if there's any code
that manipulates the pin control (SDP) bits via the GEI_CTRL or
GEI_CTRLEXT register. There may be a magic pin that has to be asserted
to apply power to the optical interfaces. You could also check to see
if there's any code to manipulate the PHY reset bit
(GEI_CTRL_PHY_RST). Normally the VxWorks driver only manipulates this
bit for the 82541 chip.
Note: earlier I had suggested cabling the two ports on the DSS card
together with a fiber patch to see what would happen, but you never
said what happened. (I don't know if you didn't try it or forgot to
report the results.)
-Bill
> Best regards
>
> Ulrik
ulrik
Hello Bill
Correcting the bug that you spotted (thanks) didn’t help, also when
manually restarting the hardware auto negotiation, it fails.
That the auto negotiation has failed seems obvious considering that
the:
- CTRL register bit LRST (Link reset) is set
- CTRL register bits RFCE (Receive Flow Control Enable) and TCFE
(Transmit Flow Control Enable) are cleared
- RXCW register bit RxConfigInvalid (Invalid Symbol during
configuration process) is set
- STATUS register bits FD (Link Full Duplex) and LU (Link Up) are
cleared
For now, I went for a rather pragmatic approach:
- GEITBIPHY driver:
A change was made to the function “geiTbiPhyModeGet” of file
“geiTbiPhy.c” to always return status active, independently of the
result of the hardware auto negotiation.
This had the effect that also the optical interfaces entered the
RUNNING state as indicated by the “ifconfig” command. However, this
was not enough to allow traffic over these interfaces but in addition
a change in the GEI driver was necessary.
- GEI driver:
A change was made to the function “geiMuxConnect” of file
“gei825xxVxbEnd.c”, forcing register bits into the desired state. Code
extract:
/* Clear the CTRL register bit LRST and set the bits RFCE and TFCE */
tmp = CSR_READ_4(pDev, GEI_CTRL);
tmp &= 0xFFFFFFF7;
tmp |= 0x18000000;
CSR_WRITE_4(pDev, GEI_CTRL, tmp);
/* Set the RXCW register to hold the same data as seen under VxWorks
5.5.1 */
CSR_WRITE_4 (pDev, GEI_RXCW, 0xc40041e0);
/* Set the STATUS register bits LU and FD */
tmp = CSR_READ_4(pDev, GEI_STS);
tmp |= 0x00000003;
CSR_WRITE_4(pDev, GEI_STS, tmp);
With this code change, the optical interfaces are operating, being
able to send and receive packets.
It was noted that if packet send/receive is started over both DSS PMC
card pairs (send/receive pair) and one or more cables are unplugged,
the packet send/receive stops on all optical interfaces (each of the
two node boards are equipped with 2 PMC cards and there are two
interfaces per PMC card -> 4 interface pairs).
However, if packet send/receive is started only between one card pair,
packet send/receive is unaffected when unplugging one or both cables
of the other card pair.
Important to note is that this problem is not present when electrical
Ethernet DSS PMC cards are deployed, but they behave as expected: Only
the interface pair for which the cable is removed, the packet send/
receive is interrupted.
As you understand, the current situation is not very satisfying and
I’ll continue to investigate on these issues. Nevertheless, at least I
can run traffic now.
Best regards
Ulrik
nois...@gmail.com
I am assuming you added this code after the call to muxDevStart(). Is
this correct?
These last two register updates should be no-ops. Per the Intel
documentation, the RXCW and STATUS registers are read only, so writing
to them should have no effect. (In particular, the link up bit should
be set automatically by the hardware to indicate that a link has been
established.) This means that the only effective change with regards
to how the hardware is configured is to enable RX and TX flow control
in the MAC. I suspect that if you remove the code that tries to write
to RXCW ans STATUS, you'll still get a link.
The value for RXCW which you say is oberved under 5.5.1 seems to
indicate that the link partner to which you have the fiber cards
connected is advertising flow control support (bits 7 and 8 are set).
The flow control bits in the TXCW register (which control whether or
not the PRO/1000 advertises flow control support too) are initialized
from the EEPROM. The geiTbiPhy driver preserves them when setting up
autonegotiation, however the GEI driver itself doesn't support it
(which is why it doesn't turn flow control on in the CTRL register).
With my Intel PRO/1000 MF card, the EEPROM sets the TXCW bits on, and
I'm assuming that the EEPROM on your PMC cards is set up the same way.
I think there may be some sort of mismatch between the link partner
and the 82546 because the NIC is advertising flow control support, but
it's not actually used. Technically, since the MAC driver doesn't
support it, it shouldn't be advertising it. Can you please try the
following test for me:
- put back the original copies of gei825xxVxbEnd.c and geiTbiPhy.c
- in geiTbiPhy.c, find the geiTbiPhyInit() routine
- Find the code that says:
miiBusRead (pDev, pDrvCtrl->miiPhyAddr, GTBI_TXCFG, &miiVal);
miiVal &= ~(GTBI_TXCFG_1000HD|GTBI_TXCFG_1000FD);
miiBusWrite (pDev, pDrvCtrl->miiPhyAddr, GTBI_TXCFG, miiVal);
- Change it to:
miiBusRead (pDev, pDrvCtrl->miiPhyAddr, GTBI_TXCFG, &miiVal);
miiVal &= ~(GTBI_TXCFG_1000HD|GTBI_TXCFG_1000FD|GTBI_TXCFG_PAUSE);
miiBusWrite (pDev, pDrvCtrl->miiPhyAddr, GTBI_TXCFG, miiVal);
- Recompile geiTbiPhy and gei825xxVxbEnd and see if you get a link
this time
What this will do is force off the RX and TX pause frame (flow
control) advertisement, which should prevent the link partner from
enabling flow control with this link. This way, the PHY, MAC and link
partner will all agree. Please let me know if this fixes the problem.
I tried this with my PRO/1000 MF card and the one gigE switch I have
available (an old Extreme Networks Summit 5i with GBICs) and it seems
to work ok (of course it worked with the old code too).
> It was noted that if packet send/receive is started over both DSS PMC
> card pairs (send/receive pair) and one or more cables are unplugged,
> the packet send/receive stops on all optical interfaces (each of the
> two node boards are equipped with 2 PMC cards and there are two
> interfaces per PMC card -> 4 interface pairs).
> However, if packet send/receive is started only between one card pair,
> packet send/receive is unaffected when unplugging one or both cables
> of the other card pair.
> Important to note is that this problem is not present when electrical
> Ethernet DSS PMC cards are deployed, but they behave as expected: Only
> the interface pair for which the cable is removed, the packet send/
> receive is interrupted.
This is a little puzzling, but may have to do with the changes you
made to geiTbiPhy. Normally, all interfaces should operate
independently, but it sounds like a link change event on one is
messing up another. I'm hoping that if you restore everything and make
the one change that I suggested above, you won't see this problem
anymore.
> As you understand, the current situation is not very satisfying and
> I’ll continue to investigate on these issues. Nevertheless, at least I
> can run traffic now.
Yeah, I do understand, and I really want to solve this. Please let me
know if my suggested change helps.
> Best regards
>
> Ulrik
ulrik
Hello Bill
Sorry for taking so long to respond!
The code, writing of registers, was added at the end of the
“geiMuxConnect” routine (after the muxDevStart() function call).
The effect of the writing to the registers RXCW and STATUS is outlined
below. The two node boards were equipped with one optical PMC card
each and each of the two interfaces was connected to the corresponding
interface of the other board/card (loop test). The reason why the unit
numbering is not what you may expect is the fact that I force a
certain numbering scheme, so that the interfaces of the upper PMC card
always have the unit numbers 5 and 6 in order to avoid that the
numbering is different when no, one or two PMC cards are attached.
I checked on the content of the STATUS and RXCW registers, at three
different points in time:
1. Register content directly before the write done in the
“geiMuxConnect” routine
2. Register content directly after the write done in the
“geiMuxConnect” routine
3. Register content some seconds later (by calling a routine that
examines the content)
What can be seen is that the write to the STATUS register has no
effect and the write to the RXCW at most a temporary one since it also
changes between point 2 and point 3 in time (without me doing anything
in the mean time).
The values within parentheses show the change in content after I have
removed the write to the STATUS and RXCW registers.
BOARD 1:
unitNumber=5
GEI_STS=dba4
GEI_RXCW=c000000
GEI_STS=dba4
GEI_RXCW=640001e0 (64000000)
GEI_STS=dba7
GEI_RXCW=c40041e0 (c40041e0)
unitNumber=6
GEI_STS=dba0
GEI_RXCW=c000000
GEI_STS=dba0
GEI_RXCW=640001e0 (64000000)
GEI_STS=dba3
GEI_RXCW=c40041e0 (c40041e0)
BOARD 2:
unitNumber=5
GEI_STS=dba4
GEI_RXCW=c000000
GEI_STS=dba4
GEI_RXCW=74000000 (640001e0)
GEI_STS=dba7
GEI_RXCW=c40041e0 (c40041e0)
unitNumber=6
GEI_STS=dba0
GEI_RXCW=c000000
GEI_STS=dba0
GEI_RXCW=74000000 (640001e0)
GEI_STS=dba3
GEI_RXCW=c40041e0 (c40041e0)
Consequently, the writing to the STATUS and RXCW registers was
dropped. However, the writing to the CTRL register is still necessary
since alone the change to the routine “geiTbiPhyInit” that you
proposed doesn’t bring the link to an operating state (seems to have
no effect).
Something else I noted is that the change I earlier did to the file
“geiTbiPy.c”, always flagging for status active in the routine
“geiTbiPhyModeGet”, is no longer needed for bringing the interfaces
into the RUNNING state (indicated by “ifconfig”).
Please note that the strange behavior "unplugging a cable effecting
also other interfaces" still remains.
Many thanks for your support so far and best regards
Ulrik
nois...@gmail.com
> Hello Bill
>
> Sorry for taking so long to respond!
>
> The code, writing of registers, was added at the end of the
> “geiMuxConnect” routine (after the muxDevStart() function call).
>
> The effect of the writing to the registers RXCW and STATUS is outlined
> below. The two node boards were equipped with one optical PMC card
> each and each of the two interfaces was connected to the corresponding
> interface of the other board/card (loop test).
This is not a "loop test." This is just connecting two targets
together back to back. I remember at some point asking you just what
it was that these two targets were connected to (i.e. what kind of
switch, if any), but you never told me. (Or if you did tell me, it was
long enough ago that I forgot.)
So I'm going to ask you this again, and I'm going to insist that you
answer this time:
- When you first posted, and you said you couldn't get a link in the
fiber optic ports, what were they plugged into? Did you just have two
targets connected back to back like you do now? Was the target
connected to a switch with fiber optic ports? If so, what kind of
switch?
> The reason why the unit
> numbering is not what you may expect is the fact that I force a
> certain numbering scheme, so that the interfaces of the upper PMC card
> always have the unit numbers 5 and 6 in order to avoid that the
> numbering is different when no, one or two PMC cards are attached.
Okay, this is very cryptic. Exactly what the heck did you do to "force
a certain numbering scheme?" I submit to you that you should have held
off on doing anything strange like this until _AFTER_ you'd figured
out the problem with the GEI driver.
> I checked on the content of the STATUS and RXCW registers, at three
> different points in time:
> 1. Register content directly before the write done in the
> “geiMuxConnect” routine
> 2. Register content directly after the write done in the
> “geiMuxConnect” routine
> 3. Register content some seconds later (by calling a routine that
> examines the content)
> What can be seen is that the write to the STATUS register has no
> effect and the write to the RXCW at most a temporary one since it also
> changes between point 2 and point 3 in time (without me doing anything
> in the mean time).
Okay, I'm going to say this again: both the GEI_STS and GEI_RXCW
registers are _read-only_. Writing to them _NEVER_ has any effect. You
only see their contents change because the underlying device status
has changed. If you don't believe me, wait until the link comes up and
then try writing a garbage value to the RXCW register manually from
the target shell, then read it back again. You'll see that nothing
happens.
It would appear from the output here that you set the unit numbers
backwards. Bit 3 in the status register is the function ID code. This
bit is used to tell the difference between different ports on a dual
port card. It should be cleared for port 0 and set for port 1. (This
is mainly used to set the MAC address: there is only one EEPROM, which
is shared between both ports, so both ports end up reading the same
station address from the EEPROM, and the one with function ID 1 flips
the last bit so that the address on the second port will be different
from the first.)
But it looks like port 0 and port 1 per card translate to unit 6 and
unit 5, instead of unit 5 and unit 6. Again, I want to know just what
the heck you did to achieve this.
> Consequently, the writing to the STATUS and RXCW registers was
> dropped. However, the writing to the CTRL register is still necessary
> since alone the change to the routine “geiTbiPhyInit” that you
> proposed doesn’t bring the link to an operating state (seems to have
> no effect).
That's the main thing I wanted to know. I still don't understand why
enabling flow control here made a difference.
> Something else I noted is that the change I earlier did to the file
> “geiTbiPy.c”, always flagging for status active in the routine
> “geiTbiPhyModeGet”, is no longer needed for bringing the interfaces
> into the RUNNING state (indicated by “ifconfig”).
That's because it wasn't actually needed in the first place: you did
it early on to force the link to appear up, but once you poked the
CTRL register to actually establish the link, this change became
redundant.
> Please note that the strange behavior "unplugging a cable effecting
> also other interfaces" still remains.
I still don't fully understant the strange behavior. I'm starting to
suspect it has to do with this hitherto unmentioned unit number
forcing.
If you would be so kind, please describe for me a complete test setup
where you experience this problem. And I mean describe all of it, from
start to finish: explain how you connect up the hardware. Explain how
you configure the VxWorks image that you load. Explain what test you
run. If you want to be really nice, show the console output where you
run the tests. (It makes it easier for me to attempt the same thing.)
Also, you might want to experiment with forcing the flow control
enable bits in the control register on at a different point within the
code. Doing it in the geiMuxConnect() routine is suboptimal, since
it's only evern called once. I would try doing it somewhere in the
geiEndStart() routine instead.
-Bill
ulrik
Hello Bill
“Loop test” may not be the right term but that is how we usually call
it when we connect one “node” board to another and run tests, before
involving any switch or other network elements.
The test setup that I have used to investigate on the fiber optical
ports and get them working was always the same, letting one board talk
to another. Sorry, for not being clear on this in the past.
In the routine “geiInstInit” the routine “vxbNextUnitGet” is normally
called. I changed that, so that the routine “vxbInstUnitSet” is called
with the desired device number, code extract (lower PMC card lower
interface -> gei3, lower PMC card upper interface -> gei4 and so one):
LOCAL void geiInstInit
(
VXB_DEVICE_ID pDev
)
{
struct vxbPciDevice *pPciDev;
pPciDev = (struct vxbPciDevice *) pDev->pBusSpecificDevInfo;
….
else if ( pPciDev->pciBus == 4 && pPciDev->pciDevId == DEVID_82546EB
&& pPciDev->pciFunc == 0 )
{
vxbInstUnitSet(pDev, 4);
}
else if ( pPciDev->pciBus == 4 && pPciDev->pciDevId == DEVID_82546EB
&& pPciDev->pciFunc == 1 )
{
vxbInstUnitSet(pDev, 3);
}
else if ( pPciDev->pciBus == 8 && pPciDev->pciDevId == DEVID_82546EB
&& pPciDev->pciFunc == 0 )
{
vxbInstUnitSet(pDev, 6);
}
else if ( pPciDev->pciBus == 8 && pPciDev->pciDevId == DEVID_82546EB
&& pPciDev->pciFunc == 1 )
{
vxbInstUnitSet(pDev, 5);
}
….
Extensive testing in the past has shown that I can consider this code
change safe and not related to the problem of the optical links not
coming up without the “CTRL register write” workaround.
I changed the above code to swap unit 5 and 6 and 3 and 4 as you
suggested and also reverted back to the original code but without the
“CTRL register write” workaround, the links don’t come up.
Test set-up remarks:
- VxWorks image: GEI driver and GEI TBI driver components included
- The interfaces of a PMC card on one board are connected to the
corresponding interfaces of the corresponding PMC card on the other
board (gei3 –gei3, gei4 – gei4, gei5 – gei5 and gei6 – gei6)
- I am using “ifconfig” and “ping” to check on status and connectivity
I agree with you that a register that is said to be read only also
should be read only. However, it depends on the design what is the
actual case. Nevertheless, what it means is that a write at most would
have some temporary effect since the content of such a registers may
change at any time.
Since, I can live with the current workaround for now, I will have to
put this issue on ice, due to higher prioritized tasks.
Many thanks for you support and ideas
Ulrik