T113 EMAC stuck in soft reset

[Dave Johnson]

Hi,

I'm trying to bring up my T113-s3 based board and cannot get the EMAC
to initialize.

Some info about my board:

I'm using a RTL8201F phy on the PG pins in RMII mode and want to use
the EPHY-25M clock output on PG11 to drive the phy so no external
crystal is needed.


Kernel: https://github.com/smaeul/linux.git at b466df90d48fb7ef03f331e0c73c8012ac03a12e

DTS snippets:

emac: ethernet@4500000 {
compatible = "allwinner,sun20i-d1-emac",
"allwinner,sun50i-a64-emac";
reg = <0x4500000 0x10000>;
interrupts = <GIC_SPI 46 IRQ_TYPE_LEVEL_HIGH>;
interrupt-names = "macirq";
clocks = <&ccu CLK_BUS_EMAC>;
clock-names = "stmmaceth";
resets = <&ccu RST_BUS_EMAC>;
reset-names = "stmmaceth";
syscon = <&syscon>;
status = "disabled";

mdio: mdio {
compatible = "snps,dwmac-mdio";
#address-cells = <1>;
#size-cells = <0>;
};
};

&pio {
rmii_pg_pins: rmii-pg-pins {
pins = "PG0", "PG1", "PG2", "PG3", "PG4",
"PG5", "PG11", "PG12", "PG13", "PG14", "PG15";
function = "emac";
};
};

&emac {
pinctrl-0 = <&rmii_pg_pins>;
pinctrl-names = "default";
phy-handle = <&rtl8201f>;
phy-mode = "rmii";
phy-supply = <&reg_vcc_3v3>;
clocks = <&ccu CLK_BUS_EMAC>, <&ccu CLK_EMAC_25M>;
clock-names = "stmmaceth", "ephy25m";
status = "okay";
};

&mdio {
status = "okay";
reset-gpios = <&pio 6 7 GPIO_ACTIVE_LOW>; /* PG7 */
reset-assert-us = <10000>;
reset-post-delay-us = <150000>;
reset-deassert-us = <150000>;
rtl8201f: ethernet-phy@0 {
compatible = "ethernet-phy-id001c.c816"; /* RTL8201F */
reg = <0>;
};
};



When the kernel driver loads, I end up with sun8i_dwmac_reset() failing because bit 0 in EMAC_BASIC_CTL1 is stuck on and never self-clears:

[ 1.984000] dwmac-sun8i 4500000.ethernet: EMAC reset timeout


I can also manually attempt to bring the EMAC online with:

$ sudo devmem 0x0200197c 32 0x00000000 #EMAC_BGR_REG reset
$ sudo devmem 0x0200197c 32 0x00010000 #EMAC_BGR_REG enable clock
$ sudo devmem 0x0200197c 32 0x00010001 #EMAC_BGR_REG release reset
$ sudo devmem 0x02001970 32 0xc0000000 #EMAC_25M_CLK_REG setup EPHY-25M
$ sudo devmem 0x03000030 32 0x00012000 #EMAC_EPHY_CLK_REG0 setup RMII mode, external PHY
$ sudo devmem 0x04500004 32 0x08000001 #EMAC_BASIC_CTL1 attempt soft reset


pinctrl is setup:
$ sudo devmem 0x02000120 32
0x1F444444
$ sudo devmem 0x02000124 32
0x44444FFF

clocks and reset:
$ sudo devmem 0x0200197c 32
0x00010001
$ sudo devmem 0x02001970 32
0xC0000000

syscon:
$ sudo devmem 0x03000030 32
0x00012000

soft reset is stuck:
$ sudo devmem 0x04500004 32
0x08000001


Has anyone succesfully got the T113/D1 EMAC to work, especially in RMII mode?




--
Dave

[Matthew Trescott]

Hi Dave,

> Some info about my board:
>
> I'm using a RTL8201F phy on the PG pins in RMII mode and want to use
> the EPHY-25M clock output on PG11 to drive the phy so no external
> crystal is needed.
>
>
> Kernel: https://github.com/smaeul/linux.git at
b466df90d48fb7ef03f331e0c73c8012ac03a12e

Do you have a scope to check this net on the board? As far as I know you need
a patch that's not in mainline or smaeul's kernel:

https://oftc.irclog.whitequark.org/linux-sunxi/2023-02-14

> Has anyone succesfully got the T113/D1 EMAC to work, especially in RMII
mode?

There are a few other reports of actual boards using RMII + 25MHz clock
output, probably only using the BSP kernel though:

https://oftc.irclog.whitequark.org/linux-sunxi/2023-07-29
https://oftc.irclog.whitequark.org/linux-sunxi/2022-06-07#30990410

I would be interested in hearing what you find out, since I'm looking to make
my own Lichee RV (D1) carrier board and I also like to be a cheapskate when it
comes to hardware. :)

Kind regards,
Matt

[Dave Johnson]

Matthew Trescott writes:
> Hi Dave,
>
> > Some info about my board:
> >
> > I'm using a RTL8201F phy on the PG pins in RMII mode and want to use
> > the EPHY-25M clock output on PG11 to drive the phy so no external
> > crystal is needed.
> >
> >
> > Kernel: https://github.com/smaeul/linux.git at
> b466df90d48fb7ef03f331e0c73c8012ac03a12e
>
> Do you have a scope to check this net on the board? As far as I know you need
> a patch that's not in mainline or smaeul's kernel:
>
> https://oftc.irclog.whitequark.org/linux-sunxi/2023-02-14

Added that patch and moved my <&ccu CLK_EMAC_25M> into the mdio/phy DT
section.

Manually setting EMAC_25M_CLK_REG does work and produces a nice 25Mhz
sine wave clock output on the pin, however the driver isn't doing this
(even with that patch). The driver does release phy reset during
init, but not enable the 25mhz clock output.

If I manually (outside of the driver) release the phy reset GPIO and
enable the 25mhz output, the phy comes online and starts generating a
50mhz clock back towards the MAC on the RMII-TXCK pin.

I verified clocks going in both directions with a scope and once the
TXCK clock makes it back to the EMAC from the PHY it looks like it
will exit soft-reset.

So at this point, getting the ephy-25mhz enabled by mac and/or mdio
drivers should fix things.

I placed these in both the mdiobus and phy clauses of the DTS but it's
not getting picked up:

clocks = <&ccu CLK_EMAC_25M>;
clock-names = "ephy25m";

>
> > Has anyone succesfully got the T113/D1 EMAC to work, especially in RMII
> mode?
>
> There are a few other reports of actual boards using RMII + 25MHz clock
> output, probably only using the BSP kernel though:
>
> https://oftc.irclog.whitequark.org/linux-sunxi/2023-07-29
> https://oftc.irclog.whitequark.org/linux-sunxi/2022-06-07#30990410
>
> I would be interested in hearing what you find out, since I'm looking to make
> my own Lichee RV (D1) carrier board and I also like to be a cheapskate when it
> comes to hardware. :)
>
> Kind regards,
> Matt
>
>

--
Dave

[Matthew Trescott]

Hi Dave,

> Added that patch and moved my <&ccu CLK_EMAC_25M> into the mdio/phy DT
> section.
>
> Manually setting EMAC_25M_CLK_REG does work and produces a nice 25Mhz
> sine wave clock output on the pin, however the driver isn't doing this
> (even with that patch). The driver does release phy reset during
> init, but not enable the 25mhz clock output.

Is it actually a sine wave on your board? I get a square wave here... had to
use a ground spring rather than the alligator clip though; otherwise the
waveform was nasty-looking. I was using PE10, not PG11, because PG11 is
already in use on the Lichee RV 86 Panel.

> If I manually (outside of the driver) release the phy reset GPIO and
> enable the 25mhz output, the phy comes online and starts generating a
> 50mhz clock back towards the MAC on the RMII-TXCK pin.
>
> I verified clocks going in both directions with a scope and once the
> TXCK clock makes it back to the EMAC from the PHY it looks like it
> will exit soft-reset.
>
> So at this point, getting the ephy-25mhz enabled by mac and/or mdio
> drivers should fix things.
>
> I placed these in both the mdiobus and phy clauses of the DTS but it's
> not getting picked up:
>
> clocks = <&ccu CLK_EMAC_25M>;
> clock-names = "ephy25m";
>

OK, I need to try it out to see if I can find the problem. Maybe someone more
experienced than me can weight in in the meantime, though.

Matt

[Dave Johnson]

Matthew Trescott writes:
> > So at this point, getting the ephy-25mhz enabled by mac and/or mdio
> > drivers should fix things.
> >
> > I placed these in both the mdiobus and phy clauses of the DTS but it's
> > not getting picked up:
> >
> > clocks = <&ccu CLK_EMAC_25M>;
> > clock-names = "ephy25m";
> >
>
> OK, I need to try it out to see if I can find the problem. Maybe someone more
> experienced than me can weight in in the meantime, though.

If I list the phy as discoverable, it isn't detected as the
clocks/reset aren't setup:

compatible = "ethernet-phy-ieee802.3-c22";

[ 1.273000] mdio_bus stmmac-0: MDIO device at address 0 is missing.


Or if I list the phy specifically, mdiobus_register_clock() does get
called, but mdio_device_toggle_clock() is never called:

compatible = "ethernet-phy-id001c.c816";

[ 1.448000] mdio_bus stmmac-0:00: Resources present before probing





--
Dave

[Dave Johnson]

Dave Johnson writes:
> If I list the phy as discoverable, it isn't detected as the
> clocks/reset aren't setup:
>
> compatible = "ethernet-phy-ieee802.3-c22";
>
> [ 1.273000] mdio_bus stmmac-0: MDIO device at address 0 is missing.
>
>
> Or if I list the phy specifically, mdiobus_register_clock() does get
> called, but mdio_device_toggle_clock() is never called:
>
> compatible = "ethernet-phy-id001c.c816";
>
> [ 1.448000] mdio_bus stmmac-0:00: Resources present before probing
>

OK, I've found the problem.

The patch mentioned https://oftc.irclog.whitequark.org/linux-sunxi/2023-02-14 seems incomplete.

The assumption that clocks only need to be enabled/disabled at
mdio_probe()/mdio_remove() isn't correct. mdio_device_reset() is also
called from phy_device_reset() and those calls are the only ones prior
to probe, so probing for the phy needs the clocks too.

When mdio_device_reset() is called from phy_device_reset() we also
need to enable/disable clocks along with the reset.

The previous mdio_device_toggle_clock() also resulted in some warnings
as it tried to disable clocks when they were already disabled,
something the clock subsystem doesn't like.

Will throw together a patch shortly.

--
Dave

[Matthew Trescott]

Hi Dave,

I made a bit of a mistake; I did not realize there is a new clocks binding
already mainline:
https://lore.kernel.org/netdev/20230605154010.4961...@collabora.com/T/#u

That patch puts the clock under control of the PHY driver, so in theory it
should have worked without Andre's patch at all, as long as the clock was a
property of the PHY node. It seems only the Realtek driver handles it for now,
but that wouldn't be a problem in your case.

> Dave Johnson writes:
> > If I list the phy as discoverable, it isn't detected as the
> > clocks/reset aren't setup:
> >
> > compatible = "ethernet-phy-ieee802.3-c22";
> >
> > [ 1.273000] mdio_bus stmmac-0: MDIO device at address 0 is missing.
> >

While looking through the netdev archives, I found someone is already working
on this, too, but it looks like it's not going to be mainline anytime soon:
https://lore.kernel.org/netdev/13590315.F0gNSz5aLb@diego/T/#t

> >
> > Or if I list the phy specifically, mdiobus_register_clock() does get
> > called, but mdio_device_toggle_clock() is never called:
> >
> > compatible = "ethernet-phy-id001c.c816";
> >
> > [ 1.448000] mdio_bus stmmac-0:00: Resources present before probing
> >
>
> OK, I've found the problem.
>
> The patch mentioned https://oftc.irclog.whitequark.org/linux-sunxi/
2023-02-14 seems incomplete.
>
> The assumption that clocks only need to be enabled/disabled at
> mdio_probe()/mdio_remove() isn't correct. mdio_device_reset() is also
> called from phy_device_reset() and those calls are the only ones prior
> to probe, so probing for the phy needs the clocks too.
>
> When mdio_device_reset() is called from phy_device_reset() we also
> need to enable/disable clocks along with the reset.
>
> The previous mdio_device_toggle_clock() also resulted in some warnings
> as it tried to disable clocks when they were already disabled,
> something the clock subsystem doesn't like.
>
> Will throw together a patch shortly.

Still haven't been able to test locally yet, but hopefully tomorrow.

Kind regards,
Matt

[Dave Johnson]

Matthew Trescott writes:
> Hi Dave,
>
> I made a bit of a mistake; I did not realize there is a new clocks binding
> already mainline:
> https://lore.kernel.org/netdev/20230605154010.4961...@collabora.com/T/#u
>
> That patch puts the clock under control of the PHY driver, so in theory it
> should have worked without Andre's patch at all, as long as the clock was a
> property of the PHY node. It seems only the Realtek driver handles it for now,
> but that wouldn't be a problem in your case.

Sorry for the delay, I'm just getting back to this.

Yes, that patch does solve the issue of using ephy-25m output for a
crystal-less PHY solution.

Note that patch is only for a few specific models of PHY. I had to
expand it to work with others, but solution is the same.

As mentioned before, the DI/T113 EMAC does require the PHY to feed
it's 50mhz clock back into the SOC for it to complete soft reset,
but the wait/timeout is already in the driver for this. It's
unfortunate that there is an external dependency on this in the
hardware design, but it is what it is.




Thanks.