Meraki MR24 /overlay problem narrowed

[Russell Senior]

We observed problems with both MR24s we deployed tonight where in dropbear did not come up when we plugged the devices in.  Upon investigation it was found that the overlay files /overlay/upper/etc/config/dropbear consisted of a bunch of NULLs instead of the correct data.  We could recover from serial console by copying the versions from squashfs (in /rom/etc/config/dropbear) and reboot.

When I got home, I poked at the device in my test bed trying to replicate the problem.  Eventually I found that it manifest only if I yanked power after the first boot.  If I "warm" rebooted, the data got flushed.  If I yanked power it did not.

It seems the problem is related to not flushing data out to persistent storage on ubifs (on NAND flash).  I think we can recover from this by doing a factory reset (i.e. boot normally, then press and hold the reset button for 5 seconds or more), then logging in and rebooting from the shell, rather than power cycling.  It still needs physical access, but it doesn't need to replace the device.

I will be pursuing a proper fix, but this is a bandaid in the meantime.

--

Russell Senior, President

rus...@personaltelco.net