It looks like the SPI freq. for the SDCard interface is too low. Line 33 of chipyard/fpga/src/main/scala/vcu118/Configs.scala is normalizing the frequency to MHz
val freqMHz = (site(SystemBusKey).dtsFrequency.get / (1000 * 1000)).toLong
but the sd.c at chipyard/fpga/src/main/resources/vcu118/sdboot is using Hz:
REG32(spi, SPI_REG_SCKDIV) = (F_CLK / 300000UL);
Solution, divide in MHz instead (300000) --> 0.3 (0.3MHz). Use 0.4 instead since that is the expected freq by the MCU within the SDcard. Chnage line 179 too in sd.c accordingly! (20MHz works ok).