Alveo U250 Not Being Recognized When Flashed With Firesim MCS

[iggy12345]

Hello,

I've been trying to get firesim setup on my lab's Alveo U250 FPGA card for the better part of 3 weeks at this point and I cannot get it to work according to your docs. 

I've followed the instructions here: https://docs.fires.im/en/latest/Getting-Started-Guides/On-Premises-FPGA-Getting-Started/Initial-Setup/Xilinx-Alveo-U250.html

But the last step using lspci won't work for me, even if I remove the usb cable prior to cold booting. I also found this information https://adaptivesupport.amd.com/s/article/71757?language=en_US referenced here https://docs.amd.com/r/en-US/ug1289-u200-u250-reconfig-accel/MCS-File-Generation-and-Alveo-Card-Programming about resetting to a gold mcs before flashing to a non-standard mcs. I tried this too with no luck. 

A colleague of mine was able to get the lspci command to work previously, but even he can't get it to work anymore and he's followed the same setup as me. I've verified that the xdma and xvsec kernel modules are loaded and that I'm source from the right firesim shell script.

I've noticed that my fpga card has a nic, but the documentation explicitly states to use the non-nic version of the mcs, does that matter?

I'm curious if I'm missing something here? I'm not sure what else I can do.

Any help is greatly appreciated.

[Josh Bodner]

Hi, I'm the colleague.  For reference, when it was working, this is what was seen:

[Chris Stratford]

If you have a usb cable plugged in it reports that xdma is attached to a Xilinx Serial Controller when you lspci -v upon issuing a 'sudo reboot'.

It is a pain in the arse to get working, very hit and miss, very much more miss than hit ! i had to try 4x different usb cables before finally cracking it as well as messing with the xdma driver and unloading xocl/xclmgmt pcie xrt drivers, it just seems all a bit random behaviour. At reboot there appears a race between seeing the jtag serial controller via usb-cable and the pcie coming up with the relevant drivers, you see one or the other. Ironically, the vivado Hardware Manager doesnt seem to have a problem connecting to the fpga jtag it sees it regardless when you auto connect. 

I have not managed to do this with any level of consistency, if you get to see the serial controller after reboot, never reset the PC, like ever !!!!!

i was using U200 with ubuntu. Good luck.

[Chris Stratford]

Setup ssh

As per instructions on Initial Setup/Installation — FireSim 1.18.0 documentation

Setup: openssh server, cable drivers, xdma, xvsec, +ssh keys.

For XDMA+XVSEC kernel drivers use : Local FPGA System Setup — FireSim HEAD documentation

NOTE: there are TWO separate repos for these drivers ! The XVSEC maintained by Xilinx does not work for recent kernel versions, its maintained by a separate alternate github source.

sudo modprobe -r xocl to remove xilinx xrt driver

sudo modprobe xdma [+xvsec] these two drivers and ensure they are present when you ‘lsmod’

try sudo reboot the machine if you did a sudo make install of the drivers and they should auto run and pick up the card after a reboot.

BASICALLY,lspci -v should reveal something like:

08:00.0 Serial controller: Xilinx Corporation Device 903f (prog-if 01 [16450]) Subsystem: Xilinx Corporation Device 0007 Flags: fast devsel, IRQ 223, IOMMU group 22 Memory at 50000000 (32-bit, non-prefetchable) [size=32M] Memory at 52000000 (32-bit, non-prefetchable) [size=64K] Capabilities: <access denied> Kernel driver in use: xdma Kernel modules: xdma

Often the Xilinx PCIe drivers xocl, xclmgmt attaches itself at 08:00.0 and 08:00.1 and the Serial controller drops away. It is UNCLEAR why the serial jtag controller doesn not get picked up. JJust keep reloading/unloading drivers and rebooting until it appears as above. It is annoying to say the least !

sudo dmesg to see the xdma log, and check in /dev that there are /dev/xdma* files. A kernel driver may be loaded but it is not bound to a device unless it has something in /dev directory.

FURTHER: DO NOT run /opt/xilinx/xrt/setup.sh to load the PCIe drivers xocl and xclmgmt as this will cause havoc. If this happens remove xocl, xclmgmt and sudo reboot the machine !!!!

Note: this is attached to the SERIAL controller, e.g. JTAG as the cable is plugged into Alveo u200 as well as the machine

Put the following into a runtime script it will be needed to be sourced often:

cd ~/.ssh
ssh-agent -s > AGENT_VARS
source AGENT_VARS
ssh-add firesim.pem
cd ~/chipyard/sims/firesim # assumed location of chipyard repo

Make sure you can: ‘ssh localhost printenv’ with no passwords asked for.

In the .bashrc add the following to ensure that vivado is enabled:

if [ -r "/opt/Vivado/2024.1/settings64.sh" ]; then source /opt/Vivado/2024.1/settings64.sh fi #if [ -r "/opt/xilinx/xrt/setup.sh" ]; then # source /opt/xilinx/xrt/setup.sh #fi case $- in i) ;; *) return;; esac

locations are assumed above, adjust as required.

On Wednesday, 24 September 2025 at 18:57:55 UTC+1 aaron...@gmail.com wrote:

[Chris Stratford]

load a bitfile first with vivado hardware manager, load xdma/xvsec drivers, remove xocl/xclmgmt drivers and sudo reboot then lspci -v to see serial controller attached to xdma driver and xdma devices appearing in /dev

[Chris Stratford]

program a firesim bitfile first via vivado hardware manager, load xdma+xvsec, remove xocl/xclmgmt and sudo reboot. It should then appear then you lspci -v

Once its there, never power down your linux box !!! Not sure why loading a bitfile before sudo reboot matters but it seems to fix it.

On Wednesday, 24 September 2025 at 18:57:55 UTC+1 aaron...@gmail.com wrote:

[Chris Stratford]

i think i found why Serial Controller doesnt always appearing. 

I had to use Vivado hardware manager to program a bit file, load the xdma/xvsec drivers, remove xocl/xclmgmt and sudo reboot.

it looks like i need to reprogram the u200 with the bitfile first to ensure after reboot the xdma driver picks up the Xilinx Serial Controller [above] 

otherwise it doesnt appear. Its a bit odd behaviour. Firesim uses the jtag via xdma driver to program the card. 

Anyway, nothing else to add. Good luck

On Thursday, 25 September 2025 at 17:38:44 UTC+1 Chris Stratford wrote:

[iggy12345]

So it looks like there is two different places to get bit files. The bit files listed in the sample_config_hwdb in firesim-staging must not be the same as the one listed in the setup documentation. Once I flashed the fpga using the correct bit stream and sudo rebooted, the fpga popped up with lspci. I got 1 step further in trying to do firesim enumeratefpgas, but now it just crashes when running a python script somewhere. I think maybe it's because I'm using the latest release of firesim, I want to go back and follow your setup for 1.18.0. I will do that and see where I can get next week.

[Chris Stratford]

Sounds promising ! 

I had a riscv cpu and toolchain.

 I had to do a bit of trickery to get the firesim .conda-env riscv-tools same as root of chipyard 

All good fun 

--
You received this message because you are subscribed to the Google Groups "FireSim" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firesim+u...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/firesim/b3498113-3a0d-4e06-87da-951ceb1ab979n%40googlegroups.com.