FireSim Infrastructure Setup Issue - Incorrect PCIe Device (0000:05:00:1)

[Ishmam]

I'm setting up FireSim on a Xilinx VCU118 FPGA, but firesim infrasetup keeps trying to modify permissions for a non-existent PCIe device 0000:05:00:1, while my actual FPGA is located at 0000:05:00:0.

What I have verified so far: 

- Checked PCIe devices with lspci -vvv -d 10ee:903f, which correctly shows:
05:00.0 Serial controller: Xilinx Corporation Device 903f
Kernel driver in use: xdma
There is no 0000:05:00:1 device present.

- Checked /opt/firesim-db.json, which correctly contains:
{
  "fpgas": [
    {
      "pcie_id": "0000:05:00:0",
      "platform": "xilinx_vcu118",
      "hardware": "xilinx_vcu118_firesim_rocket_singlecore_4GB_no_nic"
    }
  ]
}

- Searched for 0000:05:00:1 references:
Ran grep -r "0000:05:00:1" ~/chipyard/sims/firesim/
Found occurrences in FireSim logs but not in configuration files.

Why does FireSim still reference 0000:05:00:1 even though /opt/firesim-db.json only lists 0000:05:00:0? Are there additional files that could be hardcoding the incorrect PCIe device?

Any insights would be greatly appreciated!

[Karl Hallsby]

This happens because a Python script has an incorrectly-hardcoded PCIe function number. The diff below should fix this.

diff --git a/deploy/runtools/run_farm_deploy_managers.py b/deploy/runtools/run_farm_deploy_managers.py
index 3b9491e6f..01a3d7474 100644
--- a/deploy/runtools/run_farm_deploy_managers.py
+++ b/deploy/runtools/run_farm_deploy_managers.py
@@ -1198,7 +1198,7 @@ class XilinxVCU118InstanceDeployManager(InstanceDeployManager):

                 # TODO: is hardcoded cap 0x1 correct?
                 # TODO: is "Partial Reconfig Clear File" useful (see xvsecctl help)?
-                bdfs = [ { "busno": "0x" + i[:2], "devno": "0x" + i[3:5], "capno": "0x1" } for i in collect.splitlines() if len(i.strip()) >= 0 ]
+                bdfs = [ { "busno": "0x" + i[:2], "devno": "0x" + i[3:5], "capno": "0x0" } for i in collect.splitlines() if len(i.strip()) >= 0 ]
                 bdf = bdfs[slotno]

                 busno = bdf['busno']

[Lyana]

Hi Ishmam, 

Although I am not at this part yet, I am interested how you managed to properly configure the PCIe device (lspci -vvv -d 10ee:903f). In my case, it doesn't show anything, but it does recognize the Xilinx device. I believe this is because the progamming the configuration memory device for the Alve0 U250 cards did not succeed ( FPGA Setup — FireSim HEAD documentation). When doing it from Vivado Labs, the entire screen freezes and I just have to manually shut down the entire computer and card and do a cold boot. 
Do you maybe still remember the steps you did with the VCU118 from opening the biststream to programming the config memory to ensure with  lspci -vvv -d 10ee:903f?

Your help is very much appreciated!

Op zondag 16 maart 2025 om 23:05:33 UTC+1 schreef Ishmam: