Hammerblade on Alveo U250

[Amit Hiremath]

Hello All,

I am porting  hammerblade-example to Alveo U250 FPGA Data Center Card, when I run the design in the board, I am getting following output:

// bsg_zynq_pl: be sure to run as root
INFO: ps.cpp: reading three base registers
INFO: ps.cpp: dram_base=1
Creating Bitbang Driver: 4 16 1
INFO: Reset Tag Master
Resetting Tag Client 0
Setting Tag Client 0<-1
Setting Tag Client 0<-0
Idling for 50
INFO: ps.cpp: calling allocate dram with size 209715200
bsg_zynq_pl: Allocated dummy DRAM
INFO: ps.cpp: received 0x7fb1d47b9010 (phys = 7fb1d47b9010)
INFO: ps.cpp: wrote and verified base register
INFO: Starting NBF load
INFO: ps.cpp: nbf finish command, line 8129
INFO: Waiting for credit drain
INFO: Credits drained
INFO: ps.cpp: Starting scan thread
INFO: ps.cpp: Starting MC i/o polling thread
INFO: Errant request packet: 1260 0

Manycore>> Hello from core 0, 0 in group origin=(0,0).
Manycore>> Values in DRAM:ffffffff,00000001,0000000f,80000000,

Manycore stderr>> Hello!

Interrupt array at 0x0
INFO: MC finish packet received 1
INFO: ps.cpp: Starting BP i/o polling thread
mcause: 0x0000000000000004
mtval: 0xffffffffffdc0007
mepc: 0x0000000080004694
Unhandled machine-mode trap
INFO: BP finish packet received 1
bsg_zynq_pl: done() called, exiting

It is raising exception as shown in red,  the above output shown is running at 100 MHz  I did change frequency of design to 65MHz, 75MHz in these cases, it is stuck at  INFO: ps.cpp: Starting BP i/o polling thread  I am wondering, what's going wrong?  And where do I start debugging design?

Any guidance could be of great help.

Thanks,

-Amit

[Dan Petrisko]

Hi Amit,

Are you able to successfully run this same program in co-simulation? Are you seeing any critical warnings for timing?

I don't know how your shell<->host communication works but if frequency affects the behavior then that suggests a bug in your timing constraints. How are you handling the clock domain crossing from the PCIE to DUT clock?

-Dan

--
You received this message because you are subscribed to the Google Groups "black-parrot" group.
To unsubscribe from this group and stop receiving emails from it, send an email to black-parrot...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/black-parrot/68d343a3-84d8-42b7-878e-ec32d505cbd3n%40googlegroups.com.

[Amit Hiremath]

Hello Dan,

I think,  I am able to successfully run co-simulation example, it is matching with vcs simulation the output of vcs is:

bsg_zynq_pl: Exiting reset
bsg_zynq_pl: Exiting reset
bsg_zynq_pl: Exiting reset

INFO: ps.cpp: reading three base registers

INFO: ps.cpp: dram_base=0
INFO: Creating Bitbang Driver: 0xea8560 40000004 16 1
INFO: Reset Tag Master
INFO: Resetting Tag Client 0
INFO: Setting Tag Client 0<-1
INFO: Setting Tag Client 0<-0
INFO: Idling for 50
INFO: ps.cpp: calling allocate dram with size 134217728
INFO: bsg_zynq_pl: Allocated dummy DRAM
INFO: ps.cpp: received 0x2aaabf8b3010 (phys = 2aaabf8b3010)

INFO: ps.cpp: wrote and verified base register
INFO: Starting NBF load
INFO: ps.cpp: nbf finish command, line 8129
INFO: Waiting for credit drain

[INFO][RX] Unfreezing tile t=19566100000, x= 2, y= 2

INFO: Credits drained
INFO: ps.cpp: Starting scan thread
INFO: ps.cpp: Starting MC i/o polling thread
INFO: Errant request packet: 1260 0

Manycore>> Hello from core 0, 0 in group origin=(0,0).
Manycore>> Values in DRAM:ffffffff,00000001,0000000f,80000000,

Manycore stderr>> Hello!

Interrupt array at 0x0
INFO: MC finish packet received 1
INFO: ps.cpp: Starting BP i/o polling thread

Hello World!

INFO: BP finish packet received 1

INFO: bsg_zynq_pl: done() called, exiting
$finish called from file "/home/sonal/ViBram/zynq-parrot/cosim/v/bsg_nonsynth_zynq_testbench.sv", line 471.
$finish at simulation time 20506525001
V C S S i m u l a t i o n R e p o r t
Time: 20506525001 ps

except that in Alveo I am not getting  Hello World!  it is stuck at  INFO: ps.cpp: Starting BP i/o polling thread

However, I successfully  ran black-parrot-minimal example in Alveo. 

I think shell<-->host is proper. I haven't changed in anything hardware designs/DUT, except shell_read and shell_write, which I am communicating through PCIe slot. 

For 60MHz, 65Mhz and 75Mhz designs/DUT, I did not get critical timing warnings for timings, however,  I did get  critical timing warnings for 100MhZ design because of clock skew and design/DUT is not meeting timing requirements for 100MHz. 

In case of clock domain crossing from PCIe to DUT, I am using clocking wizard IP from vivado, which divides 300MHz PCIe clock to   60MHz, 65Mhz, 75Mhz and 100Mhz or any frequency of your choice. 

-Amit

[Amit Hiremath]

Hello Dan,

I ran  black-parrot-minimal  co-simulation example it is successfully running on Alveo U250.  I haven't changed shell <-> host communication, neither any of the hardware design/DUT.  The only change is in shell_read/write, I am communicating through PCIe slot. I think the communication to Hammerblade is proper because I am getting hello! from manycore (in case of hammerblade example) and also hello world! from black-parrot-minimal example. 

I haven't seen any critical warnings for timing for 75MHz, 60MHz, 65MHz designs/DUT, however, I did get critical timing warnings for 100MHZ design for clock skew. 

In case of PCIE to DUT clock domain crossing, I am using clock wizard, which I divided 300MHz PCIe frequency to   75MHz, 60MHz, 65MHz, 100MHz to Design/DUT

-Amit

On Tuesday, February 25, 2025 at 11:14:53 AM UTC+5:30 Dan Petrisko wrote:

[Amit Hiremath]

Hello Dan,

I think,  I am able to successfully run co-simulation example, it is matching with vcs simulation the output of vcs is:

bsg_zynq_pl: Exiting reset
bsg_zynq_pl: Exiting reset
bsg_zynq_pl: Exiting reset

INFO: ps.cpp: reading three base registers

INFO: ps.cpp: dram_base=0
INFO: Creating Bitbang Driver: 0xea8560 40000004 16 1
INFO: Reset Tag Master

INFO: Resetting Tag Client 0

INFO: Setting Tag Client 0<-1
INFO: Setting Tag Client 0<-0
INFO: Idling for 50
INFO: ps.cpp: calling allocate dram with size 134217728
INFO: bsg_zynq_pl: Allocated dummy DRAM
INFO: ps.cpp: received 0x2aaabf8b3010 (phys = 2aaabf8b3010)

INFO: ps.cpp: wrote and verified base register
INFO: Starting NBF load
INFO: ps.cpp: nbf finish command, line 8129
INFO: Waiting for credit drain

[INFO][RX] Unfreezing tile t=19566100000, x= 2, y= 2

INFO: Credits drained
INFO: ps.cpp: Starting scan thread
INFO: ps.cpp: Starting MC i/o polling thread
INFO: Errant request packet: 1260 0

Manycore>> Hello from core 0, 0 in group origin=(0,0).
Manycore>> Values in DRAM:ffffffff,00000001,0000000f,80000000,

Manycore stderr>> Hello!

Interrupt array at 0x0
INFO: MC finish packet received 1
INFO: ps.cpp: Starting BP i/o polling thread

Hello World!

INFO: BP finish packet received 1

INFO: bsg_zynq_pl: done() called, exiting
$finish called from file "/home/sonal/ViBram/zynq-parrot/cosim/v/bsg_nonsynth_zynq_testbench.sv", line 471.
$finish at simulation time 20506525001
V C S S i m u l a t i o n R e p o r t
Time: 20506525001 ps

except that in Alveo I am not getting  Hello World!  it is stuck at  INFO: ps.cpp: Starting BP i/o polling thread

However, I successfully  ran black-parrot-minimal example in Alveo.

I think shell<-->host is proper. I haven't changed in anything hardware designs/DUT, except shell_read and shell_write, which I am communicating through PCIe slot.
For 60MHz, 65Mhz and 75Mhz designs/DUT, I did not get critical timing warnings for timings, however,  I did get  critical timing warnings for 100MhZ design because of clock skew and design/DUT is not meeting timing requirements for 100MHz.

In case of clock domain crossing from PCIe to DUT, I am using clocking wizard IP from vivado, which divides 300MHz PCIe clock to   60MHz, 65Mhz, 75Mhz and 100Mhz or any frequency of your choice.

-Amit

On Tuesday, February 25, 2025 at 11:14:53 AM UTC+5:30 Dan Petrisko wrote: