HLS4ML - IO planning

[Charles-Etienne Granger]

Greetings!

We are currently trying to customize the HDL code generated by HLS4ML. Here are the steps we did so far :

1. Generate the HDL with HLS4ML
2. Create a new project with the sources files
   
3. Map the necessary I/O pins
   
4. Run the Synthesis
5. Run the Implementation
6. Generate a bitstream  -> always fail because of the 3rd step
   

For the third step, we are kind of lost. We dont really know to which pins we should map the ports or the signal type/voltage level required.

We are using a KCU105 Xilinx Dev Board (if this info helps).

Is there any documentation available for this use case of HLS4ML ?  

Cheers,

Charles-Etienne Granger

Research Professional

University of Sherbrooke

IMPETUS

Parc Innovation, pavillon P2

3000 boul. Université

Sherbrooke, Qc, Canada J1K 0A5

Cellphone : 1+ (438) 885-4344

[sioni summers]

Hi,

It sounds like you're now trying to execute the NN on your device, is that right? If so, can I ask where your input data will eventually be sourced from? For example, will it come from a host PC over PCIe, or maybe ethernet? Or some peripheral device like a camera that you have connected to the board? In any case, unless you are really setting the NN inputs with voltages on the GPIO pins, I suspect you don't want to be connecting those NN ports directly to the FPGA pins.

For these types of flows, you can use the new VivadoAccelerator backend in hls4ml to create an IP with various AXI interfaces (AXI-lite Slave/Master, AXI Stream), and then use the other IPs from Xilinx to connect to the outside world with Block Design / IP Integrator in Vivado.

You might take inspiration from this Block Design tcl file from the VivadoAccelerator backend to start from. That one will connect the AXI Stream NN to the PS of a Zynq ZCU104 kit. Your design would need to be a bit different since there's no PS/Zynq on the KCU105. You can probably find an example design for the KCU105 with a DMA/Ethernet AXI Stream IO online for the rest. We plan to have a complete solution for kits more like yours with an FPGA (as opposed to a Zynq SoC) but we don't have it just yet. 

I hope that helps, feel free to get back to us with any more queries.

Thanks,

Sioni

--
You received this message because you are subscribed to the Google Groups "hls4ml help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hls4mlhelp+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/hls4mlhelp/YT1PR01MB42491AD74FE12C29F1060B83C2A39%40YT1PR01MB4249.CANPRD01.PROD.OUTLOOK.COM.

[Charles-Etienne Granger]

Hello Sioni!

How are you?

You are right we are trying to execute the NN on our FPGA! As you guessed it, the data will be from the PCIe from a PC first and later from another FPGA that will be used to capture and prepare the data for the NN. The data will then be sent to a PC over Ethernet.

Concerning the pins of the neural network, this is what we were unsure of : which ports/signals did what, and why we were getting so many port errors (400ish) while trying to generate a bitstream.

We will look into the script from the VivadoAccelerator backend and try to make it work with our KCU105 to generate AXI Stream interfaces for our NN with an online example.

In the meantime, do you have an example of project to generate a bitstream with hls4ml for validation purposes ? It can be with any FPGA, we are still tinkering with HLS4ML 🙃.
Please note that we are not using Vivado HLS for this since we only have access to Vivado 2020.2 with our research group for now. The HLS4ML code runs on your docker image that contains a compatible vivado version.

Thank you for your time!

Charles

---

De : sioni summers <sions...@googlemail.com>
Envoyé : 24 septembre 2021 08:12
À : hls4m...@googlegroups.com <hls4m...@googlegroups.com>; Charles-Etienne Granger <Charles-Eti...@usherbrooke.ca>
Cc : hls4m...@gmail.com <hls4m...@gmail.com>; Mohammad Mehdi Rahimifar <Mohammad.Meh...@usherbrooke.ca>
Objet : Re: [hls4ml.help] HLS4ML - IO planning

 

[sioni summers]

Hi Charles,

I've attached a notebook that can be added like a "part 6" to the hls4ml tutorial to show how to take one of those models and generate a bitstream for a pynq-z2 board. Note that the VivadoAccelerator backend that is used is not part of a release just yet, this is using an up to date hls4ml master branch (commit 3111391). 

I included a screenshot of the "Block Design", which is what I think you will want to try to create a version of for the KCU105. You can start by creating a new Vivado project, selecting the KCU105 from the boards menu, then "Create new Block Design". It will be a bit different to our current design which is for Zynq devices, but we can work towards supporting the board you have. It seems like your final design will be a bit different anyway since the input/output will come from/go to different sources. But you can create the design in a similar way by using appropriate Xilinx IPs that take AXI data for DMA over PCIe, and ethernet.

Thanks,

Sioni

[Charles-Etienne Granger]

Hey there Sioni,

This is just a follow up to see if you had the time to look it up. We are still not able to generate the constraint files required for the pynq board with the code you sent us.
In the meantime, we will create custom docker image with vivado hls 2018 so that we can try the normal way of doing things with root access.

Cheers,

Charles-E.

 

From: Charles-Etienne Granger <Charles-Eti...@USherbrooke.ca>
Sent: October 25, 2021 11:36 AM
To: sioni summers <sions...@googlemail.com>
Cc: Mohammad Mehdi Rahimifar <Mohammad.Meh...@USherbrooke.ca>
Subject: Re: [hls4ml.help] HLS4ML - IO planning

 

Hi Sioni,

 

Just a friendly follow-up. Have you been able/had the time to try our use case? If not we are here to help if you need with anything.

Also, do you think you can give us an estimate on when you'll be able to look it up?

Thanks,
Charles-E.

 

---

De : Charles-Etienne Granger
Envoyé : 14 octobre 2021 12:52
À : sioni summers <sions...@googlemail.com>
Cc : Mohammad Mehdi Rahimifar <Mohammad.Meh...@USherbrooke.ca>
Objet : RE: [hls4ml.help] HLS4ML - IO planning

 

No worries take your time,

As we said the block design works with our little work around (since we have no root access to place the files in /opt/Xilinx/…/boards) but our floorplan is empty in the PL and we cant generate the bitstream since there was no constraint files generated for the board by the scripts.

 

If you need anything else don’t hesitate to contact us!

Cheers,

Charles-E.

 

 

De : sioni summers <sions...@googlemail.com>
Envoyé : October 14, 2021 12:05 PM
À : Charles-Etienne Granger <Charles-Eti...@usherbrooke.ca>
Cc : Mohammad Mehdi Rahimifar <Mohammad.Meh...@usherbrooke.ca>

Objet : Re: [hls4ml.help] HLS4ML - IO planning

 

Hi Charles,

 

I'll need some time to look into this for you. In my experience, placing the board files under the /opt/Xilinx/.../boards directory is enough to complete the implementation, it provides all the necessary constraints. However, I've never tried using the method of setting the path in the tcl like you mentioned. I'll try it that way myself and see if I can replicate what you see. It could be that a different error causes your design to be "optimized away", I have seen that before... I'll let you know if I find anything.

 

Thanks,

Sioni

 

On Thu, Oct 14, 2021 at 3:35 PM Charles-Etienne Granger <Charles-Eti...@usherbrooke.ca> wrote:

Hi Sioni,

Do you happen to have the constraints files of the Pynq-z2 board that fit the part_6 of the tutorial you sent us ?

To make the tutorial work we needed to add the board file of the pynq in vivado. Since we did not have the root access to copy in the /opt/Xilinx/…/boards directory, we simply added the file in the jupyter interface and added the following line at the start of  the design.tcl script:

              set_param_board.repoPaths [list “/repo/with/board/file”]

 

Now we have the same bloc design as you but there were no constraints generated with it so the floorplan of the device is empty. From my understanding, we need a constraint file for each out of context (OOC) modules so that they do not need to be resynthesized each time we change the top-level design.

 

Cheers,
Charles-E.

 

De : sioni summers <sions...@googlemail.com>
Envoyé : September 28, 2021 10:11 AM
À : Charles-Etienne Granger <Charles-Eti...@usherbrooke.ca>

Objet : Re: [hls4ml.help] HLS4ML - IO planning

 

Hi Charles,

 

It's really just because they were a bit easier to support to start with. They're pretty nice to work with since the PS part can run an operating system. We do plan to have similar push-button workflows in place for non-MPSoC devices, like Alveo accelerators and PCIe-based kits like yours. So, it's on the way but not there yet! The Pynq-Z2 is quite a nice board to work with for prototyping though, I recommend it!

 

Thanks,

Sioni

 

On Tue, Sep 28, 2021 at 3:58 PM Charles-Etienne Granger <Charles-Eti...@usherbrooke.ca> wrote:

Hi Sioni, 

Thank you for your reply! 

 

Just a quick question we have for you, Is there a reason to why you are using FPGAs with MPSoC (Zynq, Pynq) ? 

 

From our understanding, those have much less DSPs available for neural networks. 

 

Cheers, 

Charles-E.

---

De : sioni summers <sions...@googlemail.com>
Envoyé : 28 septembre 2021 09:40
À : Charles-Etienne Granger <Charles-Eti...@usherbrooke.ca>
Cc : hls4m...@googlegroups.com <hls4m...@googlegroups.com>; hls4m...@gmail.com <hls4m...@gmail.com>; Mohammad Mehdi Rahimifar <Mohammad.Meh...@usherbrooke.ca>

[sioni summers]

Hi Charles,

I haven't been able to look into it further yet I'm afraid! I'm still a bit confused at why you need some extra files, if the board files are accessed they would provide the constraints. 

Thanks, and sorry I can't offer any more info,

Sioni

[Charles-Etienne Granger]

Hi Sioni,

I am not sure I understood your last message... Are you saying that HLS4ML should generate the constraints from the board files or that we would need a constraint file prior to running HLS4ML ? 

Our understanding up to now was that HLS4ML would generate the constraint files. If that's not the case,  could you please explain to us what's the usual way of adding constraints to an HLS4ML project ?

Cheers,
Charles-E.

---

De : sioni summers <sions...@googlemail.com>
Envoyé : 5 novembre 2021 12:26
À : Charles-Etienne Granger <Charles-Eti...@usherbrooke.ca>
Cc : hls4m...@gmail.com <hls4m...@gmail.com>; Mohammad Mehdi Rahimifar <Mohammad.Meh...@usherbrooke.ca>

[sioni summers]

Hi Charles,

What I mean is that in my experience, all necessary constraints are provided just by putting the board files in the directory that Vivado can access, and pointing the project to the board (which hls4ml will do in the design.tcl script). hls4ml doesn't create any constraints files itself.

After you have "set_param_board.repoPaths [list “/repo/with/board/file”]", can you select pynq-z2 as the target in a Vivado project, like this:

That's basically what our tcl file does in script, and if the board is found that will define the pin constraints. Clock constraints I think are automatically generated when we make the Board Design in Vivado.

I did see that you can download the pin constraints for the pynq-z2 from the manufacturer's webiste: https://www.tul.com.tw/productspynq-z2.html (the "Master XDC" link on that page). But just to repeat, I've always been able to use the "board files" download with nothing else added.