OpenPiton Linux Boot in Simulation

[Isuru Meegahathenna]

Hi,

 

In my project, I have connected a DDR MIG with AXI4 interface to the openpiton design (with Ariane core) using  noc_axi4_bridge. I am using Vivado generated encrypted DDR memory model to simulate the hello world program with VCS.

There I am loading the hex file to the DDR model using a program loader module. There is a multiplexer logic to choose between program loader and the openpiton chip. This logic first selects program loader to load the program. Once it is loaded, the multiplexer logic selects the chip to execute the program.

 

 

My idea is to simulate linux booting with same setup (Load the linux image to the DDR model using program loader and then boot and run the OS). I have the following questions.

 

1. Is it possible to simulate the OS booting in this setup?
2. Where do I get/ how do I generate the linux image?
3. How do I load it into DDR model?

[Jonathan Balkind]

Hi Isuru,

This email got lost under a large pile after I was travelling. Just getting to it now.

My experience with the full Xilinx memory model is that it is very (very) slow to simulate, much moreso than simulating the rest of the OpenPiton design alone. I would expect it to take a very long time. Simulating with the OpenPiton fake memory instead, we've been able to do linux boots in times on the order of 12 hours, if I'm remembering right.

To get the image, you could use my fork of the ariane-sdk: https://github.com/Jbalkind/ariane-sdk/tree/openpiton-opensbi

You will then need to take the fw_payload.bin and turn it into whatever format is required to load it into DRAM. What format is required by Vivado?

Also just to note, however you attached the image, it is not loading for me in email or on google groups.

Thanks,

Jon

--
You received this message because you are subscribed to the Google Groups "OpenPiton Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpiton+...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/openpiton/3798be75-cb1a-4a1f-8929-9d4e436d6662n%40googlegroups.com.

[Isuru Meegahathenna]

Hi Jon,

Thanks for your reply!

The OpenPiton GitHub page has instructions only on Linux boot on FPGA board. It does not have instructions to do it with simulation.

You mentioned that, you simulated the Linux boot with OpenPiton fake memory.

Can you send me the steps to do the same?

Thanks & Regards,

Isuru Meegahathenna

[Jonathan Balkind]

You have a custom environment so I don't have specific instructions for you. I don't know what format you need to have the image in to load it into your DDR model. What format is it? That would affect the steps needed.

To view this discussion on the web, visit https://groups.google.com/d/msgid/openpiton/261d4955-2741-42e6-9024-bc1c91c53a3en%40googlegroups.com.

[Isuru Meegahathenna]

Hi Jon,

I have 2 things to ask your help with.

1. I have encountered problems when trying to get the image using your sdk (https://github.com/Jbalkind/ariane-sdk/tree/openpiton-opensbi). After I executed the git submodule update --init --recursive command, it reported an error. It seems to be a network failure in the upstream repo. I noticed that the address of the dependent warehouse changed during the build process. The address of git.qemu-project.org was changed to gitlab.com/qemu-project. However, when cloning the repository, I encountered a permission error, which may be related to this. Please see the below screenshots.

      2. This is about the file format of the image needed to load into DDR model. For fake memory, we load a hex file using $readmemh command. DDR model needs the same hex file format.

          Actually, I first want to try the simulation with fake memory. Can you send me the steps to simulate Linux boot with OpenPiton fake memory?

 

Thanks & Regards,

Isuru Meegahathenna

[Jonathan Balkind]

Hi Isuru,

Regarding your first issue: I'm not sure why that's occurring as I do know other folks have been successful in the last few weeks at cloning this, though maybe repos have moved. You could go into the submodules and change the .gitmodules then run `git submodule sync --recursive`.

Regarding the second, if you compile and create fw_payload.elf (which is usually built along with fw_payload.bin), you can probably use -precompiled to have sims directly pull it up. Otherwise, you could look at the contents of rv64_img and run a similar set of commands to create a mem.image file from fw_payload.bin, then run sims without a test argument and it will pick up that mem.image file.

Thanks,

Jon

To view this discussion on the web, visit https://groups.google.com/d/msgid/openpiton/8791a1b0-4974-4a46-881a-9ff534922d17n%40googlegroups.com.

[Isuru Meegahathenna]

Hi Jon,

I was able to clone and update the git repo using your instructions. Thank you.

I am working on generating the Linux image. I encountered some buildroot issues and fixing them.

Previously, I have cloned the SDK repository by openhwgroup (cva6-sdk) and noticed there are already fw_payload.bin and fw_payload.elf files. Is it possible to use this image to boot Linux in simulation?

Regarding the answer to the second issue mentioned above, I am not very clear on using the sims command with -precompiled switch.

I referred the OpenPiton GitHub page Readme (Running RISC-V Tests and Benchmarks section) and did the following. But I think this is not the correct approach.

1. Copy the above mentioned fw_payload.bin and fw_payload.elf files (from cva6-sdk repo) to $ARIANE_ROOT/tmp/riscv-tests/build/isa
2. cd $PITON_ROOT/build
3. sims -sys=manycore -x_tiles=1 -y_tiles=1 -vcs_build -ariane
4. sims -sys=manycore -x_tiles=1 -y_tiles=1 -vcs_run -ariane -precompiled

Can you send me the steps on how use sims -precompiled to directly pull up fw_payload.elf ?

Thanks in advance.

Regards,

Isuru Meegahathenna.

[Jonathan Balkind]

Hi Isuru,

The image from cva6-sdk is unlikely to work as it has a different configuration. It might boot partially but I would expect some problems.

The approach you mention is just what I'm suggesting. You would need to add fw_payload.elf in command 4. Also it may have to be named .riscv instead of .elf. You can check the code in sims,2.0 to be sure how -precompiled is being used.

Thanks,

Jon

To view this discussion on the web, visit https://groups.google.com/d/msgid/openpiton/1be54879-97f5-4143-978d-49c176ad106bn%40googlegroups.com.

[Isuru Meegahathenna]

Hi Jon,

I was able to run it in simulation using the method you suggested. Thank you. 

However, as you mentioned it has a different configuration. How do we give the configuration of our OpenPiton SoC during the bootable Linux image generation?

Thanks & Regards,

Isuru Meegahathenna

[Jonathan Balkind]

Oh cool. The configuration is already in the device tree which is built into the bootrom of the simulation model or FPGA bitstream. I expected they'd have some other configs that could cause issues but I guess not.

To view this discussion on the web, visit https://groups.google.com/d/msgid/openpiton/ab80add2-9958-429f-bd6d-25049f8908f8n%40googlegroups.com.

[Isuru Meegahathenna]

Hi Jon,

Sorry about the miscommunication. I was able to start the simulation but it did not work (with the image from cva6-sdk).

I was able to generate the Linux image using  your sdk (https://github.com/Jbalkind/ariane-sdk/tree/openpiton-opensbi) and start the simulation using sims -precompiled flow.

1. sims -sys=manycore -x_tiles=3 -y_tiles=1 -vcs_build -ariane
2. sims -sys=manycore -x_tiles=3 -y_tiles=1 -vcs_run -ariane fw_payload.riscv -precompiled

After the mem.image was generated, I terminated the sims run command and run it again with VCS. I am using OpenPiton fake memory.

I got the following prints but simulation does not progress beyond this point.

From what I understand, 

• The Linux image was generated to be boot from the SD card.
• I am running simulation by booting it from fake memory.

Is there any modifications I should do in the OpenPiton design to get this working?

Thanks & Regards,

Isuru Meegahathenna

[Jonathan Balkind]

This seems to be an issue that people are seeing some of the time (but not all of the time). On zulip #u-boot, Tiago said they managed to get boot working using this somewhat more updated fork of ariane-sdk, though they were trying on FPGA: https://github.com/Tiago-R/cva6-sdk/tree/buildroot_bump

You could try giving that one a shot. Otherwise if it doesn't work I have another version you could try tinkering with that I can give you a pointer to.

Thanks,

Jon

To view this discussion on the web, visit https://groups.google.com/d/msgid/openpiton/53c055c4-05c0-4498-95c7-a9d22a1d5a8fn%40googlegroups.com.

[Isuru Meegahathenna]

Hi Jon,

I am getting the same issue with the image generated by Tiago's fork also.

Can you point me to the other version you mentioned?

And also, when you tried Linux boot in simulation, did you do any changes to the OpenPiton design?

Thanks & Regards,

Isuru Meegahathenna

[Jonathan Balkind]

Hm this is puzzling. I've seen issues on FPGA where the user might need to change the version of opensbi (0.8/0.9/1.0) but usually it gets resolved and the branch you're using was from a user who is actively running on FPGA without issue. I'm wondering if there could be an issue with riscv cross-compiler versions?

Could you tell me the steps you're following with the above? Just in case this is happening for some other reason (e.g. from the output are you sure that the linux kernel was built and included in the fw_payload.bin blob?)

The branch I mentioned is here (https://github.com/cohort-project/ariane-sdk/tree/a-vinod_g2_cohort) but note that to use it you would need to disable the cohort patch in the buildroot_config: https://github.com/cohort-project/ariane-sdk/blob/a-vinod_g2_cohort/configs/buildroot_defconfig#L462

Thanks,

Jon

To view this discussion on the web, visit https://groups.google.com/d/msgid/openpiton/35494f36-73c6-440e-a2bb-2f7ada0aa2a2n%40googlegroups.com.

[Isuru Meegahathenna]

Hi Jon,

I am following the exact instructions on https://github.com/Tiago-R/cva6-sdk/tree/buildroot_bump to create the Linux image.

1. Clone the repo.
2. Change .gitmodules to correctly point to the opensbi repo (URL g...@github.com:Tiago-R/opensbi.git did not work. Used URL https://github.com/Tiago-R/opensbi.git)

1. git submodule update --init --recursive
   

1. make fw_payload.bin

This created the image successfully.

To load the linux image on OpenPiton,

1. cp fw_payload.elf $ARIANE_ROOT/tmp/riscv-tests/build/isa/fw_payload.riscv

1. sims -sys=manycore -x_tiles=3 -y_tiles=1 -vcs_build -ariane

1. sims -sys=manycore -x_tiles=3 -y_tiles=1 -vcs_run -ariane -precompiled fw_payload.riscv
   
2. Terminate sims and run with just VCS

Simulation hangs after the OpenSBI prints (the screenshot I sent in previous mail). I inspected the created mem.image file and Linux kernel image should be included there.

I checked the trace log for boot Hart and found the following exception.

I found an older mail thread discussing about a similar issue (Title: Linux hangs in special circumstances). So, the problem could be with the Ariane version?

Therefore, I updated my openpiton repo and ariane with the latest version.

But I am still getting the same issue.

Could the problem really be with Ariane? 

The trace log reports the exception for address 80200080 and then it shows fetching instructions from address ffffffe000000080. I compared the fetched instruction with mem.image file and it matches the instruction that should be mapped too address 80200080.

 

(line #65539 maps to 80200000 and line #65543 maps to 80200080)

I have got this "Instruction Page Fault" exception with every Linux image I have tried. 

Thanks & Regards,

Isuru Meegahathenna

[Jonathan Balkind]

Are you using the latest commit from openpiton or openpiton-dev? I believe we should be pointing at a new enough version of ariane that we contributed a fix to a similar issue to this one (though it may not specifically be this issue).

One thing you could try is Joan's ariane https://github.com/jfarresg/cva6/commits/master/ and corresponding openpiton fork to get a newer version of the core (I think from last Dec or so).

Thanks,

Jon

To view this discussion on the web, visit https://groups.google.com/d/msgid/openpiton/2112bfd2-1b01-42a7-a753-3aa5fac87c32n%40googlegroups.com.

[Dimuthu Asiri]

Hi Jon,

Thanks for the guidance given so far.

I am also working with Isuru in this task . I was able to load the Linux kernel in the simulation (with VCS) with the latest Openpiton Design.

Although it is progress further, I am getting this run /init process error.  Could you please help me resolving this issue? 

Thanks 

Dimuthu

[Jonathan Balkind]

Hi Dimuthu,

Could you check in buildroot/output/images/ and see if there is a file called something like rootfs.cpio there? This sort of error looks like either the rootfs wasn't compiled properly or it wasn't properly included into the Linux Image file.

Also what size is the fw_payload.bin? And could you send the complete terminal output?

Thanks,

Jon

To view this discussion on the web, visit https://groups.google.com/d/msgid/openpiton/70ed20ee-0b34-44cd-b403-464e3d56f1ddn%40googlegroups.com.

[Dimuthu Asiri]

Hi Jon, 

In the previous run I was running modified images (removed MMU) from ariane_sdk ( (https://github.com/Jbalkind/ariane-sdk/tree/openpiton-opensbi).

With the default image (without any modifications) I was able to run up to NFS Preparation skipped stage,. But I am getting some prints (not readable) in the terminal (see following Image). I have attached the terminal output as well.

What is the reason for that? Do you have any golden terminal output that I can use as a reference?

Simulation is still running. I am expecting Buildroot login: message in the terminal.

Thanks,

Dimuthu

[Jonathan Balkind]

Hi Dimuthu,

This output looks good! I think the # you see is the shell prompt you were expecting. I disabled the login prompt on my branch because it wasn't doing anything.

Note that we haven't wired up input for the UART so if you're looking to take input you'll need to add something further. If you just want it to run another command, you can probably modify the init script or overwrite it by putting your own under the rootfs directory.

I think the unprintable characters are probably because our fake_uart.v just prints whatever gets stored to address 0 of the UART. That's the transmit holding register or THR, but when the UART is in another mode, it also represents another register and so probably the linux driver is putting the driver into that mode and storing something into the register. You could read a bit more about how the UART16550 works and code up a check to track the mode and not print when it's in the other mode.

Glad to see it working :)

Thanks,

Jon

To view this discussion on the web, visit https://groups.google.com/d/msgid/openpiton/98b84d6e-7355-46be-9a9a-f1a6fb6aad03n%40googlegroups.com.

[Dimuthu Asiri]

Hi Jon,

Thanks for the support. I modified the init script as you mentioned.

Further I need to try booting Ubuntu 16.04 on Openpiton. Have you try such thing before?

What will be the steps generate a image with  Ubuntu 16.04?

Thanks,

Dimuthu

[Jonathan Balkind]

Hi Dimuthu,

Ubuntu 16.04 is much too old to run on RISC-V. Why exactly do you want to run that?

So far with our u-boot flow we have been able to boot both Debian and Fedora images on FPGA. However, to do the same in simulation would have several challenges. For one, I'm not sure how well such a large memory image would be handled. For another, when running a distro there is a lot more time in the boot process after the point that you simulated to. It could potentially take twice as long to get to a terminal.

Are you absolutely sure that you need to run a distro under RTL simulation?

Thanks,

Jon

To view this discussion on the web, visit https://groups.google.com/d/msgid/openpiton/aacc035e-c74a-4a49-af1c-0854e5ebfe48n%40googlegroups.com.