EDK2 on RISC-V Sophgo SG2042 platform

[li bing]

Topics from:Academy of Intelligent Innovation, Shandong Universiy, China.P.R.

 First of all, thank you for your attention to the work of our team on EDK2. The following is a brief introduction to some of the work done by our team on EDK2 on SG2042.

Our work can be roughly divided into several phases: Note: All the work is done on the SG2042 EVB development board.

1. Testing and development work on bootflow provided Sophgo.
   
2. Implementation according to the initial EDK2 design for RISC-V.
   Implementation steps can be found here:   https://github.com/tianocore/edk2-platforms/blob/master/Platform/RISC-V/PlatformPkg/Readme.md
   Implementation content can be found here: https://github.com/AII-SDU/AII-edk2-platforms/tree/main/Platform/Sophgo/SG2042Pkg
3. Establishment of the China UEFI on RISC-V working group to discuss the implementation of UEFI on RISC-V.
   
4. Incorporating feedback from the open-source community, discussing implementation plans with the working group, and ultimately confirming the adoption of the separation approach between openSBI and EDK2, actively submitting it to the master branch. 
   Implementation content can be found here:https://github.com/AII-SDU/edk2-platforms/blob/devel-Sophgo/SG2042Pkg/Platform/Sophgo/SG2042Pkg/About_Sophgo_platform.md
   
5. Currently conducting testing and adjustments while keeping track of the community's progress.

Current progress and status: 

• We are able to successfully boot and run Linux OS on the SG2042 EVB testing board. The general boot process is as follows: ZSBL + FSBL + openSBI + EDK2 + GRUB2 + Linux OS. The boot medium is SD card, where ZSBL is loaded and executed by an auxiliary MCU, performing initial initialization operations such as DIMM initialization. FSBL then continues with further initialization tasks. OpenSBI is used to initialize the hardware platform and kernel. Additionally, we have upgraded the OpenSBI from version V0.9 to V1.2 to align with community progress. EDK2, running in S-mode, handles configuration operations, primarily completing the SD card driver in the DXE stage, while PCIe driver testing and follow-up work are planned. GRUB2 is used to load the operating system, facilitating subsequent operations.

Current limitation:

• PCIE driver is not currently supported.
• SG2042 (Xuantie C920) MMU can be enabled in SV39 mode, but there are bugs with exception handling and MMC that need to be fixed, so MMU is not currently enabled. As mentioned in https://groups.google.com/a/riscv.org/g/fw-exchange/c/wKYnYBNmPr4/m/nY69VnxXAQAJ

These are the main milestones we have achieved so far, and we will actively monitor the progress of the community. Future plans include secure boot and multiArch support. We are looking forward to your feedback and suggestions, and I hope this feature will be a beneficial addition to the project.

Thanks to tp...@ventanamicro.com\
sun...@ventanamicro.com\
yon...@intel.com\
evan...@intel.com
Give us guidance and opinions.

Thanks,
China.P.R. SDU-AII Team

[Li, Yong]

Congratulations to SDU Team and China UEFI on RISC-V working group, thanks for sharing and really impressive work😊

 

Thanks,

Yong Li

[libin...@outlook.com]

Thank you, Yong Li, for affirming our work. We remain committed to accelerating the development of RISC-V firmware and related projects, and we are in the process of establishing a campus open-source community to further enhance the RISC-V ecosystem.

Our team (SDU Team) is planning to submit our current results to edk2-platforms. We highly value everyone's input and would greatly appreciate your feedback, which we will use to make improvements promptly.

Meanwhile, we have addressed the comments provided by sun...@ventanamicro.com and have already submitted the necessary revisions. We are truly grateful for these insightful comments, and we are dedicated to making continuous enhancements.

Thanks,
China.P.R. SDU-AII Team

[Sunil V L]

Hi!,

I left few comments in the PR. Appreciate your work on this and glad you
moved to payload design.

Is there a reason you moved to opensbi 1.2 instead of latest 1.3.1? If
no restrictions, I recommend to use 1.3.1.

Thanks,
Sunil

On Thu, Aug 17, 2023 at 06:14:34PM -0700, libin...@outlook.com wrote:
> Thank you, Yong Li, for affirming our work. We remain committed to
> accelerating the development of RISC-V firmware and related projects, and
> we are in the process of establishing a campus open-source community to
> further enhance the RISC-V ecosystem.
>
> Our team (SDU Team) is planning to submit our current results to
> edk2-platforms. We highly value everyone's input and would greatly
> appreciate your feedback, which we will use to make improvements promptly.
>
> Meanwhile, we have addressed the comments provided by
> sun...@ventanamicro.com and have already submitted the necessary
> revisions. We are truly grateful for these insightful comments, and we are
> dedicated to making continuous enhancements.
>
> Thanks,
> China.P.R. SDU-AII Team
> 在2023年8月17日星期四 UTC+8 17:08:06<yon...@intel.com> 写道：
>
> > Congratulations to SDU Team and China UEFI on RISC-V working group, thanks
> > for sharing and really impressive work😊
> >
> >
> >
> > Thanks,
> >
> > Yong Li
> >
> >
> >

> > *From:* li bing <libin...@outlook.com>
> > *Sent:* Thursday, August 17, 2023 3:56 PM
> > *To:* RISC-V Firmware Exchange <fw-ex...@riscv.org>
> > *Cc:* tp...@ventanamicro.com; sun...@ventanamicro.com; Li, Yong <
> > yon...@intel.com>; Chai, Evan <evan...@intel.com>
> > *Subject:* EDK2 on RISC-V Sophgo SG2042 platform

> >
> >
> >
> > Topics from:Academy of Intelligent Innovation, Shandong Universiy,
> > China.P.R.
> >
> > First of all, thank you for your attention to the work of our team on
> > EDK2. The following is a brief introduction to some of the work done by our
> > team on EDK2 on SG2042.
> >
> > Our work can be roughly divided into several phases: Note: All the work is
> > done on the SG2042 EVB development board.
> >

> > 1. Testing and development work on bootflow provided Sophgo.
> > 2. Implementation according to the initial EDK2 design for RISC-V.

> > Implementation steps can be found here:
> > https://github.com/tianocore/edk2-platforms/blob/master/Platform/RISC-V/PlatformPkg/Readme.md
> > Implementation content can be found here:
> > https://github.com/AII-SDU/AII-edk2-platforms/tree/main/Platform/Sophgo/SG2042Pkg

> > 3. Establishment of the China UEFI on RISC-V working group to discuss

> > the implementation of UEFI on RISC-V.

> > 4. Incorporating feedback from the open-source community, discussing

> > implementation plans with the working group, and ultimately confirming the
> > adoption of the separation approach between openSBI and EDK2, actively
> > submitting it to the master branch.
> > Implementation content can be found here:
> > https://github.com/AII-SDU/edk2-platforms/blob/devel-Sophgo/SG2042Pkg/Platform/Sophgo/SG2042Pkg/About_Sophgo_platform.md

> > 5. Currently conducting testing and adjustments while keeping track of

> > the community's progress.
> >
> > Current progress and status:
> >

> > - We are able to successfully boot and run Linux OS on the SG2042 EVB

> > testing board. The general boot process is as follows: ZSBL + FSBL +
> > openSBI + EDK2 + GRUB2 + Linux OS. The boot medium is SD card, where ZSBL
> > is loaded and executed by an auxiliary MCU, performing initial
> > initialization operations such as DIMM initialization. FSBL then continues
> > with further initialization tasks. OpenSBI is used to initialize the
> > hardware platform and kernel. Additionally, we have upgraded the OpenSBI
> > from version V0.9 to V1.2 to align with community progress. EDK2, running
> > in S-mode, handles configuration operations, primarily completing the SD
> > card driver in the DXE stage, while PCIe driver testing and follow-up work
> > are planned. GRUB2 is used to load the operating system, facilitating
> > subsequent operations.
> >
> >
> > Current limitation:
> >

> > - PCIE driver is not currently supported.
> > - SG2042 (Xuantie C920) MMU can be enabled in SV39 mode, but there

[libin...@outlook.com]

Hi,

Thanks  sun...@ventanamicro.com for  the comments in the PR. We sent out an email explaining some of the issues in the PR:

---

发件人: caiyuq...@outlook.com
发送时间: 2023年8月19日 16:57
收件人: sun...@ventanamicro.com <sun...@ventanamicro.com>
抄送: 李 冰 <libin...@outlook.com>; Li, Yong <yon...@intel.com>; Chai, Evan <evan...@intel.com>
主题: Re: EDK2 on RISC-V Sophgo SG2042 platform (PR #92) Q&A

 

Hi, sunilvl

We sincerely appreciate your comments and we have refined the code based on some of them. There were also some issues that were difficult to explain clearly in PR, so I will explain them here.

Also, thanks to yon...@intel.com for the guidance on the upstream steps of the code. We've sent latest patches against the edk2-platform repo to de...@edk2.groups.io. Maybe using email to review patches and make comments is the usual and convenient way.

There are some tricky issues that need better solutions:

Q1. About Why Copy RISC-V MMU Libraries?

A1. There are some instructions for commit information: "SG2042 (Xuantie C920) MMU can be enabled in SV39 mode, but there are bugs with exception handling and MMC that need to be fixed, so MMU is not currently enabled. Add this library is to ensure compilation pass." One of the difficult bugs can be seen at https://groups.google.com/a/riscv.org/g/fw-exchange/c/wKYnYBNmPr4/m/nY69VnxXAQAJ. It could be due to AMO PMA, but the manual for the C920 doesn't have any information about PMA, and this issue still needs to be addressed. 

So we modified the RiscVConfigureMmu function so that the satp mode is set to SATP_MODE_OFF. This version is handled this way for now, do you have any suggestions?

Q2. Error in SG2042 about initializing system memory independently of each memory node.

A2. The SG2042 EVB has four DDR slots. When three 32GB sized DDRs are plugged in, the results run as shown:

The first figure shows the result of one InitializeRamRegions for the four memory nodes as a whole. The MemoryLength in the red box is the sum of the sizes of the three memory nodes.

The second graph shows the results of InitializeRamRegions performed independently for each of the four memory nodes. The MemoryLength in the red box is just the first memory node size.

The print information added in the yellow box of the second image is in MdeModulePkg/Core/Dxe/Gcd.c:2333.Only the PhysicalStart of the HOB created by the first node is less than EfiFreeMemoryBottom, and CoreInitializeMemoryServices will only use the first eligible ResourceHob to calculate MemoryLength.

So we calculate the sum of the four memory node sizes for one InitializeRamRegions in the SEC module. Or is there a better way?

Q3. Why use opensbi v1.2 instead of the latest v1.3?

A3. The opensbi for SG2042 is maintained by Sophgo and there are issues with using opensbi v1.3. Currently, Sophgo has a plan to submit patches to upstream and is working on the issues, so it may not be long before opensbi v1.3 is ready to use. Upstream plan and status can be found at the link below:

https://github.com/sophgocommunity/SG2042-Wiki/blob/main/docs/upstream-status.md

Thanks again for your advice and help!😀

Thanks,

China.P.R. SDU-AII Team

We've addressed the comments in the PR and sent latest patches against the edk2-platform repo to de...@edk2.groups.io.

Please check out the latest patch we've submitted:

[PATCH edk2-platforms 0/8] EDK2 on RISC-V Sophgo SG2042 platform (groups.io)

[PATCH edk2-platforms 1/8] Sophgo/SG2042Pkg: Add SmbiosPlatformDxe module. (groups.io)

[PATCH edk2-platforms 2/8] Sophgo/SG2042Pkg: Add RiscVMmuLib Library. (groups.io)

[PATCH edk2-platforms 3/8] Sophgo/SG2042Pkg: Add Sophgo SDHCI driver. (groups.io)

[PATCH edk2-platforms 4/8] Sophgo/SG2042Pkg: Add base MMC driver. (groups.io)

[PATCH edk2-platforms 5/8] Sophgo/SG2042Pkg: Add SEC module. (groups.io)

[PATCH edk2-platforms 6/8] SG2042Pkg/SG2042_EVB_Board: Add Sophgo SG2042 platform. (groups.io)

[PATCH edk2-platforms 7/8] Sophgo/SG2042Pkg: Add SG2042Pkg. (groups.io)

[PATCH edk2-platforms 8/8] Sophgo/SG2042Pkg: Add platform readme and document. (groups.io)

We look forward to your feedback and we will use it to make timely improvements.

Thanks,

China.P.R. SDU-AII Team

[Sunil V L]

Hi,

Apologies for the delayed response. Please see my reply inline below.

On Thu, Aug 24, 2023 at 01:00:02AM -0700, libin...@outlook.com wrote:
> Hi,
> Thanks sun...@ventanamicro.com <https://groups.google.com/u/0/> for the

> comments in the PR. We sent out an email explaining some of the issues in
> the PR:
>

> ------------------------------
> *发件人:* caiyuq...@outlook.com
> *发送时间:* 2023年8月19日 16:57
> *收件人:* sun...@ventanamicro.com <sun...@ventanamicro.com>
> *抄送:* 李 冰 <libin...@outlook.com>; Li, Yong <yon...@intel.com>; Chai,
> Evan <evan...@intel.com>
> *主题:* Re: EDK2 on RISC-V Sophgo SG2042 platform (PR #92) Q&A

>
> Hi, sunilvl
>
> We sincerely appreciate your comments and we have refined the code based on
> some of them. There were also some issues that were difficult to explain
> clearly in PR, so I will explain them here.
>
> Also, thanks to yon...@intel.com for the guidance on the upstream steps of
> the code. We've sent latest patches against the edk2-platform repo to
> de...@edk2.groups.io. Maybe using email to review patches and make comments
> is the usual and convenient way.
>
> There are some tricky issues that need better solutions:
>

> *Q1. About Why Copy RISC-V MMU Libraries?*

> A1. There are some instructions for commit information: "SG2042 (Xuantie
> C920) MMU can be enabled in SV39 mode, but there are bugs with exception
> handling and MMC that need to be fixed, so MMU is not currently enabled.
> Add this library is to ensure compilation pass." One of the difficult bugs
> can be seen at
> https://groups.google.com/a/riscv.org/g/fw-exchange/c/wKYnYBNmPr4/m/nY69VnxXAQAJ.
> It could be due to AMO PMA, but the manual for the C920 doesn't have any
> information about PMA, and this issue still needs to be addressed.
> So we modified the RiscVConfigureMmu function so that the satp mode is set
> to SATP_MODE_OFF. This version is handled this way for now, do you have any
> suggestions?
>

My suggestion is to introduce a PCD variable to disable MMU
configuration. This probably helps in debugging also and avoids
duplicating the entire library.

Tuan, do you see any issue with that?

> *Q2. Error in SG2042 about initializing system memory independently of each
> memory node.*

> A2. The SG2042 EVB has four DDR slots. When three 32GB sized DDRs are
> plugged in, the results run as shown:
> The first figure shows the result of one InitializeRamRegions for the four
> memory nodes as a whole. The MemoryLength in the red box is the sum of the
> sizes of the three memory nodes.
> [image: Figure1]
> The second graph shows the results of InitializeRamRegions performed
> independently for each of the four memory nodes. The MemoryLength in the
> red box is just the first memory node size.
>

> The print information added in the yellow box of the second image is in
> MdeModulePkg/Core/Dxe/Gcd.c:2333.Only the PhysicalStart of the HOB created
> by the first node is less than EfiFreeMemoryBottom, and
> CoreInitializeMemoryServices will only use the first eligible ResourceHob
> to calculate MemoryLength.
>

> So we calculate the sum of the four memory node sizes for one
> InitializeRamRegions in the SEC module. Or is there a better way?
>

Do you get any error during boot or whether OS will see reduced memory?
It is not really required to add all of the memory in SEC phase. Even in
RiscVVirt there is a known issue that it adds all of the memory in the
SEC phase which makes HighMemDxe pretty much useless. I have a fix for
that at
https://github.com/vlsunil/edk2/commit/a306ca530004cb9e05afccd954e73a7ce81be4b6.

Could you please check whether this works for you?

Also, what is the guarantee that all 32G memory ranges will be
contiguous to create a single large range?

> *Q3. Why use opensbi v1.2 instead of the latest v1.3?*

> A3. The opensbi for SG2042 is maintained by Sophgo and there are issues
> with using opensbi v1.3. Currently, Sophgo has a plan to submit patches to
> upstream and is working on the issues, so it may not be long before opensbi
> v1.3 is ready to use. Upstream plan and status can be found at the link
> below:
> https://github.com/sophgocommunity/SG2042-Wiki/blob/main/docs/upstream-status.md
>

Thanks!. The primary reason I wanted you to move to 1.3/1.3.1 is, it
properly adds no-map attributes.

Thanks,
Sunil

[Tuan Phan]

>My suggestion is to introduce a PCD variable to disable MMU
>configuration. This probably helps in debugging also and avoids
>duplicating the entire library.

>Tuan, do you see any issue with that? 

 

Agree that PCD can be used to disable MMU.

[Sunil V L]

On Thu, Aug 31, 2023 at 06:31:15PM -0700, libin...@outlook.com wrote:
> Hi,
> We made the second version of the patch with the following major changes:
>
> 1. We have added a PlatformUpdateMmuDxe which contains two main
> functions. The first function is to change the page attributes
> corresponding to memory. C920 has five customizable page attributes that
> control whether the page is Strong order(SO), Cacheable(C), Bufferable(B),
> Shareable(SH), Trustable(Sec). This driver modifies the page table
> attributes to avoid exceptions based on the memory attributes of the C920.
> Update Read/Write/Strong Order attribute for memory mapped IO. And update
> Read/Write/Execute/Sharable/Cacheable attribute for system memory. The
> second function is to introduces a PCD variable PcdForceNoMMU to disable
> MMU configuration.Currently, enabling MMU results in a timeout for reading
> data blocks from the SD card, So MMU is disabled by default.
Okay, Thanks.

> 2. We tried this method at
> https://github.com/vlsunil/edk2/commit/a306ca530004cb9e05afccd954e73a7ce81be4b6.
> This method makes HighMemDxe work and adds all memory. But this method
> causes grub to run with a relocation overflow error. The method of
> InitializeRamRegions with the four memory nodes as a whole works fine when
> running grub. The reason why this happens is unclear. Also, memory nodes in
> fdt are rewritten during the FSBL phase, and the 32G memory ranges of
> neighboring memory nodes are contiguous.
>
Okay. grub2 relocation overflow error is a known issue. I am still not
sure whether there is any guarantee that you will not hit grub issue
even with your solution. But I think it is ok to go with your solution
which atleast masks the issue for the time being.

> Do you see any issue with these?Looking forward to your feedback!
>
I think the changes are reasonable. One thing I would like to point you
is, in the SEC module, please avoid using getting FDT pointer from the
Firmware context.
+ FdtPointer = (VOID *)FirmwareContext->FlattenedDeviceTree;

I know this is leveraged from the RiscVVirt. But we are trying to remove
it from RiscVVirt also. So, if you can make this change, it will reduce
the effort to cleanup later.

You can refer to Andrei's branch
https://github.com/andreiw/edk2-rv-wip/tree/remove_riscv_fw_ctx

I see multiple emails from you with same subject and it is not clear to
me whether there is any new information. If you are not getting
response, assume people might be busy. So, please give a week or so
before sending a reminder.

BTW, I will be OOO next week. If others give any feedback, please
address them. Otherwise, I will review and help with merging a week
later.

Thanks,
Sunil
> Thanks,
> China.P.R. SDU-AII Team

>
> 在2023年8月30日星期三 UTC+8 00:11:01<tp...@ventanamicro.com> 写道：
>
> > >My suggestion is to introduce a PCD variable to disable MMU
> > >configuration. This probably helps in debugging also and avoids
> > >duplicating the entire library.
> >
> > >Tuan, do you see any issue with that?
> >
> >
> >
> > Agree that PCD can be used to disable MMU.
> >
> >
> >

> > *From: *Sunil V L <sun...@ventanamicro.com>
> > *Date: *Tuesday, August 29, 2023 at 3:44 AM
> > *To: *libin...@outlook.com <libin...@outlook.com>
> > *Cc: *RISC-V Firmware Exchange <fw-ex...@riscv.org>, yon...@intel.com <

> > yon...@intel.com>, tp...@ventanamicro.com <tp...@ventanamicro.com>,
> > evan...@intel.com <evan...@intel.com>

> > *Subject: *Re: EDK2 on RISC-V Sophgo SG2042 platform

[libin...@outlook.com]

Hi, 

Thanks for the previous feedback from Sunil and Leif, we did some test and finished the third version of patch. My colleague replied to a message at https://edk2.groups.io/g/devel/message/108216 earlier, which also explains some of questions below.

1)  Remove firmware context

Referring to Andrei's branch, in the SEC module, the port avoids getting the FDT pointer from the firmware.

2) Blurb

We have added some descriptions in Blurb to explain the current status, testing situation, and limitations of the project. And added a link to the public git in Blurb at https://github.com/AII-SDU/edk2-platforms/tree/devel-Sophgo/SG2042Pkg/Platform/Sophgo

3) Milk-V Pioneer board

Our team doesn't have Milk-V Pioneer board at the moment, but we asked our partner team to help us to test it briefly. According to the test result, EVB and Pioneer board are not fully compatible. Milk-V Pioneer board running on EDK2 can boot into UEFI shell normally, but the SD driver can't recognize all the partitions correctly and can't boot Linux OS normally. Sorry, because of the lack of Milk-V Pioneer board, we don't have a plan to add support for Milk-V Pioneer board.

4) Layout

Based on Leif's comment, we moved most of the code in the port from 

Platform/Sophgo/SG2042Pkg/ 

to 

Silicon/Sophgo/SG2042Pkg/.

The specific changes can be found in the submitted third version of the patch, or in the link to the public git tree provided in Blurb.

5) Clang toolchain support

Our team tried to build the port using the CLANGDWARF toolchain (clang version 18.0.0), and updated README for CLANGDWARF support. It is able to build it successfully but the compiled binary didn't work at all. The compiled binary gets stuck in a loop when the SD card driver reads a block of data, but adding two lines of printout to the function that reads the block of data works fine and boots into the Linux OS. It looks like there are still a lot of problems with the compiled binary.

Thanks,
China.P.R. SDU-AII Team