Updating to llvm-13+ for RISCV JIT support

[developandplay]

As part of a research project I'm porting Swiftshader to RISCV.

In terms of dependencies I already ported marl to RISCV and added a PR:

https://github.com/google/marl/pull/208

While llvm-10 supports RISCV in general it seems like the JIT is not supported.

Now with llvm-13+ JIT support for RISCV is being added as far as I can tell:

https://reviews.llvm.org/D105429

Therefore I just wanted to ask for your thoughts on updating the llvm version that swiftshader is using. I'm not sure what would go into updating llvm but if I can help with that I'd be happy to.

For more context on the project:

I'm part of Prof. Michael Taylors research group at UW and for the project we want to render graphics on a GP-GPU (Hammerblade) composed of hundreds of RISCV cores. Of course we are now starting small with just rendering on a single RISCV core.

Since we have worked together with Tim Ansell (mithro) for other RISCV projects he suggested that Swiftshader could be a good fit for software rendering.

As alternative I'm considering llvm-pipe (or lava-pipe for Vulkan) but I didn't start yet with porting that to RISCV.

[developandplay]

The steps I took so far are guided by this commit: https://swiftshader-review.googlesource.com/c/SwiftShader/+/21748

1. Extended the llvm update script for RISCV and ran it for linux. This results in the following files:

modified:   ../CMakeLists.txt
        modified:   ../third_party/llvm-10.0/BUILD.gn
        modified:   ../third_party/llvm-10.0/configs/linux/include/llvm/Config/AsmParsers.def
        modified:   ../third_party/llvm-10.0/configs/linux/include/llvm/Config/AsmPrinters.def
        modified:   ../third_party/llvm-10.0/configs/linux/include/llvm/Config/Disassemblers.def
        modified:   ../third_party/llvm-10.0/configs/linux/include/llvm/Config/Targets.def
        modified:   ../third_party/llvm-10.0/configs/linux/include/llvm/Config/abi-breaking.h
        modified:   ../third_party/llvm-10.0/configs/linux/include/llvm/Config/config.h
        modified:   ../third_party/llvm-10.0/configs/linux/include/llvm/Config/llvm-config.h
        modified:   ../third_party/llvm-10.0/scripts/update.py

Untracked files:
  (use "git add <file>..." to include in what will be committed)
        ../third_party/llvm-10.0/configs/common/lib/Target/RISCV/

2. Updated marl.

3. Extend CMakelist and others with RISCV parameters as the Mips commit did.

4. Ran make as which so far has led me to this issue:

[ 91%] Linking CXX shared library ../../libvk_swiftshader.so
/usr/bin/ld: ../Reactor/libReactorLLVM.a(LLVMJIT.cpp.o): in function `.L362':
LLVMJIT.cpp:(.text._ZZN12_GLOBAL__N_110JITGlobals3getEvENKUlvE_clEv.constprop.0+0xb0): undefined reference to `LLVMInitializeRISCVTargetInfo'
/usr/bin/ld: LLVMJIT.cpp:(.text._ZZN12_GLOBAL__N_110JITGlobals3getEvENKUlvE_clEv.constprop.0+0xb4): undefined reference to `LLVMInitializeRISCVTarget'
/usr/bin/ld: LLVMJIT.cpp:(.text._ZZN12_GLOBAL__N_110JITGlobals3getEvENKUlvE_clEv.constprop.0+0xb8): undefined reference to `LLVMInitializeRISCVTargetMC'
/usr/bin/ld: LLVMJIT.cpp:(.text._ZZN12_GLOBAL__N_110JITGlobals3getEvENKUlvE_clEv.constprop.0+0xc0): undefined reference to `LLVMInitializeRISCVAsmPrinter'
/usr/bin/ld: LLVMJIT.cpp:(.text._ZZN12_GLOBAL__N_110JITGlobals3getEvENKUlvE_clEv.constprop.0+0xc8): undefined reference to `LLVMInitializeRISCVAsmParser'
collect2: error: ld returned 1 exit status
gmake[2]: *** [src/Vulkan/CMakeFiles/vk_swiftshader.dir/build.make:656: libvk_swiftshader.so] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:2117: src/Vulkan/CMakeFiles/vk_swiftshader.dir/all] Error 2
gmake: *** [Makefile:136: all] Error 2

I'm not sure if this is related to JIT support not being there in llvm-10 but LLVMInitializeRISCVTargetInfo and the other undefined refernces are part of llvm-10 even without JIT.

Therefore I would assume this is some other issue that I'm running into.

[developandplay]

I also wanted to add that building llvm-10 with gcc for RISC-V leads to a series of errors described here: https://reviews.llvm.org/D68964

Unfortunately it seems like those fixes were only landed with llvm-11.
Of course those can be backported to llvm-10 which is what I unknowingly did to get to the state described in the last post.

However now that I'm aware that an actual fix exists in llvm-11 that would be another good reason to update llvm.

[developandplay]

Since it's probably not that easy to follow without seeing the commits I pushed them to my fork of swiftshader: https://github.com/developandplay/swiftshader/tree/rv64

I would be happy if one of you could have a look at them. The updates to marl for rv64 are already integrated.

In addition I'm also in touch with the llvm jit maintainers to implement the missing pieces for RISC-V JIT: https://groups.google.com/g/llvm-dev/c/MoCSoRK4wJ8

[Nicolas Capens]

Hallo Martin!

Thanks for your interest in SwiftShader and porting it to a new architecture.

Note that we're using a minimized build of LLVM 10 because SwiftShader is built numerous times as part of CI testing for Android and other projects. So keeping the build time reasonable is important for these purposes. Unfortunately it means that upgrading to a newer version takes a bit of effort. I've attached a document which details the process in case you find it useful.

However, I would recommend a different approach. We actually also have internal projects which build SwiftShader and link it against the latest revision of LLVM. So if build time is not critical, this could be an option for you. Such a build is not readily available through our CMake build files, but should be reasonably straightforward to add. It could be a new REACTOR_BACKEND option and download a recent version of LLVM as a Git submodule, on demand.

Please go ahead and create a tracking bug at https://g.co/swiftshaderbugs for this. We'd be happy to review changes made in the main repository to avoid forking issues.

Kind regards,

Nicolas Capens

--
You received this message because you are subscribed to the Google Groups "swiftshader" group.
To unsubscribe from this group and stop receiving emails from it, send an email to swiftshader...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/swiftshader/0bf8eab4-4b44-468e-a611-0fd0f9bdb3b3n%40googlegroups.com.

[developandplay]

As Ben pointed out this was just an issue with the build files. I had added the llvm RISCV files to the .gn file but not the CMakeLists.txt...

With those added the RISC-V port builds properly: https://github.com/developandplay/swiftshader/commit/f36993f937ad3e4cb703c0ddcdade8da0a4946fe

Obviously the unit test still fail due to the missing RISC-V backend for LLVM JIT. I will work on that for the next week or two.

developandplay schrieb am Freitag, 24. Dezember 2021 um 13:43:53 UTC+1:

[developandplay]

Hi Nicolas,

Thank you for the response!

It's great to hear that you are using a recent version for LLVM already internally. (Just to know that it doesn't break anything major)

I now successfully used the `update.py` script to add the RISC-V target and compile everything with the llvm10 dependency. (Of course that does not pass the unit tests)

Using a full llvm build for linking sounds like a good idea! I'll try that.

Just to make sure: Your modifications to reduce build times only remove (don't add) stuff from the full llvm build?

I'll also check if I can modify the update script for a newer llvm version...

Regarding the bugtracker and gerrit I'll sign up for that in the next few days.

Best wishes

Martin