Having trouble running GPU in full system mode.

[Tony Niro]

Hi all,

Just new to gem5 and trying to get the gpu-fs running with an example. I've built the docker container using gem5/util/dockerfiles/gpu-fs/Dockerfile. I was able to build gem5/build/VEGA_X86/gem5.opt as well as the square application in gem5-resources. 

I had trouble building the disk-image from gem5-resources/src/gpu-fs/disk-image so I downloaded vmlinux-5.4.0-105-generic  and x86-gpu-fs-20220512.img from google:

gs://dist.gem5.org/dist/v22-0/kernels/x86/static/vmlinux-5.4.0-105-generic

gs://dist.gem5.org/dist/v22-0/images/x86/ubuntu-18-04/x86-gpu-fs-20220512.img.gz

 

I wanted to run the simulation in the docker container so I ran the following: 

 $ docker run --privileged --device=/dev/kvm --rm -v $(pwd):/repos -w /repos -it rp-gpu-fs:v22-00

I ran the following command: 

root@d89fe796c370:/repos/gem5# build/VEGA_X86/gem5.opt \ configs/example/gpufs/vega10_kvm.py \

--disk-image /repos/x86-gpu-fs-20220512.img \

--kernel /repos/vmlinux-5.4.0-105-generic \

--gpu-mmio-trace /repos/gem5-resources/src/gpu-fs/mmio_trace.log \

--app /repos/gem5-resources/src/gpu/square/bin/square

The environment seems to run but the square application fails with:

error: 'hipErrorInvalidDevice'(101) at square.cpp:66

I added the following to square.cpp before the failure to output some info:

system("/usr/bin/lspci -vvv");

system("/opt/rocm/bin/rocminfo");

system("/opt/rocm/hip/bin/hipcc --version");

system("/opt/rocm/bin/mygpu");

system("/opt/rocm/bin/mymcpu");

I've included the gem5.opt output (gpu-fs-output.txt) as well as the image output (system.pc.com_1.device).  

I'm not sure if I'm using the correct options, image or kernel. Looks like the application doesn't see the GPU. Any suggestions would be greatly appreciated.

Tony