An open-source initial implementation for RVV-1.0 in gem5

[Yin Zhang]

Hi all,

FYI, PLCT has supported and open-sourced mainly part of RVV-1.0 in gem5[1] based on the vector engine[2] designed by CRISTÓBAL RAMÍREZ, who has published a paper named A RISC-V Simulator and Benchmark Suite for Designing and Evaluating Vector Architectures[3] to present his work. 

And PLCT also updated vectorized benchmark suite[4] by using RVV-1.0 intrinsic. All instructions used in the benchmark suite are supported in vector engine.Now, with the exception of part of the Pathfinder module, all other tests in benchmark can run correctly in gem5.

There are still some issues to be resolved. First, instructions not used in the benchmark, including narrowing, widening, and the case of lmul ≠ 1, are not supported yet. Besides, the length of issue queues in the vector engine is temporarily limited to 1 to avoid scheduling error in memory queue that it cannot maintain RAW dependency. Finally, when running med-level pathfinder test, the gem5 dram module got an error.

Welcome to try and put forward suggestions. And also welcome to send PR for improving it!

CRISTÓBAL RAMÍREZ's work:
[3] paper: https://dl.acm.org/doi/10.1145/3422667
[2] gem5: https://github.com/RALC88/gem5/
     benchmark: https://github.com/RALC88/riscvvectorized-benchmark-suite

PLCT's work:
[1] plct-gem5: https://github.com/plctlab/plct-gem5
[4] benchmark: https://github.com/plctlab/riscv-vectorized-benchmark-suite

--

Regards,

Yin Zhang (张尹)

[Jerin Joy]

Hi Yin,

Thanks for posting this!

I was looking through the code and noticed that the execute functions for the vector instructions are never called. Is that because the benchmarks you've run on the model have vector instructions that don't actually affect the control flow?

Jerin

[Yin Zhang]

Hi Jerkin,

I'm not sure what you mean by “the execute functions for the vector instructions”. I can provide some examples of the definition and call of vector instruction executions in vector engine:

definition of execute functions(e.g. compute_long_int_op): https://github.com/plctlab/plct-gem5/blob/dee190f7b9805547685ee240889206a55ee4e2d9/src/cpu/vector_engine/vpu/multilane_wrapper/func_unit.hh#L474

call execute functions(e.g. compute_long_int_op):https://github.com/plctlab/plct-gem5/blob/dee190f7b9805547685ee240889206a55ee4e2d9/src/cpu/vector_engine/vpu/multilane_wrapper/datapath.cc#L658

pack execute functions:https://github.com/plctlab/plct-gem5/blob/dee190f7b9805547685ee240889206a55ee4e2d9/src/cpu/vector_engine/vpu/multilane_wrapper/datapath.cc#L407

call packed execute functions:https://github.com/plctlab/plct-gem5/blob/dee190f7b9805547685ee240889206a55ee4e2d9/src/cpu/vector_engine/vpu/multilane_wrapper/datapath.cc#L228

hope these will help you.

Regards,

Yin

[Jerin Joy]

Hi Yin,

Thanks for getting back to me. I got the simulator built and ran some code through it to find the execution path. 

I was originally looking in the generated build/RISCV/arch/riscv/generated/exec-ns.cc.inc file where the execution for the other instructions is handled:

    Fault
    Vadd_vv::execute(ExecContext *xc,
        Trace::InstRecord *traceData) const
    {
        panic("Vector instruction had it's execute() called!");
        return NoFault;
    }

I see that the model has a separate “vector engine” that the vector instructions are dispatched to. From src/cpu/minor/execute.cc

                    DPRINTF(CpuVectorIssue,"Sending vector isnt to the Vector"
                        " Engine: %s , pc: 0x%8X\n",*inst , pc);

                    completed_inst = false;
                    completed_vec_inst = false;
                    waiting_vector_engine_resp = true;

                    cpu.ve_interface->sendCommand(vector_insn,xc,src1,src2,
                        [this,inst,vector_insn]() mutable {
                        DPRINTF(CpuVectorIssue,"The instruction has been "
                        "hosted by the Vector Engine %s \n",*inst );
                        completed_vec_inst = true;

Jerin

[Yin Zhang]

Hi Jerin,

First, sorry for misspelled your name in the last email. My automatic typing error correction caused this problem.

Yes, you are right. The generated execute functions for vector instructions are never called.

In this implementation, the author takes “vector engine” as a co-processor of MinorCPU. So MinorCPU sends all vector instructions into “vector engine” for execution. The execute functions generated by decoder will not and should not be used.

Regards,

Yin

[Chen Zou]

Hi Yin,

Great work!

Is there any plan to merge to upstream gem5?

Or base plct-gem5 on the current upstream gem5 stable release V21.2?

Regards,
Chen

[Yin Zhang]

Hi Chen,

The coprocessor version only partially supports RVV, and makes major changes to the original architecture of gem5 (adding an independent vector engine module), which is greatly deviated from the official code of gem5. So it may not be suitable for merging upstream.

In addition, when exploring the follow-up support in the coprocessor version, we encountered some problems that are difficult to solve under this framework. Therefore, we did not continue to promote the implementation of this version, but turned to the scheme of CPU integration.

At present, the RVV support of CPU integrated version has made some progress. We will open source the implementation of this version after successfully running RiVEC bencmark suite. During the implementation of this version, we will pull and merge the upstream code from time to time. The CPU integration scheme is more likely to be submitted to the upstream in the future.

Regards,

Yin

[Chant Zine]

Thanks a lot for the information.

I believe that would be very beneficial to both RISCV and the Gem5 community.

[Jerin Joy]

We (Rivos) are working on adding RVV support to Gem5. I had sent an email out yesterday about the initial changes that we have released publicly:

https://www.mail-archive.com/gem5...@gem5.org/msg42420.html

We could potentially collaborate on getting RVV support in mainline Gem5 to avoid duplication of work.

Jerin

[Yin Zhang]

Hi Jerin,

Thank you for giving the initial patch publicly! But if I'm not mistaken, this patch is only focusing on AtomicSimpleCPU of gem5. We have completed some support for AtomicSimpleCPU and are exploring how to implement the timing model to support CPUs using the model. That's why we asked for help on the gem5-dev mailing list. We think supporting all CPU types should be necessary for submitting upstream. At present, we have a scheme supporting timing model and it is under coding.

In view of your willingness to cooperate, we are glad to and welcome it very much. We are discussing to give the public release of our CPU integration implementation recently. According to my personal estimation, it may be about two weeks later. At that time, we can further discuss how to cooperate based on the code of open source implementation. We are looking forward to cooperating with RIVOS!

Regards,
Yin

[Jerin Joy]

Hi Yin,

Yes I’ve been testing my changes on the SimpleCPU model. Our initial goal with this release is to get the execution path for the vector instructions in Gem5. You’ll see that much of the changes are in vector.isa and decoder.isa

Once your changes are public we should sync up to see how we can get RVV support in mainline Gem5.

Jerin

[Przemek Barański]

Hi!

Thanks for your great effort on gem5 simulator for RISCV. 

Jerin and Yin, do you have some rough timeline for more cycle accurate model?

Best regards

Przemek