Board Verification/Testing Process

[Carl Perry]

Greetings Everyone -

Per our discussion at the group call on August 15th, I wanted to start a
discussion about what we can and should be testing when we get new
boards. I looked at a Phoronix Test Suite as a possible way to easily
run these tests and some simple benchmarks, but it seems like overkill
for this particular use case. Ideally I would like something that will
capture CPU, Memory, IO, and Thermal stats during each test run. I also
think it's beneficial to have the test and the control suite distributed
in a "Ready to run" form, rather than compiling everything every time.
With those in mind, I'm looking at making a Python framework that can
coordinate the tests and collect results. But if anyone knows of
something that already exists, I'm happy to look into that instead of
re-inventing the wheel.

In terms of tests, this is what I have come up with so far and I would
love feedback:

1) The RISC-V ACT tests

2) The RISC-V Sail suite

3) glibc test suite

4) FFTW tests

5) stress-ng (for generic CPU testing to ensure thermal logging is
useful, as well as memory testing)

6) fio (for IO sub-system testing)

7) core-to-core latency https://github.com/nviennot/core-to-core-latency

8) mprime

I'm still looking for some good tests for Vector and would love
suggestions there. Especially as we get designs that implement more than
128b Vector hopefully soon.

Thoughts and feedback encouraged, let me know what you think and I'll
try to have a POC running on a few boards by the end of the weekend.

  -Carl

[Rafael Sene]

Take a look here for Vector: https://dl.acm.org/doi/fullHtml/10.1145/3422667

Obrigado | Thank You

Rafael Peria de Sene

Technical Program Manager, RISC-V

Phone: +55 19 98153 9778

E-mail: raf...@riscv.org

[Jeff Scheel]

The other thing we've struggled with is floating point formats.  IBM used to maintain a testsuite, but no longer does so.  If we could add some of these, it would be great.

-Jeff

--

Jeff Scheel (he/him/his)

Director of Technical Programs, RISC-V International

Join me at RISC-V Summit North America (Oct. 21-23)

[Kurt Keville]

Hi Jeff,

Peter Lindstrom at LLNL used to keep a list of those with some tools. Let me see if that is still active.

Kurt

To unsubscribe from this group and stop receiving emails from it, send an email to devboard-commun...@riscv.org.

[Igor Moura]

> I looked at a Phoronix Test Suite as a possible way to easily run these tests and some simple benchmarks, but it seems like overkill for this particular use case.

Wouldn't a specific test suite (like the CPU one), or even just creating our own suite with specific tests work for this case?

However, the PTS does lack some of the tests you mentioned (like the RISC-V ones, and core-to-core latency).