[llvm-dev] Running llvm test-suite

[Hongbin Zheng via llvm-dev]

Hi llvm-dev,

Are we still able to run llvm test-suite without LNT? 

Also, is there any example of configuring test-suite to run against 3rd party backend rather than llc?

Thanks

Hongbin

[Matthias Braun via llvm-dev]

Yes the test-suite can be run without lnt (you can always just look at the LNT logfiles to see what commands it is running and run those yourself).

Currently I would recommend using the cmake/lit version of the test-suite. You typically do something like:

mkdir build && cd build
cmake -GNinja -DCMAKE_C_COMPILER=/path/to/my/clang -DTEST_SUITE_ARCH_FLAGS="-arch arm64" /path/to/test-suite
ninja # to build the benchmarks
llvm-lit # to run the benchmarks

You should check "ccmake"/"cmake -LA" to learn about the available options as well as llvm-lit. You can either use llvm-lit from an llvm build or use the pypy packaged version (pip install lit). The cmake/lit version has no support to run custom backends through llc at the moment, you need to have a clang version that accepts -arch XXX for your target.

- Matthias

> _______________________________________________
> LLVM Developers mailing list
> llvm...@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

_______________________________________________
LLVM Developers mailing list
llvm...@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

[Hongbin Zheng via llvm-dev]

Hi Matthias,

Thanks a lot for the explanation. 

On Thu, Aug 11, 2016 at 11:45 AM, Matthias Braun <mbr...@apple.com> wrote:

Yes the test-suite can be run without lnt (you can always just look at the LNT logfiles to see what commands it is running and run those yourself).

Currently I would recommend using the cmake/lit version of the test-suite. You typically do something like:

mkdir build && cd build
cmake -GNinja -DCMAKE_C_COMPILER=/path/to/my/clang -DTEST_SUITE_ARCH_FLAGS="-arch arm64" /path/to/test-suite
ninja      # to build the benchmarks
llvm-lit   # to run the benchmarks

You should check "ccmake"/"cmake -LA" to learn about the available options as well as llvm-lit. You can either use llvm-lit from an llvm build or use the pypy packaged version (pip install lit). The cmake/lit version has no support to run custom backends through llc at the moment, you need to have a clang version that accepts -arch XXX for your target.

Sorry I didn't explain well. Here I want to use testsuite to test a customized compilation flow like:

clang -> opt-> <some other optimization tool> -> llc.

Is this possible by changing some of the makefile of llvm testsuite? 

Thanks 

Hongbin

[Matthias Braun via llvm-dev]

On Aug 11, 2016, at 6:00 PM, Hongbin Zheng <ethe...@gmail.com> wrote:

Hi Matthias,

Thanks a lot for the explanation. 

On Thu, Aug 11, 2016 at 11:45 AM, Matthias Braun <mbr...@apple.com> wrote:

Yes the test-suite can be run without lnt (you can always just look at the LNT logfiles to see what commands it is running and run those yourself).

Currently I would recommend using the cmake/lit version of the test-suite. You typically do something like:

mkdir build && cd build
cmake -GNinja -DCMAKE_C_COMPILER=/path/to/my/clang -DTEST_SUITE_ARCH_FLAGS="-arch arm64" /path/to/test-suite
ninja      # to build the benchmarks
llvm-lit   # to run the benchmarks

You should check "ccmake"/"cmake -LA" to learn about the available options as well as llvm-lit. You can either use llvm-lit from an llvm build or use the pypy packaged version (pip install lit). The cmake/lit version has no support to run custom backends through llc at the moment, you need to have a clang version that accepts -arch XXX for your target.

Sorry I didn't explain well. Here I want to use testsuite to test a customized compilation flow like:

clang -> opt-> <some other optimization tool> -> llc.

Is this possible by changing some of the makefile of llvm testsuite? 

It's probably possible by using one of the existing TEST.* files or writing your own. Unfortunately this system can get complicated and is not documented much. I don't really know how it works.

An alternative to this could be that you write your own little python/shell script that acts like a compiler on the outside but internally translates to clang, opt, llc commands you want. You could point CMAKE_C_COMPILER to that script.

- Matthias