IWYU+CMake?

[David Doria]

Does anyone use IWYU with CMake projects? I tried the recommended:

doriad@david-lab:~/Test$ make -k CXX=~/build/llvm/Debug+Asserts/bin/include-what-you-use

-- Configuring done

-- Generating done

-- Build files have been written to: /home/doriad/Test

Scanning dependencies of target Test

[100%] Built target Test

but as you can see IWYU is not actually run. I also tried setting CMAKE_CXX_COMPILER to /path/to/llvm/Debug+Asserts/bin/include-what-you-use, but on one machine CMake didn't like that at all (errors about that it was unable to compile a simple program), and on another machine, configuring after setting that variable seems to go into an infinite loop or something (cmake hangs with the processor at 100% usage). Does anyone know how to do this correctly?

Thanks,

David

[Zaheer Chothia]

CMake expects a real compiler when it runs its initial checks, so setting IWYU as the compiler directly doesn't work.
Following this mail is a script I cooked up a while back for just this scenario.  It invokes the compiler as usual and IWYU in addition when it makes sense.

See the header comment for usage details (... and don't forget to give execute permissions).

--Zaheer

---- BEGIN: iwyu-wrapper -------------------------------------------------------

#!/usr/bin/env bash

# Compiler wrapper script for include-what-you-use (IWYU).

#

# This is required for build systems (e.g. CMake) which perform configuration

# checks and thus require a functioning compiler.  First, the compiler is

# invoked as usual and then IWYU is called in addition. Note this will increase

# build times since each file is parsed twice.

#

# This script should be called as a prefix to the build command, for instance:

#   $ iwyu-wrapper clang -c hello.c -o hello.o

# To use this from CMake, configure with a command such as:

#   $ cmake -DCMAKE_C_COMPILER="iwyu-wrapper clang" \

#           -DCMAKE_CXX_COMPILER="iwyu-wrapper clang++" ..

#

# Build command output is not redirected, however IWYU reports are placed into

# files with the extension '.iwyu.log'.  An example of how to conveniently fix

# the includes using the given advice is as follows:

#   $ find . -name '*.iwyu.log' -exec cat {} \; > __all__.iwyu.log

#   $ python fix_includes.py --comments --nosafe_headers < __all__.iwyu.log

set -e  # Die if any command fails.

if [[ $# -lt 1 ]] ; then

  echo "Usage: $0 <build command>"

  exit 1

fi

# Execute the build command itself.

# "$@" returns all positional parameters whilst preserving quoting.

"$@"

# IWYU expects exactly one compiler job so we check for commands of the form:

#   $ clang -c hello.c -o hello.o

# The aim of this is to ignore commands which don't involve compilation, e.g.:

#   $ clang a.o b.o c.o -o xyz

# otherwise these would unnecessarily result in a build failure.

compile_only=false

output_file=

for ((i = 2; i <= $#; i++)); do

  # ${!i} returns the i-th positional command-line argument.

  case "${!i}" in

    -c)

      compile_only=true

      ;;

    -o)

      # Output file is given by next argument.

      (( i++ ))

      output_file=${!i}

      ;;

  esac

done

if [[ -z "${output_file}" ]]; then

  echo "Error: build command did not specify an output file (-o)"

  exit 1

fi

# Finally execute IWYU.

if ${compile_only} ; then

  # IWYU seems to return exit code 1 even on success, so we need to be more

  # permissive and temporarily toggle the failure mode.

  set +e

  # "${@:i}" returns all parameters from position i onwards (preserves quotes).

  include-what-you-use "${@:2}" > "${output_file}.iwyu.log" 2>&1

  if [[ $rc != 1 ]] ; then

    exit $rc

  fi

  set -e

fi

---- END: iwyu-wrapper ---------------------------------------------------------

[David Doria]

My simple project works with:

doriad@david-lab:~/Test$ cmake .

-- Configuring done

-- Generating done

-- Build files have been written to: /home/doriad/Test

but when I try with:

doriad@david-lab:~/Test$ cmake -DCMAKE_C_COMPILER="iwyu-wrapper gcc" -DCMAKE_CXX_COMPILER="iwyu-wrapper g++" .

I get pages and pages of these and it never seems to stop:

-- Configuring done

You have changed variables that require your cache to be deleted.

Configure will be re-run and you may have to reset some variables.

The following variables have changed:

CMAKE_C_COMPILER= iwyu-wrapper gcc

CMAKE_CXX_COMPILER= iwyu-wrapper g++

-- Configuring done

You have changed variables that require your cache to be deleted.

Configure will be re-run and you may have to reset some variables.

The following variables have changed:

CMAKE_C_COMPILER= iwyu-wrapper gcc

CMAKE_CXX_COMPILER= iwyu-wrapper g++

-- Configuring done

You have changed variables that require your cache to be deleted.

Configure will be re-run and you may have to reset some variables.

The following variables have changed:

CMAKE_C_COMPILER= iwyu-wrapper gcc

CMAKE_CXX_COMPILER= iwyu-wrapper g++

-- Configuring done

You have changed variables that require your cache to be deleted.

Configure will be re-run and you may have to reset some variables.

The following variables have changed:

CMAKE_C_COMPILER= iwyu-wrapper gcc

CMAKE_CXX_COMPILER= iwyu-wrapper g++

Any thoughts?

David

[Zaheer Chothia]

It looks as though you are trying to change the compiler in an existing build tree which won't work.

You may be able to get away with just deleting CMakeCache.txt, but the general recommendation is to build out-of-source instead:

    $ mkdir build && cd build

    $ cmake -DCMAKE_C_COMPILER="iwyu-wrapper gcc" -DCMAKE_CXX_COMPILER="iwyu-wrapper g++" ..

--Zaheer

[David Doria]

Ok, I stared with a fresh build (out of source) build tree as you suggested). Below I have shown the output of the commands that show I am (hopefully) in the correct configuration:

doriad@david-lab:~/Test/build$ ls

gcc.libc.imp  gcc.stl.headers.imp  gcc.symbols.imp  google.imp  iwyu.gcc.imp  iwyu-wrapper.bash  third_party.imp

doriad@david-lab:~/Test/build$ ls -lhrt iwyu-wrapper.bash 

-rwxrwxrwx 1 doriad doriad 2.3K Nov 15 13:39 iwyu-wrapper.bash

doriad@david-lab:~/Test/build$ ls ../

build  CMakeLists.txt  Test.cpp

doriad@david-lab:~/Test/build$ cmake -DCMAKE_C_COMPILER="iwyu-wrapper.bash gcc" -DCMAKE_CXX_COMPILER="iwyu-wrapper.bash g++" ..

-- The C compiler identification is unknown

-- The CXX compiler identification is unknown

-- Check for working C compiler: iwyu-wrapper.bash gcc

CMake Error: your C compiler: "iwyu-wrapper.bash gcc" was not found.   Please set CMAKE_C_COMPILER to a valid compiler path or name.

CMake Error: Internal CMake error, TryCompile configure of cmake failed

-- Check for working C compiler: iwyu-wrapper.bash gcc -- broken

CMake Error at /usr/local/share/cmake-2.8/Modules/CMakeTestCCompiler.cmake:61 (message):

  The C compiler "iwyu-wrapper.bash gcc" is not able to compile a simple test

  program.

  It fails with the following output:

  CMake will not be able to correctly generate this project.

Call Stack (most recent call first):

  CMakeLists.txt:3 (PROJECT)

CMake Error: your C compiler: "iwyu-wrapper.bash gcc" was not found.   Please set CMAKE_C_COMPILER to a valid compiler path or name.

CMake Error: your CXX compiler: "iwyu-wrapper.bash g++" was not found.   Please set CMAKE_CXX_COMPILER to a valid compiler path or name.

-- Configuring incomplete, errors occurred!

David

[Zaheer Chothia]

My bad, I now recall a CMake issue [1] which means you need to set the compiler using the CC/CXX environment variable instead:

    $ CC="/tmp/hello_cmake/iwyu-wrapper clang" CXX="/tmp/hello_cmake/iwyu-wrapper clang++" cmake -G Ninja ..

(Note: pass the full path to the script since it is likely not on the PATH environment.)

I hope that works for you now.

--Zaheer

[1] http://public.kitware.com/Bug/view.php?id=12593

[David Doria]

Ok, it builds now:

doriad@david-lab:~/Test/build$ CC="/home/doriad/Test/build/iwyu-wrapper.bash gcc" CXX="/home/doriad/Test/build/iwyu-wrapper.bash g++" cmake ..

-- The C compiler identification is unknown

-- The CXX compiler identification is unknown

-- Check for working C compiler: /home/doriad/Test/build/iwyu-wrapper.bash

-- Check for working C compiler: /home/doriad/Test/build/iwyu-wrapper.bash -- works

-- Detecting C compiler ABI info

-- Detecting C compiler ABI info - done

-- Check for working CXX compiler: /home/doriad/Test/build/iwyu-wrapper.bash

-- Check for working CXX compiler: /home/doriad/Test/build/iwyu-wrapper.bash -- works

-- Detecting CXX compiler ABI info

-- Detecting CXX compiler ABI info - done

-- Configuring done

-- Generating done

-- Build files have been written to: /home/doriad/Test/build

doriad@david-lab:~/Test/build$ make

Scanning dependencies of target Test

[100%] Building CXX object CMakeFiles/Test.dir/Test.cpp.o

Linking CXX executable Test

[100%] Built target Test

but how do I pass the 

-Xiwyu --mapping_file=iwyu.gcc.imp test.cpp

that Kim told me about (because as of now no output about which headers are unnecessary is produced)?

Thanks for your help so far,

David

[Zaheer Chothia]

You can pass additional IWYU flags when executing 'include-what-you-use' (towards the end of the script).

Bear in mind that output may be produced, but the script redirects those to '.iwyu.log' files (see same directory as the corresponding source file).

--Zaheer

[Ryan Pavlik]

The easiest thing I've found to do is to enable the "CMAKE_EXPORT_COMPILE_COMMANDS" cmake option (It's in at least 2.8.6, not sure when it was added), which creates a compile_commands.json file in the build directory.  I then have a pretty minimal python script that loads that file and uses it to build a command line to call iwyu.  Works great, though the recent mapping_file feature added a little bit of complication.  The script is here: very specific to my system because of hardcoded paths, but shouldn't be too bad to make more generic. https://gist.github.com/4121171

Ryan

[Kim Gräsman]

Hi Ryan,

On Tue, Nov 20, 2012 at 10:14 PM, Ryan Pavlik <ryan....@gmail.com> wrote:
> The easiest thing I've found to do is to enable the
> "CMAKE_EXPORT_COMPILE_COMMANDS" cmake option (It's in at least 2.8.6, not
> sure when it was added), which creates a compile_commands.json file in the
> build directory. I then have a pretty minimal python script that loads that
> file and uses it to build a command line to call iwyu. Works great, though
> the recent mapping_file feature added a little bit of complication. The
> script is here: very specific to my system because of hardcoded paths, but
> shouldn't be too bad to make more generic. https://gist.github.com/4121171

Very nice! For CMake configurations this should work well. I'd love it
if we could bundle that with IWYU.

- Kim

[Ryan Pavlik]

You're certainly welcome to - I didn't submit it as a "patch" since it's rather hacked together - hardcoded paths, no proper command line option or error handling - but you're welcome to include it under the prevailing license in IWYU - I already sent in a contributer agreement.

[Volodymyr Sapsai]

Thanks, Zaheer, Ryan, for sharing tips how to apply IWYU to projects using CMake.

I think CMake workflow is too rough. I'd like to use compile_commands.json file directly, hope that will make using IWYU easier.

Regards,

Volodymyr