[llvm-dev] Question about the build

[Paul C. Anagnostopoulos via llvm-dev]

If I rebuild TableGen and then build lib/Target/all, shouldn't Ninja rebuild all the target .inc files and then all the files that include those .inc files? I realize that identical .inc files are not written, but it's not even running TableGen on the .td files.

I swear it was rebuilding a couple of weeks ago, but it isn't now. I'm doubting my mental faculties on this issue.

_______________________________________________
LLVM Developers mailing list
llvm...@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

[Fāng-ruì Sòng via llvm-dev]

It works fine for me.

touch a file in llvm/utils/TableGen/
ninja lib/Target/all
# llvm-tblgen is rebuilt
# include/llvm/IR/IntrinsicsXcore.h
lib/Target/AArch64/AArch64GenSystemOperands.inc
lib/Target/AArch64/AArch64GenAsmWriter1.inc etc are rebuilt

--
宋方睿

[Paul C. Anagnostopoulos via llvm-dev]

When I do that, Ninja builds:

PseudoLoweringEmitter.cpp [the file I touched]
llvm-tblgen.exe

If I touch a target's .td file, then all all the .inc files are rebuilt, as expected.

About a week ago I cleared out the CMake cache and the CmakeFiles directory. Could that be related? I'm not sure why that would change file dependencies.

[Paul C. Anagnostopoulos via llvm-dev]

I deleted build.ninja and ran CMake again. It didn't help. Touching a file in utils/Tablegen and building the targets still only builds TableGen, not the .inc files.

[Paul C. Anagnostopoulos via llvm-dev]

I think my problem has something to do with NATIVE building. When I build all the targets, CMake gets run again and ends with this message:

-- Build files have been written to: D:/LLVM/build/NATIVE

Where did that come from? Here is my cmake command:

cmake -G "Ninja" -S c:\llvm\llvm-project\llvm -DCMAKE_BUILD_TYPE=Debug -DCMAKE_LINKER:FILEPATH="C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.28.29333/bin/Hostx64/x64/link.exe" -DLLVM_BUILD_DOCS=ON -DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_ENABLE_LIBXML2=OFF -DLLVM_ENABLE_PROJECTS=clang;lld;llvm;mlir -DLLVM_ENABLE_SPHINX=ON -DLLVM_HOST_TRIPLE:STRING=x86_64-pc-windows-msvc -DLLVM_OPTIMIZED_TABLEGEN=ON -DLLVM_DEFAULT_TARGET_TRIPLE:STRING=X86_64-pc-windows-msvc -DLLVM_TARGETS_TO_BUILD=AMDGPU;ARM;AVR;BPF;Lanai;Mips;NVPTX;SystemZ;WebAssembly -DSPHINX_OUTPUT_HTML=ON -DSPHINX_WARNINGS_AS_ERRORS=OFF

Did I screw up by specifying both a host triple and a target triple?

~~ Paul

[Paul C. Anagnostopoulos via llvm-dev]

Sorry to keep posting, but I'm hoping continued evidence will enable someone to understand what is going on.

If I touch a lib/TableGen file and then build bin/llvm-tblgen.exe, Ninja rebuilds build/bin/llvm-tblgen.exe

If I touch a file and then build lib/Target/all, Ninja rebuilds both build/bin/llvm-tblgen.exe and build/NATIVE/bin/llvm-tblgen.exe. In neither case does it rebuild any .inc files.

[Reid Kleckner via llvm-dev]

This seems like a bug in LLVM_OPTIMIZED_TABLEGEN. What you are describing works in the standard configuration.

[Paul C. Anagnostopoulos via llvm-dev]

Thank you, sir! Indeed, -DLLVM_OPTIMIZED_TABLEGEN=ON was the problem. Everything is back to normal with =OFF.

At 12/19/2020 08:48 PM, Reid Kleckner wrote:
>This seems like a bug in LLVM_OPTIMIZED_TABLEGEN. What you are describing works in the standard configuration.

----------------------------------------------------------------
Windfall Paul C. Anagnostopoulos
----------------------------------------------------------
Software 978 369-0839
www.windfall.com
----------------------------------------------------------------
My life has been filled with calamities,
some of which actually happened.
---Mark Twain

Guga 'mzimba, sala 'nhliziyo

[James Henderson via llvm-dev]

For what it's worth, I've recently been running into problems related to the NATIVE tablegen build for optimized tablegen usage in debug builds. In my case, it was (and may still be) unnecessarily rebuilding tablegen related files every time, even though nothing has changed, which in turn causes the rest of the build to trigger (as Visual Studio sees things as being out-of-date). I forget the exact details, but can dig them out later if someone is interested in investigating further. I haven't had the time to dig into this properly myself, but it is frequently hurting my productivity.

[Paul C. Anagnostopoulos via llvm-dev]

I was having the opposite problem: Building the native TableGen did not trigger downstream rebuilds.

Could you see if the problem is still happening and then post some details here? I know almost nothing about the build process, but I can investigate whether TableGen is unnecessarily rewriting identical output files.

At 1/4/2021 03:32 AM, James Henderson wrote:
>For what it's worth, I've recently been running into problems related to the NATIVE tablegen build for optimized tablegen usage in debug builds. In my case, it was (and may still be) unnecessarily rebuilding tablegen related files every time, even though nothing has changed, which in turn causes the rest of the build to trigger (as Visual Studio sees things as being out-of-date). I forget the exact details, but can dig them out later if someone is interested in investigating further. I haven't had the time to dig into this properly myself, but it is frequently hurting my productivity.

[James Henderson via llvm-dev]

I'm not sure what sort of details exactly you want. I use Visual Studio 2017 to build my LLVM checkout. The LLVM_OPTIMIZED_TABLEGEN setting is enabled, and when I build the Debug build it always complains that a number of the projects that use tablegen are out of date. Specifically, if I turn up the build log verbosity to "Detailed", I get a large number of projects which say something like this:

23>------ Up-To-Date check: Project: utils\TableGen\LLVM-tablegen-host.vcxproj, Configuration: Debug x64 ------

23>Project is not up-to-date: build input 'c:\llvm\build\native\release\bin\llvm-tblgen' is missing.

Subsequently, tablegen runs and emits "Building Options.inc..." and similar lines for all tablegen inputs, rather than doing nothing, resulting in virtually the whole build being run ultimately.

Note that c:\llvm\build\native\release\bin\llvm-tblgen.exe exists and the project build to build that executable says there is nothing to update. When investigating this last year, I found that applying a small change to CrossCompile.cmake to add ".exe" to the end of the output_path for the optimized tablegen output was sufficient to fix things, but I don't think that's a realistic long-term solution, as it's not cross-platform.

I don't really know enough about cmake or tablegen to have any further clue what to do to fix things.

James

[Michael Kruse via llvm-dev]

Am Di., 5. Jan. 2021 um 05:02 Uhr schrieb James Henderson via llvm-dev
<llvm...@lists.llvm.org>:

> Note that c:\llvm\build\native\release\bin\llvm-tblgen.exe exists and the project build to build that executable says there is nothing to update. When investigating this last year, I found that applying a small change to CrossCompile.cmake to add ".exe" to the end of the output_path for the optimized tablegen output was sufficient to fix things, but I don't think that's a realistic long-term solution, as it's not cross-platform.

To just get the extension, one can make use of CMAKE_EXECUTABLE_SUFFIX
(https://cmake.org/cmake/help/latest/variable/CMAKE_EXECUTABLE_SUFFIX.html)
within cmake. The idiomatic solution within the same cmake build would
be to use the `$<TARGET_FILE:llvm-tblgen>` generator expression.

Additionally, tblgen-gen should check whether content of the target
file is identical before overwriting an existing file, such that its
time stamp does not change and dependent files do not need to be
rebuilt. This also seems to not always work, but I don't know whether
this is a limitation of Visual Studio/MSBuild, e.g. does not re-check
whether files in a dependency chain actually have changed. In my
experience, it is often that the mlir-tablegen dependees are rebuilt
regardless, but clang-tblgen and llvm-tblgen seem to work.

Michael