LLVM compilation issues

[gz83]

I plan to use the tools\clang\scripts\build.py file to compile LLVM, but when this file runs for a period of time, an error occurs and interrupts, I am running this file on Windows Server 2022+VS2022+SDK22621, the specific error is reported message as follows:

Running ['C:\\Program Files (x86)\\Windows Kits\\10\\bin\\SetEnv.cmd', '/x64', '&&', 'cl.exe', 'adler32.c', 'compress.c', 'crc32.c', 'deflate.c', 'gzclose.c', 'gzlib.c', 'gzread.c', 'gzwrite.c', 'inflate.c', 'infback .c', 'inftrees.c', 'inffast.c', 'trees.c', 'uncompr.c', 'zutil.c', '/nologo', '/O2', '/DZLIB_DLL', ' /c', '/D_CRT_SECURE_NO_DEPRECATE', '/D_CRT_NONSTDC_NO_DEPRECATE']
'"C:\Program Files (x86)\Windows Kits\10\bin\SetEnv.cmd"' is not recognized as an internal or external command,
operable program or batch file.
Failed.

May I ask how should I solve this problem?

[K. Moon]

Do you have a "C:\Program Files (x86)\Windows Kits\10\bin\SetEnv.cmd" file?

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/703dbde6-d034-4074-86c2-9b5c5e23a433n%40chromium.org.

[gz83]

There is no such file in the C:\Program Files (x86)\Windows Kits\10\bin\ directory, but there is this file in the temp folder under the C drive

[K. Moon]

It sounds like you need to install something that'll provide the file in the expected location. (Perhaps the Windows 10 SDK?)

[gz83]

I am currently installing the sdk version 22621, do I need to downgrade the sdk to 20348 and then try to compile LLVM? I installed the 22621 version of the sdk to test whether the new version of the sdk can compile Chromium normally.

[K. Moon]

https://chromium.googlesource.com/chromium/src/+/HEAD/docs/windows_build_instructions.md#setting-up-windows states that you need version 10.0.20348.0 of the Windows 10 SDK installed. According to https://developer.microsoft.com/en-us/windows/downloads/sdk-archive/, 10.0.20348.0 is the latest version of the Windows 10 SDK.

10.0.22621.755 is the latest version of the Windows 11 SDK (note 10 vs. 11), and isn't supported, apparently. I'm sure it will be some day (maybe this is a "contributions welcome" area), but that doesn't seem to be the current state of things. Someone who does more Windows development may be able to chime in and say more.

[K. Moon]

I should also add that building LLVM isn't required to develop Chromium, so it's possible you're hitting pain points most developers wouldn't.

[gz83]

I'm in the process of downgrading the sdk version, Bruce is currently following up on updating the VS artifacts as well as the sdk version, I thought we might let him know and take a look at this discussion.

I'm currently planning to compile Chromium using LLVM with Polly.

[K. Moon]

The Windows 11 SDK presumably has a similar SetEnv.cmd script, so I suspect this is a matter of updating the right paths when using the Windows 11 vs. Windows 10 SDK. Anyway, Bruce is the expert on all things Windows toolchain, so if he's already involved, I'll defer to him.

[gz83]

I just re-tried compiling under sdk20348 and still got the same error

[K. Moon]

Perhaps try debugging build\vs_toolchain.py, and try and figure out how it's computing the path for SetEnv.cmd?

[gz83]

The SetEnv.cmd file on my computer is in the C:\Users\Administrator\AppData\Local\Temp\2\tmptd5by4bm\Windows Kits\10\bin\ directory, it looks like it is not in C:\Program Files (x86)\Windows Kits \10\bin\ directory.

When I manually copy this file to the directory "C:\Program Files (x86)\Windows Kits\10\bin\, an error related to cl.exe appears in the build.py file. The specific error message is as follows:

"cl.exe" is not recognized as an internal or external command,

operable program or batch file.
Failed.

[Bruce Dawson]

I've updated the tracking bug (crbug.com/1273169) with my latest results. I'm not seeing any problems so far, although I need to get permissions to upload an updated toolchain before I can say for sure.

--

Bruce Dawson, he/him

[gz83]

I tried to compile many times and the same error "Cannot find SetEnv.cmd" appeared, and there was no such file in the SDK directory

[gz83]

After I deleted the code related to setenv.cmd in the build.py file, the error related to setenv.cmd disappeared, but then an error related to cl.exe appeared. I solved this error by adding environment variables, and then there will be new errors again. In order to solve it, I used the x64 native command prompt of visual studio, and finally found that after these operations, I can successfully compile what I want, and successfully apply it to the compilation of chromium.

In the process of troubleshooting these errors, I found that Microsoft may have deleted the setenv.cmd file in the previous sdk, which caused an error. In addition, regarding the error of cl.exe, I personally think that chromium may need to re-modify the code of the relevant files so that it can automatically add relevant environment variables, or provide users with some relevant guidance on adding environment variables. Finally, I think using x64 native command prompts is also indispensable.

[Bruce Dawson]

The setenv.cmd file is not part of the Windows SDK. It is added to the packaged toolchain that is used by Google developers. You need to do this:

set DEPOT_TOOLS_WIN_TOOLCHAIN=0

That tells the scripts to not try to use the packaged toolchain (which you do not have) but instead use the installed SDK and Visual Studio.

This should work without modifying your installation or any local files. I have tested this with the Windows 11 SDK and VS 2022 and it seemed to work. Make sure you have all of the required components installed.

--

Bruce Dawson, he/him

[gz83]

This is very strange, the relevant environment variable I checked and it is set, not sure what is wrong.

[Bruce Dawson]

TL;DR - LLVM doesn't seem to support being built without the prepackaged toolchain.

My tests were all with trying to build Chromium. It uses build/toolchain/win/setup_toolchain.py which uses this script path:

script = os.path.normpath(os.path.join(sdk_dir, 'Bin/SetEnv.cmd'))

However! It only uses this script path in this case where DEPOT_TOOLS_WIN_TOOLCHAIN is not set or is set to 1.

But, that is in the context of building Chromium. If you are trying to build LLVM then the rules are different. In tools/clang/scripts/build.py I see this script path being used:

os.path.join(GetWinSDKDir(), 'bin', 'SetEnv.cmd')

Crucially this script path is used always - there is no checking of DEPOT_TOOLS_WIN_TOOLCHAIN when deciding whether or not to use that script path. The GetWinSDKDir() function returns different paths, but bin/SetEnv.cmd is used unconditionally (on Windows).

That is a problem because the SetEnv.cmd file is not a built-in component of the Windows SDK. It is created when we package the SDK. It is created by third_party/depot_tools/win_toolchain/package_from_installed.py.

What Chromium does in the case where there is no packaged toolchain is, roughly speaking, to invoke this script:

    script_path = os.path.normpath(os.path.join(
                                       os.environ['GYP_MSVS_OVERRIDE_PATH'],
                                       'VC/vcvarsall.bat'))

There are other messy details which you would have to look in build/toolchain/win/setup_toolchain.py to find.

So, the summary of all of this is that it looks like LLVM's tools/clang/scripts/build.py script currently does not support building LLVM without a packaged toolchain. I'm sure that fixing this would be worthwhile and patches would be accepted. I would recommend modifying tools/clang/scripts/build.py to incorporate some of the logic from build/toolchain/win/setup_toolchain.py. Modifying your local environment to try to make this work is not recommended because it is an unstable path that won't help others who encounter the same problem.

--

Bruce Dawson, he/him