How does clang++.exe work around the max command-line length limitation of the shell in Windows?

[Horatio Chien]

Hi all,

I am working a project which has its own Waf-based build system on Win7. It supports GCC compiler of NDK now.

Now I am trying to incorporate Clang into the build system. The NDK version is r10e.

This build system use a "read-options-from-file" technique to work around the max command-line length limitation of the shell in Win7.

It means that it create a temporary file and put options/parameters into it, and then call gcc/g++ like this:

arm-linux-androideabi-g++.exe @c:\\users\\username\\appdata\\local\\temp\\tmpms48hj

P.S. tmpms48hj is that temp file. It contains a lot of compiler options and object files when linking. One item per line.

I tried to use the same technique for Clang.exe/Clang++.exe. It works for clang-3.5 for compiling. (doesn't work for clang-3.6, but this is another problem)

But, it doesn't work for linking. I got the following error message:

clang++.exe @c:\\users\\username\\appdata\\local\\temp\\tmpix_rry

clang++.exe: error: unable to execute command: Couldn't execute program 'P:\android\android-ndk-r10e\toolchains\arm-linux-androideabi-4.8\prebuilt\windows-x86_64/lib/gcc/arm-linux-androideabi/4.8/../../../../arm-linux-androideabi/bin\ld.exe'The filename or extension is too long.

I believe that NDK has its own technique to overcome the problem of command-line length limitation. Otherwise it can't build large/complex project.

Can someone tell me how NDK achieve that? Which file I should look into?

Besides, since the same technique works for GCC but not Clang, I suppose that Clang has something different, need to be treated differently. Does anyone have clue? 

What should I do to let clang++.exe link a lot of object files? 

Thank you very much.

[Jarkko Pöyry]

I've just hit the same problem.

I know this comes too late for you, but if anybody else stumbles here searching for a solution (like I did), here's a workaround:

The problem is caused by a known miscalculation in clang's commandLineFitsWithinSystemLimits function which underestimates the command line length. To "fix" the issue, just add some dummy cpp files until so that even the incorrectly underestimated command line would exceed the platform limits. 2 - 5 dummy files should do it, or alternatively just one dummy with a long filename. This causes clang to invoke the linker with a command file like it should.

[Adam Brown]

Do you mean adding dumby files to the linker via LOCAL_LDLIBS?

[Jarkko Pöyry]

That might work too. But what I did was just add some empty files with long names to LOCAL_SRC_FILES or equivalent.

[Adam Brown]

Huh, my LOCAL_SRC_FILES is actually pretty massive already, maybe this is a different problem...

[Jarkko Pöyry]

It could be a different problem.

In my case it was caused by clang trying to invoke the linker with too long a command line, and the workaround only works for that particular case (note that long file name means usually like 200 characters, the number of characters required was IIRC about 3*number of compile units in your project + 1).

But you might have the limit exceeded earlier in the pipeline. I can only propose running with LOCAL_SHORT_COMMANDS:=true but if you are already doing it, I have no further tricks up my sleeve. Good luck.

[Adam Brown]

Thanks, ya it looks like it's a regression in the tool chain:

https://code.google.com/p/android/issues/detail?id=225119