Temporary Files with Clang on Windows

[oven...@gmail.com]

Hello, I'm still having some issues with tup getting confused about clang's temporary files on windows, even after some of the recent fixes.

My main.c file contains:

int main (int argc, char** argv) {

  return 0;

}

My Tupfile is:

: main.c |> clang -o %o %f |> main.exe

And the error from tup is:

* 0) clang -o main.exe main.c

 *** tup messages ***

tup error: File 'C:/Users/user/AppData/Local/Temp/main-6d5eec.o' was written to, but is not in .tup/db. You probably should specify it as an output

 -- Delete: C:/Users/user/AppData/Local/Temp/main-6d5eec.o

tup error: File 'C:/Users/user/AppData/Local/Temp/lnk{969DF542-7485-45FD-A867-3DA38C84A200}.tmp' was written to, but is not in .tup/db. You probably should specify it as an output

 -- Delete: C:/Users/user/AppData/Local/Temp/lnk{969DF542-7485-45FD-A867-3DA38C84A200}.tmp

 *** Command failed due to errors processing the output dependencies.

 [ ] 100%

 *** tup: 1 job failed.

Also, if I run tup twice in quick succession for this project, I get the following:

deps-31: Permission denied

tup error: Unable to clean out a file in .tup/tmp directory. Please try cleaning this directory manually. Note there may be a stuck sub-process that still has the file open (check the Task Manager).

I don't know if this is just a side effect of the previous issue or if it's a separate problem.

tup version:

tup v0.7.9-43-gf2f02bba

clang version:

clang version 10.0.0

Target: x86_64-pc-windows-msvc

Thread model: posix

InstalledDir: C:\Program Files\LLVM\bin

I've tried to reconcile LLVM's temporary file code with the system call tracking that tup is doing, but so far I haven't been able to get anywhere.  I would appreciate any help on the issue, thanks!

[Mike Shal]

On Wed, Sep 30, 2020 at 12:31 AM oven...@gmail.com <oven...@gmail.com> wrote:

Hello, I'm still having some issues with tup getting confused about clang's temporary files on windows, even after some of the recent fixes.

My main.c file contains:

int main (int argc, char** argv) {

  return 0;

}

My Tupfile is:

: main.c |> clang -o %o %f |> main.exe

And the error from tup is:

* 0) clang -o main.exe main.c

 *** tup messages ***

tup error: File 'C:/Users/user/AppData/Local/Temp/main-6d5eec.o' was written to, but is not in .tup/db. You probably should specify it as an output

 -- Delete: C:/Users/user/AppData/Local/Temp/main-6d5eec.o

tup error: File 'C:/Users/user/AppData/Local/Temp/lnk{969DF542-7485-45FD-A867-3DA38C84A200}.tmp' was written to, but is not in .tup/db. You probably should specify it as an output

 -- Delete: C:/Users/user/AppData/Local/Temp/lnk{969DF542-7485-45FD-A867-3DA38C84A200}.tmp

 *** Command failed due to errors processing the output dependencies.

 [ ] 100%

 *** tup: 1 job failed.

Thanks for the report! I installed VS2019 and the clang from there, and was able to reproduce this. It looks like the temporary file logic changed again, and now it uses the FILE_DELETE_ON_CLOSE flag in NtCreateFile(). I also updated the default VS environment variables, so clang should pick up the linker from MSVC by default if you aren't using lld.

Can you try tup-v0.7.9-49-g715e4657.zip and see if it works for you?

 

Also, if I run tup twice in quick succession for this project, I get the following:

deps-31: Permission denied

tup error: Unable to clean out a file in .tup/tmp directory. Please try cleaning this directory manually. Note there may be a stuck sub-process that still has the file open (check the Task Manager).

I don't know if this is just a side effect of the previous issue or if it's a separate problem.

I believe this is a separate problem. I've hit this before, and in my case it was VCTIP.EXE that was holding the deps file open to do some telemetry reporting. I'm not sure if there's anything tup can do in this case - it would be nice if there was a way to tell visual studio to ignore those files.

You can disable this in Visual Studio under Help -> Send Feedback -> Settings (see: https://docs.microsoft.com/en-us/visualstudio/ide/visual-studio-experience-improvement-program?view=vs-2019). I'm not sure if you'd want to keep it on for other reasons, though.

 

I've tried to reconcile LLVM's temporary file code with the system call tracking that tup is doing, but so far I haven't been able to get anywhere.  I would appreciate any help on the issue, thanks!

Usually when debugging issues like this, I find it helpful to run tup while ProcMon is running, and then filter on the filename that tup complains about. You would still have to trace through the dllinject.c code to find out how tup is handling the various system calls, but at least you'd see exactly what Windows is seeing so you know why it thinks a file is temporary, or was renamed/deleted, etc.

-Mike

[oven...@gmail.com]

Everything is working now, thanks for the quick response!

(I wasn't able to get vctip.exe to release its hold on the deps file by disabling telemetry.  However, I deleted vctip.exe and, while probably a bad idea, it fixed the second issue).