Error linking large object list on windows - 1.38.34

[John Harvey]

I am providing a large list of .bc files with a response file. 

cmd /c em++ --bind -s ALLOW_MEMORY_GROWTH=1 -s NO_EXIT_RUNTIME=1 -s FULL_ES2=1 --memory-init-file 0 -s ERROR_ON_UNDEFINED_SYMBOLS=1 -s DISABLE_EXCEPTION_CATCHING=0 -s EMBIND_STD_STRING_IS_UTF8=0 -s EXTRA_EXPORTED_RUNTIME_METHODS=[\'Pointer_stringify\'] -s AGGRESSIVE_VARIABLE_ELIMINATION=1 -O3 -s ASSERTIONS=0 -s DEMANGLE_SUPPORT=0 -DNDEBUG=1 -s DISABLE_EXCEPTION_CATCHING=0 --std=c++14 --llvm-lto 1  -o d:/scratch/pview_objects/vissrc_thingview1/js_wasm.o/js/ptc/thingview/libthingview_wasm.js @libthingview.tmp

During the linking this now runs opt.exe with a long list of .bc files expanded from the response file (in 1.38.24 it passed a response file to opt.exe) which has too long a command line for windows to run.

Traceback (most recent call last):

  File "d:\compilers\emscripten-1.38.34\fastcomp\emscripten\emcc.py", line 3391, in <module>

    sys.exit(run(sys.argv))

  File "d:\compilers\emscripten-1.38.34\fastcomp\emscripten\emcc.py", line 1944, in run

    final = shared.Building.llvm_opt(final, link_opts, DEFAULT_FINAL)

  File "d:\compilers\emscripten-1.38.34\fastcomp\emscripten\tools\shared.py", line 2110, in llvm_opt

    run_process(cmd, stdout=PIPE)

  File "d:\compilers\emscripten-1.38.34\fastcomp\emscripten\tools\shared.py", line 178, in run_process

    proc = Popen(cmd, *args, **kw)

  File "d:\compilers\emscripten-1.38.34\python\2.7.13.1_64bit\python-2.7.13.amd64\lib\subprocess.py", line 390, in __init__

    errread, errwrite)

  File "d:\compilers\emscripten-1.38.34\python\2.7.13.1_64bit\python-2.7.13.amd64\lib\subprocess.py", line 640, in _execute_child

    startupinfo)

WindowsError: [Error 206] The filename or extension is too long

EMCC_DEBUG output for this part of the linking in 1.38.34 is :-

shared:DEBUG: considering archive d:\users\jpharvey\.emscripten_cache_1.38.34\asmjs\libcompiler_rt.a

shared:DEBUG: adding object d:\users\jpharvey\appdata\local\temp\emscripten_temp_abyc_l_archive_contents\muldc3.c.o to link

shared:DEBUG: adding object d:\users\jpharvey\appdata\local\temp\emscripten_temp_abyc_l_archive_contents\mulsc3.c.o to link

shared:DEBUG: done running loop of archive d:\users\jpharvey\.emscripten_cache_1.38.34\asmjs\libcompiler_rt.a

shared:DEBUG: adding object d:\users\jpharvey\.emscripten_cache_1.38.34\asmjs\libc++abi.bc to link
emcc:DEBUG: emcc step "link" took 3.08 seconds

emcc:DEBUG: saving intermediate processing steps to D:\Users\jpharvey\AppData\Local\Temp\emscripten_temp

emcc:DEBUG: (not saving intermediate D:\Users\jpharvey\AppData\Local\Temp\emscripten_temp\emcc-0-basebc.bc because deferring linking)

mp_ictraz_archive_contents\\operations.cpp.o'] - ()

run ['d:/compilers/emscripten-1.38.34/fastcomp/fastcomp/bin\\llvm-nm.exe', 'd:\\users\\jpharvey\\appdata\\local\\temp\\emscripten_temp_abyc_l_archive_contents\\divdc3.c.o'] - ()

run ['d:/compilers/emscripten-1.38.34/fastcomp/fastcomp/bin\\llvm-nm.exe', 'd:\\users\\jpharvey\\appdata\\local\\temp\\emscripten_temp_abyc_l_archive_contents\\divsc3.c.o'] - ()

run ['d:/compilers/emscripten-1.38.34/fastcomp/fastcomp/bin\\llvm-nm.exe', 'd:\\users\\jpharvey\\appdata\\local\\temp\\emscripten_temp_abyc_l_archive_contents\\muldc3.c.o'] - ()

run ['d:/compilers/emscripten-1.38.34/fastcomp/fastcomp/bin\\llvm-nm.exe', 'd:\\users\\jpharvey\\appdata\\local\\temp\\emscripten_temp_abyc_l_archive_contents\\mulsc3.c.o'] - ()

run ['d:/compilers/emscripten-1.38.34/fastcomp/fastcomp/bin\\opt.exe', 'd:\\scratch\\pview_objects\\vissrc_thingview1\\js_wasm.o\\src_js\\tvapi\\thingview\\html\\ThingView_html.bc', 'd:\\scratch\\pview_objects\\vissrc_thingview1\\js_wasm.o\\src_js\\tvapi\\tvapi\\html\\Appearance_html.bc', 'd:\\scratch\\pview_objects\\vissrc_thingview1\\js_wasm.o\\src_js\\tvapi\\tvapi\\html\\Application_html.bc', 'd:\\scratch\\pview_objects\\vissrc_thingview1\\js_wasm.o\\src_js\\tvapi\\tvapi\\html\\BoundMarker_html.bc', 'd:\\scratch\\pview_objects\\vissrc_thingview1\\js_wasm.o\\src_js\\tvapi\\tvapi\\html\\CVApplication_html.bc', 'd:\\scratch\\pview_objects\\vissrc_thingview1\\js_wasm.o\\src_js\\tvapi\\tvapi\\html\\CV_js_Interface.bc', 'd:\\scratch\\pview_objects\\vissrc_thingview1\\js_wasm.o\\src_js\\tvapi\\tvapi\\html\\DocumentScene_html.bc', 'd:\\scratch\\pview_objects\\vissrc_thingview1\\js_wasm.o\\src_js\\tvapi\\tvapi\\html\\DocumentView_html.bc', 'd:\\scratch\\pview_objects\\vissrc_thingview1\\js_wasm.o\\src_js\\tvapi\\tvapi\\html\\HttpRequest.bc', 'd:\\scratch\\pview_objects\\vissrc_thingview1\\js_wasm.o\\src_js\\tvapi\\tvapi\\html\\ItemsList_html.bc', 'd:\\scratch\\pview_objects\\vi …..

followed by the traceback above.

For 1.38.26 it is :-

shared:DEBUG: adding object d:\users\jpharvey\appdata\local\temp\emscripten_temp_knfx1d_archive_contents\muldc3_24e31f29.c.o to link
shared:DEBUG: adding object d:\users\jpharvey\appdata\local\temp\emscripten_temp_knfx1d_archive_contents\mulsc3_0d7e9a2a.c.o to link
shared:DEBUG: done running loop of archive d:\users\jpharvey\.emscripten_cache_1.38.26\asmjs\libcompiler_rt.a
shared:DEBUG: adding object d:\users\jpharvey\.emscripten_cache_1.38.26\asmjs\libc++abi.bc to link
shared:DEBUG: using response file for llvm-link
emcc:DEBUG: emcc step "link" took 2.88 seconds
emcc:DEBUG: saving intermediate processing steps to D:\Users\jpharvey\AppData\Local\Temp\emscripten_temp
emcc:DEBUG: (not saving intermediate D:\Users\jpharvey\AppData\Local\Temp\emscripten_temp\emcc-0-basebc.bc because deferring linking)
shared:DEBUG: successfully executed d:/compilers/emscripten-1.38.26/clang/e1.38.26_64bit\opt.exe @D:\Users\jpharvey\AppData\Local\Temp\emscripten_temp\tmp1tc4dj.response -strip-debug -disable-verify -internalize -internalize-public-api-list=main,malloc,free,__errno_location,__cxa_can_catch,__cxa_is_pointer_type,realloc,_get_environ,e,m,s,c,r,i,p,t,n,_,w,b,g,l,a,k,o,x,u -std-link-opts -disable-loop-vectorization -disable-slp-vectorization -vectorize-loops=false -vectorize-slp=false -o D:\Users\jpharvey\AppData\Local\Temp\emscripten_temp\libthingview_wasm.bc
emcc:DEBUG: emcc step "post-link" took 102.98 seconds

Thanks

John

[Alon Zakai]

Thanks for the bug report! I think this should fix it, please test:

https://github.com/emscripten-core/emscripten/pull/8784

However I am puzzled by this, since I don't think we changed this behavior. Going back to 1.38.26 I don't see us emitting a response file either - perhaps it depends on other specific arguments somehow? In any case, the fix in the link above seems obvious.

- Alon

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/f3b085b7-6213-4dd5-9fb0-849c2233aae1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Alon Zakai]

Ah, maybe we used to special-case windows and only emit a response file for it, back then. That could explain why I fail to see it.

- Alon

[John Harvey]

Thank you for the quick fix. That fixes the problem

To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.