Using LLVM Directly to Generate WebAssembly Module (vanilla LLVM targeting WebAssembly)?

[Osman Zakir]

Hi, everyone.

I have some problem with Emscripten's gen_struct_info.py, so I want to try generating WebAssembly using LLVM directly.  I'd also have to know how to write the JavaScript myself, of course, but it's better than nothing.  (Note: the problem with the gen_struct_info I'm having is described in this Emscripten GitHub issue).

So yeah, how do I compile C++ code to WebAssembly using LLVM directly?  I'm using Windows 10 x64.  

[Osman Zakir]

I'll also add this here: I did try what's shown here.  It didn't work; I got an error saying that it can't find stdio.h.  I can generate regular .exe files with Clang just fine, though.

[Thomas Lively]

Hi Osman,

Your clang cannot find stdio.h because it does come with a C standard library. As part of using bare clang/lld you will have to roll your own toolchain including standard libraries. Even if you did that, in order to use stdio.h you would have to implement emulated POSIX I/O syscalls, which would not be trivial to get right.

Building your own installation of Emscripten is rather complex, as you have discovered. Why don't you use the emsdk to install emscripten instead? https://emscripten.org/docs/getting_started/downloads.htm

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/cd408530-4e23-4912-9841-f40b1baf096f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Osman Zakir]

I do have Emscripten.  I just have a problem with gen_struct_info due to which I can't generate JavaScript glue code and a WebAssembly module when not using the "-s SIDE_MODULE=1" setting and can't generate a decent, non-corrupt (I assume) WebAssembly module when using the "-s SIDE_MODULE" setting.  Didn't you look at the GitHub issue I linked to?  

This are the contents of the first post there:

This time I've built LLVM 9.0.0 with the -A x64 and -Thost=x64 flags and also built Binaryen and Emscripten's asm.js optimizer again, then tried to compile some C code to WebAssembly.  But I get the same error as before:

Traceback (most recent call last):
  File "C:\emscripten\emcc.py", line 3316, in <module>
    sys.exit(run(sys.argv))
  File "C:\emscripten\emcc.py", line 1947, in run
    final = shared.Building.emscripten(final, target + '.mem', js_libraries)
  File "C:\emscripten\tools\shared.py", line 2197, in emscripten
    emscripten.run(infile, outfile, memfile, js_libraries)
  File "C:\emscripten\emscripten.py", line 2560, in run
    shared.Settings.STRUCT_INFO = shared.Cache.get(generated_struct_info_name, ensure_struct_info)
  File "C:\emscripten\tools\cache.py", line 133, in get
    temp = creator()
  File "C:\emscripten\emscripten.py", line 2557, in ensure_struct_info
    gen_struct_info.main(['-qo', out, path_from_root('src', 'struct_info.json')])
  File "C:\emscripten\tools\gen_struct_info.py", line 529, in main
    struct_info = inspect_code(header_files, cpp_opts, structs, defines)
  File "C:\emscripten\tools\gen_struct_info.py", line 407, in inspect_code
    info = shared.run_js(js_file[1]).splitlines()
  File "C:\emscripten\tools\shared.py", line 1090, in run_js
    return jsrun.run_js(filename, engine, *args, **kw)
  File "C:\emscripten\tools\jsrun.py", line 155, in run_js
    raise Exception('Expected the command ' + str(command) + ' to finish with return code ' + str(assert_returncode) + ', but it returned with code ' + str(proc.returncode) + ' instead! Output: ' + str(ret)[:error_limit])
Exception: Expected the command ['C:\\Program Files\\nodejs\\node.exe', 'c:\\users\\osman\\appdata\\local\\temp\\tmpadbhvx.js'] to finish with return code 0, but it returned with code -1073741819 instead! Output: 

(The Node output from above is empty, by the way).  

The command I ran was:

emcc -std=c11 -Wall -pedantic main.c -Os -s WASM=1 -s SIDE_MODULE=1 -s BINARYEN_ASYNC_COMPILATION=0 -o main.wasm

And it gave me a .wasm file -- but the file is corrupted.  When I tried to click "Show WebAssembly" in the context menu that comes up when right-clicking on a .wasm file in Visual Studio Code's left pane, I had these two errors:

Unable to open 'main.wasm': resource is not available.

and

readWasm failed: 000007b: error: invalid linking metadata version: 2

Any help or advice is greatly appreciated.  Thanks in advance.

____________________________________________________________________

Anyway, yeah, I do have Emscripten; I just want to use the Wasm Backend with it instead of Fastcomp.  

[Thomas Lively]

Yes, what I meant about building your own installation of emscripten is building LLVM and BInaryen and the other tools yourself. There shouldn't be any need to do that. In particular, if you get your checkout of LLVM and the rest of the emscripten tools from different places I would not expect them to work together correctly. That is why you are seeing errors when trying to use the tools. Using emsdk will get you a complete installation of emscripten that does not require you to build anything yourself and it should not have any of the issues you are seeing.

--

You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/1320184f-e7d6-451f-8280-63a6e9742859%40googlegroups.com.

[Osman Zakir]

The reason I don't want to use Emsdk is mainly because I want to have control over the installation of Emscripten and the version of Clang I'm using.

[Thomas Lively]

Unfortunately that is a recipe for an avalanche of problems and because there are so many components being developed in parallel there is essentially no way to make that work manually without being at tip of tree of anything, which is inherently unstable. emsdk gives you complete control over the version of emscripten you have installed and lets you switch between versions easily. We cannot provide support for trying to roll your own toolchain when emsdk would already do what you need.

--

You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/5675fc59-ec8a-4908-8a55-be23a23383ca%40googlegroups.com.

[Osman Zakir]

Okay, I'll use Emsdk for WebAssembly then.  But even then, I'd still like to know how I can have Clang itself produce a Wasm module directly.  I'll use the stable version instead of the tip-of-the-tree one for it though.  

[Osman Zakir]

There should be a way to get sysroot_wasi to work under Windows, right?  I could try to use that (I need to how first, though).

[Thomas Lively]

Yes, you could try using WASI and the WASI web polyfill instead. I haven't played with that myself so I'm not sure how it works yet.

--

You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/543abae6-f7d0-401a-a625-b2551ff2371d%40googlegroups.com.

[Osman Zakir]

If I want to use my own LLVM/Clang, Python and Node with the EMSDK and only want the latest stable versions of Emscripten and Binaryen, what exact tag should I install and activate when invoking the emsdk command?  What version of those two would be good to use with the Node (latest stable version), Python (2.7 of course) downloaded from their respective official sites, and LLVM 9?  I've decided to try using my own LLVM, Node and Python with the EMSDK, using the ones from the SDK itself for Emscripten and Binaryen.  And if I have Java installed and want to use that instance of it with the closure compiler, what value should I put for the "CLOSURE_COMPILER" path in .emscripten?  

[Osman Zakir]

I also want to ask: I have both the JDK and the JRE, but I only need the latter for the closure compiler, right?  And what does the value for the "CLOSURE_COMPILER" have to be in the .emscripten file?  The "JAVA" variable with the value set to the path to the JRE installation?

[Osman Zakir]

Okay, I think I got the right tags and such; I just need to know how to change the command that the EMSDK runs when calling CMake.  I want to fix a certain mistake I've found.