Guidance on debugging LLVM errors

[Tom Short]

The Julia language [1] uses LLVM's JIT to dynamically create
code. It's closer to being able to generate static code [2],
including (I think) LLVM bitcode.

As a trial, I used Julia to generate a large bitcode
file (sys.bc) for much of Julia's standard set of functions. For
simple functions that use standard LLVM types, a recent
Emscripten (fastcomp) generates good JavaScript. For more complex
functions that use other types, I'm getting a variety of errors.
Here is an example of a common error:

Value: %Complex.16 %0
LLVM ERROR: Unrecognized struct value
Traceback (most recent call last):
  File "/usr/bin/emcc", line 1200, in <module>

The disassembled code for the function that triggered this error
looks like this:


define %Complex.15 @"julia_sin;108403"(%Complex.15) {
top:
  %1 = load %jl_value_t** @"*Main.Core.Float3267512", align 8
  %2 = getelementptr inbounds %jl_value_t* %1, i64 1, i32 0
  %3 = load %jl_value_t** %2, align 8
  %4 = extractvalue %Complex.15 %0, 0, !julia_type !98
  %5 = call float @"julia_convert;85919"(%jl_value_t* %3, i16 %4)
  %6 = load %jl_value_t** @"*Main.Core.Float3267512", align 8
  %7 = getelementptr inbounds %jl_value_t* %6, i64 1, i32 0
  %8 = load %jl_value_t** %7, align 8
  %9 = extractvalue %Complex.15 %0, 1, !julia_type !98
  %10 = call float @"julia_convert;85919"(%jl_value_t* %8, i16 %9)
  %11 = insertvalue %Complex.16 undef, float %5, 0
  %12 = insertvalue %Complex.16 %11, float %10, 1, !julia_type !103
  %13 = call %Complex.16 @"julia_sin;89568"(%Complex.16 %12), !julia_type !103
  %14 = load %jl_value_t** @"*Main.Core.Float1667510", align 8
  %15 = getelementptr inbounds %jl_value_t* %14, i64 1, i32 0
  %16 = load %jl_value_t** %15, align 8
  %17 = extractvalue %Complex.16 %13, 0
  %18 = call i16 @"julia_convert;85922"(%jl_value_t* %16, float %17), !julia_type !98
  %19 = load %jl_value_t** @"*Main.Core.Float1667510", align 8
  %20 = getelementptr inbounds %jl_value_t* %19, i64 1, i32 0
  %21 = load %jl_value_t** %20, align 8
  %22 = extractvalue %Complex.16 %13, 1
  %23 = call i16 @"julia_convert;85922"(%jl_value_t* %21, float %22), !julia_type !98
  %24 = insertvalue %Complex.15 undef, i16 %18, 0
  %25 = insertvalue %Complex.15 %24, i16 %23, 1, !julia_type !104
  ret %Complex.15 %25
}

The type mentioned in the error was defined earlier as:

%Complex.16 = type { float, float }

My main question is: where do I start to try to debug LLVM errors
like this? Are there LLVM references that'd help? Any way to use
Clang or other LLVM tool to try to narrow it down or test this?
The .bc code from Julia could well be wrong or be missing code,
but I don't really know where to start here.



[1] http://julialang.org
[2] https://groups.google.com/d/msg/julia-dev/qdnggTuIp9s/BoQSNGNXxV0J

[Alon Zakai]

This is likely a limitation on the IR we can currently process. We use some of the PNaCl lowering passes to simplify LLVM IR, then run on that, which works for basically everything clang emits, but some things clang doesn't emit, might not work. This looks like that type of bug.

You can grep for that string ("Unrecognized struct value") in the fastcomp llvm repo, to see where it happens. Likely in lib/Transform/NaCl somewhere.

- Alon

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Tom Short]

I tried compiling several of the functions with fastcomp off, and that
allowed many more functions to be compiled to JavaScript successfully.

Also, Julia is switching to MCJIT at some point, and that may change
the bitcode generation.

> You received this message because you are subscribed to a topic in the
> Google Groups "emscripten-discuss" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/emscripten-discuss/Gxk678Fv2rc/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to

[JF Bastien]

Tom,

Please let us know if we can help improve the PNaCl passes so that Emscripten as well as PNaCl can be targets for Julia. native-client-discuss@ may be a better forum for the PNaCl passes, but we do work with Alon to make sure Emscripten works well :)

JF

[Tom Short]

Thanks for the offer, JF. I'll see if I can find a reduced bitcode
file that displays some of the issues I'm running into.

Tom

[JF Bastien]

That would be very helpful, thanks. We definitely want to support more than just bitcode generated by Clang.

FWIW switching to MCJIT won't change anything if you only do static code generation: the "stable" bitcode you give Emscripten/PNaCl doesn't go through MCJIT before becoming "stable".

[Tom Short]

For the "Unrecognized struct value" error, I wasn't able to find a
reduced case, so I'm including a link to a large emcc-0-basebc.bc
file:

http://1drv.ms/1vE5fXs

Here is the error I get with Emscripten:

[tshort@bridgelinux julia]$ emcc -v
/tmp/emscripten_temp/emcc-0-basebc.bc -o out.js -s
EXPORTED_FUNCTIONS="['_julia_abs;104547']"
WARNING root: invocation: /usr/bin/emcc -v
/tmp/emscripten_temp/emcc-0-basebc.bc -o out.js -s
EXPORTED_FUNCTIONS=['_julia_abs;104547'] (in
/home/tshort/emjl/usr/lib/julia)
INFO root: (Emscripten: Running sanity checks)
INFO root: (Emscripten: Running sanity checks)
DEBUG root: compiling to bitcode
DEBUG root: emcc step "parse arguments and setup" took 0.11 seconds
DEBUG root: using bitcode file: /tmp/emscripten_temp/emcc-0-basebc.bc
DEBUG root: emcc step "bitcodeize inputs" took 0.00 seconds
DEBUG root: emcc step "process inputs" took 0.00 seconds
DEBUG root: will generate JavaScript
DEBUG root: emcc step "calculate system libraries" took 2.69 seconds
DEBUG root: emcc step "link" took 0.04 seconds
DEBUG root: saving intermediate processing steps to /tmp/emscripten_temp
DEBUG root: emcc: LLVM opts: -strip-debug -internalize
-internalize-public-api-list=julia_abs;104547,malloc,free,malloc
-globaldce -pnacl-abi-simplify-preopt -pnacl-abi-simplify-postopt
-enable-emscripten-cxx-exceptions
Value: %27 = call { i64, i1 } @llvm.ssub.with.overflow.i64(i64 %26, i64 1)

LLVM ERROR: Unrecognized struct value

INFO root: emcc saved files are in:/home/tshort/emjl/usr/lib/julia/mtmp

Traceback (most recent call last):
File "/usr/bin/emcc", line 1200, in <module>

shared.Building.llvm_opt(final, link_opts)
File "/usr/lib/emscripten/tools/shared.py", line 1358, in llvm_opt
assert os.path.exists(target), 'Failed to run llvm optimizations: ' + output
AssertionError: Failed to run llvm optimizations:


This started with a sys.bc generated by Julia. To create this, I took
a recent Julia with the following changes made to the main Julia
Makefile:

- $(call spawn,$(JULIA_EXECUTABLE)) --build $(call
cygpath_w,$(build_private_libdir)/sys) \
+ $(call spawn,$(JULIA_EXECUTABLE)) --build $(call
cygpath_w,$(build_private_libdir)/sys) --dump-bitcode=yes
--compile=all \

Tom


On Fri, Aug 29, 2014 at 3:48 PM, 'JF Bastien' via emscripten-discuss

[Tom Short]

Here's another error I've come across ("some i64 thing we can't
legalize yet"). This happens later in the compile process, so I can
provide a link to a reduced case. Here is a link to an emcc-1-.bc
file:

http://1drv.ms/1sUR77k

Here is the error for this case:

[tshort@bridgelinux julia]$ emcc -v

/tmp/emscripten_temp/emcc-1-linktime.bc -o out.js -s
EXPORTED_FUNCTIONS="['_julia_abs;104545']"

WARNING root: invocation: /usr/bin/emcc -v

/tmp/emscripten_temp/emcc-1-linktime.bc -o out.js -s
EXPORTED_FUNCTIONS=['_julia_abs;104545'] (in

/home/tshort/emjl/usr/lib/julia)
INFO root: (Emscripten: Running sanity checks)
INFO root: (Emscripten: Running sanity checks)
DEBUG root: compiling to bitcode
DEBUG root: emcc step "parse arguments and setup" took 0.11 seconds

DEBUG root: using bitcode file: /tmp/emscripten_temp/emcc-1-linktime.bc

DEBUG root: emcc step "bitcodeize inputs" took 0.00 seconds
DEBUG root: emcc step "process inputs" took 0.00 seconds
DEBUG root: will generate JavaScript

DEBUG root: emcc step "calculate system libraries" took 0.01 seconds
DEBUG root: emcc step "link" took 0.00 seconds

DEBUG root: saving intermediate processing steps to /tmp/emscripten_temp
DEBUG root: emcc: LLVM opts: -strip-debug -internalize

-internalize-public-api-list=julia_abs;104545,malloc,free -globaldce
-pnacl-abi-simplify-preopt -pnacl-abi-simplify-postopt
-enable-emscripten-cxx-exceptions
DEBUG root: emcc step "post-link" took 0.01 seconds
DEBUG root: LLVM => JS
DEBUG root: emscript: llvm backend: /opt/emscripten-fastcomp/llc
/home/tshort/emjl/usr/lib/julia/mtmp/out.bc -march=js -filetype=asm -o
/tmp/emscripten_temp/tmp7QmDVP.4.js -emscripten-assertions=1
-emscripten-no-aliasing-function-pointers -O0
-emscripten-max-setjmps=20
%3 = alloca [5 x %jl_value_t*], align 8
LLVM ERROR: 0 && "some i64 thing we can't legalize yet"
DEBUG root: emscript: llvm backend took 0.00503993034363 seconds

Traceback (most recent call last):

File "/usr/lib/emscripten/emscripten.py", line 1506, in <module>
_main(environ=os.environ)

Like the previous error, this one works fine with fastcomp turned off.

Tom

[JF Bastien]

Hi Tom,

Could you file a bug for each of these two issues on the bug tracker? I'll make sure they get to the right people (I can also file the bugs, but then you won't get updates or requests for more info).

I think both of these are hitting limitations that clang never exercises and were therefore left unimplemented. The Rust port recently had similar issues, so you're in good company (and we're getting these issues fixed!).

Thanks,

JF

[Tom Short]

Done (I think). Thanks JF!

On Mon, Sep 1, 2014 at 11:25 AM, 'JF Bastien' via emscripten-discuss

[JF Bastien]

Thanks Tom.

For those following at home:

https://code.google.com/p/nativeclient/issues/detail?id=3931

https://code.google.com/p/nativeclient/issues/detail?id=3932

[Alon Zakai]

>   %3 = alloca [5 x %jl_value_t*], align 8
> LLVM ERROR: 0 && "some i64 thing we can't legalize yet"

Hmm, that line doesn't contain any i64s - unless the pointer is 64-bit? Are you building the code for the emscripten/asm.js target, which is 32-bit?

- Alon

[Tom Short]

I'll look again at that. I thought I had set a 32-bit target, but I
might have messed it up.