Altering LLVM instructions during codegen

[rpre...@gmail.com]

I'm experimenting with Julia's interaction with LLVM for my  GSoC project (those interested in the proposal can read it here). As part of this, I am trying to manipulate some Julia code via an LLVM pass however I've run into some difficulties that I hope someone with some more experience working with Julia's code generation/runtime might be able to provide some insight towards.

As the simplest example of altering an LLVM instruction during codegen, I decided to attempt to remove an instruction from a BasicBlock. I identify a certain instruction and then do the following:

i.removeFromParent()

where i is a valid instruction (I can print it, access all its information etc.). However, whenever I attempt to remove a single instruction (in this case, I chose a CallInst that calls the julia_cos_ function), I get the following error:

Instruction:   %1 = call %jl_value_t* @julia_cos_63281(%jl_value_t* %0) #0, !dbg !41, !vectorize !4

signal (11): Segmentation fault: 11
while loading no file, in expression starting on line 0
runOnBasicBlock at /Volumes/andromeda/julia/src/llvm-vectorize.cpp:85
_ZN12_GLOBAL__N_113BBPassManager13runOnFunctionERN4llvm8FunctionE at /Volumes/andromeda/julia/usr/lib/libLLVM-3.7.dylib (unknown line)
_ZN4llvm13FPPassManager13runOnFunctionERNS_8FunctionE at /Volumes/andromeda/julia/usr/lib/libLLVM-3.7.dylib (unknown line)
_ZN4llvm13FPPassManager11runOnModuleERNS_6ModuleE at /Volumes/andromeda/julia/usr/lib/libLLVM-3.7.dylib (unknown line)
_ZN4llvm6legacy15PassManagerImpl3runERNS_6ModuleE at /Volumes/andromeda/julia/usr/lib/libLLVM-3.7.dylib (unknown line)
operator() at /Volumes/andromeda/julia/src/./jitlayers.cpp:334
operator() at /Library/Developer/CommandLineTools/usr/bin/../include/c++/v1/functional:1437
operator() at /Library/Developer/CommandLineTools/usr/bin/../include/c++/v1/functional:1817
addModule at /Volumes/andromeda/julia/src/./jitlayers.cpp:442
jl_finalize_function at /Volumes/andromeda/julia/src/./jitlayers.cpp:710
jl_finalize_function at /Volumes/andromeda/julia/src/./jitlayers.cpp:719
jl_generate_fptr at /Volumes/andromeda/julia/src/codegen.cpp:1031
jl_call_method_internal at /Volumes/andromeda/julia/src/./julia_internal.h:169
do_call at /Volumes/andromeda/julia/src/interpreter.c:66
eval at /Volumes/andromeda/julia/src/interpreter.c:190
jl_toplevel_eval_flex at /Volumes/andromeda/julia/src/toplevel.c:560
jl_toplevel_eval_in_warn at /Volumes/andromeda/julia/src/builtins.c:573
eval at ./boot.jl:234
unknown function (ip: 0x109a5450f)
jl_call_method_internal at /Volumes/andromeda/julia/src/./julia_internal.h:172
eval_user_input at ./REPL.jl:62
unknown function (ip: 0x31b3db8b6)
jl_call_method_internal at /Volumes/andromeda/julia/src/./julia_internal.h:172
macro expansion; at ./REPL.jl:92
unknown function (ip: 0x31b3d200f)
jl_call_method_internal at /Volumes/andromeda/julia/src/./julia_internal.h:172
jl_apply at /Volumes/andromeda/julia/src/./julia.h:1389
Allocations: 1880913 (Pool: 1879818; Big: 1095); GC: 1
[1]    28873 segmentation fault  /Volumes/andromeda/julia/julia

I am printing the instruction before I attempt to remove it.

Clearly, removing even just a single instruction in a basic block is influencing the Julia backend in some way that I haven't taken into account. I've traced this back up as best as I can but I still can't track down a particular area of the codegen process that would be affected by such a change. If anyone wants a more detailed view of what is going on, the code is available on Github.

Does anyone have any ideas or suggestions about where I might look to track down this issue (I have exhausted my ideas and ability to isolate this issue) or any ideas about what could be causing it? All ideas and directions will be most appreciated.

Thank you in advance!

[Yichao Yu]

The patch you link to doesn't have any `eraseFromParent` in it so it
is hard to tell what exactly went wrong (note that the place you added
the pass is after all the processing on our side so it is most likely
a LLVM issue. Julia is a frontend, not a backend).
A few cases I've seen that can cause LLVM segfault.

* Make sure you've replaced all use before you delete the instruction.
* Make sure you don't invalidate the iterator.

[rpre...@gmail.com]

I updated the link - it should show the removeFromParent now. (I used the term Julia backend to refer to the C++ code generation and optimization processes of Julia - full understanding that this "backend" is just a frontend for LLVM).

[Yichao Yu]

On Tue, Jul 12, 2016 at 1:02 AM, <rpre...@gmail.com> wrote:
> I updated the link - it should show the removeFromParent now. (I used the

Hmm, not really?
If the link address changed, it would be better if you could just
repost the link since updating post doesn't work for emails.

> term Julia backend to refer to the C++ code generation and optimization
> processes of Julia - full understanding that this "backend" is just a
> frontend for LLVM).
>

[Remy Prechelt]

The link now points to the correct PR comparison with the instruction erase.  

[Yichao Yu]

On Tue, Jul 12, 2016 at 11:45 AM, Remy Prechelt <rpre...@gmail.com> wrote:
> The link now points to the correct PR comparison with the instruction erase.

The problem is what I mentioned in the first reply.

1. You can't just delete an instruction, that will leave all use of it
invalid (unless it has no use)

See here[1] for an example of that.

2. I believe removing instructions invalidate the BasicBlock iterator.

See here[2] for an example of how that could be done, note how the
iterator is used. (I'm not sure if this is the most idiomatic way)

[1] https://github.com/JuliaLang/julia/blob/f881efb5e8ee4b3c04d90895a92c03c075805147/src/codegen.cpp#L3497
[2] https://github.com/JuliaLang/julia/blob/f881efb5e8ee4b3c04d90895a92c03c075805147/src/llvm-gcroot.cpp#L669

[Yichao Yu]

Also `Accelerate::cos` looks really strange. You probably don't want
to use `jl_value_t` for vectors and you absolutely don't want to call
`jl_valueof(5)`.

[Remy Prechelt]

The trick of manually incrementing the iterator before removing the instruction worked - thanks for the thoughts!