[LLVMdev] Compile Linux Kernel module into LLVM bitcode

[Kevin Boos]

Greetings,

I am trying to compile a linux kernel module (currently a small part of the gpu driver) into the bitcode ".bc" format so that I can run a pass on it using the "opt" command. This pass will count the number of times copy_to/from_user() is invoked.

Compiling the gnu driver kernel modules works with clang as a front end. However, I am unable to get the "-emit-llvm" flag to work.

I am using the typical Makefile system (kbuild) included with the kernel, as such:
$ make V=1 CC=clang CFLAGS="-emit-llvm -c" drivers/gpu/drm/drm_drv.o

I have tried setting KBUILD_CFLAGS or KCFLAGS instead of CFLAGS, but nothing I do gets it to work. Trying these few different variations gets me to the point where it doesn't actually use the "-emit-llvm" flag in the clang (previously gcc) commands that are shown with the verbose V=1 option. If the "-emit-llvm" flag *does* make it into the clang command, it doesn't create the .bc file and sometimes errors out like this:

====================
objdump: scripts/mod/.tmp_empty.o: File format not recognized
if [ "-pg" = "-pg" ]; then if [ scripts/mod/empty.o != "scripts/mod/empty.o" ]; then /home/kevin/split_io_Linux/scripts/recordmcount "scripts/mod/empty.o"; fi; fi;
gcc -Wp,-MD,scripts/mod/.mk_elfconfig.d -Wall -Wmissing-prototypes -Wstrict-prototypes -O2 -fomit-frame-pointer -o scripts/mod/mk_elfconfig scripts/mod/mk_elfconfig.c
scripts/mod/mk_elfconfig < scripts/mod/empty.o > scripts/mod/elfconfig.h
Error: not ELF
make[2]: *** [scripts/mod/elfconfig.h] Error 1
make[1]: *** [scripts/mod] Error 2
make: *** [scripts] Error 2
====================


Essentially, I need to know how to use LLVM/clang with a linux kernel makefile to output the .bc bitcode file. I thought clang was a good drop-in replacement for gcc, but it hasn't worked out for me yet. How can I change the "make" command so that it produces .bc?

Thanks in advance,
Kevin
_______________________________________________
LLVM Developers mailing list
LLV...@cs.uiuc.edu http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

[Duncan Sands]

Hi Kevin,

On 01/06/13 09:59, Kevin Boos wrote:
> Greetings,
>
> I am trying to compile a linux kernel module (currently a small part of the gpu driver) into the bitcode ".bc" format so that I can run a pass on it using the "opt" command. This pass will count the number of times copy_to/from_user() is invoked.
>
> Compiling the gnu driver kernel modules works with clang as a front end. However, I am unable to get the "-emit-llvm" flag to work.
>
> I am using the typical Makefile system (kbuild) included with the kernel, as such:
> $ make V=1 CC=clang CFLAGS="-emit-llvm -c" drivers/gpu/drm/drm_drv.o
>
> I have tried setting KBUILD_CFLAGS or KCFLAGS instead of CFLAGS, but nothing I do gets it to work. Trying these few different variations gets me to the point where it doesn't actually use the "-emit-llvm" flag in the clang (previously gcc) commands that are shown with the verbose V=1 option. If the "-emit-llvm" flag *does* make it into the clang command, it doesn't create the .bc file

-emit-llvm doesn't change the name of the file output, it changes what is put
in the output file (bitcode rather than object code). So it is normal not to
see a file called xyz.bc, it will still be called xyz.o. If you look inside
it you should see that it starts with BC (followed by nasty stuff). You can
use llvm-dis to turn this into a file containing human readable IR. The
following error output shows that indeed bitcode is being output:

and sometimes errors out like this:
>
> ====================
> objdump: scripts/mod/.tmp_empty.o: File format not recognized
> if [ "-pg" = "-pg" ]; then if [ scripts/mod/empty.o != "scripts/mod/empty.o" ]; then /home/kevin/split_io_Linux/scripts/recordmcount "scripts/mod/empty.o"; fi; fi;
> gcc -Wp,-MD,scripts/mod/.mk_elfconfig.d -Wall -Wmissing-prototypes -Wstrict-prototypes -O2 -fomit-frame-pointer -o scripts/mod/mk_elfconfig scripts/mod/mk_elfconfig.c
> scripts/mod/mk_elfconfig < scripts/mod/empty.o > scripts/mod/elfconfig.h
> Error: not ELF

Yup, it's not ELF, it's bitcode! This is going to be a problem for something
like the linux kernel that does a lot of tricky stuff with generated object
files. I suggest you only try compiling your particular module as bitcode,
and compiling everything else normally. For example, compile everything using
clang but without -emit-llvm, and tell make to produce verbose output. This
should show you the clang command line it uses to compile your module. Run
that command line by hand, but adding -emit-llvm. This should produce bitcode
for your module.

> make[2]: *** [scripts/mod/elfconfig.h] Error 1
> make[1]: *** [scripts/mod] Error 2
> make: *** [scripts] Error 2
> ====================
>
>
> Essentially, I need to know how to use LLVM/clang with a linux kernel makefile to output the .bc bitcode file. I thought clang was a good drop-in replacement for gcc, but it hasn't worked out for me yet.

You didn't just drop it in, you made a major change, telling it to output
bitcode rather than ELF.

Ciao, Duncan.

[chenping]

Greatings,

  

     I am encountering the same issue with Kevin. Does anyone know how to compile the linux kernel to .bc file?

     Is there any oneline guide for this? Thanks!~

Bests,

Ping

     

> Date: Sat, 1 Jun 2013 20:15:56 +0200
> From: bald...@free.fr
> To: llv...@cs.uiuc.edu
> Subject: Re: [LLVMdev] Compile Linux Kernel module into LLVM bitcode

[Kevin Boos]

Ping,

Per Duncan's suggestion, I just ended up running a verbose make command 
   "make V=1  drivers/gpu/drm/drm_drv.o"

on the kernel module that I needed (which was drm_drv). That make command prints out  a series of long "gcc" commands, so you can see exactly how each file is compiled. I then copied and pasted each gcc command for each file (you could script this instead) and changed each "gcc" invocation to "clang -emit-llvm" instead. 

At this point, you have an output file (which by default is drm_drv.o), but you can change the file extension to be .bc if you want. That drm_drv.o file is the actual bitcode file that you can use with "opt" or "llvm-dis". 

I never bothered figuring out how to do it with the default kbuild Makefiles, if that's even possible.

Best of luck,
Kevin