Problem when including external/llvm headers in art/compiler

[Paschalis Mpeis]

Problem

I want to include the external/llvm module in the art/compiler module and use some of its functionality.

I have updated:

  1. the relevant makefiles (.mk) in art/build and art/compiler to include the LLVM module

  2. the LLVM module so it can export its includes, but still the headers cannot be found.

 

It seems that LOCAL_EXPORT_C_INCLUDES or LOCAL_EXPORT_C_INCLUDE_DIRS do not work.

In a second approach, I have tried to manually expand the include directory in art's makefiles, but this causes other problems!

You can find the changes I made below:

Approach 1:

art/build/Android.common_build.mk:

 ..
 LLVM_ROOT_PATH := external/llvm
 include $(LLVM_ROOT_PATH)/llvm.mk
 ..

art/compiler/Android.mk

 ..
 LOCAL_SHARED_LIBRARIES += libLLVM
 ..

external/llvm/ llvm-host-build.mk and llvm-device-build.mk:

Right after of [LOCAL_C_INCLUDES (link in aosp code):

..
# tested all of these individually:
# test1
LOCAL_EXPORT_C_INCLUDE_DIRS := $(LLVM_ROOT_PATH)
# test2
LOCAL_EXPORT_C_INCLUDE_DIRS := $(LOCAL_C_INCLUDES)
# test3
LOCAL_EXPORT_C_INCLUDE :=  $(LOCAL_C_INCLUDES)
..

Then I built the external/llvm module, and subsequently the art/compiler module, which had an additional source file with the following line:

 #include <llvm/LinkAllPasses.h> // also tried with double quotes -> "
 // this inclusion is just an example.

It seems that the export that should have happened when llvm was built never occurred, so the above include resulted in the compilation error:

"file not found"

I have also tried to add in external/llvm/shared_llvm.mk the following:

Right after the lines here and here:

 LOCAL_MODULE_CLASS := SHARED_LIBRARIES

Approach 2:

Kept makefiles the same, and appended on the following makefile:

art/compiler/Android.mk

LOCAL_C_INCLUDES += external/llvm/include
LOCAL_CFLAGS += -D_GNU_SOURCE -D__STDC_LIMIT_MACROS
# code that checks if a device build:
LOCAL_C_INCLUDES += external/llvm/device/include
# code that checks if a host build:
LOCAL_C_INCLUDES += external/llvm/host/include

The headers can now be found but this causes lot of other problems.

The included headers should be included in the exact state they had after the preprocessor ran over them. Manually appending CFLAGS in the compiler's makefile (line 2 of above snippet) does not solve this problem. Also, I don't think this is the recommended way to include the header files of another module.

Additional info

• I am working on the Marshmallow branch
  
• I 've seen how other modules successfully include and use llvm, like lldb, but still I haven't noticed what they do differently to successfully use the llvm module
  
• I 've read the build/core/build-system.html, and digged into the build sources in general
• I 've seen how this was included some time ago when there was an experimental llvm backend in art/compiler

Question

What is the proper way to include and use llvm module from the art/compiler module in AOSP?

[Colin Cross]

LOCAL_EXPORT_C_INCLUDE_DIRS := $(LLVM_ROOT_PATH)/include plus
LOCAL_SHARED_LIBRARIES += libLLVM should work. Adding it to
llvm-host-build.mk and llvm-device-build.mk is overkll, those are used
for building everything in external/llvm, but you only need to affect
the libLLVM module itself. I'd suggest adding
LOCAL_EXPORT_C_INCLUDE_DIRS to the two modules in shared_llvm.mk.

If it is still not working, you can see if the export is working correctly with:
cat $OUT/obj/SHARED_LIBRARIES/libLLVM_intermediates/export_includes
And if the import is working using:
cat $OUT/obj/SHARED_LIBRARIES/libart-compiler_intermediates/import_includes

If that is still not working, post the exact command that fails with
make showcommands and the output.

> --
> --
> You received this message because you are subscribed to the "Android
> Building" mailing list.
> To post to this group, send email to android-...@googlegroups.com
> To unsubscribe from this group, send email to
> android-buildi...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/android-building?hl=en
>
> ---
> You received this message because you are subscribed to the Google Groups
> "Android Building" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to android-buildi...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Paschalis Mpeis]

Thanks for helping me out Colin Cross.

So, as per your suggestions I did just the following changes:

art/compiler/Android.mk: (after this line)

..
LOCAL_SHARED_LIBRARIES += libLLVM
..

On this makefile I have also added an additional llvm_compiler.cc file in the LIBART_COMPILER_SRC_FILES variable.

For the reference here is the additional file:

art/compiler/llvm/llvm_compiler.cc:

 #include "llvm_compiler.h"
// just to test whether it finds the header
// tried also with < instead of double quotes
#include "llvm/LinkAllPasses.h"

namespace art {
  class TestLLVM {
    public:
        void Test() {
        // no code for now..
        }
  };
}

And its header:
art/compiler/llvm/llvm_compiler.h:

#ifndef _ART_COMPILER_LLVM_LLVM_COMPILER_H_
#define _ART_COMPILER_LLVM_LLVM_COMPILER_H_

#ifndef __STDC_CONSTANT_MACROS
#define __STDC_CONSTANT_MACROS
#endif

#ifndef __STDC_LIMIT_MACROS
#define __STDC_LIMIT_MACROS
#endif
 namespace art {
// quite empty
}
#endif

On the LLVM module:

external/llvm/shared_llvm.mk after this and this line (that set of the LOCAL_MODULE variables):

..
LOCAL_EXPORT_C_INCLUDE_DIRS := $(LLVM_ROOT_PATH)/include
..

Export includes seem to be working:

cat $OUT/obj/SHARED_LIBRARIES/libLLVM_intermediates/export_includes:

-I external/llvm/include

cat $OUT/obj/SHARED_LIBRARIES/libart-compiler_intermediates/import_includes:

-I external/llvm/include

Make command:

mmma -j8 art/compiler/ showcommands

Result: link 

Since this did not work, I have also tried doing these extra modifications (still no luck):

art/build/Android.common_build.mk: after this line:

LLVM_ROOT_PATH := external/llvm
include $(LLVM_ROOT_PATH)/llvm.mk

Result: link

[Colin Cross]

The failing command is for the host build of libart-compiler. The
host version of libLLVM comes from prebuilts/sdk/tools/Android.mk as a
prebuilt. Try building with make FORCE_BUILD_LLVM_COMPONENTS=true to
rebuild the host version of libLLVM.so from source and pick up your
LOCAL_EXPORT_C_INCLUDE_DIRS.

[Paschalis Mpeis]

Great. The inclusion problem seems to be solved now!

BTW, I don't know if it affects but in the LOCAL_EXPORT_C_INCLUDE_DIRS variable for the two modules (device/host) in shared_llvm.mk I have also included the device/include and host/include dirs accordingly.

In the particular example that I gave, the inclusion of llvm/LinkAllPasses.h needs some preprocessor definitions, otherwise it leads to errors:

external/llvm/include/llvm/Support/DataTypes.h:49:3: error: "Must #define __STDC_LIMIT_MACROS before #including Support/DataTypes.h"

For this reason I have appended in CFLAGS in art/compiler/Android.mk as follows:

LIBART_COMPILER_CFLAGS += `llvm-config --cppflags`

However there are still some issues, some warnings that are escalated into errors, e.g:

..
external/llvm/include/llvm/ADT/Statistic.h:113:39: error: unused parameter 'Val' [-Werror,-Wunused-parameter]
..
external/llvm/include/llvm/ADT/ilist_node.h:81:13: error: declaration shadows a field of 'ilist_node<NodeTy>' [-Werror,-Wshadow]
..

Now what I want is to ignore these errors (make them again warnings) but I want this to be applied just to the included LLVM sources, and not to the rest of the art/compiler sources. Is there a mechanism to do this?

If there is I should also apply it for the flags that are produced by llvm-config, as there could be interferences with the rest of the art/compiler sources.

Thanks once again Collin.