NDK build fails with: fatal error: string: No such file or directory

[Benjamin Kohler-Crowe]

Hi,

I'm trying to get the sqlite android bindings (https://www.sqlite.org/android/doc/trunk/www/index.wiki) building in BUCK.

I've put a simple BUCK file in the same directory as the Android.mk and Application.mk files, that looks like this:

ndk_library(
    name = 'sqlite_android_bindings',
    #deps = [],
    #flags = [],
    #is_asset = False,
    visibility = ['PUBLIC'],
)

The documentation for this rule at https://buckbuild.com/rule/ndk_library.html doesn't seem to list anything else I should be specifying.

If I build it with buck like this:
$ ndk-build clean && ndk-build
it builds successfully.  However, if I try to build it with buck:
---
$ buck build j/3rdparty/sqlite_android_bindings/jni:sqlite_android_bindings
Not using buckd because watchman isn't installed.
No Android platform target specified. Using default: Google Inc.:Google APIs:21
[-] PROCESSING BUCK FILES...FINISHED 0.1s [100%]
[+] BUILDING...0.5s [50%] (0/1 JOBS, 0 UPDATED, 0.0% CACHE MISS)
 |=> IDLE
 |=> IDLE
 |=> //j/3rdparty/sqlite_android_bindings/jni:sqlite_android_bindings...  0.1s (running ndk_build[0.0s])
sqlite/JNIHelp.cpp:28:18: fatal error: string: No such file or directory
 #include <string>
                  ^
compilation terminated.
sqlite/android_database_SQLiteConnection.cpp:44:18: fatal error: string: No such file or directory
 #include <string>
                  ^
compilation terminated.
---

Searching for that particular build error found me a bunch of pages about needing to configure the android NDK build correctly to use STL, like http://stackoverflow.com/questions/4893403/cant-include-c-headers-like-vector-in-android-ndk
However, as far as I can tell the project is already configured the way that says -- and the fact that invoking ndk-build directly works supports that belief. :)

Are there some other arguments I need to pass to ndk_library to get it to build with APP_STL:=stlport_static?  Or, does anyone know what else I might be missing?

Thanks!
-Ben

[Shawn Wilsher]

In general, we discourage the use of ndk_library. Buck can't build
things with it efficiently (because it shells out to ndk-build). We
generally recommend that you use cxx_library instead to get a much
better caching experience.

With that said, you should be able to add APP_STL:=stlport_static to
your Application.mk.

Cheers,

Shawn

[Benjamin Kohler-Crowe]

Ah, I'll look into cxx_library.

> should be able to add APP_STL:=stlport_static to your Application.mk.

That's the thing, it's already there.

[Benjamin Kohler-Crowe]

OK, based on the documentation of cxx_library, I've converted the existing Android.mk into the following buck rule.  However, I have two problems (so far).
* As I understand things, preprocessor_flags and compiler_flags are collectively analogous to LOCAL_CFLAGS in the NDK build files.  However, I don't see a way to do the equivalent of LOCAL_CPPFLAGS.  The sqlite library contains both c and cpp files.  Now, the only C++ specific flag it's using is
LOCAL_CPPFLAGS += -Wno-conversion-null
so maybe I can just delete that.
* My other problem is with the header_namespace thing -- I think.  When the library's source code tries to include its own headers, I get errors -- No such file or directory -- presumably because this setup requries the header to be referred to as <$HEADER_NAMESPACE/$HEADER_NAME> but all the code just includes as <$HEADER_NAME>.  I've tried both setting the header_namespace to empty string and leaving it unset, but either way I get the same errors.  On a lark I've tried instead putting the headers in the srcs glob, but that's rejected immediately.


cxx_library(
    name = 'sqlite_android_bindings_cxx',
    srcs = glob(['sqlite/**/*.cpp', 'sqlite/**/*.c']),
    headers = glob(['sqlite/**/*.h']),
    exported_headers = glob(['sqlite/**/*.h']),
    header_namespace = '',  # I've tried with this commented out or empty-string
    preprocessor_flags =
    [
        '-DSQLITE_TEMP_STORE=3',
        '-DHAVE_CONFIG_H',
        '-DKHTML_NO_EXCEPTIONS',
        '-DGKWQ_NO_JAVA',
        '-DNO_SUPPORT_JS_BINDING',
        '-DQT_NO_WHEELEVENT',
        '-DKHTML_NO_XBL',
        '-U__APPLE__',
        '-DHAVE_STRCHRNUL=0',
    ],
    platform_preprocessor_flags =
    [
        ('.*arm.*', ['-DPACKED="__attribute__ ((packed))"']),
    ],
    compiler_flags =
    [
        '-Wno-unused-parameter',
        '-Wno-int-to-pointer-cast',
        '-Wno-maybe-uninitialized',
        '-Wno-parentheses',
        #'-Wno-conversion-null',  # Commenting out this line gets rid of one error
    ],
    linker_flags =
    [
        '-ldl',
        '-llog',
    ],

    #link_style = 'static',  # this is default
    #link_style = 'dynamic', # this is alternative

    soname = 'libsqliteX.$(ext)',

    visibility = ['PUBLIC'],
)

And here's the errors I'm getting:

cc1: error: command line option '-Wno-conversion-null' is valid for C++/ObjC++ but not for C [-Werror]
cc1: all warnings being treated as errors

j/3rdparty/sqlite_android_bindings/jni/sqlite/android_database_SQLiteGlobal.cpp:24:21: fatal error: JNIHelp.h: No such file or directory
 #include <JNIHelp.h>
                     ^
compilation terminated.

j/3rdparty/sqlite_android_bindings/jni/sqlite/android_database_SQLiteDebug.cpp:24:21: fatal error: JNIHelp.h: No such file or directory
 #include <JNIHelp.h>
                     ^
compilation terminated.

j/3rdparty/sqlite_android_bindings/jni/sqlite/android_database_SQLiteGlobal.cpp:24:21: fatal error: JNIHelp.h: No such file or directory
 #include <JNIHelp.h>
                     ^
compilation terminated.

In file included from j/3rdparty/sqlite_android_bindings/jni/sqlite/android_database_SQLiteCommon.cpp:21:0:
j/3rdparty/sqlite_android_bindings/jni/sqlite/android_database_SQLiteCommon.h:25:21: fatal error: JNIHelp.h: No such file or directory
 #include <JNIHelp.h>
                     ^
compilation terminated.


Any insight?

Thanks,
Ben

[Benjamin Kohler-Crowe]

I was able to get around the headers problem by making an explicit map of the header files like so:

    headers =
    {
        'ALog-priv.h' : 'sqlite/ALog-priv.h',
        'android_database_SQLiteCommon.h' : 'sqlite/android_database_SQLiteCommon.h',
        'jni.h' : 'sqlite/nativehelper/jni.h',
        'JniConstants.h' : 'sqlite/nativehelper/JniConstants.h',
        'JNIHelp.h' : 'sqlite/nativehelper/JNIHelp.h',
        'ScopedLocalRef.h' : 'sqlite/nativehelper/ScopedLocalRef.h',
        'sqlite3.h' : 'sqlite/sqlite3.h',
    },

Awkward, but it worked.  I feel like that shouldn't be necessary, though.

Removing '-Wno-conversion-null' fixed one error, and then adding '-Wno-unused-variable' got me past several more errors.
And it built!  And then I tried buck install, it built it over again, and failed.  Definitely not a behavior I'm used to seeing from buck!

j/3rdparty/sqlite_android_bindings/jni/sqlite/android_database_SQLiteConnection.cpp: In function 'jlong android::nativeExecuteForCursorWindow(JNIEnv*, jclass, jlong, jlong, jobject, jint, jint, jboolean)':
j/3rdparty/sqlite_android_bindings/jni/sqlite/android_database_SQLiteConnection.cpp:777:54: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
   for(i=0; i<(sizeof(aMethod)/sizeof(struct CWMethod)); i++){
                                                      ^
cc1plus: all warnings being treated as errors

j/3rdparty/sqlite_android_bindings/jni/sqlite/android_database_SQLiteConnection.cpp: In function 'jlong android::nativeExecuteForCursorWindow(JNIEnv*, jclass, jlong, jlong, jobject, jint, jint, jboolean)':
j/3rdparty/sqlite_android_bindings/jni/sqlite/android_database_SQLiteConnection.cpp:777:54: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
   for(i=0; i<(sizeof(aMethod)/sizeof(struct CWMethod)); i++){
                                                      ^
cc1plus: all warnings being treated as errors

j/3rdparty/sqlite_android_bindings/jni/sqlite/android_database_SQLiteConnection.cpp: In function 'jlong android::nativeExecuteForCursorWindow(JNIEnv*, jclass, jlong, jlong, jobject, jint, jint, jboolean)':
j/3rdparty/sqlite_android_bindings/jni/sqlite/android_database_SQLiteConnection.cpp:777:54: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
   for(i=0; i<(sizeof(aMethod)/sizeof(struct CWMethod)); i++){
                                                      ^
cc1plus: all warnings being treated as errors


Adding yet another -Wno line, '-Wno-sign-compare', seems to have it building consistently now.  I hope. :)


Cheers,
Ben

[Mike Kaplinskiy]

Hey Benjamin,

Just a quick tip: if you don't want to list out all the headers by hand, you can use subdir_glob to fix it up:

headers=subdir_glob([

    ('sqlite', '*.h'),

    ('sqlite/nativehelper', '*.h')

]),

header_namespace='',

Alternatively: BUCK files are python, so you can import os.path and manipulate paths returned by glob as you want, e.g..

import os.path

...
headers={os.path.basename(f): f for f in glob(['sqlite/**/*.h'])}

header_namespace='',

Mike.

[Benjamin Kohler-Crowe]

Oh neat!  That works nicely.  :)

Thanks!

[Shawn Wilsher]

On 12/18/2015 15:36, Benjamin Kohler-Crowe wrote:
> That's the thing, it's already there.

That is surprising. I'm honestly not sure why that isn't working, but
it looks like you are unblocked anyway with the cxx_library stuff.

Cheers,

Shawn

[Shawn Wilsher]

On 12/18/2015 16:31, Benjamin Kohler-Crowe wrote:
> * As I understand things, preprocessor_flags and compiler_flags are
> collectively analogous to LOCAL_CFLAGS in the NDK build files. However,
> I don't see a way to do the equivalent of LOCAL_CPPFLAGS. The sqlite
> library contains both c and cpp files. Now, the only C++ specific flag
> it's using is
> LOCAL_CPPFLAGS += -Wno-conversion-null
> so maybe I can just delete that.

We have `lang_preprocessor_flags`, but no `lang_compiler_flags` it looks
like. If you wanted to add that, we'd take a PR for it for sure! I'm
not sure when we'd get to it otherwise. I'm happy to give you the
details about how to implement that.

Cheers,

Shawn