build OpenSSL for arm64-v8a and x86_64 with r10c

[Michael Moussa]

Hi, I am trying to figure out how to build a 64 bit open ssl for android ndk's use with 64 bit support. Everything i am finding does not support these newer ABIs, and just adding the ABIs doesnt seem to help as the builds don't seem to work with r10c. 

[Gilles Olivier Vollant]

Le dimanche 2 novembre 2014 05:20:00 UTC+1, Michael Moussa a écrit :

Hi, I am trying to figure out how to build a 64 bit open ssl for android ndk's use with 64 bit support. Everything i am finding does not support these newer ABIs, and just adding the ABIs doesnt seem to help as the builds don't seem to work with r10c. 

what is your project file?

I've my own android.mk which compile curl and openssl (and other code).

64 bits buils produce a .so, but I don't have tested it...

[MCDV test]

Hi,

I am facing similar issue. 

Trying to compile openssl with android-ndk-r10c for 64bit support (on Mac).

Any pointers will be appreciated..

regards,

[Michael Moussa]

Here is the official openSSL documentation on how to build for Android: http://wiki.openssl.org/index.php/Android 

The documentation though is a bit old and does not make mention of how to support 64 bit.

Here is a github repo that tracks against AOSP with instructions on how to pull in the latest AOSP code; however there were way too many merge conflicts to handle:

https://github.com/guardianproject/openssl-android

[MCDV test]

Thanks Michael

You are right -- the openssl documentation doesn't provide info w.r.t. arm 64bit.

Tried with the Wiki options, but it generated the default i386 build for Mac (though 'arm' & similar options were set in the 'setenv' file).

https://github.com/guardianproject/openssl-android

This has a old version of openssl (101a). The build instructions doesn't provide info w.rt. arm64bit too.

It indicates compile via 'android-ndk-r10c/ndk-build'.

Tried the same, but it generated the default 32bit lib crypto and ssl files.

#1: How to enable 64bit using NDK compile?

i.e. if we compile via android-ndk-r10c/ndk-build .... how to ensure that it uses 64bit?

#2: Which arch to select in ./Configure?

We can try via: "./Configure android/android-armv7" as well.

But, there is no option to use/select 64bit there too.

Ref: "pick os/compiler from:" :: Configure provides various os/compiler options, but none correspond to arm64.

regards,

[George Metaxas]

Hello,

AOSP produces arm64 builds for openssl. If you follow the instructions on building AOSP from source, you will be able to get
the libraries you want for the corresponding ABIs.

For reference, the AOSP openssl code can be found here:
https://android.googlesource.com/platform/external/openssl

However, you have to use the AOSP tools to build the above.

Best regards.
George

[Michael Moussa]

I was trying to build it from the NDK. But building AOSP is a reasonable workaround. Is there any differences in the binary produced by AOSP than one that would be produced with the NDK? That is, can I use the openssl libcrypto.so that was built with AOSP in a regular NDK project?

Thanks,
Mike

[George Metaxas]

You should be able to use it without any problems.

Regards

[Michael Moussa]

Looking at my project I just realized that I need a static library. Could AOSP produce static files?

Thanks

[George Metaxas]

Yes it can and it does, but you are better off using the shared library versions.

Regards

[Michael Moussa]

Why is the dynamic library better than the static library?

And I did build it, I could only find dynamic libraries in the target output. There were only static libraries in the intermediate output for the host environment. I cannot use those.

Mike

[George Metaxas]

You get smaller binaries, because your libraries depend on the system-provider libraries. More importantly,
if you use shared libraries, you will not be responsible for patching a major security flaw found in openssl  (e.g. heatbleed). If you use
static libraries, you will have to recompile your library with an updated version of openssl, and redistribute your application in order to
fix the flaw.

Now, if you still want to go down the static library route, if you change to external/openssl and run: make crypto_static ssl_static, you will get
the static versions you want.

Regards

[Michael Moussa]

Hi George, thanks for the details.

As for linking against the system shared library I did not know you could use the system shared openssl library from the Android NDK within an app. Any article/example you could point me to that does this? Also do you think there could be any issues with the same app working with different versions of Android; and therefore using different versions of OpenSSL?

As for potential upside with static, I could at least make sure that new openSSL patches get into the app when running on old versions of Android, but I am not sure if this is enough of an upside to warrant a static library.

Thanks

Mike

[George Metaxas]

Hello,

You won't find these libraries in the NDK. However, you can either compile them yourself from AOSP, or pull them from an emulator image.
Once you have them, just use them like any other PREBUILT_SHARED_LIBRARY. I do not think that you will have a problem with
using different versions of OpenSSL, unless you want to use a very specific feature. In any case have a look at what openssl versions
are included in the platforms you are targeting.

In any case, the decision of whether to use a shared or static version of a library, depends on your specific application needs.

Regards.

[Nicolas Raby]

Hi,

sorry to update a 2 years old thread, but I need to build openSSL for arm64 & x86_64

From the docs here https://wiki.openssl.org/index.php/Android , I am able to find how to change the setenv-android.sh for building armv7 & x86, but I failed building 64 bits versions.looks like the script provided by the openSSL wiki does not fit for 64 bits. any pointers ? I will share the scripts on github here https://github.com/ad34/openssl-android-build-scripts if I manage to compile this.

I m on mac os el capitan with ndk 12b

Le dimanche 2 novembre 2014 05:20:00 UTC+1, Michael Moussa a écrit :

[Dex]

It's been about another 5 months since the last post, and I can't believe with as many ARM64 devices as there are out there, that building openSSL for them is THIS difficult. Has anyone had any luck with pulling this off?

dex

[Kenneth Geisshirt]

Dex,

Which version of openssl are you trying to build? I routinely build 1.0.2 for ARM64 using r10e (gcc). I must admit that it requires a hacks ;-)

- kneth

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk...@googlegroups.com.
To post to this group, send email to andro...@googlegroups.com.
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/493ab5b3-464d-464f-9f73-5ecd29d915de%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Kenneth Geisshirt, M.Sc., Ph.D.
Majbøl Allé 18, DK-2770 Kastrup, +45 60 62 71 82

[Kenneth Geisshirt]

Hey Paul,

OpenSSL 1.0.2 comes with a script "config" to generate a Makefile. But 1.0.2 doesn't support Android/gcc so the Makefile isn't fit for NDK.

1. Changes -O3 to -Os (-O3 is known to produce incorrect binaries)

2. Remove option -mandroid (messy up make depend)

3. Allow to add to DEPFLAG from the command line

4. Use the proper ar command (from NDK)

5. Use the proper ranlib command (from NDK)

You can do all of above in one big sed command :-) If I recall correctly, -mandroid became an issue in 1.0.2 as that series is running makedepend (which I haven't used for anything else than OpenSSL for about two decades).

You might have to remove OPENSSL_clense from sha256.c - again sed can do it.

Building is now trivial:

make depend DEPFLAG=-I${ANDROID_TOOLCHAIN_ROOT}/lib/gcc/${ANDROID_TOOLCHAIN_MACHINE_NAME}/4.9/include

make build_crypto

make build_ssl

ANDROID_TOOLCHAIN_ROOT = root of the NDK's location (mine is /usr/local/Cellar/android-ndk/r10e/toolchains/aarch64-linux-android-4.9/prebuilt/darwin-x86_64, ...)

ANDROID_TOOLCHAIN_MACHINE_NAME = aarch64-linux-android, ...

Both libcrypto.a and libssl.a must be move to an ABI specific folder, and you have to do through the above for each ABI.

- kneth

On Wed, Aug 23, 2017 at 2:17 PM <pcus...@wickr.com> wrote:

Kneth,

Can you elaborate on the hacks you needed to make to get it to build for ARM64?

thanks

Paul

[pcus...@wickr.com]

Kneth,

Can you elaborate on the hacks you needed to make to get it to build for ARM64?

thanks

Paul

On Tuesday, January 31, 2017 at 2:58:53 PM UTC-5, Kenneth Geisshirt wrote:

[Andrew Esh]

This is how I build openssl version 1.0.2k for Android with NDK 15 for Android Oreo. The target device is the Pixel C, which is an arm64 architecture.

source this:

  # Pixel C/dragon/ryu

  export ANDROID_HOME=$HOME/android-sdk

  export ANDROID_NDK_ROOT=$HOME/android-ndk-r15

  export ANDROID_NDK_API=26

  export ANDROID_NDK_ARCH=arm64

  export ANDROID_NDK_HOST=aarch64-linux-android

  export ANDROID_NDK_ABI=arm64-v8a

bash script:

#!/bin/bash

echo "Checking for Android Standalone NDK..."

if [ ! -e $ANDROID_NDK_ROOT-$ANDROID_NDK_ARCH-$ANDROID_NDK_API ]; then

  echo "Creating the Android Standalone NDK..."

  if [ ! -e $ANDROID_NDK_ROOT ]; then

    echo "The NDK directory $ANDROID_NDK_ROOT does not exist, aborting!"

    exit 1

  fi

  $ANDROID_NDK_ROOT/build/tools/make_standalone_toolchain.py --arch=$ANDROID_NDK_ARCH --api=$ANDROID_NDK_API --install-dir=$ANDROID_NDK_ROOT-$ANDROID_NDK_ARCH-$ANDROID_NDK_API --stl=libc++

fi

EXTRA=`pwd`

SOURCE=$EXTRA/source

AUTOTOOLS=$EXTRA/data/autotools

PATCHES=$EXTRA/patches

SYSROOT=$EXTRA/sysroot

DOWNLOAD=$EXTRA/download

JNILIBS=$EXTRA/jniLibs/$ANDROID_NDK_ABI

SHARED_LIB=`find $ANDROID_NDK_ROOT-$ANDROID_NDK_ARCH-$ANDROID_NDK_API -name "*shared.so" | head -1`

mkdir -p $SOURCE $SYSROOT $DOWNLOAD $JNILIBS

echo "Copying shared library to jniLibs..."

cp -f $SHARED_LIB $JNILIBS

echo "Checking for openssl..."

OPENSSL_VERSION=1_0_2k

if [ ! -e $SYSROOT/lib/libssl.so ] || [ ! -e $SYSROOT/lib/libcrypto.so ] || [ ! -e $SYSROOT/include/openssl ]; then

  if [ ! -e $SOURCE/openssl-OpenSSL_$OPENSSL_VERSION ]; then

    echo "Building openssl..."

    if [ ! -e $DOWNLOAD/OpenSSL_$OPENSSL_VERSION.tar.gz ]; then

      echo "Downloading openssl..."

      pushd $DOWNLOAD

      wget https://github.com/openssl/openssl/archive/OpenSSL_$OPENSSL_VERSION.tar.gz

      popd

    fi

    pushd $SOURCE

    tar -xvf $DOWNLOAD/OpenSSL_$OPENSSL_VERSION.tar.gz

    popd

    if [ "$ANDROID_NDK_ARCH" == "x86" ]; then

      export MACHINE=i686

    else

        if [ "$ANDROID_NDK_ARCH" == "arm64" -o "$ANDROID_NDK_ARCH" == "aarch64" ]; then

            export MACHINE=aarch64

        else

            export MACHINE=armv7

        fi

    fi

    export ARCH=$ANDROID_NDK_ARCH

    export CROSS_COMPILE=$ANDROID_NDK_HOST-

    export SYSTEM=android

    export ANDROID_DEV=$ANDROID_NDK_ROOT-$ANDROID_NDK_ARCH-$ANDROID_NDK_API/sysroot/usr

    pushd $SOURCE/openssl-OpenSSL_$OPENSSL_VERSION

    if [ "$ANDROID_NDK_ARCH" == "arm64" -o "$ANDROID_NDK_ARCH" == "aarch64" ]; then

        patch < $PATCHES/openssl-config.patch

        patch < $PATCHES/openssl-Configure.patch

    fi

    ./config shared no-ssl2 no-ssl3 no-comp no-hw no-engine no-dso no-asm --prefix=$SYSROOT --openssldir=$SYSROOT

    patch < $PATCHES/openssl-Makefiles.patch

    make clean

    make depend

    make CALC_VERSIONS="SHLIB_COMPAT=; SHLIB_SOVER=" MAKE="make -e" all

    popd

  fi

  echo "Installing openssl..."

  mkdir -p $SYSROOT/lib

  pushd $SOURCE/openssl-OpenSSL_$OPENSSL_VERSION

  echo "place-holder make target for avoiding symlinks" >> $SYSROOT/lib/link-shared

  make SHLIB_EXT=.so install_sw

  rm $SYSROOT/lib/link-shared

  popd

fi

openssl-config.patch:

--- config 2017-01-26 07:22:36.000000000 -0600

+++ /home/andrewes/openssl_mods/config 2017-05-08 10:08:02.275587001 -0500

@@ -862,6 +862,7 @@

   *-*-qnx6) OUT="QNX6" ;;

   x86-*-android|i?86-*-android) OUT="android-x86" ;;

   armv[7-9]*-*-android) OUT="android-armv7" ;;

+  aarch64-*-android) OUT="android-aarch64" ;;

   *) OUT=`echo $GUESSOS | awk -F- '{print $3}'`;;

 esac

openssl-Configure.patch 

--- Configure 2017-01-26 07:22:36.000000000 -0600

+++ /home/andrewes/openssl_mods/Configure 2017-05-09 11:16:17.585722000 -0500

@@ -474,6 +474,7 @@

 "android","gcc:-mandroid -I\$(ANDROID_DEV)/include -B\$(ANDROID_DEV)/lib -O3 -fomit-frame-pointer -Wall::-D_REENTRANT::-ldl:BN_LLONG RC4_CHAR RC4_CHUNK DES_INT DES_UNROLL BF_PTR:${no_asm}:dlfcn:linux-shared:-fPIC::.so.\$(SHLIB_MAJOR).\$(SHLIB_MINOR)",

 "android-x86","gcc:-mandroid -I\$(ANDROID_DEV)/include -B\$(ANDROID_DEV)/lib -O3 -fomit-frame-pointer -Wall::-D_REENTRANT::-ldl:BN_LLONG ${x86_gcc_des} ${x86_gcc_opts}:".eval{my $asm=${x86_elf_asm};$asm=~s/:elf/:android/;$asm}.":dlfcn:linux-shared:-fPIC::.so.\$(SHLIB_MAJOR).\$(SHLIB_MINOR)",

 "android-armv7","gcc:-march=armv7-a -mandroid -I\$(ANDROID_DEV)/include -B\$(ANDROID_DEV)/lib -O3 -fomit-frame-pointer -Wall::-D_REENTRANT::-ldl:BN_LLONG RC4_CHAR RC4_CHUNK DES_INT DES_UNROLL BF_PTR:${armv4_asm}:dlfcn:linux-shared:-fPIC::.so.\$(SHLIB_MAJOR).\$(SHLIB_MINOR)",

+"android-aarch64","clang:-I\$(ANDROID_DEV)/include -B\$(ANDROID_DEV)/lib -O3 -fomit-frame-pointer -Wall::-D_REENTRANT::-ldl:SIXTY_FOUR_BIT_LONG RC4_CHAR RC4_CHUNK DES_INT DES_UNROLL BF_PTR:${aarch64_asm}:linux64:dlfcn:linux-shared:-fPIC::.so.\$(SHLIB_MAJOR).\$(SHLIB_MINOR)",

 "android-mips","gcc:-mandroid -I\$(ANDROID_DEV)/include -B\$(ANDROID_DEV)/lib -O3 -Wall::-D_REENTRANT::-ldl:BN_LLONG RC4_CHAR RC4_CHUNK DES_INT DES_UNROLL BF_PTR:${mips32_asm}:o32:dlfcn:linux-shared:-fPIC::.so.\$(SHLIB_MAJOR).\$(SHLIB_MINOR)",

 

 #### *BSD [do see comment about ${BSDthreads} above!]

---

I removed references to the part of the build where I rename the library to avoid colliding with the system version of the same library. Other scripts copy the libraries into a jnilibs directory under the Java project app/src/main directory, where the normal APK packaging in a gradle build gets all the JNI libraries and puts them into the .apk.

Note that under Android Oreo, the path under which the libraries are made available to the application contains a hashed directory component. This component changes each time the apk is installed. The system must be queried to find the library path if a dlopen is to be done. 

[anil aryan]

Hi Andrew,

I have tried using the steps from the official website and find no luck.

Can you please share the exact script file and the patch files which you are using in this so that I can try myself and will let you know the output of mine.

Your cooperation is much appreciated.