Building "Fat" Mac Binary, or Cross-Compiling from ARM arch to x86-64

[Nick]

Hi all,

I'm running on a MacBook Pro with M1 chip (apple silicon) with MacOS Big
Sur 11.2.

I built my own clang/llvm v18.1.8 from source and it's setup to build
both ARM64 and x86-64.

The compiler successfully creates "fat" mac binaries which contain both
the arm64 and x64 binaries for other open source projects, as well as my
own code.  I accomplish this by passing "-arch x86_64 -arch arm64" to
the relevant compiler flags (CFLAGS for the open source projects).


I'm trying to do the same for OpenSSL v1.1.1w, but I get the following
error:

   crypto/arm_arch.h:55:6: error: "unsupported ARM architecture"


Even if I try to build OpenSSL only for x64 (by passing CFLAGS=-arch
x86_64), the OpenSSL compile sets both "-arch arm64 -arch x86_64".  Note
the order of these params is the opposite from what I pass, so tells me
something in the configuration script is setting these.

If I pass no params for CFLAGS, OpenSSL builds successfully, but for
only ARM64.


If this issue is already fixed in a newer version of OpenSSL I'm fine to
upgrade.


Thanks in advance!


Regards,

Nick

[Viktor Dukhovni]

On Thu, Jan 02, 2025 at 12:24:42PM -0500, Nick wrote:

> The compiler successfully creates "fat" mac binaries which contain both
> the arm64 and x64 binaries for other open source projects, as well as my
> own code.  I accomplish this by passing "-arch x86_64 -arch arm64" to
> the relevant compiler flags (CFLAGS for the open source projects).

Unfortunately, that can't trivially work for OpenSSL, because
architecture-specific assembly targets are included in the build. You'd
need to build *both* the ARM *and* X86_64 assembly targets, invoking the
appropriate assembler on each one, and producing either object code for
that particular platform for inclusion in the "fat" binary, or somehow
producting a "fat" objec that is "empty" for one of the targets.

How to do that, requires more detailed knowledge of fat binaries than I
can muster.

> I'm trying to do the same for OpenSSL v1.1.1w, but I get the following
> error:
>
>    crypto/arm_arch.h:55:6: error: "unsupported ARM architecture"
>
> Even if I try to build OpenSSL only for x64 (by passing CFLAGS=-arch
> x86_64), the OpenSSL compile sets both "-arch arm64 -arch x86_64".  Note
> the order of these params is the opposite from what I pass, so tells me
> something in the configuration script is setting these.
>
> If I pass no params for CFLAGS, OpenSSL builds successfully, but for
> only ARM64.
>
> If this issue is already fixed in a newer version of OpenSSL I'm fine
> to upgrade.

I don't expect this is supported or "fixed".

--
Viktor.

[Mounir IDRASSI]

On 1/2/2025 10:59 PM, Viktor Dukhovni wrote:
> Unfortunately, that can't trivially work for OpenSSL, because
> architecture-specific assembly targets are included in the build. You'd
> need to build *both* the ARM *and* X86_64 assembly targets, invoking the
> appropriate assembler on each one, and producing either object code for
> that particular platform for inclusion in the "fat" binary, or somehow
> producting a "fat" objec that is "empty" for one of the targets.
>
> How to do that, requires more detailed knowledge of fat binaries than I
> can muster.

To create a fat binary on macOS, you need to use lipo command to combine
both x64 and arm64 libraries that must be built separately as Viktor
explained:
lipo -create -output libcrypto.dylib libcrypto.dylib.x64
libcrypto.dylib.arm64

We have a script that automates this process. It fetches the OpenSSL
tarball from GitHub, performs separate x64 and arm64 builds, invokes
lipo, updates the install_name and handles code signing. The script is
too long to include here so I’ve shared it as a GitHub gist:

https://gist.github.com/idrassi/cf4ef1a81f9bb5b7d7b3e8382b942665

Take a look at the function lipo_dylib that calls lipo:

-----------------------------------------------------------------

lipo_dylib() {
    local lib_name="\$1"
    local src_x86_64="${INSTALL_PREFIX_X86_64}/lib/${lib_name}"
    local src_arm64="${INSTALL_PREFIX_ARM64}/lib/${lib_name}"
    local dest="${INSTALL_PREFIX_UNIVERSAL}/lib/${lib_name}"

    if [[ -f "${src_x86_64}" && -f "${src_arm64}" ]]; then
        echo "Creating universal binary for ${lib_name}..."
        lipo -create -output "${dest}" "${src_x86_64}" "${src_arm64}"

        # Use install_name_tool to set the correct install_name
        echo "Updating install_name for ${lib_name}..."
        install_name_tool -id "${INSTALL_PATH}/lib/${lib_name}"
"${dest}" || {
            echo "Failed to update install_name for ${lib_name}"
            exit 1
        }

        # If libssl depends on libcrypto, update the dependency path
        if [[ "${lib_name}" == "libssl.3.dylib" ]]; then
            echo "Updating dependency of libssl.3.dylib to point to
${INSTALL_PATH}/lib/libcrypto.3.dylib..."
            install_name_tool -change
"/tmp/openssl_x86_64/lib/libcrypto.3.dylib"
"${INSTALL_PATH}/lib/libcrypto.3.dylib" "${dest}" || {
                echo "Failed to update libssl.3.dylib dependency for
x86_64"
                exit 1
            }
            install_name_tool -change
"/tmp/openssl_arm64/lib/libcrypto.3.dylib"
"${INSTALL_PATH}/lib/libcrypto.3.dylib" "${dest}" || {
                echo "Failed to update libssl.3.dylib dependency for
arm64"
                exit 1
            }
        fi
    else
        echo "Error: Missing ${lib_name} for one or both architectures."
        exit 1
    fi
}
--------------------

Mounir IDRASSI

[Blumenthal, Uri - 0553 - MITLL]

FWIW, I’m working a lot with Macs, both Intel-based and Apple Silicon-based. 

In my experience, it’s been never useful to build “fat” binaries. They brought nothing but problems. 

In our place we build x86_64 binaries for Intel, and AARCH64 for Silicon - and is been perfectly satisfactory here. I recommend considering the same (proven) approach. 

P.S. Can’t comment on cross-compiling - haven’t done that - but would imagine it should be reasonably straightforward. 

—

Regards,

Uri

Secure Resilient Systems and Technologies

MIT Lincoln Laboratory

On Jan 2, 2025, at 19:17, 'Mounir IDRASSI' via openssl-users <openss...@openssl.org> wrote:

!-------------------------------------------------------------------|
This Message Is From an External Sender
This message came from outside the Laboratory.
|-------------------------------------------------------------------!

--
You received this message because you are subscribed to the Google Groups "openssl-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openssl-user...@openssl.org.
To view this discussion visit https://groups.google.com/a/openssl.org/d/msgid/openssl-users/30a11935-2688-44f8-86e2-52d48462f3f6%40idrix.net.

[Mounir IDRASSI]

On 1/3/2025 3:01 AM, Blumenthal, Uri - 0553 - MITLL wrote:
> In my experience, it’s been _never_ useful to build “fat” binaries.

> They brought nothing but problems.
>
> In our place we build x86_64 binaries for Intel, and AARCH64 for
> Silicon - and is been perfectly satisfactory here. I recommend
> considering the same (proven) approach.

On my side, using "fat" binaries on macOS for OpenSSL linked products
has been a positive experience.
I have been using this approach for various macOS products since Mac OS
X Tiger to provide single installation packages that target all
architectures (including ppc, i386).
Of course, it depends on the type and complexity of the application
logic but usually it is possible to sort out issues without much
difficulty, and OpenSSL is almost never the cause of any issues.

--
Mounir IDRASSI

[Blumenthal, Uri - 0553 - MITLL]

I hear you. But given that there are two (or three at most!) architectures – I fail to see why building one complicated package with guaranteed 50% garbage (x86 machines don’t need PPC or AARCH64, and vs. versa) would be better than building two (or three) straightforward packages, one for each architecture, with no fluff.

 

--

V/R,

Uri

 

There are two ways to design a system. One is to make it so simple there are obviously no deficiencies.

The other is to make it so complex there are no obvious deficiencies.

                                                                                                                                     -  C. A. R. Hoare

 

 

From: 'Mounir IDRASSI' via openssl-users <openss...@openssl.org>
Date: Friday, January 3, 2025 at 04:16
To: openss...@openssl.org <openss...@openssl.org>
Subject: Re: [EXT] Re: Building "Fat" Mac Binary, or Cross-Compiling from ARM arch to x86-64

!-------------------------------------------------------------------|
  This Message Is From an External Sender
  This message came from outside the Laboratory.
|-------------------------------------------------------------------!

--
You received this message because you are subscribed to the Google Groups "openssl-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openssl-user...@openssl.org.

To view this discussion visit https://groups.google.com/a/openssl.org/d/msgid/openssl-users/a9a300d1-fe63-48ae-b38e-ea4bd1aacc1c%40idrix.net.

[Nick]

On 1/2/25 4:59 PM, Viktor Dukhovni wrote:

Unfortunately, that can't trivially work for OpenSSL, because

architecture-specific assembly targets are included in the build.

Yes, that makes sense. After sending my last message I noticed that the
OpenSSL "C" files (.c) were building fine with both archs; it was when
it got to the assembly file (.S) that it started failing.

You'd
need to build *both* the ARM *and* X86_64 assembly targets, invoking the
appropriate assembler on each one, and producing either object code for
that particular platform for inclusion in the "fat" binary, or somehow
producting a "fat" objec that is "empty" for one of the targets.

How to do that, requires more detailed knowledge of fat binaries than I
can muster.

This can be done with the "lipo" command.  It was Plan B, which is now turning into Plan A.


Regards,
Nick

[Nick]

On 1/2/25 7:17 PM, 'Mounir IDRASSI' via openssl-users wrote:
> To create a fat binary on macOS, you need to use lipo command to
> combine both x64 and arm64 libraries that must be built separately as
> Viktor explained:
> lipo -create -output libcrypto.dylib libcrypto.dylib.x64
> libcrypto.dylib.arm64
>
> We have a script that automates this process. It fetches the OpenSSL
> tarball from GitHub, performs separate x64 and arm64 builds, invokes
> lipo, updates the install_name and handles code signing. The script is
> too long to include here so I’ve shared it as a GitHub gist:
>
> https://gist.github.com/idrassi/cf4ef1a81f9bb5b7d7b3e8382b942665
>
> Take a look at the function lipo_dylib that calls lipo:

I already knew about lipo, but this is even better.  Thanks!


Regards,

Nick

[Nick]

On 1/2/25 9:01 PM, Blumenthal, Uri - 0553 - MITLL wrote:

FWIW, I’m working a lot with Macs, both Intel-based and Apple Silicon-based. 

In my experience, it’s been never useful to build “fat” binaries. They brought nothing but problems. 

In our place we build x86_64 binaries for Intel, and AARCH64 for Silicon - and is been perfectly satisfactory here. I recommend considering the same (proven) approach.

We considered this approach.  For context, the software I'm working on is a commercial closed-source application library, containing only .so/.dylib/.dll files, delivered in a SDK+Runtime package.  Our customers integrate our software into their application and that gets deployed to either the cloud or on-prem.  Originally we planned to have 2 packages for Mac:  1 for Intel, 1 for Apple Silicon.  But as we thought about the use case of deploying to Mac endpoints, it seemed like Universal binaries would be easier (ie. "One size fits all").

Your experience is interesting to me.  Can you elaborate on the problems you've faced with Universal ("fat") binaries?

Regards,

Nick

[Nick]

On 1/3/25 4:13 AM, 'Mounir IDRASSI' via openssl-users wrote:
> On my side, using "fat" binaries on macOS for OpenSSL linked products
> has been a positive experience.
> I have been using this approach for various macOS products since Mac
> OS X Tiger to provide single installation packages that target all
> architectures (including ppc, i386).
> Of course, it depends on the type and complexity of the application
> logic but usually it is possible to sort out issues without much
> difficulty, and OpenSSL is almost never the cause of any issues.
>

I appreciate this.  We're reading up on pros and cons of Universal
binaries for MacOS.  I'd appreciate any insight you have, especially any
issues you've seen.


Regards,

Nick