Building with CMake for iOS Simulator on ARM64 fails with undefined reference to _VP8FiltersInitNEON

305 views
Skip to first unread message

Jan Niklas Hasse

unread,
Jun 26, 2021, 1:26:27 PM6/26/21
to WebP Discussion
I'm linking WebP to my CMake-based project using

FetchContent_Declare(webp
     GIT_TAG 6a9916d73480e820ea46316ec8b5bd4c7d04d1f5)
FetchContent_MakeAvailable(webp)

and then linking webpdecode. This works great for all platforms, but recently, after switching from an x86 to an ARM Mac I can no longer build for the simulator:

Undefined symbols for architecture arm64:
  "_VP8FiltersInitNEON", referenced from:
      _VP8FiltersInit_body in libjngl.a(filters.o)
  "_WebPRescalerDspInitNEON", referenced from:
      _WebPRescalerDspInit_body in libjngl.a(rescaler.o)
  "_WebPInitUpsamplersNEON", referenced from:
      _WebPInitUpsamplers_body in libjngl.a(upsampling.o)
  "_WebPInitConvertARGBToYUVNEON", referenced from:
      _WebPInitConvertARGBToYUV_body in libjngl.a(yuv.o)
  "_WebPInitAlphaProcessingNEON", referenced from:
      _WebPInitAlphaProcessing_body in libjngl.a(alpha_processing.o)
  "_WebPInitSharpYUVNEON", referenced from:
      _WebPInitConvertARGBToYUV_body in libjngl.a(yuv.o)
  "_VP8LDspInitNEON", referenced from:
      _VP8LDspInit_body in libjngl.a(lossless.o)
  "_VP8DspInitNEON", referenced from:
      _VP8DspInit_body in libjngl.a(dec.o)
ld: symbol(s) not found for architecture arm64

(libjngl.a is my own library which needs webp)

Building for iOS normally still works. It seems that there's a header which thinks NEON optimizations should be used while the actually library is build without it. Any ideas where I should look?

James Zern

unread,
Jun 28, 2021, 9:04:54 PM6/28/21
to WebP Discussion
Hi,

For the arm64 simulator I'd expect we could use NEON. Are you able to see how cmake is being invoked in the webp project? It's possible there's a bug in the sources included. NEON support is detected automatically [1] so in this case it looks like filters_neon.c, etc. are not being included. Those are always safe to build since they'll produce a stub function if NEON isn't present.

 
--
You received this message because you are subscribed to the Google Groups "WebP Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to webp-discuss...@webmproject.org.
To view this discussion on the web visit https://groups.google.com/a/webmproject.org/d/msgid/webp-discuss/e1c3e923-75bb-4ab2-b736-a3827166ff92n%40webmproject.org.

Jan Niklas Hasse

unread,
Jul 5, 2021, 5:31:44 PM7/5/21
to WebP Discussion, James Zern
Hi,

I invoke CMake like this to generate an Xcode project:

cmake -Bbuild-ios -GXcode -DCMAKE_TOOLCHAIN_FILE=cmake/iOS.toolchain.cmake -DIOS_PLATFORM=SIMULATOR

James Zern

unread,
Jul 10, 2021, 8:03:53 PM7/10/21
to WebP Discussion
On Mon, Jul 5, 2021 at 2:31 PM Jan Niklas Hasse <jha...@gmail.com> wrote:
Hi,

I invoke CMake like this to generate an Xcode project:

cmake -Bbuild-ios -GXcode -DCMAKE_TOOLCHAIN_FILE=cmake/iOS.toolchain.cmake -DIOS_PLATFORM=SIMULATOR

Could you provide the contents of the toolchain file?

On a x86-64 Mac using:
cmake .. -GXcode -DCMAKE_SYSTEM_NAME=iOS -DCMAKE_OSX_ARCHITECTURES=arm64 \
  -DWEBP_BUILD_DWEBP=0 -DWEBP_BUILD_CWEBP=0 \
  -DWEBP_BUILD_IMG2WEBP=0 -DWEBP_BUILD_WEBPINFO=0 \
  -DWEBP_BUILD_WEBPMUX=0 -DWEBP_BUILD_EXTRAS=0

will generate a libwebpdecoder.a with the correct symbols, even if I force the -simulator flag (-DIOS_PLATFORM=SIMULATOR wasn't recognized for me using cmake 3.20.5).

Can you try using xcframeworkbuild.sh [2] to make sure the library can be created correctly? I don't have an ARM Mac to test with.

Jan Niklas Hasse

unread,
Jul 13, 2021, 4:18:10 AM7/13/21
to WebP Discussion, James Zern
On Sunday, 11 July 2021 at 02:03:53 UTC+2 James Zern wrote:
Could you provide the contents of the toolchain file?

 
I've noticed that the IOS_ARCH specified at the bottom is wrong, but removing those lines or adding arm64 to the platform IOS_ARCH doesn't fix the undefined symbols. Also most of the other options doesn't seem tha have an effect, I couldn't find out what results in the NEON definitions missing (see below).

On a x86-64 Mac using:
cmake .. -GXcode -DCMAKE_SYSTEM_NAME=iOS -DCMAKE_OSX_ARCHITECTURES=arm64 \
  -DWEBP_BUILD_DWEBP=0 -DWEBP_BUILD_CWEBP=0 \
  -DWEBP_BUILD_IMG2WEBP=0 -DWEBP_BUILD_WEBPINFO=0 \
  -DWEBP_BUILD_WEBPMUX=0 -DWEBP_BUILD_EXTRAS=0

will generate a libwebpdecoder.a with the correct symbols, even if I force the -simulator flag (-DIOS_PLATFORM=SIMULATOR wasn't recognized for me using cmake 3.20.5).

That works for me on an arm64 Mac, too. I've compared the CMake output of this with my project and noticed the following in my output only:

-- Performing Test WEBP_HAVE_FLAG_NEON - Failed

So there's some flags or settings that make this test fail. Regardless of the test, shouldn't it still work? As you said the files add stub definitions for those functions when NEON support isn't available.

After digging in the CMake sources I found that those files will be removed if the check failed (or WEBP_ENABLE_SIMD is set to OFF). The header will still add the declarations if __aarch64__ is set, resulting in those undefined references.

I've added a patch which no longer removes SIMD files from the generated CMake project, even if SIMD is disabled or unavailable, so that the stub definitions are added either way. Is this the right approach?
neon_on_arm64.patch

James Zern

unread,
Jul 14, 2021, 5:03:54 PM7/14/21
to WebP Discussion
On Tue, Jul 13, 2021 at 1:18 AM Jan Niklas Hasse <jha...@gmail.com> wrote:
On Sunday, 11 July 2021 at 02:03:53 UTC+2 James Zern wrote:
Could you provide the contents of the toolchain file?

 
I've noticed that the IOS_ARCH specified at the bottom is wrong, but removing those lines or adding arm64 to the platform IOS_ARCH doesn't fix the undefined symbols. Also most of the other options doesn't seem tha have an effect, I couldn't find out what results in the NEON definitions missing (see below).

On a x86-64 Mac using:
cmake .. -GXcode -DCMAKE_SYSTEM_NAME=iOS -DCMAKE_OSX_ARCHITECTURES=arm64 \
  -DWEBP_BUILD_DWEBP=0 -DWEBP_BUILD_CWEBP=0 \
  -DWEBP_BUILD_IMG2WEBP=0 -DWEBP_BUILD_WEBPINFO=0 \
  -DWEBP_BUILD_WEBPMUX=0 -DWEBP_BUILD_EXTRAS=0

will generate a libwebpdecoder.a with the correct symbols, even if I force the -simulator flag (-DIOS_PLATFORM=SIMULATOR wasn't recognized for me using cmake 3.20.5).

That works for me on an arm64 Mac, too. I've compared the CMake output of this with my project and noticed the following in my output only:

-- Performing Test WEBP_HAVE_FLAG_NEON - Failed

Could you attach the full console output, CMakeFiles/CMakeOutput.log, CMakeFiles/CMakeError.log and src/webp/config.h? This should be checking for -mfpu=neon which isn't necessary for aarch64, but it could explain the difference as you note below.
 

So there's some flags or settings that make this test fail. Regardless of the test, shouldn't it still work? As you said the files add stub definitions for those functions when NEON support isn't available.

After digging in the CMake sources I found that those files will be removed if the check failed (or WEBP_ENABLE_SIMD is set to OFF). The header will still add the declarations if __aarch64__ is set, resulting in those undefined references.

I've added a patch which no longer removes SIMD files from the generated CMake project, even if SIMD is disabled or unavailable, so that the stub definitions are added either way. Is this the right approach?

As the build and source files are constructed currently yes this is probably correct. The issue comes from the auto-detection in the header like you mentioned. All the webp sources are always safe to build (configure does this), they'll have stubs if a micro-architecture isn't available at build time. This is a bug, however, in that the resulting output may not respect the CMake and configure options since the flags (WEBP_HAVE_*) aren't tested in isolation from the presence of the micro-architecture. 

James Zern

unread,
Jul 19, 2021, 3:47:48 PM7/19/21
to WebP Discussion
After my change to better handle the values in config.h [1] this shouldn't be necessary, though it still might be a reasonable clean up. An open issue after my change is that the cmake build was not designed with multi-configuration in mind, so the simulators will use C-code only when they could instead use some of the optimizations. Because all of the device targets are arm, however, it won't be a problem in that case.

 

Jan Niklas Hasse

unread,
Jul 20, 2021, 1:52:41 PM7/20/21
to WebP Discussion, James Zern
On Monday, 19 July 2021 at 21:47:48 UTC+2 James Zern wrote:
After my change to better handle the values in config.h [1] this shouldn't be necessary, though it still might be a reasonable clean up. An open issue after my change is that the cmake build was not designed with multi-configuration in mind, so the simulators will use C-code only when they could instead use some of the optimizations. Because all of the device targets are arm, however, it won't be a problem in that case.

With your change the undefined references disappear and it works perfectly in the simulator :) Thank you so much!

As for missing out on optimizations: libwebp is already fast enough for my use case anyway :)

James Zern

unread,
Jul 20, 2021, 5:02:05 PM7/20/21
to WebP Discussion
Thanks for testing it out, I'm glad it's working for you. I'll leave the simulator + optimizations as a low priority TODO, but I figured in most cases the C code would be enough for testing.
 
Reply all
Reply to author
Forward
0 new messages