Build failure on macOS 11 and earlier due to use of `readlink -f` (Issue #25675)
Ryan Carsten Schmidt
ryandesign created an issue (wxWidgets/wxWidgets#25675)Build System Used
I build wxWidgets and/or my application using:
- CMake
- configure
- mingw32-make with makefile.gcc
- MSBuild (Microsoft Visual Studio solution file)
- nmake with makefile.vc
- Xcode
Description
wxWidgets 3.2.8.1 doesn't build on macOS versions earlier than 12, at least when installed using MacPorts. This problem didn't occur in 3.2.4, the previous version we used in MacPorts.
The error is:
readlink: illegal option -- f
usage: readlink [-n] [file ...]
followed by errors from install_name_tool. If you want a full build log, see https://trac.macports.org/ticket/72751.
I believe the problem was introduced in 65ad684, which introduced the use of readlink -f. readlink doesn't have a -f flag until macOS 12.
Platform and version information
- wxWidgets version you are building: 3.2.8.1
- wxWidgets port you are building: wxOSX
- OS and its version: macOS 11 and earlier
- Compiler being used: clang
- Non-default compiler options, if any : probably not relevant
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.
VZ
vadz left a comment (wxWidgets/wxWidgets#25675)Thanks for reporting this and sorry for missing that -f wasn't available on earlier macOS versions.
This StackOverflow answer proposes to use perl -MCwd -e 'print Cwd::abs_path shift' instead of readlink -f, which should work on all macOS versions, I think, but we probably don't actually need to canonicalize anything and this should be an even simpler solution:
diff --git a/configure.in b/configure.in index 9e119382a8..75c9ec7dd1 100644 --- a/configure.in +++ b/configure.in @@ -4142,15 +4142,16 @@ if test "$wxUSE_SHARED" = "yes"; then DYLIB_RPATH_POSTLINK="${HOST_PREFIX}install_name_tool -id \$@ \$@" cat <<EOF >change-install-names #!/bin/sh +libdir=\$(cd lib ; pwd -P) libnames=\$(cd lib ; ls -1 libwx*${WX_RELEASE}.dylib) changes='' for dep in \${libnames} ; do - target=\$(readlink -f \${4}/\${dep}) - changes="\${changes} -change \${target} \${3}/\${dep}" + target=\$(readlink \${4}/\${dep}) + changes="\${changes} -change \${libdir}\${target} \${3}/\${dep}" done for i in \${libnames} ; do - lib=\$(readlink -f \${1}/\${i}) - ${HOST_PREFIX}install_name_tool \${changes} -id \${3}/\${i} \${lib} + lib=\$(readlink \${1}/\${i}) + ${HOST_PREFIX}install_name_tool \${changes} -id \${3}/\${i} \${libdir}\${lib} done if test -f "\${2}/wxrc-${WX_RELEASE}" ; then
Could you please check if this works for you (please let me know if you need a diff to configure itself as well or if you can rerun autoconf on your own)?
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.
Ryan Carsten Schmidt
ryandesign left a comment (wxWidgets/wxWidgets#25675)Could you please check if this works for you
Thanks; I'll try it.
proposes to use
perl -MCwd -e 'print Cwd::abs_path shift'instead ofreadlink -f, which should work on all macOS versions
Apple announced in 2019 that "Scripting language runtimes such as Python, Ruby, and Perl are included in macOS for compatibility with legacy software. Future versions of macOS won’t include scripting language runtimes by default" so I'm glad you were able to avoid adding a dependency on perl here.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.
Ryan Carsten Schmidt
ryandesign left a comment (wxWidgets/wxWidgets#25675)+ ${HOST_PREFIX}install_name_tool \${changes} -id \${3}/\${i} \${libdir}\${lib}
Almost: you forgot the / between \${libdir} and \${lib}.
error: /Library/Developer/CommandLineTools/usr/bin/install_name_tool: can't open file: /opt/local/var/macports/build/wxWidgets-3.2-b818c50a/work/wxWidgets-3.2.8.1/build/liblibwx_baseu-3.2.0.dylib (No such file or directory)
But the build doesn't stop there; it proceeds to install libraries that haven't had their install_names fixed. This can be avoided by adding || exit $? at the end of lines that run commands that could fail.
--- configure.orig 2025-05-25 12:15:56.000000000 -0500 +++ configure 2025-08-01 05:10:49.000000000 -0500 @@ -34271,19 +34271,20 @@
DYLIB_RPATH_POSTLINK="${HOST_PREFIX}install_name_tool -id \$@ \$@"
cat <<EOF >change-install-names
#!/bin/sh
+libdir=\$(cd lib ; pwd -P)
libnames=\$(cd lib ; ls -1 libwx*${WX_RELEASE}.dylib)
changes=''
for dep in \${libnames} ; do
- target=\$(readlink -f \${4}/\${dep})
- changes="\${changes} -change \${target} \${3}/\${dep}"
+ target=\$(readlink \${4}/\${dep}) || exit \$?+ changes="\${changes} -change \${libdir}\${target} \${3}/\${dep}" done for i in \${libnames} ; do - lib=\$(readlink -f \${1}/\${i}) - ${HOST_PREFIX}install_name_tool \${changes} -id \${3}/\${i} \${lib}
+ lib=\$(readlink \${1}/\${i}) || exit \$? + ${HOST_PREFIX}install_name_tool \${changes} -id \${3}/\${i} \${libdir}/\${lib} || exit \$?
done
if test -f "\${2}/wxrc-${WX_RELEASE}" ; then
- ${HOST_PREFIX}install_name_tool \${changes} \${2}/wxrc-${WX_RELEASE} + ${HOST_PREFIX}install_name_tool \${changes} \${2}/wxrc-${WX_RELEASE} || exit \$? fi EOF chmod +x change-install-names
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.
VZ
vadz left a comment (wxWidgets/wxWidgets#25675)Oops, thanks for testing and correcting this!
Adding exit $? might be a good idea, but maybe we should add set -e in the beginning of the script instead? For something as short as this, it should work fine, I think.
So the current version of my proposed patch is
diff --git a/configure.in b/configure.in index 9e119382a8..1b4e2b385e 100644 --- a/configure.in +++ b/configure.in @@ -4142,15 +4142,17 @@ if test "$wxUSE_SHARED" = "yes"; then DYLIB_RPATH_POSTLINK="${HOST_PREFIX}install_name_tool -id \$@ \$@" cat <<EOF >change-install-names #!/bin/sh +set -e
+libdir=\$(cd lib ; pwd -P) libnames=\$(cd lib ; ls -1 libwx*${WX_RELEASE}.dylib) changes='' for dep in \${libnames} ; do - target=\$(readlink -f \${4}/\${dep}) - changes="\${changes} -change \${target} \${3}/\${dep}" + target=\$(readlink \${4}/\${dep})
+ changes="\${changes} -change \${libdir}\${target} \${3}/\${dep}" done for i in \${libnames} ; do - lib=\$(readlink -f \${1}/\${i}) - ${HOST_PREFIX}install_name_tool \${changes} -id \${3}/\${i} \${lib} + lib=\$(readlink \${1}/\${i})
+ ${HOST_PREFIX}install_name_tool \${changes} -id \${3}/\${i} \${libdir}/\${lib}
done
if test -f "\${2}/wxrc-${WX_RELEASE}" ; thenDoes this work?
Thanks again!
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.
nieder
nieder left a comment (wxWidgets/wxWidgets#25675)I can confirm that the latest patch there works for me on quite old macOS 10.14.6
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.
Ryan Carsten Schmidt
ryandesign left a comment (wxWidgets/wxWidgets#25675)Adding
exit $?might be a good idea, but maybe we should addset -ein the beginning of the script instead? For something as short as this, it should work fine, I think.
I agree set -e is fine here.
I suggest making it set -ex to also print the commands being run. This makes it easier to debug when problems occur.
Almost: you forgot the
/between\${libdir}and\${lib}.
Does this work?
Unfortunately no. The build and install succeeds but the install names have not been fixed as you intended.
First, there's another missing slash: \${libdir}\${target} needs to be \${libdir}/\${target}.
Second, there's a mismatch between the install name used at build time and the changes being attempted at install time, so the change doesn't happen.
For example, at build time, the build system creates the library with an install name of the final install location, as is customary:
/usr/bin/clang++ -std=gnu++11 -dynamiclib -single_module -headerpad_max_install_names -o /opt/local/var/macports/build/wxWidgets-3.2-b818c50a/work/wxWidgets-3.2.8.1/build/lib/libwx_baseu-3.2.0.4.1.dylib ... -install_name /opt/local/Library/Frameworks/wxWidgets.framework/Versions/wxWidgets/3.1/lib/libwx_baseu-3.2.0.dylib ...
The install name is now /opt/local/Library/Frameworks/wxWidgets.framework/Versions/wxWidgets/3.1/lib/libwx_baseu-3.2.0.dylib.
The build system then immediately changes the install name to the build path, which is weird:
install_name_tool -id /opt/local/var/macports/build/wxWidgets-3.2-b818c50a/work/wxWidgets-3.2.8.1/build/lib/libwx_baseu-3.2.0.4.1.dylib /opt/local/var/macports/build/wxWidgets-3.2-b818c50a/work/wxWidgets-3.2.8.1/build/lib/libwx_baseu-3.2.0.4.1.dylib
The install name is now /opt/local/var/macports/build/wxWidgets-3.2-b818c50a/work/wxWidgets-3.2.8.1/build/lib/libwx_baseu-3.2.0.4.1.dylib.
At install time, it then tries to change the install name back to the final install path:
install_name_tool -change /opt/local/var/macports/build/wxWidgets-3.2-b818c50a/work/wxWidgets-3.2.8.1/build/lib/libwx_baseu-3.2.0.dylib /opt/local/Library/Frameworks/wxWidgets.framework/Versions/wxWidgets/3.1/lib/libwx_baseu-3.2.dylib ..Ryan Carsten Schmidt
ryandesign left a comment (wxWidgets/wxWidgets#25675)A MacPorts user provided a different patch
Ok I've looked at that patch:
--- configure.in.orig 2025-05-26 02:15:56.000000000 +0900 +++ configure.in 2025-07-29 22:57:00.000000000 +0900 @@ -4142,15 +4142,13 @@
DYLIB_RPATH_POSTLINK="${HOST_PREFIX}install_name_tool -id \$@ \$@"
cat <<EOF >change-install-names
#!/bin/sh
-libnames=\$(cd lib ; ls -1 libwx*${WX_RELEASE}.dylib) +libnames=\`cd lib ; ls -1 | grep '\.[0-9][0-9]*\.dylib\$'\`
changes=''
for dep in \${libnames} ; do
- target=\$(readlink -f \${4}/\${dep})
- changes="\${changes} -change \${target} \${3}/\${dep}"
+ changes="\${changes} -change \${4}/\${dep} \${3}/\${dep}"
done
for i in \${libnames} ; do
- lib=\$(readlink -f \${1}/\${i})
- ${HOST_PREFIX}install_name_tool \${changes} -id \${3}/\${i} \${lib}
+ ${HOST_PREFIX}install_name_tool \${changes} -id \${3}/\${i} \${1}/\${i}
done
if test -f "\${2}/wxrc-${WX_RELEASE}" ; thenIt avoids the question of whether the libraries are symlinks or which of them was used as the install name by changing libnames to all the dylibs, both real files and symlinks. This means a bunch of unnecessary -change clauses will be in there, but the necessary ones will be too. And it means each library's -id will be changed multiple times, but since the output of ls is sorted the last one will be the right one. So while this might not be the optimal way to do it, it does at least work.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.
VZ
vadz left a comment (wxWidgets/wxWidgets#25675)I don't have time to look at this in details now, but I'm afraid the alternative patch just restores the original version of this code which means that it reintroduces #23143 fixed 7fdc132 (Fix dylib symlinks on Apple Silicon Macs, 2024-05-09) in 3.2.
See also the subsequent 65ad684 (Fix install names of macOS shared libraries, 2025-02-19) which introduced the use of readlink -f for the explanation why it is needed.
Again, I won't be able to look at this soon, but we really need to skip the symlinks. It looks like the real problem is just the mismatch between using 3.2.0 vs 3.2.0.4.1 and this needs to be fixed. FWIW it's convenient to have RPATH set to the build location as it allows to use the library without installing it, so this is something we also want to keep.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.
Ryan Carsten Schmidt
ryandesign left a comment (wxWidgets/wxWidgets#25675)but I'm afraid the alternative patch just restores the original version of this code which means that it reintroduces #23143 fixed 7fdc132 (Fix dylib symlinks on Apple Silicon Macs, 2024-05-09) in 3.2.
Ah. The user did not disclose that their patch was taken from your previous code.
I see this change was in order to work around a bug in Xcode 12.2's install_name_tool that replaced symlinks with copies. In #23143 you link to the explanation of that bug that I wrote on a MacPorts mailing list. Hopefully the number of people using that buggy version of Xcode is small today, but it's a noble goal to work around it.
See also the subsequent 65ad684 (Fix install names of macOS shared libraries, 2025-02-19) which introduced the use of
readlink -ffor the explanation why it is needed.Again, I won't be able to look at this soon, but we really need to skip the symlinks.
I already committed the user's patch to MacPorts after verifying that it seemed to produce reasonable libraries and binaries and we had successful builds on all automated builders running x86_64 OS X 10.11 and later and arm64 macOS 13 and later. Using the shorter WX_RELEASE version number in the cross-library references would be ideal but I didn't think using the long version number would cause any problems since they should get resolved through the symlinks. The install name is using the short version number so it does not impact other software that links with the library.
It looks like the real problem is just the mismatch between using 3.2.0 vs 3.2.0.4.1 and this needs to be fixed.
Where does the number 3.2.0.4.1 come from? Is there a variable we can use that contains that value, like WX_RELEASE contains 3.2?
One option is, as you say, to change change-install-names so that it can change the long version number into the short one.
A second option is to fix how install_name_tool is used at build time so that it sets the install name to the shorter version number. Option 2a is to eliminate the use of install_name_tool at build time and set the build-path install name when the library is created.
Option 3 is to go back to the normal way of creating libraries, setting their final install name just once at build time and getting rid of change-install-names since implementing it correctly is obviously a challenge and it shouldn't be needed.
FWIW it's convenient to have RPATH set to the build location
We're not talking about rpath; we're talking about library install names. rpath is an unrelated dylib feature that is not involved here.
as it allows to use the library without installing it, so this is something we also want to keep.
The normal way to use a library before installing it is to set the DYLD_LIBRARY_PATH environment variable, e.g.:
DYLD_LIBRARY_PATH=path/to/lib/dir ./prog-that-uses-lib
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.
Adam
moubctez left a comment (wxWidgets/wxWidgets#25675)I had to fix configure this way (for PkgSrc):
--- configure.orig 2025-12-07 22:59:42.000000000 +0000
+++ configure
@@ -34272,16 +34272,13 @@ rm -f core conftest.err conftest.$ac_obj
cat <<EOF >change-install-names
#!/bin/sh
set -e
-libdir=\$(cd lib ; pwd -P)
-libnames=\$(cd lib ; ls -1 libwx*${WX_RELEASE}.dylib)
+libnames=\$(cd lib ; ls -1 libwx*${WX_RELEASE}.[0-9].[0-9].[0-9].dylib)
changes=''
for dep in \${libnames} ; do
- target=\$(readlink \${4}/\${dep})
- changes="\${changes} -change \${libdir}\${target} \${3}/\${dep}"
+ changes="\${changes} -change \${4}/\${dep} \${3}/\${dep}"
done
for i in \${libnames} ; do
- lib=\$(readlink \${1}/\${i})
- ${HOST_PREFIX}install_name_tool \${changes} -id \${3}/\${i} \${libdir}/\${lib}
+ ${HOST_PREFIX}install_name_tool \${changes} -id \${3}/\${i} \${1}/\${i}
done
if test -f "\${2}/wxrc-${WX_RELEASE}" ; then
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.