Understanding macOS build environments

[Albrecht Schlosser]

All,

I need your help to (better) understand other users' macOS build
environments.

As you may know I'm trying to maintain our build support files, mainly
CMake but also autoconf as far as I can do.

As a new Mac owner I'm now able to improve our build files for macOS. My
first goal is to make the build files compatible with different users'
build environments, for instance Homebrew, Fink, certainly more,
*without* having to hardcode paths like /sw, /opt/sw, /opt/homebrew etc.
directly in the build files. I'm making progress but I need more info,
that's why I'm asking here.

I know this may involve some work on your side, but maybe you can help
and I'd appreciate this very much. I'll post questions prefixed by "Q: "
and my own answers as an example prefixed by "A: " (Answer/Albrecht).

If you reply, please cite the Questions (Q:) but trim my answers for
brevity and clarity. TIA.


I learned that you need the "Xcode commandline tools" for any builds so
you have compilers, git, and build tools. This is what I did too and I
assume this as a general requirement for commandline builds - unless you
install Xcode and build and test with that.

I learned also that there are some tools that act like "package
managers" to install software packages that are not included in macOS.
Popular tools are Homebrew and Fink, but there are certainly others. I
decided to install Homebrew and that's what I'm now working with.


Q: Which "package management" tool(s) did you install and which tool(s)
are you actually using?

A: Homebrew


Q: How are you mainly building FLTK? Options are configure/make or
CMake. If CMake, please add your "Generator" as well, like "Unix
Makefiles", "Ninja", "Xcode", ...

A: autoconf + CMake/Ninja + CMake/Xcode (I did not yet try CMake/"Unix
Makefiles").

Note: my first build with configure/make worked fine w/o installing
autoconf. I copied a current 'configure' file from my Linux build system
but you can also use a FLTK release tarball or snapshot which contain
configure scripts. No additional software beyond the FLTK download and
the "commandline tools" are required. I'm trying to use as many
different build environments as possible to test FLTK builds.


Q: If you are using autoconf/configure, how did you install 'autoconf'?

A: Homebrew (brew install autoconf)


Q: If you are using Homebrew, what is your Homebrew install directory?

A: /opt/homebrew

FYI: Homebrew installs its packages on new "arm64" Macs to /opt/homebrew
and places symbolic links in /usr/local whereas it installs to
/usr/local on Intel Macs.


Q: Do you have pkg-config installed; if yes: by which package manager or
self-built or ...?

A: yes, Homebrew


Q: How are you building/using the bundled libs (jpeg, png, zlib), i.e.
system or bundled (aka "local") versions?

A: all three either system (WIP) or bundled, i.e. all combinations of
system and/or bundled libs work with either configure/make or
CMake/Ninja (see below for "WIP").

Note: I like Ninja (brew install ninja). It's much faster than "Unix
Makefiles" particularly if only a few files were changed.


Q: What build environments do you use, particularly do you use Xcode to
edit/build/test/debug (FLTK) and your FLTK application code?

A: VS Code (Visual Studio Code) is now my preferred editor and
build/test environment.

I'm using VS Code on Linux and macOS for editing and debugging. There
are currently some limits on M1 (arm64) Macs: debugging with VS Code
works only for x86_64 builds (using Rosetta which makes it somewhat
slower), not for native arm64 but I'm confident this will be resolved
soon (there are open GitHub Issues). Meanwhile I can build x86_64 target
code for debugging with VS Code or I can use Xcode to debug arm64 code.
Universal binaries (arm64 + x86_64) build but not yet tested with
debugger (lldb).


Q: What else should I know about the build environments on your Macs WRT
building FLTK? Which extra build options work, which ones don't?

A1: I installed Cairo (brew install cairo) and I can now build the Cairo
demo - either with configure/make or CMake (after a recent commit).

A2: I'm working on support for using the system versions of all bundled
libs (jpeg, png, zlib). The current status is that I can build (see
answer above) but only with modified CMake files, suited to my Homebrew
setup. AFAICT in previous versions only the bundled jpeg and png libs
worked and zlib was usually "system".

A3: Building universal apps (arm64 + x86_64) basically works but all my
installed Homebrew libs are (currently) arm64 only, hence I can build
apps with system libs (jpeg, png, zlib, cairo, etc.) only as arm64
targets. I hope this can be changed, maybe by building universal libs
myself or if Homebrew can support universal libs in the future.

[Albrecht Schlosser]

On 3/3/21 2:37 PM Albrecht Schlosser wrote:

I forgot an important question, please "insert" after this one:

> Q: If you are using Homebrew, what is your Homebrew install directory?
>
> A: /opt/homebrew

Q: If you are using another "package manager", what is its install
directory and how are packages found (symbolic links anywhere, PATH, etc.)?

A: n/a

[Greg Ercolano]

Q: Which "package management" tool(s) did you install and which tool(s) are you actually using?
A: Homebrew

        Yeah, brew for me quite a few times. Fink maybe once years ago?

        I generally try to avoid installing any package managers on Mac OSX, OSX, macOS (lol, pick a name, Apple)
        just because it can bring in all kinds of stuff that messes around with environment variables that can affect
        Q/C testing for my commercial apps, and tries to stick things in weird places.

Q: How are you mainly building FLTK? Options are configure/make or CMake. If CMake, please add your "Generator" as well, like "Unix Makefiles", "Ninja", "Xcode", ...
A: autoconf + CMake/Ninja + CMake/Xcode (I did not yet try CMake/"Unix Makefiles").

    I've always used 'configure', and now sometimes 'cmake'.

Q: If you are using autoconf/configure, how did you install 'autoconf'?
A: Homebrew (brew install autoconf)

    Hmm, I don't remember having to do special stuff for autoconf.
    I think installing Xcode and/or Xcode tools or some such pulled all that in.

    I never use Xcode myself, but the package brings in all the command line stuff, IIRC.

    You can also JUST install the command line tools.. I think I've done that from time to time.
    But usually I just install "Xcode".

    Others will probably have better info.

Q: If you are using Homebrew, what is your Homebrew install directory?
A: /opt/homebrew

    Ya, that's why I don't like homebrew; creating directories in root just seems wrong,
    and I think in recent OS versions (Big Sur, Catalina), it's seriously frowned apon by the OS
    and actively prevented by having the root system mounted read only.

    No longer can you be root and use 'mkdir' in the root directory. Even in safe mode, creating
    entries in the root directory on the OS volume magically disappear on reboot. To pull it off
    in Big Sur, you have to mess around with some hacky thing called /etc/synthetic.conf,
    or whatever it's called, and it will create a directory or some new magic symbolic links,
    and recommend you put stuff you'd normally put in root in a new directory called /System/Volume/Data
    or some such appley crazy thing that has nothing to do with the unix world. Don't get me started 😠

FYI: Homebrew installs its packages on new "arm64" Macs to /opt/homebrew and places symbolic links in /usr/local whereas it installs to /usr/local on Intel Macs.

Q: Do you have pkg-config installed; if yes: by which package manager or self-built or ...?
A: yes, Homebrew

    I don't build a lot of other libs, so I don't know this one really.

Q: How are you building/using the bundled libs (jpeg, png, zlib), i.e. system or bundled (aka "local") versions?
A: all three either system (WIP) or bundled, i.e. all combinations of system and/or bundled libs work with either configure/make or CMake/Ninja (see below for "WIP").

    I always use the ones that come with FLTK.

Note: I like Ninja (brew install ninja). It's much faster than "Unix Makefiles" particularly if only a few files were changed.

    'make -j 4' works so well, I've never needed anything faster myself.

    But I'm slow to replace old tools cause I hate learning new stuff..
    I need less things that get in the way of daily maintenance.

Q: What build environments do you use, particularly do you use Xcode to edit/build/test/debug (FLTK) and your FLTK application code?
A: VS Code (Visual Studio Code) is now my preferred editor and build/test environment.

    I'm a pure VI/make guy, and only use VS when I have to.
    I never use Xcode, except once I think to figure out how to help someone on the FLTK forum to do something in Xcode.

    I leave my windows machines off as much as possible.

I'm using VS Code on Linux and macOS for editing and debugging. There are currently some limits on M1 (arm64) Macs: debugging with VS Code works only for x86_64 builds (using Rosetta which makes it somewhat slower), not for native arm64 but I'm confident this will be resolved soon (there are open GitHub Issues). Meanwhile I can build x86_64 target code for debugging with VS Code or I can use Xcode to debug arm64 code. Universal binaries (arm64 + x86_64) build but not yet tested with debugger (lldb).


Q: What else should I know about the build environments on your Macs WRT building FLTK? Which extra build options work, which ones don't?
A1: I installed Cairo (brew install cairo) and I can now build the Cairo demo - either with configure/make or CMake (after a recent commit).

    I generally find I have to do very little when setting up new macs to build FLTK other than installing Xcode.
    Maybe I'm forgetting something. I think I've had to build git from source a few times, but I think it "comes with" these days.

[Greg Ercolano]

Q: If you are using Homebrew, what is your Homebrew install directory?
A: /opt/homebrew

    Ya, that's why I don't like homebrew; creating directories in root just seems wrong [..]

    But to answer the question, I leave it at the default which is apparently /opt.

    I hate it though, and would surely prefer something in /usr/local/,
    though I'm not sure what. Maybe just /usr/local, or /usr/local/brew.

    I don't know why this /opt thing ever caught on, /usr/local was always the
    unixy way to install local bins/libs/source.

[Evan Laforge]

On Wed, Mar 3, 2021 at 5:37 AM Albrecht Schlosser wrote:

> Q: Which "package management" tool(s) did you install and which tool(s)
> are you actually using?

nix, and nixpkgs

> Q: How are you mainly building FLTK? Options are configure/make or
> CMake. If CMake, please add your "Generator" as well, like "Unix
> Makefiles", "Ninja", "Xcode", ...
>
> A: autoconf + CMake/Ninja + CMake/Xcode (I did not yet try CMake/"Unix
> Makefiles").

I use the nixpkgs bundled build, but override it to get the specific
fltk commit I want:
https://github.com/elaforge/karya/blob/work/default.nix#L97

I think the nixpkgs build uses `./configure && make` but it also
supports cmake. I think its default behaviour is to detect autotools
and then cmake and use whatever one it finds first. It uses its own
gcc or clang, so there's no need to install the commandline tools
stuff, though I always do that anyway since it's necessary for all
non-nix oriented development.

For the rest of the questions, nixpkgs deals with those things internally.

> Q: What build environments do you use, particularly do you use Xcode to
> edit/build/test/debug (FLTK) and your FLTK application code?

No xcode, I build with shake (it's make-like).

> Q: If you are using another "package manager", what is its install directory and how are packages found (symbolic links anywhere, PATH, etc.)?

Nix puts stuff in `/nix`, and it works by directly supplying
dependencies via absolute paths, so no finding is needed. I hadn't
heard about the "no editing /" thing, I suppose the nix people have
either found a way around that, or moved from `/nix` to somewhere
else. I'm still on High Sierra and I never look forward to upgrades
but someday I suppose I'll have to.

[Rob McDonald]

On Wednesday, March 3, 2021 at 5:37:21 AM UTC-8 Albrecht Schlosser wrote:

Q: Which "package management" tool(s) did you install and which tool(s)
are you actually using?

 

RM: Homebrew

Unlike Erco, I've drunk the coolaid with Homebrew.  I don't like that it installs stuff in /opt, but aside from that, it is far and away the easiest way to install and maintain programs on a Mac.  I've taken to installing just about anything I can through Homebrew.  Not just random small Unixy stuff (where a package manager is far and away the easiest way), but also big applications where normally you would directly download from there website.

The exception here is for any libraries I build against -- I have never installed a library for my own development via Homebrew.  I also do not distribute my application via Homebrew.  So perhaps I'm a hypocrite here.

The  other popular package manager I haven't seen mentioned is MacPorts.

 Q: How are you mainly building FLTK? Options are configure/make or

CMake. If CMake, please add your "Generator" as well, like "Unix
Makefiles", "Ninja", "Xcode", ...

RM: CMake, Xcode compiler, Unix Makefiles as the generator.  I then use CLion as my IDE -- which uses CMake natively, but also really works well with the Makefiles on the back end.  For example, I can build from the command line (make -j8) or from the IDE interchangably.

I used to use other compilers -- in particular Homebrew installed / maintained CLang and GCC.  However, MacOS's reliance on Objective C and Swift now make it practically impossible to build FLTK with anything except for XCode on Mac.

In general, I would prefer to use CLang or GCC vs. XCode -- primarily because I like having explicit control over the version numbers of my toolchain.  The way Apple constantly, vaguely, and silently updates XCode makes it harder to know what you actually used to build with.

Q: What build environments do you use, particularly do you use Xcode to

edit/build/test/debug (FLTK) and your FLTK application code?

RM: As mentioned earlier -- CLion.  It works directly with CMake as its project files, so I use Unix Makefiles as my generator with XCode as my compiler. 

Q: What else should I know about the build environments on your Macs WRT
building FLTK? Which extra build options work, which ones don't?

RM:  I'm not sure if this has improved recently as I haven't updated in a while.  However, historically, the FindFLTK.cmake bundled with CMake was utter garbage.   When I've mentioned this before, people have suggested using autoconf (this is before the recent effort to improve CMake support).  Or people have suggested other workarounds.

My project uses the 'SuperProject' pattern in CMake.  This means all the dependent libraries can be built as subprojects of a Libraries project via the 'ExternalProject_Add' capability in CMake.  In addition to the Libraries subproject, there is an Application subproject and a SuperProject.

A developer can choose to run CMake on the SuperProject (one make command should build all libraries and the application in one huge go) -- great for people building the application, but not actively developing on it.  Or they can run on the Libraries and Application separately.  That way, they aren't constantly rebuilding the Libraries, but the Application is exposed for development.

Even though the libraries are packaged and built by my own system, the Application subproject still uses FindXXXX.cmake scripts to 'find' what it needs from the Libraries subproject.  This is done to allow all the individual libraries in the Libraries project to be optional -- you can use the bundled library or instead use a system-installed and maintained library.  Therefore, the FindXXXX.cmake scripts need to work equally well for both cases (with some prompting for the location of the Libraries subproject).

In practice, we typically use bundled libraries on Mac and Windows and use system installed libraries on Linux wherever possible.

When FindFLTK.cmake fails, it is usually around finding fluid separately from the headers and libs.  Also, this general mode of operation is fragile to extra-long paths.  The SuperProject and ExternalProject_Add end up creating some really long command lines for CMake.  If you set up your build directory in a deep directory, the long path can show up multiple times in some commands and you run into a character length limit for CMake or your shell.  Very frustrating, but not really something FLTK can fix...

In short, I'd love to see some TLC applied to the FindFLTK.cmake script.  We could distribute our own version with FLTK and I am sure the CMake project would pick up any updates quickly.

Rob

 

[Albrecht Schlosser]

On 3/3/21 9:16 PM Rob McDonald wrote:
> On Wednesday, March 3, 2021 at 5:37:21 AM UTC-8 Albrecht Schlosser wrote:
>
>
> Q: Which "package management" tool(s) did you install and which tool(s)
> are you actually using?
>
> RM: Homebrew

Rob and others, thanks for taking the time to reply. It's very much
appreciated. I need some time to read and reply to details ...

However, some short notes to:

> RM:  I'm not sure if this has improved recently as I haven't updated in
> a while.  However, historically, the FindFLTK.cmake bundled with CMake
> was utter garbage.

AFAICT it is still. And it's not very likely that this will change.
Don't use it!

> When I've mentioned this before, people have
> suggested using autoconf (this is before the recent effort to improve
> CMake support).  Or people have suggested other workarounds.

Our documentation (README.CMake.txt) tells you how to invoke
find_package(FLTK ...) correctly in "config mode", not in "module mode".
Unfortunately this requires that FLTK has been built with CMake and
created (and ultimately installed) the CMake Config script.

> In practice, we typically use bundled libraries on Mac and Windows and
> use system installed libraries on Linux wherever possible.

That's the best choice for Linux, definitely. There are cases where
other shared libraries (xft, cairo, pango) pull in the system libs which
can lead to problems.

Exactly for that same reason I want to make it at least *possible* to
use the system libs on macOS (*) rather than the bundled libs to avoid
conflicts. The bundled libs are now updated (not yet released, maybe not
even in Git for 1.3.6 though), but that will be the case soon.

(*) whatever "system libs" you install, be it Homebrew, Fink, MacPorts,
others.

There are still cases that can make it unavoidable to build your own
libs - not only the bundled jpeg, png, zlib, but also more libs (and
link statically) to avoid system dependencies. But that's another story.

On Windows there are no standard system libs so it's clear that you need
to build other dependencies as well.

> In short, I'd love to see some TLC applied to the FindFLTK.cmake
> script.

What is TLC? https://www.acronymfinder.com finds 237 results.

> We could distribute our own version with FLTK and I am sure the
> CMake project would pick up any updates quickly.

I'm not sure they'd want this. Here are some links that show why they
don't want to distribute a FindSDL2 module:

https://stackoverflow.com/questions/54005991/what-happened-to-findsdl2-in-cmake

See Brad King's reply here:
https://github.com/Kitware/CMake/pull/149#issuecomment-87666029

... and the linked discussion:

The gist being (citation): "Package configuration files that come with
the package are technically superior to find modules because they know
exactly what comes with the package and where it is located...".

Another drawback is (CMake) backwards compatibility, I mean: we want to
support the oldest CMake versions possible (to *build* FLTK). I know
it's a different issue, but we can't push FindFLTK modules to older
CMake versions than the most current, i.e. the next version at best.
This will never work.

That said, I think we will need to produce CMake config files that can
be distributed with FLTK so find_package(FLTK CONFIG) aka "NOMODULE"
works. The FindFLTK module in CMake is really only for FLTK 1.1, not
1.3, nor any future release.

[Albrecht Schlosser]

On 3/3/21 9:59 PM Albrecht Schlosser wrote:
>
> I think we will need to produce CMake config files that can
> be distributed with FLTK so find_package(FLTK CONFIG) aka "NOMODULE"
> works.

Typo correction, should read "NO_MODULE" (which is a synonym for "CONFIG").

https://cmake.org/cmake/help/latest/command/find_package.html#full-signature-and-config-mode

[Greg Ercolano]

On 3/3/21 12:16 PM, Rob McDonald wrote:

The  other popular package manager I haven't seen mentioned is MacPorts.

    Oh yeah, I've used ports from time to time too. Thanks for the reminder!

    I think only one of these, brew or ports, uses /opt as a default.
    The other uses /usr/local.. I think?

    Sniffing around, might it actually be that (mac)ports uses /opt,
    and (home)brew uses /usr/local/opt?

    The times I use package managers are when I'm installing tools that have complex
    dependencies on other projects, and magically pulls in the right versions and builds
    it all with the least pain, avoiding having to carefully pick through the package specific
    README/INSTALL docs on how to build for each platform. There's so much variance,
    and so often build errors incompatible with the local compiler.. it's great that brew/ports
    handles all that.

    I don't find myself installing packages from source with wide dependencies often.
    Most of the libs I build against are minimal dependency-wise, like FLTK.

    This is on purpose, so my apps don't get forced to specific hardware/OS versions;
    the more dependencies, the more restrictions.. esp. with things like movie libraries
    that sometimes depend on specific compiler features, hardware for playback, or
    take advantage of particular features only available in newer OS versions.

    And any apps that have many dependencies, like image/movie viewers, photo
    manipulation tools like GIMP, etc. I often install as prepackaged bins; it's usually
    utter madness to build those from source, due to the massive dependencies.

   

[Rob McDonald]

On Wednesday, March 3, 2021 at 12:59:17 PM UTC-8 Albrecht Schlosser wrote:

Rob and others, thanks for taking the time to reply. It's very much
appreciated. I need some time to read and reply to details ...

Seems a small thing I can do to help out.

 

> RM:  I'm not sure if this has improved recently as I haven't updated in
> a while.  However, historically, the FindFLTK.cmake bundled with CMake
> was utter garbage.

AFAICT it is still. And it's not very likely that this will change.
Don't use it!

I hope you don't mind, but I find this answer very unsatisfying.

I will have to parse and understand everything you say later about the alternatives, but I submit that using FindXXXX.cmake is still the default mode of using CMake.

Particularly if the alternative only works if FLTK itself was built and installed with CMake, then only supporting that mode is a non-starter.  Particularly since we still insist on supporting autoconf & co.  By supporting two meta-build systems, we are committing ourselves to always fighting this.  I understand the needs of the installed base (including our core developers), but it is a substantial chunk of technical debt to take on. 

> When I've mentioned this before, people have
> suggested using autoconf (this is before the recent effort to improve
> CMake support).  Or people have suggested other workarounds.

Our documentation (README.CMake.txt) tells you how to invoke
find_package(FLTK ...) correctly in "config mode", not in "module mode".
Unfortunately this requires that FLTK has been built with CMake and
created (and ultimately installed) the CMake Config script.

I'll need to dig more into this to fully understand it.

I too hate toolchain problems.  When I get it mostly working, I do everything I can to not touch it until it is a raging dumpster fire.   Consequently, I haven't seriously touched our build system in years.

I suspect the default build for most Linux distributions is still autoconf -- why would they change if it is still working.  Autoconf used to be the only sane way to go.  Consequently, I suspect this means that FindFLTK.cmake (even in config mode) will not work on the majority of Linux distributions -- which is exactly the most important target for this kind of thing (i.e. the platform most likely to try to use a system-installed library vs. a developer controlled one).

 That's the best choice for Linux, definitely. There are cases where

other shared libraries (xft, cairo, pango) pull in the system libs which
can lead to problems.

Exactly for that same reason I want to make it at least *possible* to
use the system libs on macOS (*) rather than the bundled libs to avoid
conflicts. The bundled libs are now updated (not yet released, maybe not
even in Git for 1.3.6 though), but that will be the case soon.

(*) whatever "system libs" you install, be it Homebrew, Fink, MacPorts,
others.

There are still cases that can make it unavoidable to build your own
libs - not only the bundled jpeg, png, zlib, but also more libs (and
link statically) to avoid system dependencies. But that's another story.

On Windows there are no standard system libs so it's clear that you need
to build other dependencies as well.

Agreed on all points.  I generally use the caveman solution as much as I can.  I do all that I can to statically link everything I can on Mac and Windows.

I used to use libjpeg -- but for some reason I don't remember, I used my own version outside of FLTK.   libjpeg's build system seemed constantly broken on one or more platforms.  For a supposedly simple library, it was taking a disproportionate amount of work to maintain.  One solution could have been to switch to the FLTK bundled version -- but instead (perhaps in a fit of rage), I switched to stb_image and could not be more happy with the result.  Single-file header-only library, no toolchain problems, and I get a bunch of file formats all in one go.  Since then (in this and other projects) I've ditched libjpeg, libpng, libtiff, and others and have adopted stb_image_write where needed -- best decision ever.

> In short, I'd love to see some TLC applied to the FindFLTK.cmake
> script.

What is TLC? https://www.acronymfinder.com finds 237 results.

Tender loving care

 

> We could distribute our own version with FLTK and I am sure the
> CMake project would pick up any updates quickly.

I'm not sure they'd want this. Here are some links that show why they
don't want to distribute a FindSDL2 module:

https://stackoverflow.com/questions/54005991/what-happened-to-findsdl2-in-cmake

See Brad King's reply here:
https://github.com/Kitware/CMake/pull/149#issuecomment-87666029

... and the linked discussion:

The gist being (citation): "Package configuration files that come with
the package are technically superior to find modules because they know
exactly what comes with the package and where it is located...".

I get that this can be a more sophisticated and complex solution.  However, do we really need that complexity?  Is there no way to duplicate that functionality?

99% of what we need is the path to includes, the path to libraries, and the path to fluid.

Flags to identify build options -- jpeg, png, cairo, zlib, etc.  Are next and should be doable somehow.

Do we need to communicate anything super complex?

 

Another drawback is (CMake) backwards compatibility, I mean: we want to
support the oldest CMake versions possible (to *build* FLTK). I know
it's a different issue, but we can't push FindFLTK modules to older
CMake versions than the most current, i.e. the next version at best.
This will never work.

That is why I suggested you carry your own version with FLTK.  In my own project, I carry (or at least used to) some FindXXXX.cmake scripts for various things.  If we have an 'official' FLTK carried version, then application developers can copy that FindFLTK.cmake into their project and carry that until the commonly installed CMake on their target platforms catches up.

FindFLTK.cmake has been broken for at least 10 years.  If we had fixed it back then, I am pretty sure it would have been picked up by CMake upstream and be widely available by now.  Instead, CMake is distributing a known broken script.

 

That said, I think we will need to produce CMake config files that can
be distributed with FLTK so find_package(FLTK CONFIG) aka "NOMODULE"
works. The FindFLTK module in CMake is really only for FLTK 1.1, not
1.3, nor any future release.

I think we're saying the same thing here (but possibly different in some technicality).  

Rob

 

[Greg Ercolano]

In short, I'd love to see some TLC applied to the FindFLTK.cmake script.

What is TLC? https://www.acronymfinder.com finds 237 results.

    Tender Loving Care -- it's an old, pre-internet acronym, if I remember cor.. uh, IIRC.
    I'm not sure when it came about, but it's a really common english thing, seems
    as old as the hills.

[Albrecht Schlosser]

On 3/3/21 10:58 PM Rob McDonald wrote:
>
> > RM:  I'm not sure if this has improved recently as I haven't
> > updated in a while.  However, historically, the FindFLTK.cmake
> > bundled with CMake was utter garbage.
>
> AFAICT it is still. And it's not very likely that this will change.
> Don't use it!
>
> I hope you don't mind, but I find this answer very unsatisfying.

No, it's okay, points taken. Thanks for all constructive criticism.
We're all (and particularly I am) still learning how to deal with CMake.
I don't like autoconf so I'm striving to make the best of our CMake support.

> I will have to parse and understand everything you say later about the
> alternatives, but I submit that using FindXXXX.cmake is still the
> default mode of using CMake.

AFAIC it's officially find_package(NAME options ...) which will
eventually call FindXXXX.cmake (which is called "module mode") or use
XXXXConfig.cmake (which is called "config mode"). See the CMake docs for
find_package(). The way you call it decides which option is chosen or if
you leave it to find_package() to choose - whatever it finds.

> Particularly if the alternative only works if FLTK itself was built and
> installed with CMake, then only supporting that mode is a non-starter.

> ...

> I suspect the default build for most Linux distributions is still
> autoconf -- why would they change if it is still working.  Autoconf used
> to be the only sane way to go.  Consequently, I suspect this means that
> FindFLTK.cmake (even in config mode) will not work on the majority of
> Linux distributions -- which is exactly the most important target for
> this kind of thing (i.e. the platform most likely to try to use a
> system-installed library vs. a developer controlled one).

Good points.

Someone asked (maybe it's an open issue, I don't remember) if we could
provide FLTKConfig.cmake even with autoconf builds. This is something we
(I?) might consider. The effect would be that all Linux distributions
would provide it, whether they build FLTK with autoconf or CMake.

Another option (also a user question, IIRC) was if we could provide
pkg-config (.pc) files. That's also something that would likely help
some people.

Your idea to provide FindFLTK.cmake with FLTK sources (or somewhere
installed by the build?) is also something that might be doable. I need
to make myself familiar with "how to write a CMake find module" to be
able to do this. But if it helps FLTK users to use FLTK in their
projects it might be worth it.

All valid points and a lot of work to do. Particularly to do it *right*.

> > In short, I'd love to see some TLC applied to the FindFLTK.cmake
> > script.
>
> What is TLC? https://www.acronymfinder.com
> <https://www.acronymfinder.com> finds 237 results.
>
> Tender loving care

Thanks for explanation. Also, Greg Ercolano wrote:

> ... it's an old, pre-internet acronym, if I remember cor.. uh, IIRC.

> I'm not sure when it came about, but it's a really
> common english thing, seems as old as the hills.

Yeah, "common english thing". Hard to know for a non-native english
speaker. But now I know, thanks.

And I'll try to do my best.

> FindFLTK.cmake has been broken for at least 10 years.  If we had fixed
> it back then, I am pretty sure it would have been picked up by CMake
> upstream and be widely available by now.  Instead, CMake is distributing
> a known broken script.

The problem was (and is still) that none of the active FLTK devs knew
how to write such a FindFLTK script correctly. But I'll look into it...

[Rob McDonald]

No, it's okay, points taken. Thanks for all constructive criticism.
We're all (and particularly I am) still learning how to deal with CMake.
I don't like autoconf so I'm striving to make the best of our CMake support.

I'm all for the CMake support.  Your efforts (and those of others) in this direction are greatly appreciated.

> I will have to parse and understand everything you say later about the
> alternatives, but I submit that using FindXXXX.cmake is still the
> default mode of using CMake.

AFAIC it's officially find_package(NAME options ...) which will
eventually call FindXXXX.cmake (which is called "module mode") or use
XXXXConfig.cmake (which is called "config mode"). See the CMake docs for
find_package(). The way you call it decides which option is chosen or if
you leave it to find_package() to choose - whatever it finds.

I've read through the CMake docs and a few other resources on this.  I understand why CMake added config mode, but I largely wish they hadn't.  It seems to me that a FindXXXX.cmake script could be made to search first for a config file -- and if it does not find one, continue the old fashioned search process.  It would seem to me to be a more flexible option.

CMake has to work with libraries that weren't built with CMake -- so the config mode will never cover all cases.  FLTK makes it harder on ourselves by allowing multiple different modes for building.

Having autoconf write a CMake config file would certainly help one part of the problem.

However, (when considering compatibility) I don't think we can ever get away from the need for fixing FindFLTK.cmake.  Then again, it has been broken forever, so perhaps nobody really cares.

Someone asked (maybe it's an open issue, I don't remember) if we could
provide FLTKConfig.cmake even with autoconf builds. This is something we
(I?) might consider. The effect would be that all Linux distributions
would provide it, whether they build FLTK with autoconf or CMake.

I agree, this would help with a chunk of the problems.

 

Another option (also a user question, IIRC) was if we could provide
pkg-config (.pc) files. That's also something that would likely help
some people.

I don't know anything about these.

 

Your idea to provide FindFLTK.cmake with FLTK sources (or somewhere
installed by the build?) is also something that might be doable. I need
to make myself familiar with "how to write a CMake find module" to be
able to do this. But if it helps FLTK users to use FLTK in their
projects it might be worth it.

All valid points and a lot of work to do. Particularly to do it *right*.

> > In short, I'd love to see some TLC applied to the FindFLTK.cmake
> > script.
>
> What is TLC? https://www.acronymfinder.com
> <https://www.acronymfinder.com> finds 237 results.
>
> Tender loving care

Thanks for explanation. Also, Greg Ercolano wrote:

> ... it's an old, pre-internet acronym, if I remember cor.. uh, IIRC.
> I'm not sure when it came about, but it's a really
> common english thing, seems as old as the hills.

Yeah, "common english thing". Hard to know for a non-native english
speaker. But now I know, thanks.

And I'll try to do my best.

Your english is superb -- and from your emails, it often feels like you live in the Pacific time zone.  So you have nothing to worry about.

Rob

 

[Manolo]

On Wednesday, March 3, 2021 at 2:37:21 PM UTC+1 Albrecht Schlosser wrote:

Q: Which "package management" tool(s) did you install and which tool(s)
are you actually using?

 A: Fink. Although it doesn't work yet with macOS 11, so I may have to change.

It's installed in /opt/sw

Q: How are you mainly building FLTK? Options are configure/make or
CMake. If CMake, please add your "Generator" as well, like "Unix
Makefiles", "Ninja", "Xcode", ...
A:

A:  my own Xcode project for development

autoconf or CMake/Xcode for verifying they work

 

Q: If you are using autoconf/configure, how did you install 'autoconf'?

A: with Fink

Q: Do you have pkg-config installed; if yes: by which package manager or
self-built or ...?

A: self-built

Q: How are you building/using the bundled libs (jpeg, png, zlib), i.e.
system or bundled (aka "local") versions?

A: FLTK versions for jpeg and png, macOS version for zlib

Q: What build environments do you use, particularly do you use Xcode to
edit/build/test/debug (FLTK) and your FLTK application code?

A: I use Xcode to build and debug macOS, X11 and Windows versions of FLTK

and Virtualbox to run the Windows version.

P.S. I have since yesterday an M1-based MacBok Pro and have problems with XQuartz:

slow and unpredictable, whereas it had been beautiful on Intel for many years.

I have noticed, as expected, that FLTK apps work directly both in arm64 and x86_64 architectures

(provided, in the latter case, Rosetta is installed).

[Albrecht Schlosser]

On 3/4/21 11:10 AM Manolo wrote:
>
> On Wednesday, March 3, 2021 at 2:37:21 PM UTC+1 Albrecht Schlosser wrote:
>
> Q: Which "package management" tool(s) did you install and which tool(s)
> are you actually using?
>
>  A: Fink. Although it doesn't work yet with macOS 11, so I may have to
> change.
> It's installed in /opt/sw

[...]

Thanks to you and everybody else for your answers. I can't reply to
everybody but I appreciate your help. I'll try to improve the build
files and maybe I'll come back with specific questions or ask for help
with testing.

> A: I use Xcode to build and debug macOS, X11 and Windows versions of FLTK
> and Virtualbox to run the Windows version.
>

> P.S. I have since yesterday an M1-based MacBok Pro ...

Great! Now we can share our (FLTK) build experiences on M1 Macs!

BTW: Rumors say (and I think this is sensible) that there are no M1
(arm_64) versions of Virtualbox and maybe many or all the other VM tools
(particularly parallels) - and these VM tools are not expected to come
(at least not soon) since they would have to *emulate* an Intel
processor, at least for Windows. Windows is officially available for ARM
but (again, rumors say) is only distributed to/by OEM's and can't be
bought by single users. Did you research this already for your own
development? If it's true you'd certainly need to keep an older Intel
Mac for your VM's.

> and have problems with XQuartz:
>
> slow and unpredictable, whereas it had been beautiful on Intel for many
> years.

I did not yet install XQuartz, so I can't say anything about this. Too
much to do with other (not only FLTK) stuff, but I'll try as soon as
time permits.

> I have noticed, as expected, that FLTK apps work directly both in arm64
> and x86_64 architectures
> (provided, in the latter case, Rosetta is installed).

Yes, it worked even for me as a new macOS user pretty straight-forward.
The only issues I'm having (so far):

(1) VS Code can't debug arm64 (or universal) apps (using lldb), but this
is a VS Code problem and will hopefully be fixed soon. I can circumvent
this issue by building for x86_64 and debug with VS Code - unless I need
"other" libs (see (2)). My alternative option to debug is to use Xcode
(I built FLTK with CMake/Xcode, tested, works) but I'm not yet familiar
with it. It's very similar to what I used previously though. I'll try
again later when I have a real use case.

(2) I can't build x86_64 or universal apps if I need to link to
libraries (e.g. Cairo) installed with Homebrew because these libs are
all arm64 only - which means I can't debug such apps with VS Code, see
(1). But that's a minor issue for now and can be resolved later. I think
that I can build such libs myself (with Homebrew) and then hopefully as
universal libs. Maybe, we'll see.

[Manolo]

On Thursday, March 4, 2021 at 12:18:22 PM UTC+1 Albrecht Schlosser wrote:

Great! Now we can share our (FLTK) build experiences on M1 Macs!

For now,  I see only deficiencies relatively to my intel MacBook Pro.

BTW: Rumors say (and I think this is sensible) that there are no M1
(arm_64) versions of Virtualbox and maybe many or all the other VM tools
(particularly parallels) - and these VM tools are not expected to come
(at least not soon) since they would have to *emulate* an Intel
processor, at least for Windows. Windows is officially available for ARM
but (again, rumors say) is only distributed to/by OEM's and can't be
bought by single users. Did you research this already for your own
development?

These are my  conclusions too.

If it's true you'd certainly need to keep an older Intel  Mac for your VM's.

Yes indeed.

The only issues I'm having (so far):

(1) VS Code can't debug arm64 (or universal) apps (using lldb), but this
is a VS Code problem and will hopefully be fixed soon. I can circumvent
this issue by building for x86_64 and debug with VS Code - unless I need
"other" libs (see (2)). My alternative option to debug is to use Xcode
(I built FLTK with CMake/Xcode, tested, works) but I'm not yet familiar
with it. It's very similar to what I used previously though. I'll try
again later when I have a real use case.

(2) I can't build x86_64 or universal apps if I need to link to
libraries (e.g. Cairo) installed with Homebrew because these libs are
all arm64 only - which means I can't debug such apps with VS Code, see
(1). But that's a minor issue for now and can be resolved later. I think
that I can build such libs myself (with Homebrew) and then hopefully as
universal libs. Maybe, we'll see.

I have just tried macports since fink isn't ready for macOS 11, and it seems quite

good. Noticeably you can install binaries in arm or in universal form.
I jut did that for pkg-config, as an example of package, which I had first installed in arm form :

% file /opt/local/bin/pkg-config
/opt/local/bin/pkg-config: Mach-O 64-bit executable arm64

% sudo port install  pkgconfig +universal    <-- the install command in its universal flavor

--->  Fetching archive for pkgconfig
--->  Attempting to fetch pkgconfig-0.29.2_0+universal.darwin_20.arm64-x86_64.tbz2 from https://lil.fr.packages.macports.org/pkgconfig
--->  Attempting to fetch pkgconfig-0.29.2_0+universal.darwin_20.arm64-x86_64.tbz2 from https://packages.macports.org/pkgconfig
--->  Attempting to fetch pkgconfig-0.29.2_0+universal.darwin_20.arm64-x86_64.tbz2.rmd160 from https://packages.macports.org/pkgconfig
--->  Installing pkgconfig @0.29.2_0+universal
--->  Deactivating pkgconfig @0.29.2_0
--->  Cleaning pkgconfig
--->  Activating pkgconfig @0.29.2_0+universal
--->  Cleaning pkgconfig

% file /opt/local/bin/pkg-config
/opt/local/bin/pkg-config: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64:Mach-O 64-bit executable arm64]
/opt/local/bin/pkg-config (for architecture x86_64):    Mach-O 64-bit executable x86_64
/opt/local/bin/pkg-config (for architecture arm64):    Mach-O 64-bit executable arm64

[Albrecht Schlosser]

On 3/4/21 2:02 PM Manolo wrote:
>
> On Thursday, March 4, 2021 at 12:18:22 PM UTC+1 Albrecht Schlosser wrote:
>
> Great! Now we can share our (FLTK) build experiences on M1 Macs!
>
> For now,  I see only deficiencies relatively to my intel MacBook Pro.

Did you expect it to work flawlessly from day 0 on? ;-) 8-)

> I have just tried macports since fink isn't ready for macOS 11, and it
> seems quite
> good. Noticeably you can install binaries in arm or in universal form.

That's good news. Maybe I should try this as well. I don't know why but
I had gotten the impression that MacPorts was not as up-to-date as
Homebrew. Maybe that impression was wrong. (?)

> I jut did that for pkg-config, as an example of package, which I had
> first installed in arm form :
>
> % file /opt/local/bin/pkg-config
> /opt/local/bin/pkg-config: Mach-O 64-bit executable arm64
>
> % sudo port install  pkgconfig +universal    <-- the install command in
> its universal flavor
> --->  Fetching archive for pkgconfig
> --->  Attempting to fetch
> pkgconfig-0.29.2_0+universal.darwin_20.arm64-x86_64.tbz2 from
> https://lil.fr.packages.macports.org/pkgconfig
> --->  Attempting to fetch
> pkgconfig-0.29.2_0+universal.darwin_20.arm64-x86_64.tbz2 from
> https://packages.macports.org/pkgconfig
> --->  Attempting to fetch
> pkgconfig-0.29.2_0+universal.darwin_20.arm64-x86_64.tbz2.rmd160 from
> https://packages.macports.org/pkgconfig
> --->  Installing pkgconfig @0.29.2_0+universal
> --->  Deactivating pkgconfig @0.29.2_0
> --->  Cleaning pkgconfig
> --->  Activating pkgconfig @0.29.2_0+universal
> --->  Cleaning pkgconfig

OK, that sounds very good.

> % file /opt/local/bin/pkg-config
> /opt/local/bin/pkg-config: Mach-O universal binary with 2 architectures:
> [x86_64:Mach-O 64-bit executable x86_64] [arm64:Mach-O 64-bit executable
> arm64]
> /opt/local/bin/pkg-config (for architecture x86_64):    Mach-O 64-bit
> executable x86_64
> /opt/local/bin/pkg-config (for architecture arm64):    Mach-O 64-bit
> executable arm64

Some questions:

What is your PATH after installation and configuration of MacPorts?

What does `which pkg-config' output ?

Are there potential conflicts with other tools / symbolic links in
/usr/local or elsewhere?

Finally, pkg-config is a nice example but having a universal executable
is less important than being able to *build* universal apps. The main
purpose of installing 'universal' packages is IMHO for libraries so they
can be linked in universal binaries. Did you try this too? Is it possible?

[Greg Ercolano]

On 3/4/21 2:10 AM, Manolo wrote:

On Wednesday, March 3, 2021 at 2:37:21 PM UTC+1 Albrecht Schlosser wrote:

Q: Which "package management" tool(s) did you install and which tool(s)
are you actually using?

 A: Fink. Although it doesn't work yet with macOS 11, so I may have to change.

It's installed in /opt/sw

    I'm guessing Fink and other tools that use /opt will get what they want using
    /etc/synthetic.conf, which is apple's workaround to making / read-only.

    I'm guessing they'll use the 'special link' approach, and put their actual data
    in either /usr/local/opt or /System/Volumes/Data/opt (again, just a guess!).
    I would hope they choose the former, as the latter is something  Apple could
    change in future releases (as they often do with stuff they just make up).

    /etc/synthetic.conf tells the OS on boot to create empty dirs and/or special kinds of
    symbolic links to another place. This is apparently is Apple's workaround to making
    root completely unwriteable. With this approach, apparently Apple can supervise
    (and deny) what can be done in the root dir.

    I had to make use of synthetic.conf to keep intact my NFS mounting scheme that
    I use across platforms, and refuse to change, which is /net/<volnames>, so I
    created a single line in the file:

/net    /some/where/else

    ..which lets me put my actual mounts in /some/where/else.
    Note that's a /tab/ between /net and /some/where/else; I believe this is to allow
    for spaces in the names (yuck) without having to use quoting.

[Manolo]

On Thursday, March 4, 2021 at 3:00:06 PM UTC+1 Albrecht Schlosser wrote:

Some questions:

What is your PATH after installation and configuration of MacPorts?

My PATH was not correctly set by the macports installation, may be because I use /bin/csh

I had to manually add /opt/local/bin and /opt/local/sbin to my .tcshrc

and then all when smooth.

I also had to edit file xxxxx/misc/fonts.dir and add one line to it because X cursors where missing.
But that's a bug with XQuartz, not with macports.

% echo $PATH
.:/Users/xxx/bin:/Library/Frameworks/Python.framework/Versions/3.7/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin:/opt/X11/bin:/Library/Apple/usr/bin:/opt/local/bin:/opt/local/sbin:/Users/xxx/bin:/usr/local/bin

(there are a few useless remains)

 

What does `which pkg-config' output ?

% which pkg-config
/opt/local/bin/pkg-config

 

Are there potential conflicts with other tools / symbolic links in
/usr/local or elsewhere?

None, because /usr/local is not used. Macports puts all in /opt/local

Finally, pkg-config is a nice example but having a universal executable
is less important than being able to *build* universal apps. The main
purpose of installing 'universal' packages is IMHO for libraries so they
can be linked in universal binaries. Did you try this too? Is it possible?

Yes, it is, as shown below :

% file /opt/local/lib/libglib-2.0.0.dylib
/opt/local/lib/libglib-2.0.0.dylib: Mach-O 64-bit dynamically linked shared library arm64

% sudo port install glib2 +universal
Password:
--->  Fetching archive for ncurses
--->  Attempting to fetch ncurses-6.2_1+universal.darwin_20.arm64-x86_64.tbz2 from https://lil.fr.packages.macports.org/ncurses
… several more fetches …
--->  Fetching archive for glib2
--->  Attempting to fetch glib2-2.58.3_1+universal+x11.darwin_20.arm64-x86_64.tbz2 from https://lil.fr.packages.macports.org/glib2
--->  Attempting to fetch glib2-2.58.3_1+universal+x11.darwin_20.arm64-x86_64.tbz2 from https://packages.macports.org/glib2
--->  Attempting to fetch glib2-2.58.3_1+universal+x11.darwin_20.arm64-x86_64.tbz2.rmd160 from https://packages.macports.org/glib2
--->  Installing glib2 @2.58.3_1+universal+x11
--->  Deactivating glib2 @2.58.3_1+x11
--->  Cleaning glib2
--->  Activating glib2 @2.58.3_1+universal+x11
--->  Cleaning glib2

% file /opt/local/lib/libglib-2.0.0.dylib
/opt/local/lib/libglib-2.0.0.dylib: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit dynamically linked shared library x86_64] [arm64]
/opt/local/lib/libglib-2.0.0.dylib (for architecture x86_64):    Mach-O 64-bit dynamically linked shared library x86_64
/opt/local/lib/libglib-2.0.0.dylib (for architecture arm64):    Mach-O 64-bit dynamically linked shared library arm64

I add that you download binaries with macports, which makes it orders of magnitude faster than with homebrew where

you must recompile everything.

[Manolo]

On Thursday, March 4, 2021 at 4:28:08 PM UTC+1 er...@seriss.com wrote:

On 3/4/21 2:10 AM, Manolo wrote:

 A: Fink. Although it doesn't work yet with macOS 11, so I may have to change.

It's installed in /opt/sw

    I'm guessing Fink and other tools that use /opt will get what they want using
    /etc/synthetic.conf, which is apple's workaround to making / read-only.

I'm not aware of this /etc/synthetic.conf file and its function.

There's no such file here neither in my Intel nor my M1 macOS machines (macOS 11.2.2)

    I'm guessing they'll use the 'special link' approach, and put their actual data
    in either /usr/local/opt or /System/Volumes/Data/opt (again, just a guess!).
    I would hope they choose the former, as the latter is something  Apple could
    change in future releases (as they often do with stuff they just make up).

There's no  /usr/local/opt nor /System/Volumes/Data/opt here either.

    /etc/synthetic.conf tells the OS on boot to create empty dirs and/or special kinds of
    symbolic links to another place. This is apparently is Apple's workaround to making
    root completely unwriteable. With this approach, apparently Apple can supervise
    (and deny) what can be done in the root dir.

My understanding, which must be partial, is different. macOS now uses 2 "intertwined

filesystems" (my wording for this concept), one holding the system and one for user files.

That for the system is readonly.

It's readonly in a strong sense: even root cannot write to it or mount it rw.

It used to be possible to bypass that:

mount the system disk on another, earlier, macOS machines and you can write to /

then reboot and you had your /xxx folder. I did that for /sw to get my fink.

But then fink was improved to use /opt/sw and /opt is not in the system filesystem

but in the user filesystem. So root can write to it.

That those filesystems are interwined is visible with /opt which is a top-level directory

but in the writable part. It's also visible when you install an app, say Firefox, in /Applications:

there you have some files of /Applications in the readonly filesystem, and others in the writable filesystem.

I have once found in the web the name of the file that lists those directories which are writable

and don't remember that info. But it's mostly useless because that file is unchangeable.

With macOS 11, it's much worse: the system part of the file system is cryptographically signed,

so it's impossible to change anything in it, unless you recreate the signature, with obscure

and not well documented means.

My understanding is also that Apple intends to leave /opt a part of the filesystem that users

are free to colonize. That's visible with XQuartz, Macports, Fink that are all put there.

But this may of course change at some future time.

[Manolo]

On Thursday, March 4, 2021 at 5:10:27 PM UTC+1 Manolo wrote:

I have once found in the web the name of the file that lists those directories which are writable

and don't remember that info. But it's mostly useless because that file is unchangeable.

This is that file:  /usr/share/firmlinks

It mentions /opt, /Applications (and /Users) which explains the intertwinning mentionned above.

But /net is not in it. So it's not possible to put extra files in /net, even as root.

I don't know if /etc/synthetic.conf is a way to bypass these constraints, but I fear it's become

much harder with macOS 11.

[Greg Ercolano]

On 3/4/21 8:10 AM, Manolo wrote:

On Thursday, March 4, 2021 at 4:28:08 PM UTC+1 er...@seriss.com wrote:

    I'm guessing Fink and other tools that use /opt will get what they want using
    /etc/synthetic.conf, which is apple's workaround to making / read-only.

I'm not aware of this /etc/synthetic.conf file and its function.

There's no such file here neither in my Intel nor my M1 macOS machines (macOS 11.2.2)

    Like /etc/exports, the file doesn't exist until you create it.
    Check out 'man synthetic.conf' for details.

There's no  /usr/local/opt nor /System/Volumes/Data/opt here either.

    /System/Volumes/Data/ is where apple wants you to put stuff that would
    normally go into the root. It's a writeable directory.

    /usr/local/opt and/or /System/Volumes/Data/opt wouldn't exist until created
    by a user or tool.

    /usr/local/opt is apparently a common place for brew to put /opt stuff
    so it isn't in the root dir.

    Big Sur and Catalina made large changes to the OS to enhance security,
    enforcing intergrity with cryptographically signed features.

    Regular unix folks may be confused by all this (and other stuff Apple has done).
    While I agree with their goal of increasing security, it's ugly and feels hacky as
    to how they've gone about it.

    /etc/synthetic.conf tells the OS on boot to create empty dirs and/or special kinds of
    symbolic links to another place. This is apparently is Apple's workaround to making
    root completely unwriteable. With this approach, apparently Apple can supervise
    (and deny) what can be done in the root dir.

My understanding, which must be partial, is different. macOS now uses 2 "intertwined

filesystems" (my wording for this concept), one holding the system and one for user files.

That for the system is readonly.

It's readonly in a strong sense: even root cannot write to it or mount it rw.

    Yes; mounting root readonly has always been a design goal of unix, but it's often
    not found implemented.

    Given the extreme security climate these days with state sponsored hacking,
    OS vendors have had to start cracking down. Can't blame them.

It used to be possible to bypass that:

mount the system disk on another, earlier, macOS machines and you can write to /

then reboot and you had your /xxx folder. I did that for /sw to get my fink.

But then fink was improved to use /opt/sw and /opt is not in the system filesystem

but in the user filesystem. So root can write to it.

    I think in Catalina you could boot into Recovery mode, go into the Terminal,
    and manipulate the root volume of the boot drive, which is usually mounted rw
    as /Volumes/Xxx. (Since the root directory in recovery mode is the recovery OS,
    not the regular boot OS)

    However, in Big Sur, for sure if you try to create subdirs on the bootable root
    volume, your changes are REMOVED on boot. They just disappear.

    The OS is /really/ enforcing integrity of the OS, not only read-only,
    but insisits on no modification (see above link regarding crypto-signed features)

That those filesystems are interwined is visible with /opt which is a top-level directory

but in the writable part. It's also visible when you install an app, say Firefox, in /Applications:

there you have some files of /Applications in the readonly filesystem, and others in the writable filesystem.

    Ya, notice OS applications are now in /System/Applications, and /Applications is empty.
    That scared the crap outta me one day.. I thought I'd been hacked.

With macOS 11, it's much worse: the system part of the file system is cryptographically signed,

so it's impossible to change anything in it, unless you recreate the signature, with obscure

and not well documented means.

    Yes, exactly.

My understanding is also that Apple intends to leave /opt a part of the filesystem that users

are free to colonize. That's visible with XQuartz, Macports, Fink that are all put there.

But this may of course change at some future time.

    Well, as long as they leave some kind of workaround.
    I'm not sure if apple intends to leave /opt specifically, but rather "provide a workaround
    to create empty dirs and/or symlinks in the root directory" that are up to the user to name.

    And as long as the names don't conflict with Apple's design, they'll be 'allowed' to be created.

    Or at least that is the impression I've gotten. I've not actually dug down deep into
    Apple's documentation on the changes.. but it seems clear that's the intention.

[Albrecht Schlosser]

On 3/3/21 5:46 PM Greg Ercolano wrote:

>> Q: If you are using Homebrew, what is your Homebrew install directory?
>> A: /opt/homebrew
>
>     Ya, that's why I don't like homebrew; creating directories in root
> just seems wrong,
>     and I think in recent OS versions (Big Sur, Catalina), it's
> seriously frowned apon by the OS
>     and actively prevented by having the root system mounted read only.

And that's why I *like* Homebrew; creating directories in /opt just
seems right. ;-)

Well, /opt is well-defined in the "Filesystem Hierarchy Standard (FHS)"
for years and used to store "Optional application software packages", see
https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard

It's a good way to separate different software packages like Homebrew,
Fink, and MacPorts:

- Homebrew: /opt/homebrew
- Fink: /opt/sw
- MacPorts: /opt/local

I do like specifically /opt/homebrew because it's a qualified name.
/opt/sw is IMHO questionable but if no other package selects it, then
it's fine. I don't like /opt/local though because "local" can be
anything. /opt/macports would have been a *much* better decision.

Installing everything in /usr/local is likely to cause conflicts and one
package install overwriting another package.

Specifically for my testing (as a FLTK developer, not a user!) I like
the idea that I can switch development environments by just changing the
PATH environment variable. I hope I can install MacPorts in parallel and
switch between Homebrew and MacPorts as I like and need.

That's the theory, I need to install MacPorts before I can finally judge.

>> Note: I like Ninja (brew install ninja). It's much faster than "Unix
>> Makefiles" particularly if only a few files were changed.
>
>     'make -j 4' works so well, I've never needed anything faster myself.
>
>     But I'm slow to replace old tools cause I hate learning new stuff..
>     I need less things that get in the way of daily maintenance.

Sure, I understand. But once I got curious what that "Ninja" build
system was and how fast it was said to be I tried it and I was convinced.

Here my suggestion (try yourself, it's simple):

On a Mac with homebrew: see above. On a Debian/Ubuntu based Linux
system: `sudo apt install ninja-build' (note it's not 'ninja', that's
something different).

With CMake: use the generator "Ninja" instead of "Unix Makefiles" (the
latter is the default):

$ mkdir -p build/ninja
$ cd build/ninja
$ cmake -G Ninja ../..
$ cmake --build .

alternatively for the build step execute:

$ ninja

Advantages: ninja is super fast when determining what to build. If
nothing is to do it's almost instantaneous. Try this with Makefiles
instead and wait ...

Admittedly, it's /only/ a factor of 2.7 an my MacBook (0.168s vs 0.062s)
but on a slower machine it's noticeable.

I like also that ninja writes the progress messages in one line,
overwriting the count and message. Warnings and error messages are
output clearly in one block, not intertwined with messages from
unrelated build tasks, even in highly parallel builds (Ninja uses -j N
like make, but the default is N == number or processors).

The only difference you need to know is to select -G Ninja and to run
'ninja' instead of 'make'.

Here's a log from my MacBook with a full FLTK build with shared libs ON:

$ time ninja clean
[1/1] Cleaning all built files...
Cleaning... 879 files.

real 0m0.131s
user 0m0.037s
sys 0m0.058s

$ time ninja
[735/735] Linking CXX executable
bin/examples/howto-browser-with-icons.app/Contents/MacOS/howto-browser-with-icons

[ ^^^ no warnings: shows only the last message ]

real 0m13.300s
user 1m17.530s
sys 0m14.980s

$ time ninja
ninja: no work to do.

real 0m0.062s
user 0m0.033s
sys 0m0.027s

> I generally find I have to do very little when setting up new macs
> to build FLTK other than installing Xcode.

Yes, that's the goal for all possible build environments. And as little
setup as possible if you need to enable configure or CMake options.

> Maybe I'm forgetting something. I think I've had to build git from
> source a few times, but I think it "comes with" these days.

git comes with the Xcode commandline tools but autoconf was not
included. The only 'autoconf' on my system is from Homebrew.

[duncan]

AFAIC it's officially find_package(NAME options ...) which will
eventually call FindXXXX.cmake (which is called "module mode") or use
XXXXConfig.cmake (which is called "config mode"). See the CMake docs for
find_package(). The way you call it decides which option is chosen or if
you leave it to find_package() to choose - whatever it finds.

I've read through the CMake docs and a few other resources on this. 

Part of the CMake docs is the wiki page about "Using CMake to build an FLTK application"

which was "state of the art" and "limit of my knowledge" when I put it together in 2008 but

which is probably now contributing to the confusion. If I can still edit it[*], maybe it would be

useful to modify / annotate it to make it consistent with the current README.CMake.txt

build advice, no?

D.

[*] They moved it from its original location, so have the ownership/permissions changed?

https://gitlab.kitware.com/cmake/community/-/wikis/doc/tutorials/ForFLTK and

https://www.fltk.org/articles.php?L834

 

 

 

[Albrecht Schlosser]

On 3/4/21 9:48 PM duncan wrote:
>
> AFAIC it's officially find_package(NAME options ...) which will
> eventually call FindXXXX.cmake (which is called "module mode")
> or use
> XXXXConfig.cmake (which is called "config mode"). See the CMake
> docs for
> find_package(). The way you call it decides which option is
> chosen or if
> you leave it to find_package() to choose - whatever it finds.
>
>
> I've read through the CMake docs and a few other resources on this.
>
>
> Part of the CMake docs is the wiki page about "Using CMake to build an
> FLTK application" which was "state of the art" and "limit of my
> knowledge" when I put it together in 2008 but which is probably now
> contributing to the confusion.

In 2008 this was indeed "state of the art", which means FLTK 1.1. I
don't know who wrote these early CMake files but I read recently that
someone used them even now to build FLTK 1.1.

FLTK 1.3 had just started development in 2008.

> If I can still edit it[*], maybe it would be
> useful to modify / annotate it to make it consistent with the current
> README.CMake.txt build advice, no?

If you can, it would be useful to annotate it; say it's outdated now
(i.e. usable for outdated FLTK 1.1), and refer users to the current
README.CMake.txt

Just don't duplicate (too much) information of README.CMake.txt - I'm
pretty sure this will change before we release 1.4.0.

[duncan]

 

> If I can still edit [the CMake ForFLTK wiki page], maybe it would be

> useful to modify / annotate it to make it consistent with the current
> README.CMake.txt build advice, no?

If you can, it would be useful to annotate it; say it's outdated now
(i.e. usable for outdated FLTK 1.1), and refer users to the current
README.CMake.txt

I've registered for the CMake Gitlab system and my request to be able

to edit a Wiki page is currently under review...

D

 

[duncan]

> If I can still edit [the CMake ForFLTK wiki page], maybe it would be

> useful to modify / annotate it to make it consistent with the current
> README.CMake.txt build advice, no?

If you can, it would be useful to annotate it; say it's outdated now
(i.e. usable for outdated FLTK 1.1), and refer users to the current
README.CMake.txt

I've kept it short and simple and prefaced the ForFLTK page with:

    Disclaimer:

        This page was originally created in 2008 because the even older

        FindFLTK module, written for FLTK-1.1, was never really updated.

        Users should consult the README.CMake.txt in the FLTK-1.4 weekly

        snapshots, or the git repository for up-to-date information.

I haven't touched the FindFLTK module page