New to Meson: Questions and suggestions for docs
571 views
Skip to first unread message
GertyP
Aug 27, 2022, 2:52:10 PM8/27/22
to The Meson Build System
Hello.
I'm new to meson but I want to share a little feedback as well as to ask for some help and clarification.
I'm aware of how some build systems can have a whole load of hidden, sometimes complex machinery to try to automatically try, guess, or figure out what compiler/linkers and associated build infrastructure is available, what host/target settings to use, and where to find standard platform headers and libs. However, I'm really averse to that whole approach for many reasons (adds complexity, permutations, problems, difficulty finding or even fighting against hidden steps, reproduction and portability problems, etc.). I prefer build systems to be fully explicit and not try to go and guess and figure things out (and maybe get it wrong or conflict with a user's attempts to specify something else). There aren't too many things in a toolchain that it's impractial for people to explicitly configure. If a compiler, linker, system lib/include path ends up being needed but isn't specified, I don't think many people would be unable to simply explicitly point to what they want to use. I do know of many struggles to find out why the wrong automatically detected compilers/linkers/header-paths/lib-paths are being used or why, in some cases, explicit settings/paths are being ignored/overidden/etc. and then trying ugly hacks to work-around the increasingly complex search/find/guess automation in other build systems.
For these reason, I think it's undesireable and unfortunate if the following output from '>meson setup ...' really is a requirement -
"Visual Studio environment is needed to run Ninja. ..."
The power of a clean and portable build system would be one in which you can just unzip an entire directory structure of source, build system, toolchain, and platform sdk headers and libs, etc. and then build. So forcing users to have to run installers (with complications from multiple versions) would seem to me to unnecessarily devalue, and in some cases complicate, a build system. It clearly is possible to compile, link, and build without requiring a 'visual studio environment', by invoking the individual compiler and linker commands, pointing to and pathing everything needed into a custom directory structure of compiler/linker/binaries, headers, and libs. I'm hoping that the above meson output is something I've misunderstood or is not accurate.
I hoped to use meson to compile and link code in a clean, self-contained, and portable way, without the need to have a bunch of other things 'installed'. I'm hoping this would be possible and straightforward through a suitable native (or cross compile) config file, along with a few accompanying directories containing toolchain binaries and standard library and sdk headers and libs. However, I've been struggling to figure out how to set this up.
To try and work towards and test this setup, I can hide "C:\Program Files\Microsoft Visual Studio", and use a --cross-file that fully points to a cpp compiler, a linker, and sets the machine architecture/ABI. Then doing -
>meson setup --cross-file mytoolchainconfig.ini build
appears to invoke and parse 'vswhere.exe', which fails. This feels like the kind of automatic, find/guess/hope and flail machinery that one usually wouldn't want to be involved when trying to set up a build system's toolchain and standard headers/libs as explicitly as possible.
So, is what I'm trying to do possible with meson?
If not, would it take much work (and does it seem a reasonable thing to have this goal in a build system) to be able to let the user simply explicitly specify all toolchain aspects of their build configuration?
I'm aware of the paradox of more and lengthier questions tending to elicit fewer answers so I apologies for the length of this; I don't have the skill to further condense it, while keeping some rationale. So the above are my main questions.
For those who still have the will power, I've added some secondary questions and thoughts below.
I'm new to meson but I want to share a little feedback as well as to ask for some help and clarification.
I'm aware of how some build systems can have a whole load of hidden, sometimes complex machinery to try to automatically try, guess, or figure out what compiler/linkers and associated build infrastructure is available, what host/target settings to use, and where to find standard platform headers and libs. However, I'm really averse to that whole approach for many reasons (adds complexity, permutations, problems, difficulty finding or even fighting against hidden steps, reproduction and portability problems, etc.). I prefer build systems to be fully explicit and not try to go and guess and figure things out (and maybe get it wrong or conflict with a user's attempts to specify something else). There aren't too many things in a toolchain that it's impractial for people to explicitly configure. If a compiler, linker, system lib/include path ends up being needed but isn't specified, I don't think many people would be unable to simply explicitly point to what they want to use. I do know of many struggles to find out why the wrong automatically detected compilers/linkers/header-paths/lib-paths are being used or why, in some cases, explicit settings/paths are being ignored/overidden/etc. and then trying ugly hacks to work-around the increasingly complex search/find/guess automation in other build systems.
For these reason, I think it's undesireable and unfortunate if the following output from '>meson setup ...' really is a requirement -
"Visual Studio environment is needed to run Ninja. ..."
The power of a clean and portable build system would be one in which you can just unzip an entire directory structure of source, build system, toolchain, and platform sdk headers and libs, etc. and then build. So forcing users to have to run installers (with complications from multiple versions) would seem to me to unnecessarily devalue, and in some cases complicate, a build system. It clearly is possible to compile, link, and build without requiring a 'visual studio environment', by invoking the individual compiler and linker commands, pointing to and pathing everything needed into a custom directory structure of compiler/linker/binaries, headers, and libs. I'm hoping that the above meson output is something I've misunderstood or is not accurate.
I hoped to use meson to compile and link code in a clean, self-contained, and portable way, without the need to have a bunch of other things 'installed'. I'm hoping this would be possible and straightforward through a suitable native (or cross compile) config file, along with a few accompanying directories containing toolchain binaries and standard library and sdk headers and libs. However, I've been struggling to figure out how to set this up.
To try and work towards and test this setup, I can hide "C:\Program Files\Microsoft Visual Studio", and use a --cross-file that fully points to a cpp compiler, a linker, and sets the machine architecture/ABI. Then doing -
>meson setup --cross-file mytoolchainconfig.ini build
appears to invoke and parse 'vswhere.exe', which fails. This feels like the kind of automatic, find/guess/hope and flail machinery that one usually wouldn't want to be involved when trying to set up a build system's toolchain and standard headers/libs as explicitly as possible.
So, is what I'm trying to do possible with meson?
If not, would it take much work (and does it seem a reasonable thing to have this goal in a build system) to be able to let the user simply explicitly specify all toolchain aspects of their build configuration?
I'm aware of the paradox of more and lengthier questions tending to elicit fewer answers so I apologies for the length of this; I don't have the skill to further condense it, while keeping some rationale. So the above are my main questions.
For those who still have the will power, I've added some secondary questions and thoughts below.
. . . .
Similar to what was mentioned here (see #4), I think a lot of the questions/confusion I have would be helped by having a more comprehensive example. Preferably one that uses a native build config file with a custom toolchain and sdk binaries, headers, and libs directory structure (not to mention that such a set-up is generally better for portable, reproducible, and stable build infrastructure). In fact, it seems that the 'Persistent native environments' pretty much describes what I'm trying to do -
"To build with a non-default native tool chain (such as clang instead of gcc)"
but then it doesn't really elaborate and isn't particularly helpful in illustrating how this is set up, referring to cross files and config-files section, but not clearly saying how a native file is constructed and how it's explicitly used.
E.g.
- There's a suggestion that native files might get implicitly used if they're found under certain directories ($XDG_DATA_DIRS), but it's not at all made clear. Does a native file only get used through these XDG_ path variables?
- Can native settings files be used explicitly through a 'meson setup ...' flag, like with '--cross-file'? If so, why not say what the option is? Perhaps the '--cross-file' option confusingly doubles up as also configuring a native, custom toolchain setup, but if that were the case, why not say this and why take pains to repeatedly distinguish native from cross building but name the command line option very specifically only one of these options?
- A toolchain (along with specifying the compiler and linker tool paths and traits) almost always requires specifying the locations for a set of system or platform sdk headers and libs, which is something I'd expect to be present in a cross / native config file, but there's documentation on a cross/native config file's [paths] section that says -
Deprecated in 0.56.0 use the built-in section instead.
As of 0.50.0 paths and directories such as libdir can be defined in the native and cross files in a paths section. These should be strings.
... How does one use the built-in section instead? Where is this section? Why don't you link to it? Are the same values in the deprecated example just useable exactly the same under a different section or are the 'libdir' and 'prefix' value also also deprecated? Also, to avoid confusion, does it really mean, "... use the [built-in options] section instead"? There's an awful lot of references to 'built-in' throughout the docs but I'm guessing only one [built-in options] section of the cross/native settings file we're talking about here. This could easily lead to unnecessary confusion. Apart from the deprecated example use of 'libdir' in the [paths] section, I see no sign of how to fully explicitly configure a toolchain's standard header and lib directories. How is this done?
- Similarly, at the top of project specific options it says "Path options are not allowed, those must be set in the [paths] section.", but you've just told me that [paths] section is deprecated. Then, at the bottom of the next section, we have -
"An incomplete list of options is: pkg_config_path cmake_prefix_path"
Why are the docs giving an incomplete list of options for the [built-in options] section of a cross/native config file? I don't think I've ever come across docs that leave it to the user to try to figure out what they can and can't do with a specific bit of functionality. Why not tell the user or provide a link to the full set of values that can be used in a cross/native config file under each section, right where the section is introduced?
Other unrelated thoughts/questions I've noted when reading the documentation are -
- When the 'Cross and native file reference' introduces the [paths] section, what is the purpose and meaning of the 'prefix' value? Shouldn't documentation explain each thing it illustrates or at least link to where those values are documented elsewhere?
- subdir('src'): This says to go off and execute the '/src/meson.build' file, but can subdir be above the CWD? I.e. subdir('../../pkg_deps/somethingorother')? If it can, then the name is arguably a little misleading (i.e. dir('src') would convey the same behaviour without possible confusion from SUBdir) and so would be useful to have this explicitly stated in the reference for this command. However, if subdir really does limit what further meson.build files you can go off and execute, then this also is something that would benefit from clarification.
- When I see documentation that refers to -
install_headers('foolib.h')
I wonder what does 'install' mean in the framework of a build system? I may have overlooked where this concept is properly introduced before seeing the term used without explanation. Is it really just a build step that copys files from one place to another or is there some other reason for using the term 'install'? I know other build systems allow you to perform arbitrary file copying steps, but these are always through a general 'copy'-like command, so it seems very intentional to use the term 'install', not 'copy', and to also have distinct uses for data, directories, and headers. Have I missed an introduction to this concept or is the documentation missing an overview to answer all these questions?
"To build with a non-default native tool chain (such as clang instead of gcc)"
but then it doesn't really elaborate and isn't particularly helpful in illustrating how this is set up, referring to cross files and config-files section, but not clearly saying how a native file is constructed and how it's explicitly used.
E.g.
- There's a suggestion that native files might get implicitly used if they're found under certain directories ($XDG_DATA_DIRS), but it's not at all made clear. Does a native file only get used through these XDG_ path variables?
- Can native settings files be used explicitly through a 'meson setup ...' flag, like with '--cross-file'? If so, why not say what the option is? Perhaps the '--cross-file' option confusingly doubles up as also configuring a native, custom toolchain setup, but if that were the case, why not say this and why take pains to repeatedly distinguish native from cross building but name the command line option very specifically only one of these options?
- A toolchain (along with specifying the compiler and linker tool paths and traits) almost always requires specifying the locations for a set of system or platform sdk headers and libs, which is something I'd expect to be present in a cross / native config file, but there's documentation on a cross/native config file's [paths] section that says -
Deprecated in 0.56.0 use the built-in section instead.
As of 0.50.0 paths and directories such as libdir can be defined in the native and cross files in a paths section. These should be strings.
... How does one use the built-in section instead? Where is this section? Why don't you link to it? Are the same values in the deprecated example just useable exactly the same under a different section or are the 'libdir' and 'prefix' value also also deprecated? Also, to avoid confusion, does it really mean, "... use the [built-in options] section instead"? There's an awful lot of references to 'built-in' throughout the docs but I'm guessing only one [built-in options] section of the cross/native settings file we're talking about here. This could easily lead to unnecessary confusion. Apart from the deprecated example use of 'libdir' in the [paths] section, I see no sign of how to fully explicitly configure a toolchain's standard header and lib directories. How is this done?
- Similarly, at the top of project specific options it says "Path options are not allowed, those must be set in the [paths] section.", but you've just told me that [paths] section is deprecated. Then, at the bottom of the next section, we have -
"An incomplete list of options is: pkg_config_path cmake_prefix_path"
Why are the docs giving an incomplete list of options for the [built-in options] section of a cross/native config file? I don't think I've ever come across docs that leave it to the user to try to figure out what they can and can't do with a specific bit of functionality. Why not tell the user or provide a link to the full set of values that can be used in a cross/native config file under each section, right where the section is introduced?
Other unrelated thoughts/questions I've noted when reading the documentation are -
- When the 'Cross and native file reference' introduces the [paths] section, what is the purpose and meaning of the 'prefix' value? Shouldn't documentation explain each thing it illustrates or at least link to where those values are documented elsewhere?
- subdir('src'): This says to go off and execute the '/src/meson.build' file, but can subdir be above the CWD? I.e. subdir('../../pkg_deps/somethingorother')? If it can, then the name is arguably a little misleading (i.e. dir('src') would convey the same behaviour without possible confusion from SUBdir) and so would be useful to have this explicitly stated in the reference for this command. However, if subdir really does limit what further meson.build files you can go off and execute, then this also is something that would benefit from clarification.
- When I see documentation that refers to -
install_headers('foolib.h')
I wonder what does 'install' mean in the framework of a build system? I may have overlooked where this concept is properly introduced before seeing the term used without explanation. Is it really just a build step that copys files from one place to another or is there some other reason for using the term 'install'? I know other build systems allow you to perform arbitrary file copying steps, but these are always through a general 'copy'-like command, so it seems very intentional to use the term 'install', not 'copy', and to also have distinct uses for data, directories, and headers. Have I missed an introduction to this concept or is the documentation missing an overview to answer all these questions?
Gerion Entrup
Aug 28, 2022, 9:17:02 PM8/28/22
to The Meson Build System, GertyP
Hi,
I'm not a Meson developer, just someone reading this list.
Am Samstag, 27. August 2022, 20:52:09 CEST schrieb GertyP:
> Hello.
>
> I'm new to meson but I want to share a little feedback as well as to ask
> for some help and clarification.
>
> I'm aware of how some build systems can have a whole load of hidden,
> sometimes complex machinery to try to automatically try, guess, or figure
> out what compiler/linkers and associated build infrastructure is available,
> what host/target settings to use, and where to find standard platform
> headers and libs. However, I'm really averse to that whole approach for
> many reasons (adds complexity, permutations, problems, difficulty finding
> or even fighting against hidden steps, reproduction and portability
> problems, etc.). I prefer build systems to be fully explicit and not try
> to go and guess and figure things out (and maybe get it wrong or conflict
> with a user's attempts to specify something else). There aren't too many
> things in a toolchain that it's impractial for people to explicitly
> configure. If a compiler, linker, system lib/include path ends up being
> needed but isn't specified, I don't think many people would be unable to
> simply explicitly point to what they want to use. I do know of many
> struggles to find out why the wrong automatically detected
> compilers/linkers/header-paths/lib-paths are being used or why, in some
> cases, explicit settings/paths are being ignored/overidden/etc. and then
> trying ugly hacks to work-around the increasingly complex search/find/guess
> automation in other build systems.
Please not. Automated educated guessing is _the_ core advance of Meson.
If you want a build system without (much) guessing, just use plain Make.
For example finding libLLVM is different for nearly every Linux Distribution
(not to speak of a Windows install). Meson can simply find that for you.
The alternative is to figure this out by yourself every time for every system
you want to support.
[...]
1. configure (check, that all dependencies are there and the compiler is ready).
In Meson via `meson setup`.
2. compilation (compile the actual libraries or binaries).
In Meson via `meson compile` (or often `ninja`).
3. installation (copy the compiled files to the platform specific location).
In Meson via `meson install` (or often `ninja install`).
The installation usually needs extended rights (root on Linux, no clue on Windows).
install_headers is a command for the third step. This is necessary since headers
are not needed for running the binary but usually necessary to build against a
(system) library.
Best,
Gerion
I'm not a Meson developer, just someone reading this list.
Am Samstag, 27. August 2022, 20:52:09 CEST schrieb GertyP:
> Hello.
>
> I'm new to meson but I want to share a little feedback as well as to ask
> for some help and clarification.
>
> I'm aware of how some build systems can have a whole load of hidden,
> sometimes complex machinery to try to automatically try, guess, or figure
> out what compiler/linkers and associated build infrastructure is available,
> what host/target settings to use, and where to find standard platform
> headers and libs. However, I'm really averse to that whole approach for
> many reasons (adds complexity, permutations, problems, difficulty finding
> or even fighting against hidden steps, reproduction and portability
> problems, etc.). I prefer build systems to be fully explicit and not try
> to go and guess and figure things out (and maybe get it wrong or conflict
> with a user's attempts to specify something else). There aren't too many
> things in a toolchain that it's impractial for people to explicitly
> configure. If a compiler, linker, system lib/include path ends up being
> needed but isn't specified, I don't think many people would be unable to
> simply explicitly point to what they want to use. I do know of many
> struggles to find out why the wrong automatically detected
> compilers/linkers/header-paths/lib-paths are being used or why, in some
> cases, explicit settings/paths are being ignored/overidden/etc. and then
> trying ugly hacks to work-around the increasingly complex search/find/guess
> automation in other build systems.
If you want a build system without (much) guessing, just use plain Make.
For example finding libLLVM is different for nearly every Linux Distribution
(not to speak of a Windows install). Meson can simply find that for you.
The alternative is to figure this out by yourself every time for every system
you want to support.
[...]
> - When I see documentation that refers to -
> *install_headers('foolib.h')*
> I wonder what does 'install' mean in the framework of a build system? I
> may have overlooked where this concept is properly introduced before seeing
> the term used without explanation. Is it really just a build step that
> copys files from one place to another or is there some other reason for
> using the term 'install'? I know other build systems allow you to perform
> arbitrary file copying steps, but these are always through a general
> 'copy'-like command, so it seems very intentional to use the term
> 'install', not 'copy', and to also have distinct uses for data,
> directories, and headers. Have I missed an introduction to this concept or
> is the documentation missing an overview to answer all these questions?
Nearly all build systems provide three phases:
> may have overlooked where this concept is properly introduced before seeing
> the term used without explanation. Is it really just a build step that
> copys files from one place to another or is there some other reason for
> using the term 'install'? I know other build systems allow you to perform
> arbitrary file copying steps, but these are always through a general
> 'copy'-like command, so it seems very intentional to use the term
> 'install', not 'copy', and to also have distinct uses for data,
> directories, and headers. Have I missed an introduction to this concept or
> is the documentation missing an overview to answer all these questions?
1. configure (check, that all dependencies are there and the compiler is ready).
In Meson via `meson setup`.
2. compilation (compile the actual libraries or binaries).
In Meson via `meson compile` (or often `ninja`).
3. installation (copy the compiled files to the platform specific location).
In Meson via `meson install` (or often `ninja install`).
The installation usually needs extended rights (root on Linux, no clue on Windows).
install_headers is a command for the third step. This is necessary since headers
are not needed for running the binary but usually necessary to build against a
(system) library.
Best,
Gerion
GertyP
Aug 29, 2022, 8:18:00 AM8/29/22
to The Meson Build System
Hi Gerion.
Thanks for taking the time to try to answer some questions.
> Please not. Automated educated guessing is _the_ core advance of Meson.
> If you want a build system without (much) guessing, just use plain Make.
Automated, educated guessing is often well-intentioned and useful for some but a huge pain in the neck for others, particularly when it guesses wrong and the extra guessing functionality ends up adding extra complexity that introduces problems that conflict or compete with the intentions of the user, who then has to apply hacks and work-arounds to fight the guessing machinery.
I don't claim that my preference for a 'fully explicit, simple, loose folders toolchain cross/native config file' approach should be to the exclusion of those who like a load of hidden automation, making choices that may conflict with the user's intentions - each to their own - but the idea that guessing automation does and should preclude explicit control of some build fundamentals doesn't sound sensible to me, so I hope you won't mind if I hold out hope for a clearer answer on how (if possible) to do this with meson.
> Nearly all build systems provide three phases
> ...
> install_headers is a command for the third step...
Thanks. Maybe what you describe, in my line of work, is just probably more often describe as a 'deploy' phase, which I mention is often done with copy-like commands, but is often all part of the same build script (and main build phase).
Also, you say copying (installing) headers would be something for the 3rd phase, after the main executables and binaries have been built: Is this just a rather niche, edge-case example? I can't imagine many people needing to copy/deploy a load of header files alongside an application they've just compiled and linked. Is there a more complete example of 'install_headers' to better convey and document it's purpose and rationale?
Thanks again.
Gerty
Thanks for taking the time to try to answer some questions.
> Please not. Automated educated guessing is _the_ core advance of Meson.
> If you want a build system without (much) guessing, just use plain Make.
Consider the scenario where you have 2 or 3 different visual studio versions installed, as well as a loose folder containing an experimental compiler toolchain you want to test out. Are you really saying those users who simply want to select one of the toolchains, then build, profile, and test, and then select another compiler and build, profile, and test should just give up on the prospect of doing something so fundamental with meson? It would be rather extreme to say that the addition of a convenience feature for some should exclude the access to some fundamental basic functionality to others. Furthermore, it sounds to me as though you're simply advocating a way you prefer to configure builds and possibly suggesting those who do it any other way are misguided... But then what is the purpose of meson's cross-compile and native setting files? The fact meson appears to support them, allowing users to more explicitly configure their build environment (which I can't imagine being more than around 10 lines in a config file of settings and pointing to toolchain binaries; something that should be pretty straightforward for most devs) would suggest this use case is a fairly reasonable, supported use case, even if it's distasteful to you.
> Nearly all build systems provide three phases
> install_headers is a command for the third step...
Thanks. Maybe what you describe, in my line of work, is just probably more often describe as a 'deploy' phase, which I mention is often done with copy-like commands, but is often all part of the same build script (and main build phase).
So perhaps copying is synonymous with meson's 'install' ... except, confusingly, there are already additional references to 'copy'-like commands in the meson reference, so I'm no clearer on the need for a distinction between copying and installing. But then, from what you've said, it's not clear why the copying work in the 3rd install phase would need to be separate from the main compile/link/build phase; if it can all be rolled up into one build operation (compile -> link -> copy), why would someone separate out a final manual copying step (meson install ...)? Is the ONLY reason to separate it out into an extra phase so that it can copying can be done under different user credentials/permissions?
Thanks again.
Gerty
Eli Schwartz
Aug 29, 2022, 10:25:42 AM8/29/22
to GertyP, The Meson Build System
On 8/27/22 2:52 PM, GertyP wrote:
> Hello.
>
> I'm new to meson but I want to share a little feedback as well as to ask
> for some help and clarification.
>
> I'm aware of how some build systems can have a whole load of hidden,
> sometimes complex machinery to try to automatically try, guess, or figure
> out what compiler/linkers and associated build infrastructure is available,
> what host/target settings to use, and where to find standard platform
> headers and libs. However, I'm really averse to that whole approach for
> many reasons (adds complexity, permutations, problems, difficulty finding
> or even fighting against hidden steps, reproduction and portability
> problems, etc.). I prefer build systems to be fully explicit and not try
> to go and guess and figure things out (and maybe get it wrong or conflict
> with a user's attempts to specify something else). There aren't too many
> things in a toolchain that it's impractial for people to explicitly
> configure. If a compiler, linker, system lib/include path ends up being
> needed but isn't specified, I don't think many people would be unable to
> simply explicitly point to what they want to use. I do know of many
> struggles to find out why the wrong automatically detected
> compilers/linkers/header-paths/lib-paths are being used or why, in some
> cases, explicit settings/paths are being ignored/overidden/etc. and then
> trying ugly hacks to work-around the increasingly complex search/find/guess
> automation in other build systems.
>
> For these reason, I think it's undesireable and unfortunate if the
> following output from '>*meson setup ...*' really is a requirement -
> Hello.
>
> I'm new to meson but I want to share a little feedback as well as to ask
> for some help and clarification.
>
> I'm aware of how some build systems can have a whole load of hidden,
> sometimes complex machinery to try to automatically try, guess, or figure
> out what compiler/linkers and associated build infrastructure is available,
> what host/target settings to use, and where to find standard platform
> headers and libs. However, I'm really averse to that whole approach for
> many reasons (adds complexity, permutations, problems, difficulty finding
> or even fighting against hidden steps, reproduction and portability
> problems, etc.). I prefer build systems to be fully explicit and not try
> to go and guess and figure things out (and maybe get it wrong or conflict
> with a user's attempts to specify something else). There aren't too many
> things in a toolchain that it's impractial for people to explicitly
> configure. If a compiler, linker, system lib/include path ends up being
> needed but isn't specified, I don't think many people would be unable to
> simply explicitly point to what they want to use. I do know of many
> struggles to find out why the wrong automatically detected
> compilers/linkers/header-paths/lib-paths are being used or why, in some
> cases, explicit settings/paths are being ignored/overidden/etc. and then
> trying ugly hacks to work-around the increasingly complex search/find/guess
> automation in other build systems.
>
> For these reason, I think it's undesireable and unfortunate if the
> * "Visual Studio environment is needed to run Ninja. ..."*
> The power of a clean and portable build system would be one in which you
> can just unzip an entire directory structure of source, build system,
> toolchain, and platform sdk headers and libs, etc. and then build. So
> forcing users to have to run installers (with complications from multiple
> versions) would seem to me to unnecessarily devalue, and in some cases
> complicate, a build system. It clearly is possible to compile, link, and
> build without requiring a 'visual studio environment', by invoking the
> individual compiler and linker commands, pointing to and pathing everything
> needed into a custom directory structure of compiler/linker/binaries,
> headers, and libs. I'm hoping that the above meson output is something
> I've misunderstood or is not accurate.
It's possible on Linux, to be sure. I do it often enough. Native/cross
> can just unzip an entire directory structure of source, build system,
> toolchain, and platform sdk headers and libs, etc. and then build. So
> forcing users to have to run installers (with complications from multiple
> versions) would seem to me to unnecessarily devalue, and in some cases
> complicate, a build system. It clearly is possible to compile, link, and
> build without requiring a 'visual studio environment', by invoking the
> individual compiler and linker commands, pointing to and pathing everything
> needed into a custom directory structure of compiler/linker/binaries,
> headers, and libs. I'm hoping that the above meson output is something
> I've misunderstood or is not accurate.
files are great.
Note there is a distinction between autodetection of:
- the compiler/linker paths (which are intended to be easily overridden
and manually specified via machine file specifications, but if none
are specified, will do a search to find one)
- the name/version of the selected compiler binaries, which is crucial
to determine stuff like what flags are valid, which argument passing
convention to use (MSVC style /flag or Unix style --flag, whether the
compiler supports asneeded/lto/allow-undefined/etc.)
The "following output" you refer to is an informational log message, by
the way. Meson doesn't enforce it. The reason it mentions it at all, is
because my understanding is that Microsoft, in their grace, has designed
MSVC in such a way that you "need" to run vcvars*.bat before cl.exe /
link.exe will actually work as intended.
Meson has internal functionality to run vswhere.exe in order to try to
find a Visual Studio development environment, and activate it. This will
- check if you're already in such an environment (cl.exe is in %PATH%,
or VSINSTALLDIR is set), and if so, do nothing.
If you're not in one, it will:
- check if the --vsenv option was passed to Meson, and only try to set
up a VS environment
- check on %PATH% for some known compilers (gcc, clang, clang-cl), and
if it finds one, it assumes that you don't want to use VS
- run vswhere to find a VS for you, then run that VS's vcvars*.bat and
print out its variables, parse those variables into python, and set
them in Meson's environment
After that is all done with, it returns boolean "did we end up doing
work to set up a VS environment, or was it already there".
- if it tries, and fails, simply log a warning and continue setting up
the project
- if the build was configured with `meson setup --vsenv`, and that
failed, make the error fatal
There are no consequences to either finding or not finding or not
checking for, a VS environment. No matter what happens, Meson continues
on, and tries to use your machine file specified compiler/linker.
But, if you didn't specify the absolute path to a compiler/linker, then
successfully setting up a VS environment *might* be the only reason
Meson can later autodetect one.
Again, if you already had one set up, or GCC/clang was found first, then
Meson simply doesn't bother warning you any such thing, because it
assumes things will "just work".
Either way, you can run ninja directly. If you have gone and set up your
own VS environment, that will work fine. If the compiler/linker you
defined in your machine files works correctly without a VS environment,
then that, too, will work fine.
> >*meson setup --cross-file mytoolchainconfig.ini build*
> appears to invoke and parse 'vswhere.exe', which fails. This feels like
> the kind of automatic, find/guess/hope and flail machinery that one usually
> wouldn't want to be involved when trying to set up a build system's
> toolchain and standard headers/libs as explicitly as possible.
The vswhere failure should log a warning:
> the kind of automatic, find/guess/hope and flail machinery that one usually
> wouldn't want to be involved when trying to set up a build system's
> toolchain and standard headers/libs as explicitly as possible.
```
WARNING: Failed to activate VS environment: <reason>
```
and Meson should continue on to the compiler/linker autodetection -- in
which, again, the one you specified in the cross file shall always win.
> I'm aware of the paradox of more and lengthier questions tending to elicit
> fewer answers so I apologies for the length of this; I don't have the skill
> to further condense it, while keeping some rationale. So the above are my
> main questions.
> For those who still have the will power, I've added some secondary
> questions and thoughts below.
>
> . . . .
>
> Similar to what was mentioned here
> lot of the questions/confusion I have would be helped by having a more
> comprehensive example. Preferably one that uses a native build config file
> with a custom toolchain and sdk binaries, headers, and libs directory
> structure (not to mention that such a set-up is generally better for
> portable, reproducible, and stable build infrastructure). In fact, it
> seems that the 'Persistent native environments
> <https://mesonbuild.com/Native-environments.html>' pretty much describes
> comprehensive example. Preferably one that uses a native build config file
> with a custom toolchain and sdk binaries, headers, and libs directory
> structure (not to mention that such a set-up is generally better for
> portable, reproducible, and stable build infrastructure). In fact, it
> seems that the 'Persistent native environments
> what I'm trying to do -
> *"To build with a non-default native tool chain (such as clang instead
> of gcc)"*
> but then it doesn't really elaborate and isn't particularly helpful in
> illustrating how this is set up, referring to cross files and config-files
> section, but not clearly saying how a native file is constructed and how
> it's explicitly used.
I do indeed think we could benefit from giving a complete example. The
> illustrating how this is set up, referring to cross files and config-files
> section, but not clearly saying how a native file is constructed and how
> it's explicitly used.
underlying information is there, but piecing together the different
sections could probably be made easier...
> E.g.
> - There's a suggestion that native files might get implicitly used if
> all made clear. Does a native file only get used through these XDG_ path
> variables?
```
> variables?
The order of locations tried is as follows:
- A file relative to the local dir
- The user local location
- The system wide locations in order
```
Note that XDG standards aren't relevant to Windows, so they won't be
used there at all.
These directories are (on Unix) *always* used, as the lookup method for
any --native-file or --cross-file, but only if:
- the specified file is *not* found as a relative path
- the specified file is *not* found as an absolute path
So, you can do
```
meson setup --cross-file foo.ini
```
And, if there's a foo.ini in your current directory, it will be used. If
there isn't one in the current directory then Meson will check in
/usr/share/meson/cross/foo.ini, and finally in
~/.local/share/meson/cross/foo.ini
You can force it by specifying the actual full path, but it's quite
possible that people reuse certain cross files across many projects, so
having one in a user configuration directory and specifying the "short
name" can save on typing.
> - Can native settings files be used explicitly through a '*meson setup ...*'
> flag, like with '*--cross-file*'? If so, why not say what the option is?
> Perhaps the '*--cross-file*' option confusingly doubles up as also
> configuring a native, custom toolchain setup, but if that were the case,
> why not say this and why take pains to repeatedly distinguish native from
> cross building but name the command line option very specifically only one
> of these options?
That seems like a documentation oversight. Just like the cross file page
> why not say this and why take pains to repeatedly distinguish native from
> cross building but name the command line option very specifically only one
> of these options?
mentions the command line argument to use, the native file page should
as well... we should fix that.
That being said, it is listed in the `meson setup --help` output.
> - A toolchain (along with specifying the compiler and linker tool paths and
> traits) almost always requires specifying the locations for a set of system
> or platform sdk headers and libs, which is something I'd expect to be
> present in a cross / native config file, but there's documentation on a
> <https://mesonbuild.com/Machine-files.html#paths-and-directories> that says
> -
>
> * Deprecated in 0.56.0 use the built-in section instead. As of 0.50.0
> paths and directories such as libdir can be defined in the native and cross
> files in a paths section. These should be strings.*
> ... How does one use the built-in section instead? Where is this section?
> Why don't you link to it? Are the same values in the deprecated example
> just useable exactly the same under a different section or are the 'libdir'
> and 'prefix' value also also deprecated? Also, to avoid confusion, does it
> really mean, "... use the *[built-in options]* section instead"?
> Why don't you link to it? Are the same values in the deprecated example
> just useable exactly the same under a different section or are the 'libdir'
> and 'prefix' value also also deprecated? Also, to avoid confusion, does it
Yes, that is indeed the section it means.
Please note that this does not specify the location for a set of
platform sdk headers and libs -- neither the [built-in options] nor the
deprecated [paths].
They are equivalent to the -Dprefix, -Dlibdir, -Dincludedir command line
options, and they specify installation locations for
get_option('libdir'), the default install_headers() location, etc.
> There's
> an awful lot of references to 'built-in' throughout the docs but I'm
> file we're talking about here. This could easily lead to unnecessary
> confusion. Apart from the deprecated example use of 'libdir' in the
> *[paths]* section, I see no sign of how to fully explicitly configure a
> confusion. Apart from the deprecated example use of 'libdir' in the
> toolchain's standard header and lib directories. How is this done?
The [built-in options] section directly maps to arguments that can be
passed as -D<something> to `meson setup`.
In this case, -Dc_args='-I/path/to/include -I/path/to/other/include'
should do the trick, or in a cross file:
```
[built-in options]
c_args = ['-I/path/to/include', '-I/path/to/other/include']
c_link_args = ['-L/path/to/libs', '-L/path/to/other/libs']
```
> - Similarly, at the top of project specific options
> says "*Path options are not allowed, those must be set in the [paths]
> section.*", but you've just told me that *[paths]* section is deprecated.
Yeah that's just wrong, sorry.
> Then, at the bottom of the next section, we have -
> Why are the docs giving an incomplete list of options for the *[built-in
> options] *section of a cross/native config file? I don't think I've ever
> come across docs that leave it to the user to try to figure out what they
> can and can't do with a specific bit of functionality. Why not tell the
> user or provide a link to the full set of values that can be used in a
> cross/native config file under each section, right where the section is
> introduced?
This "incomplete list" is of options that can have different cross and
> can and can't do with a specific bit of functionality. Why not tell the
> user or provide a link to the full set of values that can be used in a
> cross/native config file under each section, right where the section is
> introduced?
native values (other than compiler options, which always can). The
canonical reference is in python code as `BUILTIN_OPTIONS_PER_MACHINE`
and currently it is just those two. I wonder if it's possible to write a
CI test that compares the two to make sure it always stays in sync...
> Other unrelated thoughts/questions I've noted when reading the
> documentation are -
> section, what is the purpose and meaning of the 'prefix' value? Shouldn't
> documentation explain each thing it illustrates or at least link to where
> those values are documented elsewhere?
It could be clarified to point out that they are installation paths.
> documentation explain each thing it illustrates or at least link to where
> those values are documented elsewhere?
> - *subdir('src')*: This says to go off and execute the '/src/meson.build'
> file, but can subdir be above the CWD? I.e.
> *subdir('../../pkg_deps/somethingorother')*? If it can, then the name is
> arguably a little misleading (i.e. dir('src') would convey the same
> behaviour without possible confusion from *SUB*dir) and so would be useful
> to have this explicitly stated in the reference for this command. However,
> if subdir really does limit what further meson.build files you can go off
> and execute, then this also is something that would benefit from
> clarification.
The documentation author probably assumed it was implied by the name.
> if subdir really does limit what further meson.build files you can go off
> and execute, then this also is something that would benefit from
> clarification.
At runtime if you do try it, it will error out with:
ERROR: Subdir contains ..
This is actually not as clear as it could be, because e.g. cmake also
calls it "subdirectory" but allows using it on parent or sibling
directories. I think it's pretty reasonable for Meson to clarify that.
> - When I see documentation that refers to -
> I wonder what does 'install' mean in the framework of a build system? I
> may have overlooked where this concept is properly introduced before seeing
> the term used without explanation. Is it really just a build step that
> copys files from one place to another or is there some other reason for
> using the term 'install'? I know other build systems allow you to perform
> arbitrary file copying steps, but these are always through a general
> 'copy'-like command, so it seems very intentional to use the term
> 'install', not 'copy', and to also have distinct uses for data,
> directories, and headers. Have I missed an introduction to this concept or
> is the documentation missing an overview to answer all these questions?
It's actually fairly common with build systems, particularly those that
> may have overlooked where this concept is properly introduced before seeing
> the term used without explanation. Is it really just a build step that
> copys files from one place to another or is there some other reason for
> using the term 'install'? I know other build systems allow you to perform
> arbitrary file copying steps, but these are always through a general
> 'copy'-like command, so it seems very intentional to use the term
> 'install', not 'copy', and to also have distinct uses for data,
> directories, and headers. Have I missed an introduction to this concept or
> is the documentation missing an overview to answer all these questions?
are compatible with Unix systems.
Probably the best description of these would be
https://www.gnu.org/prep/standards/html_node/Directory-Variables.html
A general "copy" command wouldn't be nearly as useful, because dedicated
installation keywords have special properties:
- You may want to build multiple times for local testing, and only
deploy/install once, so whatever you call it, it cannot be done by
"build all targets"
- on Unix, there are designated destination locations for each "type" of
file, e.g. "executables" vs. "libraries" vs. "public headers for
libraries" vs. "runtime data files" vs. "documentation files"
- Unix designated destination locations are often owned by the system,
not by a limited user account, and transferring those files needs to
be done after obtaining admin credentials
- even on Windows, if you're deploying a library with something like
vcpkg or conan, you're going to want to structure that in some
standard layout such as <projectname>/lib/ and <projectname>/include/
- you can tag each type of file and ask to "install headers and
libraries, but not documentation" or various other combinations
That being said, not everyone uses the installation functions. For
example, Meson's own source code has a docs/ directory with a default
build target that generates the website HTML documentation. Nothing is
installed, so running `meson install` is meaningless there. But there is
a `ninja upload` deploy target defined as `run_target('upload', ...)`
which will git push the currently built version of the docs to
https://github.com/mesonbuild/mesonbuild.github.io
--
Eli Schwartz
GertyP
Aug 29, 2022, 11:51:26 AM8/29/22
to The Meson Build System
Thanks very much Eli.
This is all a huge help (and hopefully something that can help fill out a few areas of the existing docs for meson newbies like me).
I also had to double-take at your example use of the '--native-file' meson flag because I honestly wasn't aware it was even a thing; in all the introductory guides/tutorials/notes on the subject of cross-compiling and native compiling, I'd never seen it mentioned or used (I only ever saw mention or use of '--cross-file') and '--native-file', to the best of my knowledge, can only be found hidden away in comments in various release notes.
It may take a while to digest and experiment with what what I learn from your answers and clarifications but instead of waiting, I thought I should fire of a quick thanks for going to the trouble of providing answers.
Cheers,
Gerty
Eli Schwartz
Aug 29, 2022, 12:11:51 PM8/29/22
to GertyP, The Meson Build System
On 8/29/22 11:51 AM, GertyP wrote:
> Thanks very much Eli.
> This is all a huge help (and hopefully something that can help fill out a
> few areas of the existing docs for meson newbies like me).
>
> I also had to double-take at your example use of the '--native-file' meson
> flag because I honestly wasn't aware it was even a thing; in all the
> introductory guides/tutorials/notes on the subject of cross-compiling and
> native compiling, I'd never seen it mentioned or used (I only ever saw
> mention or use of '--cross-file') and '--native-file', to the best of my
> knowledge, can only be found hidden away in comments in various release
> notes.
Yeah, while writing my previous response I did a git grep for mentions
> Thanks very much Eli.
> This is all a huge help (and hopefully something that can help fill out a
> few areas of the existing docs for meson newbies like me).
>
> I also had to double-take at your example use of the '--native-file' meson
> flag because I honestly wasn't aware it was even a thing; in all the
> introductory guides/tutorials/notes on the subject of cross-compiling and
> native compiling, I'd never seen it mentioned or used (I only ever saw
> mention or use of '--cross-file') and '--native-file', to the best of my
> knowledge, can only be found hidden away in comments in various release
> notes.
of it.
It's only present in 4 release note snippets, and an even more obscure
statement in the contributing-to-Meson docs that "nativefile.ini in a
unittest will be passed to the unittest configure step with --native-file".
I was pretty shocked.
BTW: I just pushed a couple of quick doc fixups:
https://github.com/mesonbuild/meson/compare/d4733984e...4f05f877f
The website should be updated momentarily.
> It may take a while to digest and experiment with what what I learn from
> your answers and clarifications but instead of waiting, I thought I should
> fire of a quick thanks for going to the trouble of providing answers.
>
> Cheers,
> Gerty
--
Eli Schwartz
Reply all
Reply to author
Forward
0 new messages