Porting Meson to Nim?

[lucatrv]

I wonder if anybody has ever thought about porting Meson to Nim. IMHO the big advantages for Meson would be:

• compilation to standalone native binary
• better portability (no Python dependency)
• much better performance (see for instance benchmarks)
• syntax very similar to Python

Thanks

[Jussi Pakkanen]

On Sat, Jun 23, 2018 at 1:21 AM, lucatrv <luc...@gmail.com> wrote:

> I wonder if anybody has ever thought about porting Meson to Nim. IMHO the
> big advantages for Meson would be:

The reasons why this can't really be done are outlined in the FAQ:

http://mesonbuild.com/FAQ.html#why-is-meson-implemented-in-python-rather-than-programming-language-x

[lucatrv]

On Saturday, June 23, 2018 at 1:46:17 AM UTC+2, Jussi Pakkanen wrote:

The reasons why this can't really be done are outlined in the FAQ:

http://mesonbuild.com/FAQ.html#why-is-meson-implemented-in-python-rather-than-programming-language-x

I read the FAQ, but IMHO Nim should be added to the list of languages because:

• Nim code compiles to C (besides C++, Objective-C and JavaScript), and can therefore be built anywhere a C compiler is available:
• https://nim-lang.org/features.html
  
• https://nim-lang.org/docs/backends.html
  
• https://github.com/nim-lang/Nim/blob/devel/lib/system/platforms.nim
  
  
• Bootstrapping the Nim compiler is straightforward, starting from the pre-generated C sources:
• https://github.com/nim-lang/csources
  
  
• Instructions for using Python in VS Code look to me very similar to those for Nim:
• https://code.visualstudio.com/docs/python/python-tutorial
  
• https://marketplace.visualstudio.com/items?itemName=kosz78.nim

[Jussi Pakkanen]

On Sat, Jun 23, 2018 at 3:06 PM, lucatrv <luc...@gmail.com> wrote:

> I read the FAQ, but IMHO Nim should be added to the list of languages
> because:
>
> Nim code compiles to C (besides C++, Objective-C and JavaScript), and can
> therefore be built anywhere a C compiler is available:

The point is not whether it can be built. The point is that the
compiler _needs_ to be available. This is an extra dependency. As an
example we have had several people complain about the use of Python
because it is "too big of a dependency" even though _absolutely
everything_ uses Python.

> Bootstrapping the Nim compiler is straightforward, starting from the
> pre-generated C sources:

Again, it does not matter how easy the bootstrapping is. The point is
that it needs to be done, which adds a dependency to a full compiler
toolchain. This by itself is a blocker for some.

> Instructions for using Python in VS Code look to me very similar to those
> for Nim:

The original post did not mention VS Code at all, only "real" Visual
Studio. That is (for better or for worse) what most Windows developers
use. Not supporting it natively out of the box is a major hurdle.

Using languages like Nim for apps and possibly even shared libraries
is great and you should totally do it. However build systems that need
to work at the very lowest levels of the software stack have special,
weird and unexpected requirements that make them a poor match for
non-mainstream programming languages.

[lucatrv]

On Sunday, June 24, 2018 at 8:39:59 PM UTC+2, Jussi Pakkanen wrote:

 

The point is not whether it can be built. The point is that the
compiler _needs_ to be available. This is an extra dependency.

 

Again, it does not matter how easy the bootstrapping is. The point is

that it needs to be done, which adds a dependency to a full compiler
toolchain. This by itself is a blocker for some.

 

If depending on a full compiler toolchain is an issue, then also C and C++ should be removed from the list of allowable languages in the FAQ.

 

The original post did not mention VS Code at all, only "real" Visual
Studio. That is (for better or for worse) what most Windows developers
use. Not supporting it natively out of the box is a major hurdle.

 

I see, so if the requirement is to ease Meson development on Windows with Visual Studio out of the box, then I agree that Python should be preferred to Nim. The drawback however is to require Python everywhere Meson is actually used as a build system. With Nim you would get a standalone executable with no dependencies.

[mikkelfj]

Well,

I wasted to days on trying to get specific meson version running on a Travis build.

But the python versioning system is totally <censored>

and end the end it was not possible because Travis ran Linux v. x and it only ran Python v. y and below

while meson v. z only ran with something else.

In order to get the Python thing going, more knowledge of the 17 ways Python cannot be installed needs to be known

and this includes at least unofficial and no longer supported hacks.

A C compiler would have worked.

[lucatrv]

I add that in my opinion more importance should be given to the fact that Meson could run faster, and could be better portable, than warring about inexpert Windows developers that are only able to work on Visual Studio. BTW I wonder how many commits Meson has received so far from those developers. This is just my personal opinion of course. I can see other reasons to prefer Python over Nim, for instance the user base, but this is not mentioned in the FAQ.

[Jussi Pakkanen]

On Tue, Jun 26, 2018 at 11:29 PM, lucatrv <luc...@gmail.com> wrote:

> I add that in my opinion more importance should be given to the fact that
> Meson could run faster

The biggest cause of slowness we currently have is invoking compiler
tests and other such calls to external programs. That can't really be
made faster by changing out programming language because the slowness
comes from the outside.

Python is surprisingly fast for some use cases. Meson is mostly about
string manipulation and putting stuff in hash tables. Those are
exactly those things that Python handles well. This probably can not
be scaled to Google level repositories but for now we don't need to
and can consider the rewrite once people start demanding it.

> and could be better portable

We have people using Meson on Solaris and even AIX (no reports on
HP-UX yet). Those are not listed on Nim's platform page so I'm
guessing they are not supported.

Portability is even more about the libraries in use than just the
plain language itself. Python's standard library is awesome and we use
a fair chunk of it for stuff like https requests, tar and zip file
processing and the like. Adding equivalent functionality in other
languages almost always means hunting for tens of dependency packages
and keeping them all up to date.

If you can stick to Python and its standard library (which we do), the
result is _amazingly_ portable. There is almost nothing like it
without spending a ton of effort yourself.

[mikkelfj]

At least I have a name for meson in c: mesonic

[mikkelfj]

And for nim: mesonim

[Tobias Hunger]

On Wed, Jun 27, 2018 at 8:54 AM mikkelfj <mik...@dvide.com> wrote:
> And for nim: mesonim

Great, we have a name, so the hard part is done! :-)

Note that many real-world meson build systems include python scripts,
so those can not be built by mesonic or mesonim -- at least not
without also having a python interpreter available at the machine
running the ported meson. At that point you could just use
python-meson.

Best Regards,
Tobias

[mikkelfj]

Note that many real-world meson build systems include python scripts,
so those can not be built by mesonic or mesonim -- at least not
without also having a python interpreter available at the machine
running the ported meson. At that point you could just use
python-meson.

 

No, because build scripts might run on any Python version whereas Meson is rather version specific and changes frequently.

[Tobias Hunger]

On Wed, Jun 27, 2018 at 12:02 PM mikkelfj <mik...@dvide.com> wrote:
> No, because build scripts might run on any Python version whereas Meson is rather version specific and changes frequently.

The key word here is *might*. I doubt many test with any but the
python interpreter used by meson.

Plus it is probably easier to make meson less picky about the python
interpreter than it is to port the whole thing to a new language.

Best Regards,
Tobias

[lucatrv]

On Tuesday, June 26, 2018 at 11:45:39 PM UTC+2, Jussi Pakkanen wrote:

We have people using Meson on Solaris and even AIX (no reports on
HP-UX yet). Those are not listed on Nim's platform page so I'm
guessing they are not supported.

Portability is even more about the libraries in use than just the
plain language itself. Python's standard library is awesome and we use
a fair chunk of it for stuff like https requests, tar and zip file
processing and the like. Adding equivalent functionality in other
languages almost always means hunting for tens of dependency packages
and keeping them all up to date.

If you can stick to Python and its standard library (which we do), the
result is _amazingly_ portable. There is almost nothing like it
without spending a ton of effort yourself.

 

Ok, I see your point. Just to add food for thought:

• Both Solaris and AIX are listed in platforms.nim
• Support for internet protocols is included in the Nim Standard Library
• Modules are available to access tar and zip files, whose only dependency is zlib (available on most platforms)

I acknowledge however that relying on Python alone may simplify things (at the expense of performance). So if this is an important decision factor I suggest to report it in the FAQ.

[lucatrv]

On Wednesday, June 27, 2018 at 10:56:17 AM UTC+2, Tobias Hunger wrote:

Note that many real-world meson build systems include python scripts,
so those can not be built by mesonic or mesonim -- at least not
without also having a python interpreter available at the machine
running the ported meson. At that point you could just use
python-meson.

Yes, unless they are written in NimScript or Nim itself.

[Andre Renaud]

Would using an embedded python interpreter, such as micropython be a suitable middle ground? It would allow meson to be fully standalone, whilst still retaining the ability to run python-style build scripts.

The issue would be if Meson is using large amounts of the standard library that micropython doesn't support, which is entirely likely.

I use Meson, but have not looked at it's internal code in depth, so take my question here with a suitable grain of salt.

Regards,

Andre 

[mikkelfj]

Why not add a Lua interpreter if portable scripting is an objective?

Not stating that meson script itself should be programmable.

[Jussi Pakkanen]

On Thu, Jun 28, 2018 at 12:03 AM, Andre Renaud <an...@ignavus.net> wrote:

> Would using an embedded python interpreter, such as micropython be a
> suitable middle ground? It would allow meson to be fully standalone, whilst
> still retaining the ability to run python-style build scripts.
>
> The issue would be if Meson is using large amounts of the standard library
> that micropython doesn't support, which is entirely likely.
>
> I use Meson, but have not looked at it's internal code in depth, so take my
> question here with a suitable grain of salt.

We use _a lot_ of Python's standard library so Micropython might not
work. I don't think anyone has tried that yet. If someone does, let us
know how it went.

Related to this, the MSI installers we ship have an embedded Python
installation so they look like regular Windows executables, but have
no dependency on Python and can even run Python scriptst transparently
with the embedded Python interpreter.

[Jussi Pakkanen]

On Fri, Jun 29, 2018 at 6:09 PM, mikkelfj <mik...@dvide.com> wrote:

> Why not add a Lua interpreter if portable scripting is an objective?
> Not stating that meson script itself should be programmable.

Lua is not a good fit for us because its standard library is
nonexisting compared to Python. We would have to reimplement almost
everything needed has from scratch.

The language is also not as good as Python (personal opinion).
Changing to Lua would mean a ton of work for no benefit and in fact
would make several things worse.

[lucatrv]

Maybe I should have also mentioned that Nim and Python can easily interact both ways using the nimpy module.

[mikkelfj]

I was referring to scripting as an alternative to user shell scripts in the build using a Lua filesystem module, not as an implementation language for Meson.

This would ensure builds scripts would independent of the meson implementation.

[ardi]

On Sunday, June 24, 2018 at 8:39:59 PM UTC+2, Jussi Pakkanen wrote:

>

> The point is not whether it can be built. The point is that the 

> compiler _needs_ to be available. This is an extra dependency. As an 

> example we have had several people complain about the use of Python 

> because it is "too big of a dependency" even though _absolutely 

> everything_ uses Python.

That's the only thing I dislike about Meson: it aims to be a system suitable for building critical OS tools in early stages of the OS build process (that's what the FAQ expresses as a reason for requiring "minimal dependencies"). But then... one of such "minimal dependencies" is Python. Like if when you get your first kernel booting, you were supposed to have a kernel, a C compiler, a shell, vi, and... Python. No, sorry, but no: I'll have Python, but later in the process. And you could argue that well, if you don't like building Python so early in the process, you can use other tool instead of Meson... but then the Korn shell (https://github.com/att/ast) moves to Meson and you get having to build Python before ksh. At the end, it all ends up like "don't bother about your kernel, nor the compiler, nor vi, nor the shell, nor nothing... just use Python, it's the only thing that matters". Cool if Python is your system, but no so cool when Python is just a tool for you and not your system.

Yes, I'm disappointed.

ardi

[Jussi Pakkanen]

On Thu, Jul 18, 2019 at 11:37 AM ardi <ardillas...@gmail.com> wrote:

> That's the only thing I dislike about Meson: it aims to be a system suitable for building critical OS tools in early stages of the OS build process (that's what the FAQ expresses as a reason for requiring "minimal dependencies"). But then... one of such "minimal dependencies" is Python. Like if when you get your first kernel booting, you were supposed to have a kernel, a C compiler, a shell, vi, and... Python. No, sorry, but no: I'll have Python, but later in the process.

You are forgetting Perl, which is (currently at least) required on a
minimal system and is roughly equivalent in size and scope to Python.
But even then, unfortunately, there are very few people doing build
system development. There are fewer that even care about system
bootstrapping leading to things like choosing Java as the
implementation language. Going further there are zero people in the
entire world who are willing to do all that and implement a new build
system with nothing but Perl and C [1].

The unfortunate fact of the matter is that the core bootstrap
dependency set has gotten bigger and now contains Python. This is
called progress. Sometimes progress leads to changes you personally do
not like. This can't really be helped, you just have to deal with it.
Sorry.

[1] Though I have heard about some people planning to write a minimal
Meson implementation in C just to get around this bootstrapping issue.
Which is cool.

[ardi]

On Thu, Jul 18, 2019 at 9:15 PM Jussi Pakkanen <jpak...@gmail.com> wrote:
> [...]

> [1] Though I have heard about some people planning to write a minimal
> Meson implementation in C just to get around this bootstrapping issue.
> Which is cool.

Very cool indeed! Are they in planning stage only, or is it there any
repository already?

Thanks!

ardi

[Jussi Pakkanen]

On Thu, Jul 18, 2019 at 10:58 PM ardi <ardillas...@gmail.com> wrote:

> Very cool indeed! Are they in planning stage only, or is it there any
> repository already?

Sadly I have only heard rumors about this, nothing concrete.

[Kurtis Rader]

On Thu, Jul 18, 2019 at 1:37 AM ardi <ardillas...@gmail.com> wrote:

That's the only thing I dislike about Meson: it aims to be a system suitable for building critical OS tools in early stages of the OS build process (that's what the FAQ expresses as a reason for requiring "minimal dependencies"). But then... one of such "minimal dependencies" is Python. Like if when you get your first kernel booting, you were supposed to have a kernel, a C compiler, a shell, vi, and... Python. No, sorry, but no: I'll have Python, but later in the process. And you could argue that well, if you don't like building Python so early in the process, you can use other tool instead of Meson... but then the Korn shell (https://github.com/att/ast) moves to Meson and you get having to build Python before ksh.

I happen to be one of the people trying to keep ksh alive and did most of the work to replace the legacy Nmake build tool with Meson. We also considered CMake. We rejected staying with the legacy Nmake because it was unacceptably slow (the configure phase was ~20 times slower than using Meson), it had no documentation, and there is no one outside of the old AT&T AST team who knew how it worked. Even if we had chosen CMake there would still be lots of other dependencies. For example, the project used to run this humungous /bin/sh script during the configure phase:

https://github.com/att/ast/blob/2016-01-10-beta/src/lib/libast/comp/conf.sh

That script depends on quite a few external commands such as getconf, cat, egrep, sed, tr, etc. You can certainly argue that those dependencies are qualitatively different from the dependency on python. But you can't get around the fact that even before the switch to Meson you needed a fairly complete OS to build ksh.

--

Kurtis Rader

Caretaker of the exceptional canines Junior and Hank

[ardi]

On Thursday, July 18, 2019 at 11:52:36 PM UTC+2, Kurtis Rader wrote:

> [...] Even if we had chosen CMake there would still be lots of other dependencies. For 

> example, the project used to run this humungous /bin/sh script during the configure phase:

The only dependencies to build CMake are a C++ compiler and make. And, if you did the work of removing the additional dependencies carried on by that shell script when moving to Meson, then the removal of such dependencies would have been analogous for CMake. The dependencies after your choice are 4:  Meson, Python, a C/C++ toolchain+runtime, and ninja (because Meson has no make backend). The dependencies if you had chosen CMake would be 3: CMake, a C/C++ toolchain+runtime, and make (yes, there are make implementations with no extra dependencies, or you could instead opt for ninja if you prefer because CMake supports ninja too).

Which leads us to the fact that the choice is a matter of personal preferences (Meson vs CMake). Personally (and this is of course subjective) I expect to have Python available when I'm building end-user applications, not when I'm assembling the OS bare bones.

Anyway, there are minimal shells that can be built even without make nor ninja, so it's possible to move ksh to a later stage in the OS building process. I'd prefer to have ksh up and running soon, but your choice for Meson means that, at least for me, ksh will appear in the show at the same time as Blender for example, not before.

I mean, you didn't break my planned pipeline, just changed the early shell.

ardi

[Michal Orsák]

The thing you are describing sounds more like building of image to me. And this is good job for yocto-project/buildroot.

Yes, I'm disappointed.

Maybe you are disappointed because you do not know that it is not so hard to build initial image with python and meson preinstalled.

I am using meson in embedded systems and I have to say it does it's job well unlike most of the buildtools.

ardi

--
You received this message because you are subscribed to the Google Groups "The Meson Build System" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mesonbuild+...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/mesonbuild/9c066376-b465-4fc3-8d07-ecc2f336c416%40googlegroups.com.

Michal