Makefiles to Ninja?

[Ingwie Phoenix]

Hello everyone! - also Hey, i'm new to the list. :)

But, let's get to the topic. So, I am working on a bigger project which utilizes Chromium and PHP - in one place. Now, thru Chromium I found Ninja to be a very great tool for building stuff - very clear and clean. So I have even made concepts of a PHP based program to produce ninja build-files. But, speaking of PHP...it still uses Makefile. Nothing against Makefile, but, is it actually possible to convert a Makefile to a ninja.build file? Because that'd be very useful, and would provide me the ability to use subninja/include for building - and having a speedier rebuild time.

So, does somebody know of a method on converting an existing Makefile to a build.ninja file? I should also add, I don't know how Makefiles work, but I know how build.ninja files work.

Kind regards,

Ingwie!

[Evan Martin]

Anything is possible with enough code, but I don't think a converter
currently exists.
Part of the reason for Ninja's existence is that it is much weaker
than Makefiles so it would likely be challenging unless the Makefiles
were pretty simple.

Occasionally I daydream about writing something that could take
configure.ac+Makefile.am and produce Ninja files. It seems to me that
might be an easier problem. But these days I find myself rarely
encountering autotools based projects so it's probably an idea that
will become less relevant over time.

(Also, though this is sorta secondary to the above points, I don't
think integrating two different builds together is as easy as
sprinkling some 'subninja' or 'include' lines, as the two projects
likely have different ideas about where output files go.)

> --
> You received this message because you are subscribed to the Google Groups
> "ninja-build" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to ninja-build...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

[Neil Mitchell]

I have written an interpreter for Makefiles in Shake. Shake is more
powerful (in terms of what it can express) than either Ninja or
Makefile, which made it easier than it would be for Ninja to do.
However, it was a fair bit of work, I learnt more than I ever wanted
to know about Makefile's, and it isn't useful at all in practice. If
you have simple rules it all works out, but any large project is
likely to be relying on complex substitution macros and things like
automatic reloading of makefiles that change in the middle. I can
execute most tutorials, and even a good chunk of the Make test suite,
but any real project (the kind that would actually benefit from moving
to Shake/Ninja) is unlikely to work. If you do want to start with the
Makefile translation, I can point you at the code, but I wouldn't
encourage it.

If you want to build PHP with Ninja, I recommend going from scratch. I
found that running with -j1 and using verbose flags to print every
command listed was sufficient to mostly reverse engineer what existing
build systems do, without understanding the Makefiles.

Thanks, Neil

[Matthew Woehlke]

On 2013-10-28 12:54, Neil Mitchell wrote:
> If you want to build PHP with Ninja, I recommend going from scratch. I
> found that running with -j1 and using verbose flags to print every
> command listed was sufficient to mostly reverse engineer what existing
> build systems do, without understanding the Makefiles.

You could do that (you could even add -n so that you get the same output
without actually building anything), but you lose any configuration and
platform logic.

For that matter, ninja is (as I understand it) designed to not have any
of that anyway, which means you'll end up with a build.ninja that is
only useful for your specific system and configuration.

IMHO you'd do better to rewrite the build system to use a configuration
system that supports ninja, e.g. CMake. You'll get both a better end
result, I think, and also probably more chance of convincing upstream to
adopt it.

--
Matthew

[Bill Hoffman]

On 10/28/2013 1:02 PM, Matthew Woehlke wrote:
>
> IMHO you'd do better to rewrite the build system to use a configuration
> system that supports ninja, e.g. CMake. You'll get both a better end
> result, I think, and also probably more chance of convincing upstream to
> adopt it.

There was an attempt to move php to CMake in the past:

https://wiki.php.net/internals/cmake

-Bill

[Kevin Ingwersen]

Hello, and thank you all for your help!

I have just taken a sneak peak into the Makefile that PHP generates when configured with the --disable-all switch - which really, disables pretty much everything except the pure core. Reading this up and down tells me that it is not very hard to convert a Makefile. In fact, I have it half-way converted using simple RegExp search and replace pattern. These may just need outlining. Reading a little further, I found how the files in a Makefile are structured:

ext/pcre/pcrelib/pcre_refcount.lo: /Users/Ingwie/Work/drag0n-php/ext/pcre/pcrelib/pcre_refcount.c
$(LIBTOOL) --mode=compile $(CC) -DHAVE_CONFIG_H -I/Users/Ingwie/Work/drag0n-php/ext/pcre/pcrelib -Iext/pcre/ -I/Users/Ingwie/Work/drag0n-php/ext/pcre/ $(COMMON_FLAGS) $(CFLAGS_CLEAN) $(EXTRA_CFLAGS) -c /Users/Ingwie/Work/drag0n-php/ext/pcre/pcrelib/pcre_refcount.c -o ext/pcre/pcrelib/pcre_refcount.lo

Looking at this told me, that just two simple words, and a re-wrap, is missing. And that made me get closer to the idea of having a script that actually parses lines like these - and makes them change to simething like this:

CC = clang
CFLAGS = ...
# snip...
LIBTOOL_CC_FLAGS = -DHAVE_CONFIG -H

rule LIBTOOL_CC
command $LIBTOOL --mode=compile $CC $LIBTOOL_CC_FLAGS $INCLUDES $COMMON_FLAGS $CFLAGS_CLEAN $EXTRA_CFLAGS -c $in -o $out

build ext/pcre/pcrelib/pcre_refcount.lo: LIBTOOL_CC /Users/Ingwie/Work/drag0n-php/ext/pcre/pcrelib/pcre_refcount.c

Now, we can use a generator (such as the one I am coding rightnow) to change CC and alike - actually, we could use the include statement. But what I want to point out here, is that we're only moving around some parts. One can easily get said parts using things like <?php $strArr = explode(" ",'$LIBTOOL --mode=compile ...'); ?> and then we'd have to iterate over it.

I think I finally found my solution here - and I think I can actually just convert the makefile. What will be harder however, arte the targets - such as:

all: $(all_targets)
@echo
@echo "Build complete."
@echo "Don't forget to run 'make test'."
@echo

Does anybody have a suggestion how I may handle the @ statements? Can I actually have multiple command entries in a build block? Like:

build foo: bar
command = cmd1
command = cmd2

That way, I could just convert the @ into a command = output.

Any suggestion is helpful. For now, I am going to start on the direct Makefile converter. Let's see where we get!

Also, I looked into the cmake thing. It apepars to me that the cmake thing is sort of broken. I will investigate further - maybe I am just missing something out o.o

Thanks again for all your feedback! ^_^

Kind regards, Ingwie

[Nico Weber]

Hi Kevin,

(see inline)

On Mon, Oct 28, 2013 at 11:56 PM, Kevin Ingwersen <ingwi...@googlemail.com> wrote:

Hello, and thank you all for your help!

I have just taken a sneak peak into the Makefile that PHP generates when configured with the --disable-all switch - which really, disables pretty much everything except the pure core. Reading this up and down tells me that it is not very hard to convert a Makefile. In fact, I have it half-way converted using simple RegExp search and replace pattern. These may just need outlining. Reading a little further, I found how the files in a Makefile are structured:

ext/pcre/pcrelib/pcre_refcount.lo: /Users/Ingwie/Work/drag0n-php/ext/pcre/pcrelib/pcre_refcount.c
        $(LIBTOOL) --mode=compile $(CC) -DHAVE_CONFIG_H -I/Users/Ingwie/Work/drag0n-php/ext/pcre/pcrelib -Iext/pcre/ -I/Users/Ingwie/Work/drag0n-php/ext/pcre/ $(COMMON_FLAGS) $(CFLAGS_CLEAN) $(EXTRA_CFLAGS) -c /Users/Ingwie/Work/drag0n-php/ext/pcre/pcrelib/pcre_refcount.c -o ext/pcre/pcrelib/pcre_refcount.lo

Looking at this told me, that just two simple words, and a re-wrap, is missing. And that made me get closer to the idea of having a script that actually parses lines like these - and makes them change to simething like this:

CC = clang
CFLAGS = ...
# snip...
LIBTOOL_CC_FLAGS = -DHAVE_CONFIG -H

rule LIBTOOL_CC
command $LIBTOOL --mode=compile $CC $LIBTOOL_CC_FLAGS $INCLUDES $COMMON_FLAGS $CFLAGS_CLEAN $EXTRA_CFLAGS -c $in -o $out

build ext/pcre/pcrelib/pcre_refcount.lo: LIBTOOL_CC /Users/Ingwie/Work/drag0n-php/ext/pcre/pcrelib/pcre_refcount.c

Now, we can use a generator (such as the one I am coding rightnow) to change CC and alike - actually, we could use the include statement. But what I want to point out here, is that we're only moving around some parts. One can easily get said parts using things like <?php $strArr = explode(" ",'$LIBTOOL --mode=compile ...'); ?> and then we'd have to iterate over it.

I think I finally found my solution here - and I think I can actually just convert the makefile. What will be harder however, arte the targets - such as:

all: $(all_targets)
        @echo
        @echo "Build complete."
        @echo "Don't forget to run 'make test'."
        @echo

Does anybody have a suggestion how I may handle the @ statements? Can I actually have multiple command entries in a build block? Like:

build foo: bar
command = cmd1
command = cmd2

This isn't supported, but a command can contain arbitrary shell commands (on POSIX). So you could do

build foo: bar

  command = cmd1 ; cmd 2

(I haven't seen a command= set on a build block before, only on rule blocks. Note that the first element after the ':' in a build block names the rule block to use to build "foo", so the example above means "to build 'foo', use rule 'bar'" and there are no inputs. If "bar" is supposed to be an input, do this instead:

rule cmd1cmd2

  command = cmd1 ; cmd2

build foo: cmd1cmd2 bar

)

Nico

[Kevin Ingwersen]

Hello - and thank you for the information :)

So now I have actually converted the file - it just seems to have minor syntax errors by itself. Have a look :)

[Ingwie Phoenix]

Now, I actually got a little tool done alongside. o.o Just a little PoC for those that are curios.

https://github.com/IngwiePhoenix/autoNinja