[tools] CMake (aka Cross-Platform Make)

[Andrew Hill]

Apparently its my turn to do one of these and I'm telling you all about CMake... so here goes:

Sales Pitch: What is it, in 25 words or less?

It uses scripts to build code projects, so you don't have to. It supports many target project formats (Makefile/Visual Studio/Xcode/...), and most platforms (Win, *nix) and even cross-platform development.

Who would use it?

Here are some use cases I think it's useful for:

* any cross-platform code lib/app — autoconf and the like do okay with POSIX, but what about windows? screw maintaining two different code projects

* any shared code project — because even if we both use POSIX systems, I'm an Apple zealot and only use Xcode (which can't import Makefiles) and you want to use Eclipse some shit like that... ew.

* any shared code project — again, but this time because I put my 3rd party or in-house (non-apt) libs in ~/code/3rd/ and you put yours in /usr/local/lib, and the new guy has his randomly distributed in C:\Program Files\ and C:\dev and C:\Users\noob\Code\libraries\3rdparty\installed\WorkingVersion\

* long lasting code projects — because you'll switch to Xcode eventually (;

* rapidly changing code projects — because when you refactor the whole thing and add 100 new files and change the directory structure, wouldn't you prefer if your IDE/project file just figured it out after you'd moved the files?

And... once you're familiar with CMake:

* any code project — I reckon it's easier to set up a CMake project (generally by copy/pasting from an existing CMake project) than building a new Xcode/VS/... project from scratch

Okay, so how does it work?

Here's how I usually make projects:

1. I keep the build and source separate (I write almost exclusively C++, and share development with several other people, who use different IDEs/OSes); only the CMake scripts and the source code are checked into the shared repository. The project is built externally (by CMake) and can be easily trashed/rebuilt, so is never checked in anywhere. This isn't strictly required, and you can definitely do in-source builds, I just prefer not to to keep the source/cmake cleaner (I'd only bother checking in the project files if you've got a really standard setup on everyone's machines, or a project with very standard/no dependencies).

2. It's all based around a file called "CMakeLists.txt", which is the main CMake script that tells it important things like

* what dependencies must/can be used (for common libs, this is a line or two of code, as the script to find them and set up include/link dirs is all included)

* various options for how to build the project (e.g. you might have options for static/dynamic libs, or whether you build some/all binaries or lib files)

* where the source & include files are (typically with globs, so maybe Foo/Bar/*.cpp so as to pick up new/removed files if you add/remove them)

* what targets to build (generally libraries, applications, documentation such as doxygen)

* what files to install (for an 'install' target in the generated project) and how to structure the install

* anything platform specific (e.g. you might pull in Foo/Bar/MacOSX/*.cpp vs Foo/Bar/Win32/*.cpp for seriously different implementations)

* include other CMake files (e.g. you might write some functions that you use in a few projects, or split up the scripts for different modules of your project to make it easier to read)

It can be useful to make a template, or copy the CMake structure from one project to the next when making a new project.

Also there's examples & tutorials here to get you started.

3. You then run CMake (either command line script, command-line UI or GUI as cmake, ccmake, cmake-gui respectively), e.g. if I've put a CMake project in my git repo (and...@lilypad.borkt.com/Foo.git)

# mkdir -p ~/Code/source && cd ~/Code/source

# git clone and...@lilypad.borkt.com/Foo.git

# mkdir -p  ~/Code/projects/libFoo && cd ~/Code/projects/libFoo

# ccmake -G Xcode ~/Code/source/Foo

at this point, the console UI is running, and will generate an Xcode project in ~/Code/projects/libFoo based on the CMake scripts & source files in ~/Code/source/Foo

("-G Xcode" = Generate for Xcode, the supplied directory is the location of the main CMakeLists.txt file for the project, the current directory is the build location)

4. Within CMake, there are two stages:

a) "configure" — where you select any customisable project options (e.g. static vs dynamic libs, or components to include in the project), point it to dependencies, set install prefix if relevant, etc

b) "generate" — where it makes the project files for you

The cmake script (ie no user interface) does (a) then (b) without letting you change things; this is useful if you've got your dependencies in very standard locations, and just want to accept defaults for any options in your CMake scripts (or if the answers are being pulled out of environment variables by your CMake scripts). Generally you'd use ccmake or cmake-gui the first time, and after that simply regenerate with cmake (actually in most cases, the project will regenerate itself from within the IDE if you add/remove files, though not all IDEs handle this especially well, so you shouldn't need to run cmake again if you're staying within the globs/assumptions made in your CMake scripts)

5. Now you have your project, so open it and build/run, or make && make install or whatever it is you do.

Hey, go back to Step 2... that sounds hard...

Yeah okay, you weren't going to have projects get built magically for free, now were you... (actually, you might even get that free, but i've not tried either of those utils)

There's a lot of commands, and how much of the scripting you need to learn to make a project really depends on how complex the project is.

A really basic project, which builds one library and one binary/exe is shown here:

http://www.cmake.org/cmake/help/examples.html

I'm not going to write a tutorial on this, because there's so much info already out there, though I can probably answer your questions.

Anything else?

* you don't have to write makefiles or make VS projects. Ever.

* if you fuck shit up in your IDE, you can delete and regenerate.

* you can set up a doxygen target to build your doxygen code from the same project file

* it can run arbitrary commands as build targets (which is useful for CTest/CPack)

* CPack — builds packages (windows installers, shell scripts, tar.gz files) to package & distribute your project (more advanced topic)

* CTest — unit testing! with results reported as the build output, or submitted to a CDash dashboard/server (more advanced topic)

* you can convert from some other build systems to CMake semi-automatically using these converters

* cross compiling (e.g. building on x86 for ARM, or even just against libs different to those installed in /usr/local/)

I'd recommend reading through this intro page from the CMake wiki if you want more specific info (assuming you haven't already been to it, since I've provided about 3 links to it so far).

http://www.cmake.org/Wiki/CMake

It's a good hub for info / introduction / tutorials / resources on CMake.

Where can I get it?

http://www.cmake.org/

What does it run on?

Windows, Mac, any Linux (try sudo apt-get install cmake), QNX, BSD, probably more (that's all I've used personally)

What are the alternatives?

I'm not an expert in this area by any means, as most of my C++ coding life I've used CMake, but here's a few...

* hand-writing Makefiles (lol)

* making Xcode/Visual Studio/Eclipse projects by hand (okay for simple things, can be hard to share though)

* GNU Autotools (autoconf, libtool) — i believe limited in how portable they are?

* bjam / BoostBuild, qmake — i think these are targetted quite heavily at boost & qt respectively?

* SCons, Waf, Premake, Ant (Java), Rake (Ruby), mk-configure, pymake (Python) — (these are just stolen from this StackOverflow thread)

By all means, check some of these out, and if you do, report back — I've only used CMake, and there are valid criticisms so maybe one of these is better (that said, I think the criticisms from the above SO thread are largely solved these days, except maybe Documentation, because you can always have better docs...)

[Matthew Fernandez]

Thanks, Andrew. CMake sounds much better than our current system of
Makefiles (that barf out and destroy some poor Mac user's environment
because the cp arguments are different...). Do you know if CMake is
extensible to other platforms? I'd like to give it a go for building
projects on our kernel which doesn't have a POSIX layer. My further
comments below:

On 1 November 2011 12:02, Andrew Hill <and...@thefrog.net> wrote:
> * long lasting code projects — because you'll switch to Xcode eventually (;

I might hit you up for an Xcode blurb soon. People keep trying to get
me to switch.

> * GNU Autotools (autoconf, libtool) — i believe limited in how portable they
> are?

Yes. Autotools is tied to POSIX and even non-POSIX Linux extensions :(

> * SCons
SCons is like Make, but with a different format for Makefiles. It has
some cool things like hashing your targets so it will know not to
rebuild things that depend on them if a rebuild of them does not
actually change the hash. The last version of it that we used at work
still had significant bugs (exactly what you don't want from a build
system...), but it seems to be quite popular in the embedded
community. I assume because of its flexibility.

[Andrew Hill]

If your kernel isn't compatible with POSIX or Win32 then I'd guess you're not going to be able to run CMake under your kernel.

I've no idea how easy it'd be to port across, probably depends how POSIX-like your kernel is...

Can you run make under your kernel? If so, you might be able to get CMake to generate a Makefile project (from another OS) and then build that project under your kernel?

Alternatively, if you have build chain tools (compiler, linker, archiver, ...) that run on POSIX/win32, then you can build targets for your OS entirely from the POSIX/win32 system (though I get this isn't really what you're after).

What build tools do you have?

Also, here's my Xcode blurb

just go here, there's no way i can do a better job than them at selling it

http://developer.apple.com/technologies/tools/ (xcode + various related dev tools)

http://developer.apple.com/technologies/tools/whats-new.html (or just xcode specifically)

[Matthew Fernandez]

On 3 November 2011 16:41, Andrew Hill <and...@thefrog.net> wrote:
> If your kernel isn't compatible with POSIX or Win32 then I'd guess you're
> not going to be able to run CMake under your kernel.
> I've no idea how easy it'd be to port across, probably depends how
> POSIX-like your kernel is...

Not very...

> Can you run make under your kernel? If so, you might be able to get CMake to
> generate a Makefile project (from another OS) and then build that project
> under your kernel?

Nope. This would literally be ripping out (or instrumenting) things
like file system calls and replacing them with the relevant
mechanisms.

> Alternatively, if you have build chain tools (compiler, linker, archiver,
> ...) that run on POSIX/win32, then you can build targets for your OS
> entirely from the POSIX/win32 system (though I get this isn't really what
> you're after).
> What build tools do you have?

Nil. All current dev and compiling is done on a different (Linux) host
and then bootstrapped onto the device you're working on. This is a bit
of a pipe dream for me at the moment, but one of my todos is
configuring this kernel for my laptop and porting some useful bits
like vim, so it would be cool if I could actually build projects on
there as well.

>
> Also, here's my Xcode blurb
> just go here, there's no way i can do a better job than them at selling it
> http://developer.apple.com/technologies/tools/ (xcode + various related dev
> tools)
> http://developer.apple.com/technologies/tools/whats-new.html (or just xcode
> specifically)

These tell me why I should be super psyched about Xcode if I'm
developing for OSX or iOS, but I am doing neither. It's a general IDE,
yes? I can use it for things other than C/C++ and call a compiler
other than LLVM?

[Andrew Hill]

On Thu, Nov 3, 2011 at 4:54 PM, Matthew Fernandez <matthew....@gmail.com> wrote:

On 3 November 2011 16:41, Andrew Hill <and...@thefrog.net> wrote:
> If your kernel isn't compatible with POSIX or Win32 then I'd guess you're
> not going to be able to run CMake under your kernel.
> I've no idea how easy it'd be to port across, probably depends how
> POSIX-like your kernel is...

Not very...

> Can you run make under your kernel? If so, you might be able to get CMake to
> generate a Makefile project (from another OS) and then build that project
> under your kernel?

Nope. This would literally be ripping out (or instrumenting) things
like file system calls and replacing them with the relevant
mechanisms.

I'd guess getting CMake to run under your kernel would be even harder than getting make to run.

 

> Alternatively, if you have build chain tools (compiler, linker, archiver,
> ...) that run on POSIX/win32, then you can build targets for your OS
> entirely from the POSIX/win32 system (though I get this isn't really what
> you're after).
> What build tools do you have?

Nil. All current dev and compiling is done on a different (Linux) host
and then bootstrapped onto the device you're working on. This is a bit
of a pipe dream for me at the moment, but one of my todos is
configuring this kernel for my laptop and porting some useful bits
like vim, so it would be cool if I could actually build projects on
there as well.

The build tools (that run on linux) are what I was referring to here — if you have the build tools running on linux (to create targets for your OS) then you can use CMake and point it at those tools (instead of the default gcc/ld/...)

You won't be running CMake on your non-posix OS, nor building on the OS (but CMake wouldn't actually help you there anyway, if you don't have native build tools)... but you can use CMake on Linux and whatever IDE you like to generate your targets.

 

>
> Also, here's my Xcode blurb
> just go here, there's no way i can do a better job than them at selling it
> http://developer.apple.com/technologies/tools/ (xcode + various related dev
> tools)
> http://developer.apple.com/technologies/tools/whats-new.html (or just xcode
> specifically)

These tell me why I should be super psyched about Xcode if I'm
developing for OSX or iOS, but I am doing neither. It's a general IDE,
yes? I can use it for things other than C/C++ and call a compiler
other than LLVM?

Yep, it's an IDE and you can specify arbitrary build targets — a build target in Xcode, just like any other IDE, just boils down to a command to run.

Actually CMake makes it very easy to specify the compiler, linker, etc if you don't want gcc/llvm, and I believe you can make fairly arbitrary scripted targets too for things that aren't compiling at all.

The built in custom targets are things like "install", "uninstall", "package" (ie make an executable installer package). And then there's unit testing. The unit testing system uses CTest (part of CMake) and is a "build target", but instead of running a compiler, it runs a script that calls each test, and outputs the results (or posts them to a CDash server).

You've probably done the same thing with Makefiles, setting up arbitrary commands to run when you do 

"make test" or "make package"

right?

[Matthew Fernandez]

On 3 November 2011 17:12, Andrew Hill <and...@thefrog.net> wrote:
> The build tools (that run on linux) are what I was referring to here — if
> you have the build tools running on linux (to create targets for your OS)
> then you can use CMake and point it at those tools (instead of the default
> gcc/ld/...)
> You won't be running CMake on your non-posix OS, nor building on the OS (but
> CMake wouldn't actually help you there anyway, if you don't have native
> build tools)... but you can use CMake on Linux and whatever IDE you like to
> generate your targets.

Mm, good point. I would probably just be calling out of CMake to
compiler/linker/etc. Will investigate and see how I go.

> Yep, it's an IDE and you can specify arbitrary build targets — a build
> target in Xcode, just like any other IDE, just boils down to a command to
> run.
> Actually CMake makes it very easy to specify the compiler, linker, etc if
> you don't want gcc/llvm, and I believe you can make fairly arbitrary
> scripted targets too for things that aren't compiling at all.
> The built in custom targets are things like "install", "uninstall",
> "package" (ie make an executable installer package). And then there's unit
> testing. The unit testing system uses CTest (part of CMake) and is a "build
> target", but instead of running a compiler, it runs a script that calls each
> test, and outputs the results (or posts them to a CDash server).
> You've probably done the same thing with Makefiles, setting up arbitrary
> commands to run when you do
> "make test" or "make package"
> right?

Indeed. It does sound like CMake handles this much better. Will
definitely try that out. Xcode on the other hand looks Mac only, which
is a bit of a deal breaker for me. Then again, it should be legal to
run OSX in a VM on Apple hardware right? Maybe I can try it that way.

[Andrew Hill]

Xcode only runs on Mac, yes.
But that doesn't stop you building for any platform with it, and the
debugger is gdb.

Honestly, Visual Studio is still probably the best IDE, so if you're
doing all this for Xcode, you could equally do so for VS.

You can quite legally run 10.7 in a VM. Or 10.6 server if you have that.

On 03/11/2011, at 6:10 PM, Matthew Fernandez

[Matthew Fernandez]

On 3 November 2011 20:02, Andrew Hill <and...@thefrog.net> wrote:
> Honestly, Visual Studio is still probably the best IDE, so if you're
> doing all this for Xcode, you could equally do so for VS.

Yeah, Visual Studio is still far and away the best comprehensive IDE
I've used. The only thing I've seen come close to its GUI designer is
Qt Creator and that was very much betaware. Can I call for volunteers
for next month to give us a rundown on Visual Studio? Maybe someone
who's used it long term, like Venus :)

[David Wood]

So apparrently I can't post with my Ocular e-mail... Now to figure out
if I can get my Ocular address rights to send without me
double-recieving everything.

---------- Forwarded message ----------
From: David Wood <d.w...@ocularrobotics.com>
Date: 4 November 2011 10:59
Subject: Re: [tools] CMake (aka Cross-Platform Make)
To: code-p...@googlegroups.com

In my current position I'm using CMake and VS to do pretty much
everything. My VS Project build will not only build the source, but
run Doxygen over it to export all our release documentation to TeX
source, then PDFTeXify a larger project of which the Doxy TeX is a
component, then include that PDF output along with all the text
applications and the library into a windows installer package....

I didn't actually set the existing system up, so I'm still trying to
figure out just how the hell all this works, but as soon as I've
gotten my head around it I'd be happy to whip up some sort of
contrived example (I somehow doubt my boss wants me to hand out all
our source to my mates...). Maybe put me down for doing one of these
things in a month or so.

-- DKW

--
Cheers,
 Dr David Wood
 Product Development Engineer
 Ocular Robotics
 www.ocularrobotics.com

--
-- DKW

[Matthew Fernandez]

On 4 November 2011 12:32, David Wood <the.b...@gmail.com> wrote:
> So apparrently I can't post with my Ocular e-mail... Now to figure out
> if I can get my Ocular address rights to send without me
> double-recieving everything.

Just join using your ocular address and select "No Email" in the Edit
My Membership bit.

> source, then PDFTeXify a larger project of which the Doxy TeX is a

Does PDFTeXify do what I think it does? I.e. same as pdflatex?

> our source to my mates...). Maybe put me down for doing one of these
> things in a month or so.

Thanks! Will do :)