Project description syntax in Lua

[Rodolfo Lima]

Well, we should start by defining how to specify a project in Lua.
There's two branches to follow:
1) Make it resemble cmake's script syntax
2) Do something more appropriate to Lua's syntax

First option is good if one want to keep the learn curve flat, since
people accustomed to cmake's script won't have difficulties learning
it. This might facilitate future merge of our work back into cmake's
code base, and this is a big plus, IMHO. The negative side is that
maybe the syntax will be made a little bit clunkier because the two
languages are very different, and the resulting syntax won't be the
simplest and more direct way to write projects, which brings us to the
second option...

Creating a new syntax can make things more logical to existing Lua
programmers, potential new users of CMakeLua. They outnumber cmake's
users, making this option a big plus.

As an example of the first option syntax, we have the one currently
implemented:
cm_add_library("simple_lib", "STATIC", "simpleLib.cxx",
"simpleCLib.c", "simpleWe.cpp")
which could be modified to:
cm.add_library("simple_lib", STATIC, "simpleLib.cxx", "simpleCLib.c",
"simpleWe.cpp")

The second syntax could be (my suggestion)

simple_lib = static_library("simpleLib.cxx", "simpleCLib.c",
"simpleWe.cpp");
another_sources = {"source1.cpp", "source2.cpp"}
another_lib = library(another_sources)

What you think?

Regards,
rod

[E. Wing]

So I have a general task list I've been putting together. I need to get that up.

> Well, we should start by defining how to specify a project in Lua.
> There's two branches to follow:
> 1) Make it resemble cmake's script syntax
> 2) Do something more appropriate to Lua's syntax

So my feeling is we do both, but I obviously need to qualify that. If
you look at the original Kitware CMake/Lua page in the comments
section, my proposal is to use Lua to wrap around the C-api bindings
to make things a little nicer for Lua syntax while still adhering to
the official API.

I have already integrated that little experiment into the Mercurial
repository. It was a very quick/dirty integration just to prove it
could be done and only supports cm.add_library and cm.STATIC, but I
think it can be easily extended. I'm also thinking it can be
implemented more cleverly so this can all be done dynamically in a
little for-loop without the need to write an explicit function for
each API entry. (The current Lua script is in the Modules directory.)

> First option is good if one want to keep the learn curve flat, since
> people accustomed to cmake's script won't have difficulties learning
> it. This might facilitate future merge of our work back into cmake's
> code base, and this is a big plus, IMHO. The negative side is that
> maybe the syntax will be made a little bit clunkier because the two
> languages are very different, and the resulting syntax won't be the
> simplest and more direct way to write projects, which brings us to the
> second option...

So I believe keeping API parity with the official CMake is the way to
go. Simply put, I don't want to write the documentation for anything
we invent. And I'm not sure how wise it is to change an API without
very careful consideration. Anything we invent, we take responsibility
for and must maintain for a period of time, even if it turns out to be
a bad API change.

But I make the distinction that we can keep API parity while still
making syntactic conveniences. As in my example, I support the
intermixing of tables and strings for file lists. The API is pretty
much the same as before, but just supports more natural Lua
expression.

> As an example of the first option syntax, we have the one currently
> implemented:
> cm_add_library("simple_lib", "STATIC", "simpleLib.cxx",
> "simpleCLib.c", "simpleWe.cpp")
> which could be modified to:
> cm.add_library("simple_lib", STATIC, "simpleLib.cxx", "simpleCLib.c",
> "simpleWe.cpp")

As you can see in my checked in example, I already do this and go
beyond. You can use tables here. I also already changed the cm_ to cm.
for namespacing. Thinking about it though and looking at Lua itself,
I'm thinking we should use 'cmake.' instead of 'cm.' (table, system,
math). In the age of autocompletion, I don't think the two extra
letters mean anything. I also namespaced the constant STATIC as
cm.STATIC. I think we should namespace the constants too. I'm not sure
what the namespace should be though. (cmake., CMAKE.?)

> The second syntax could be (my suggestion)
>
> simple_lib = static_library("simpleLib.cxx", "simpleCLib.c",
> "simpleWe.cpp");
> another_sources = {"source1.cpp", "source2.cpp"}
> another_lib = library(another_sources)

I think changing and inventing new function names is bad. Table
support is fine (as above). But don't see how separate functions for
static_library and presumably dynamic_library and module_library are
an improvement or even more 'Lua like'. As it stands now, some people
do add_library(foo, ${USER_SETTING}, ), meaning they use a variable
and let the end user decide if they want a shared or static library.
Changing this API would probably create more code for Lua users
because you need to introduce conditionals. If somebody wants to
invent their own function called static_library, they are free to
implement that themselves for their own scripts. (It's easy enough to
do.)

-Eric

[drahosp]

Hi,
I think there is another point that should be considered now because
its bound to come up later on. I'm interested in providing a "CMake"
module to Lua so users could take advantages of CMake in their own
projects or use CMake with other Lua modules (such as much requested
regular expressions, sockets or god knows what :P), These are
structural changes and I believe very easy to implement. So please
consider this approach as it would open more possibilities for the use
of the project.

Now, back to your points.

On Dec 28, 1:32 pm, Rodolfo Lima <rodl...@gmail.com> wrote:
> Well, we should start by defining how to specify a project in Lua.
> There's two branches to follow:
> 1) Make it resemble cmake's script syntax
> 2) Do something more appropriate to Lua's syntax
>
> First option is good if one want to keep the learn curve flat, since
> people accustomed to cmake's script won't have difficulties learning
> it. This might facilitate future merge of our work back into cmake's
> code base, and this is a big plus, IMHO. The negative side is that
> maybe the syntax will be made a little bit clunkier because the two
> languages are very different, and the resulting syntax won't be the
> simplest and more direct way to write projects, which brings us to the
> second option...

My opinion is to go separate ways, thats why I want to use Lua in the
first place, so we can make things simpler/better and more flexible. I
still would like to support the old CMake syntax but not directly .. I
think it would be better to provide a converter from CMake script to
CMakeLua that would also be helpful in converting CMake modules. Best
this conversion would be automatic so CMakeLua could be used instead
of CMake even when the input file is not Lua completely without CMake
script implementation. The conversion would of course be written in
Lua.

>
> Creating a new syntax can make things more logical to existing Lua
> programmers, potential new users of CMakeLua. They outnumber cmake's
> users, making this option a big plus.

I agree, no point making a fork otherwise.

>
> As an example of the first option syntax, we have the one currently
> implemented:
> cm_add_library("simple_lib", "STATIC", "simpleLib.cxx",
> "simpleCLib.c", "simpleWe.cpp")
> which could be modified to:
> cm.add_library("simple_lib", STATIC, "simpleLib.cxx", "simpleCLib.c",
> "simpleWe.cpp")
>
> The second syntax could be (my suggestion)
>
> simple_lib = static_library("simpleLib.cxx", "simpleCLib.c",
> "simpleWe.cpp");
> another_sources = {"source1.cpp", "source2.cpp"}
> another_lib = library(another_sources)
>

I'm for the changes as they are relatively easy to implement as
additional lua only modules based on standard cmake functions. But I
would like to keep them as separate Lua only module so it would
override the default cmake behavior/functions only when required.

example:
-- Default CMake behavior used from Lua as module
cm = require "cmake" -- use cmake module directly
cm.add_library("mylib","STATIC","mylib.cxx") -- cmake way of doing
things

-- CMakeLua behavior used from Lua as module
cm = require "cmakelua" -- based on cmake module
mylib = cm.static_lib("mylib.cxx") -- convenient and more Lua-ish

Of course the project will provide a "cmake" exectable as it is now
but with all cmake functions already existing in its global lua state
so requiring cmake module can be skipped. Users wanting CMakeLua
behavior would just have to call require. That way we can get very
close to cmake-like behavior and still be able to use convenient
functions if needed.

As for your second point I would not go that way, it would not be Lua
we end up using and that would be pain as we could not point to
lua.org for documentation. Best we leave the language syntax in the
hands of Lua creators as Roberto is already doing research on the
matter (token filters/macros).

pd

[E. Wing]

On 12/28/07, drahosp <geran...@gmail.com> wrote:
>
> Hi,
> I think there is another point that should be considered now because
> its bound to come up later on. I'm interested in providing a "CMake"
> module to Lua so users could take advantages of CMake in their own
> projects or use CMake with other Lua modules (such as much requested
> regular expressions, sockets or god knows what :P), These are
> structural changes and I believe very easy to implement. So please
> consider this approach as it would open more possibilities for the use
> of the project.

So I'm open to this idea as long as this is transparent to the
conventional usage of CMake. Maybe you can elaborate on this.

> My opinion is to go separate ways, thats why I want to use Lua in the
> first place, so we can make things simpler/better and more flexible. I
> still would like to support the old CMake syntax but not directly .. I
> think it would be better to provide a converter from CMake script to
> CMakeLua that would also be helpful in converting CMake modules. Best
> this conversion would be automatic so CMakeLua could be used instead
> of CMake even when the input file is not Lua completely without CMake
> script implementation. The conversion would of course be written in
> Lua.

I like the idea of a converter script. I had talked to Gonzolo about
supporting interoperation of both scripts. The technical difficulty
seems to lie in how to share variables between VMs. Everything is kept
in C++ I'm told so it should be possible, but would require some
effort. In the interest of getting something running sooner as opposed
to later, I like the idea of a converter script if it can be done
quickly.

> I'm for the changes as they are relatively easy to implement as
> additional lua only modules based on standard cmake functions. But I
> would like to keep them as separate Lua only module so it would
> override the default cmake behavior/functions only when required.
>
> example:
> -- Default CMake behavior used from Lua as module
> cm = require "cmake" -- use cmake module directly
> cm.add_library("mylib","STATIC","mylib.cxx") -- cmake way of doing
> things
>
> -- CMakeLua behavior used from Lua as module
> cm = require "cmakelua" -- based on cmake module
> mylib = cm.static_lib("mylib.cxx") -- convenient and more Lua-ish
>

So one, I do agree that I like the idea of using return values.
Passing return values through arguments is hokey. As long as we're
consistent about it, where we can write a one-line documentation note
that covers all of them, I think we're good. But in add_library's
case, "mylib" is not a return value, it is a string parameter that
specifies what you want your library name to be, correct? The above
example is invalid.

And again, I think changing the API so you have separate
static_library, dynamic_library, etc. is a bad idea. This doesn't have
anything to do with Lua in my opinion nor does it make things any
nicer. It makes things worse in my opinion because there is already a
huge ignorance between SHARED and MODULE and this will only make the
problem harder to deal with. This will also make it hard to deal with
new features in the proposed queue like FRAMEWORK (though we might be
going a different route with FRAMEWORK).

Public API writing is something to not take lightly. A good Public API
is generally vetted and discussed over long periods of time and
experience. I rather not take that responsibility and leave it to the
CMake community as a whole.

Thanks,
Eric

[Peter Drahoš]

On 29Dec, 2007, at 2:23, E. Wing wrote:

On 12/28/07, drahosp <geran...@gmail.com> wrote:

Hi,

I think there is another point that should be considered now because

its bound to come up later on. I'm interested in providing a "CMake"

module to Lua so users could take advantages of CMake in their own

projects or use CMake with other Lua modules (such as much requested

regular expressions, sockets or god knows what :P), These are

structural changes and I believe very easy to implement. So please

consider this approach as it would open more possibilities for the use

of the project.

So I'm open to this idea as long as this is transparent to the

conventional usage of CMake. Maybe you can elaborate on this,

No problem .. I first have to get familiar with the sources but I believe it can be done quickly.

My opinion is to go separate ways, thats why I want to use Lua in the

first place, so we can make things simpler/better and more flexible. I

still would like to support the old CMake syntax but not directly .. I

think it would be better to provide a converter from CMake script to

CMakeLua that would also be helpful in converting CMake modules. Best

this conversion would be automatic so CMakeLua could be used instead

of CMake even when the input file is not Lua completely without CMake

script implementation. The conversion would of course be written in

Lua.

I like the idea of a converter script. I had talked to Gonzolo about
supporting interoperation of both scripts. The technical difficulty
seems to lie in how to share variables between VMs. Everything is kept
in C++ I'm told so it should be possible, but would require some
effort. In the interest of getting something running sooner as opposed
to later, I like the idea of a converter script if it can be done
quickly.

Well I believe using a converter would be much faster and cleaner approach (in implementation). Maybe we could get rid of cmake script entirely but I'm not sure exactly how deep the cmake script implementation is and how (and if) its tied to cmake api.

I'm for the changes as they are relatively easy to implement as

additional lua only modules based on standard cmake functions. But I

would like to keep them as separate Lua only module so it would

override the default cmake behavior/functions only when required.

example:

-- Default CMake behavior used from Lua as module

cm = require "cmake" -- use cmake module directly

cm.add_library("mylib","STATIC","mylib.cxx") -- cmake way of doing

things

-- CMakeLua behavior used from Lua as module

cm = require "cmakelua" -- based on cmake module

mylib = cm.static_lib("mylib.cxx") -- convenient and more Lua-ish

So one, I do agree that I like the idea of using return values.
Passing return values through arguments is hokey. As long as we're
consistent about it, where we can write a one-line documentation note
that covers all of them, I think we're good. But in add_library's
case, "mylib" is not a return value, it is a string parameter that
specifies what you want your library name to be, correct? The above
example is invalid.

Yeah it is :p just noticed. Well, my point was to illustrate the usage of require to add convenient functions to the API rather than to show how the API  should look. Think of it as cmakes find_package which adds functions instead of macros and variables.

And again, I think changing the API so you have separate
static_library, dynamic_library, etc. is a bad idea. This doesn't have
anything to do with Lua in my opinion nor does it make things any
nicer. It makes things worse in my opinion because there is already a
huge ignorance between SHARED and MODULE and this will only make the
problem harder to deal with. This will also make it hard to deal with
new features in the proposed queue like FRAMEWORK (though we might be
going a different route with FRAMEWORK).

Public API writing is something to not take lightly. A good Public API
is generally vetted and discussed over long periods of time and
experience. I rather not take that responsibility and leave it to the
CMake community as a whole.

We really don't need to touch the API at all. Lets simply wrap the basic API in lua modules. This would be very similar to macros in cmake find_package modules (eg FindQT4). 

I try to illustrate it on my prev.example.

-- module cmakelua

package "cmakelua"

local cm = require "cmake"

function static_lib(...) -- wrap around default cmake api

local lib

cm.add_library(lib,"STATIC",...)

return lib

end

all you need to do is require "cmakelua" in you project .. if you don't you only have the standard API. I'd like to see all the cmake modules work this way too so modules can provide their functions and not just macros. Simply think of require as find_package in my examples.

pd

[Rodolfo Lima]

On 28 dez, 22:38, "E. Wing" <ewmail...@gmail.com> wrote:

> So I believe keeping API parity with the official CMake is the way to
> go. Simply put, I don't want to write the documentation for anything
> we invent. And I'm not sure how wise it is to change an API without
> very careful consideration. Anything we invent, we take responsibility
> for and must maintain for a period of time, even if it turns out to be
> a bad API change.

After a second thought, I'm not really convinced to keep cmake <->
cmakelua API correspondence.
Let me try to substitantiate this: The elements in a cmake script
(targets, sources, ...) are in fact objects in an OO paradigm. The
syntax adopted by cmake doesn't take that into account. It's the same
as programming OOP in C, where we have to pass the object to every
function that manipulates its internals. Having said that, Lua enables
us to think and code in a more OO-oriented fashion. A Lua build system
could use this on its behalf, so we could write

libsources = {....}
mylib = library("mylib", libsources)

projsources = {...}
myproj = executable("myproj", projsources)
myproj:add_link_libraries(mylib)
myproj.project_label = "My Project"

I don't know if I'm too OO oriented and I'm accustomed to look at
this kind code, but I think that's pretty simple and follows cmake's
core ideas, i.e: we have target objects (libraries, executables), we
can plug them together as needed, change some properties, and
dependencies arrise naturally.

IMHO the most important aspect of a build system is how it is
designed, what are its objects, how they relate together. This cmake
does pretty well. Objects are clearly delimited, etc. The syntax
naturally arises from this concept. Of course, a stable and strong API
is of utmost importance, but once the concept is well understood, the
API follows as a consequence.

Having said that, I think we should strive for a simple and elegant
syntax that maps nicely to the concept of build systems used by
cmake.

Another point: a possible syntax for add_custom_command following
cmake syntax would be:
add_custom_command(OUTPUT, "file.cpp", COMMAND, "touch", "file.cpp",
COMMAND,"othercmd","param1","param2")

OUTPUT and COMMAND are used to define what the next parameters mean,
instead of actually passing some values to the function. This might
confuse a potential user. The problem is that we must use commas to
delimit each parameter. In cmake script, this makes sense because of
the absence of commas. The command to be executed must be passed as
couple of strings, because at least in Windows were executables with
spaces are common, the following wouldn't work "my command param1
param2". We should pass "my command", "param1","param2". Even "param1
param2" wouldn't be valid because that's ONE parameter with a space,
not two as intended.

In a lua-ish syntax, we could do:
file_cpp = generate_file("file.cpp", Command("touch", "file.cpp"),
Command("othercmd", "param1", "param2"))

Let's forget about 'generate_file' name (this is another stage of
discussion). This syntax can be understood by Lua programmers (and
everyone, per definition).

> As you can see in my checked in example, I already do this and go
> beyond. You can use tables here. I also already changed the cm_ to cm.
> for namespacing. Thinking about it though and looking at Lua itself,

I don't know if we should use a namespace to cmake-related tokens.
Namespaces should be used by libraries that get included in a script
to cope with name clashes, which isn't the case of CMakeLua script.
The only parser for it will be cmakelua executable, so there won't be
a risk of name clashes since its functions and variables won't be used
outside CMakeLua. Even lua has those global functions (print, require,
load, tostring, ...). The functions in other namespaces (math, io,
file, ...) are just libraries and are in a proper namespace.

Regards,
rod

[Peter Drahoš]

I agree that OO approach would feel more natural but I would not force
it on the users. There are two main reasons for not doing so. First,
familiarity with cmakescript should be encouaged so users can use
cmakelua right away. Second, converting cmake scripts to lua would be
straightforward and simple if the APIs would match. Still, the OO
system you proposed can be built upon the cmake API in plain Lua, that
would carry the benefits of both approaches.

> Another point: a possible syntax for add_custom_command following
> cmake syntax would be:
> add_custom_command(OUTPUT, "file.cpp", COMMAND, "touch", "file.cpp",
> COMMAND,"othercmd","param1","param2")
>
> OUTPUT and COMMAND are used to define what the next parameters mean,
> instead of actually passing some values to the function. This might
> confuse a potential user. The problem is that we must use commas to
> delimit each parameter. In cmake script, this makes sense because of
> the absence of commas. The command to be executed must be passed as
> couple of strings, because at least in Windows were executables with
> spaces are common, the following wouldn't work "my command param1
> param2". We should pass "my command", "param1","param2". Even "param1
> param2" wouldn't be valid because that's ONE parameter with a space,
> not two as intended.
>
> In a lua-ish syntax, we could do:
> file_cpp = generate_file("file.cpp", Command("touch", "file.cpp"),
> Command("othercmd", "param1", "param2"))
>
> Let's forget about 'generate_file' name (this is another stage of
> discussion). This syntax can be understood by Lua programmers (and
> everyone, per definition).

I would leave this for later discussion, but I see your point.

>
>
>> As you can see in my checked in example, I already do this and go
>> beyond. You can use tables here. I also already changed the cm_ to
>> cm.
>> for namespacing. Thinking about it though and looking at Lua itself,
>
> I don't know if we should use a namespace to cmake-related tokens.
> Namespaces should be used by libraries that get included in a script
> to cope with name clashes, which isn't the case of CMakeLua script.
> The only parser for it will be cmakelua executable, so there won't be
> a risk of name clashes since its functions and variables won't be used
> outside CMakeLua. Even lua has those global functions (print, require,
> load, tostring, ...). The functions in other namespaces (math, io,
> file, ...) are just libraries and are in a proper namespace.

This is not an issue in my opinion. Its possible to use any namespace
you want (eg, cm = require "cmake" or cmake = require "cmake" when
used as module) or in case of the cmake executable the package can be
loaded into _G so no prefix or require would be needed.

pd

[Rodolfo Lima]

On Dec 29, 12:57 pm, Peter Drahoš <geraniu...@gmail.com> wrote:

> I agree that OO approach would feel more natural but I would not force
> it on the users. There are two main reasons for not doing so. First,
> familiarity with cmakescript should be encouaged so users can use
> cmakelua right away. Second, converting cmake scripts to lua would be
> straightforward and simple if the APIs would match. Still, the OO
> system you proposed can be built upon the cmake API in plain Lua, that
> would carry the benefits of both approaches.

So you're saying to write:
mylib = add_library("mylib", libsources)
myproj = executable("myproj", sources)
myproj:link_libraries(mylib)

in terms of cmake API, which would be
add_library("mylib", libsources)
add_executable("myproj", sources)
target_link_libraries("myproj", "mylib") ?

I think this could be done. The opposite would be more troublesome
because you have to map strings (target names) to actual target
objects.

Have you heard of luabind (http://www.rasterbar.com/products/
luabind.html)? It's a library to import and export variables,
functions, classes to/from Lua from C++. It's a valuable tool that
I've been using this whole year in a project I'm working in. It
resembles boost_python, but with Lua.

> This is not an issue in my opinion. Its possible to use any namespace
> you want (eg, cm = require "cmake" or cmake = require "cmake" when
> used as module) or in case of the cmake executable the package can be
> loaded into _G so no prefix or require would be needed.

I didn't know about it. I would vote for a 'cmake' namespace.

Regards,
rod

[Peter Drahoš]

On 29Dec, 2007, at 17:55, Rodolfo Lima wrote:

On Dec 29, 12:57 pm, Peter Drahoš <geraniu...@gmail.com> wrote:

I agree that OO approach would feel more natural but I would not force

it on the users. There are two main reasons for not doing so. First,

familiarity with cmakescript should be encouaged so users can use

cmakelua right away. Second, converting cmake scripts to lua would be

straightforward and simple if the APIs would match. Still, the OO

system you proposed can be built upon the cmake API in plain Lua, that

would carry the benefits of both approaches.

So you're saying to write:
mylib = add_library("mylib", libsources)
myproj = executable("myproj", sources)
myproj:link_libraries(mylib)

in terms of cmake API, which would be
add_library("mylib", libsources)
add_executable("myproj", sources)
target_link_libraries("myproj", "mylib")  ?

Yes, you got my point. I don't care about the CMakeLua API being OO or like you described, I leave that up to you.

I think this could be done. The opposite would be more troublesome
because you have to map strings (target names) to actual target
objects.

I think there is no need .. we only need to be backwards compatible with cmake :P. But you can always store additional data like the target name string in the returned object:

mylib = add_library("mylib", libsources) -- returns a table/object describing the cmake library

print (mylib.target) -- get cmake target name

> mylib

Adding more properties to the returned object can be done using metatables or by directly calling functions  so changes to the values will call the appropriate cmake api functions:

mylib:link(libs)

mylib.filename = "mylib_renamed"

mylib:install(path)

Take the above example only as inspirational. I'm only trying to show possibilities.

Have you heard of luabind (http://www.rasterbar.com/products/
luabind.html)? It's a library to import and export variables,
functions, classes to/from Lua from C++. It's a valuable tool that
I've been using this whole year in a project I'm working in. It
resembles boost_python, but with Lua.

Yes, I do prefer to use tolua++ or binding by hand but luabind is fine too.

This is not an issue in my opinion. Its possible to use any namespace

you want (eg, cm = require "cmake" or cmake = require "cmake" when

used as module) or in case of the cmake executable the package can be

loaded into _G so no prefix or require would be needed.

I didn't know about it. I would vote for a 'cmake' namespace.

Well, lets settle on cmake for now. Making a prototype that would behave as I described is trivial once the binding exists.

pd

[E. Wing]

Okay, a lot has been discussed here. Here's my attempt to respond and
move to the next stage:

On API parity:

So I repeat myself, but I am under the feeling that we should keep API
parity with official CMake. This means that for every API CMake has in
its documentation, we also have. But this doesn't mean that we can't
support Lua-isms, like supporting tables, or using return values. But
for an API function in CMake, we need to have a corresponding one in
Lua by the same name.

There are several motivations for this.
1) Code simplicity. The current binding system is actually pretty
elegant. There is one function that dynamically binds Lua to every
function in CMake. We basically get this for free from Ken. Why fight
it? Furthermore, every change we make puts us at risk for a merge
conflict with the official source code base.

2) Documentation simplicity. We don't have to write massive amounts of
documentation. I know very few open source projects that write good
documentation that is up-to-date. I can assure you we will not be one
of the projects that excels in the manner :P

3) User adoption simplicity. Since we don't have major differences and
documentation to worry about, people can pick up CMakeLua very easily
and still get help from the main CMake list since APIs and techniques
are the same.
A great case study is the failed Apple Cocoa/Java bridge vs. the
open-source PyObjC bridge and open-source RubyCocoa bridge. PyObjC and
RubyCocoa kept API parity with the Objective-C Cocoa API and people
using these bridges were able to produce good working products and
communicate cleanly with Obj-C developers. The Cocoa/Java bridge tried
to do things their own way and created an API that was neither
pure-Cocoa nor pure-Java. These people generally suffered because they
could not communicate to with the larger community because of the API
mismatch. There were also tons of bugs introduced in Cocoa/Java
because the system deviated too much from a straight binding.
Cocoa/Java was such a disaster, it has been EOL'd. Meanwhile PyObjC
and RubyCocoa are such successes, they are now officially supported by
Apple.

4) Politics. The closer we adhere to existing CMake, the better the
chance we can be reintegrated into the core CMake distribution and
won't have to bear the burden of the fork.

> Think of it as cmakes find_package which adds functions instead of macros and variables.

So I actually don't make the distinction between functions implemented
in CMake-script vs C++. That's an implementation detail I haven't
thought much of. For me the "API" is what is documented in the
documentation page. They don't document what's in C++ or CMake-script
I believe, and I suspect most users don't think about this either.

New API functions:
So in the context of things like "static_library", I think this is a mistake.

There is an adage that goes something like:
'A design is perfect not when there is no longer anything you can add
to it, but if there is no longer anything you can take away.'

Yes, this is a generalization, but I think it is an important one to
always have in the back of your mind. Too many ways to do something
makes things confusing. I think there was a recent discussion on
ADD_SUBDIRS vs ADD_SUBDIRECTORIES. Even with one marked deprecated in
the documentation, it still seems to be enough to illicit confusion.

And as far as 'Lua-likeness', I would disagree that adding API falls
in this category. Lua is a minimalist language. It is unique because
it doesn't try to do everything under the sun. It is focused and
concise. There was a comparison I really liked where somebody compared
Lua to a Haiku. If you look at the Lua API, there is very little
redundancy. lua_getfield/lua_setfield are the only ones that come to
mind at the moment, and they are new to 5.1 which suggests they were
mulling over the decision for a long time. I think the PIL book also
feels the need to justify them since they are redundant.

Finally, I return to my original argument. I don't think we should be
making API decisions that are not directly Lua-integration related.
Some of these types of things you propose are not specific to Lua and
could be very well implemented in existing/standard CMake. I say this
should be the litmus test. In these cases, you should pitch these
ideas to the main CMake community and have the community vet these
APIs as to whether they are good or bad. People may point out there is
a fundamental design flaw in the worst case and you can't do it. I
think our group is too small with too many other things to consider to
make these kind of decisions. Our focus should be on the Lua-specific
issues. The problem with public API is once you add it, it is
extremely hard to remove it if it doesn't work.

Object oriented:

I am ambivalent about this one. On one hand, I think it introduces
additional complexity where the payoff doesn't seem to be that big. It
also introduces a complex documentation problem for us because nobody
is going to know about this, and people are going to be confused about
the OO-style vs the traditional style. Furthermore, not everything in
CMake really strikes me as suitable for OO. And for the things that
might lend themselves to OO, I'm not convinced it is an improvement
because you lose a degree of transparency and explicitness which is
not necessarily a bad thing, especially in small/short scripts which a
build system typically is comprised of. And then there is the fact
that Lua itself makes OO only an option, and the way to do it can
vary. I'm not convinced that Lua users will automatically see this as
'natural'. (I don't use OO much in Lua.)

On the other hand, I can kind of see where you are going with this
one. I admit, it might be interesting to see if this could work fully
across CMake without problems and if there is indeed an overall
benefit. One area that I've been thinking about is the SET function.
Since we have the assignment operator, it seems like providing a way
to use the assignment operator would be reasonable. But since SET has
parameters that classify the variable, maybe something OO might help
here. (But then again, variables are currently not bound to Lua in the
fork, and the variables already push into the C++ core, so
thought/investigation needs to be done here.)

So two points. First, I think this should be implemented completely in
Lua. If it can't, I would be extremely nervous about it to the point I
may oppose it. One of my concerns is that we will constantly need to
merge in upstream official CMake changes. Despite how good distributed
SCMs are at merging, merge conflicts are hard no matter what. Not
knowing the official CMake code very well (in my case) makes it even
more painful. And the fact that the official CMake authors currently
have no intention to help us makes it even more complicated.

Second point, because there is so much other stuff to do, and the fact
I consider this more experimental without a clear metric of how useful
this will be, I would like to suggest tabling this idea until after we
get a solid working version 1.0, and then either consider again, or
consider making an experimental fork. (Unless it becomes clear that we
*must* implement OO in order to make something work, in which case we
just do it.)

Comment on require "cmakelua":

So my context of focus is on the 'regular way' of using CMake so this
comment may not apply to the LuaDist stuff. But for the 'regular way',
I don't think users should have to do 'require "cmakelua"'. That
should be built in and ready to go because by definition, they are
already using CMakeLua. That's the point of using the fork.

In the context of using CMakeLua as a package for what you are after
in LuaDist/LuaRocks, I think that is reasonable. But I was bothered by
the notion that you might do 'require "CMakeLua"' to import helper API
functions that are not standard API functions. CMakeLua *is* the
project. If you want helper, non-canonical functions, then I suggest
something like 'require "CMakeLuaConvenienceAPIs"' or something else
distinct from "CMakeLua".

LuaBind, lua++:

Have you looked at the actual code yet? One of the really cool things
about it is that Ken managed to create the entire C++ API interface
glue in basically one function. We should be rolling with that instead
of trying to reinvent the binding. I also think LuaBind would be a
mistake because of the Boost dependency. If we make CMakeLua to hard
to build/install or too bloated, nobody will want to use our version.

Also, I know Gonzolo spoke of trying to refactor the code in such a
way that it would be easier for somebody else to bind a different
language to CMake as well.

So I get the sense that hand bindings will be the best way to go.

Features that might be reasonable (aka, I'm not a total prude, really!):

So I noticed there has been a lot of activity concerning full blown
Perl style Regular Expressions. This is something I think would be a
good feature to add to CMakeLua. This is not an API thing easily added
to existing CMake. This seems to be a popular feature that the
community is demanding and it sounds like CMake is going to eventually
add it. For us, it should be a trivial matter of including lrexlib.

I know Roberto has also been talking about LPEG. I don't know anything
about PEGs and barely remember context free grammars from college, but
it sounds like the 'correct' way of doing things over regular
expressions, and also sounds like Roberto might be placing it as a
future killer feature of Lua. So I'm wondering if we should also
include this in. I do worry about bloating our distribution too much
though, though this implementation doesn't seem too big. Another
possibility is to use this in lieu of lrexlib, but I don't think the
regex part of the module does Perl-compatible style regex which is the
problem. (But I could be mistaken because the page doesn't explicitly
say.)

Anyway, I feel bad about saying 'no' so repeatedly to you guys. (Not
that I am any authority and can actually prevent you guys from doing
it anyway.) But my proposal is we start on the basics that we can
agree on and produce a fully capable/usable implementation that lacks
many of the 'frills' we've been discussing. Then after that 1.0, we
re-evaluate and look to the future. My mentality is that I have seen
too many projects fail because they were too ambitious to start with.
I say we focus on the basics, and we have a chance of actually
succeeding if we keep it simple.

I have a task list I've been working on which I would hope is the
subset of things we can agree on.

Thanks,
Eric

[Peter Drahoš]

First of all, thanks for a great summary.

I completely agree.

>> Think of it as cmakes find_package which adds functions instead of
>> macros and variables.
>
> So I actually don't make the distinction between functions implemented
> in CMake-script vs C++. That's an implementation detail I haven't
> thought much of. For me the "API" is what is documented in the
> documentation page. They don't document what's in C++ or CMake-script
> I believe, and I suspect most users don't think about this either.

True, but still its a good way to build lua-side functions/experiments
or addons while keeping the cmake API unchanged.

Experimenting with OO approaches can be done later but talking about
SET we could tie it with lua variables with a metatable on the cmake
namespace so new variables in the namespace automatically call SET.
Still such implementation can be done on the Lua side. This is one of
the cases that I would agree to include to cmakelua as its not an API
change rather than a different way of calling SET from the cmake API.
As long as the variables dont clash with package function names it
should work. If they clash maybe we can use different table/namespace
for it. (cmake.var for example)

eg: setting src variable in different syntax
in cmake :SET(mysrc file1.c file2.c)
in cmakelua API: cmake.set(mysrc,{"file1.c","file2.c"})
in cmakelua API with metatables: cmake.mysrc = {"file1.c", "file2.c"}

Still this approach needs a get function too get values from cmake
which were not pushed in from lua and a way to set its TYPE.

> Comment on require "cmakelua":
>
> So my context of focus is on the 'regular way' of using CMake so this
> comment may not apply to the LuaDist stuff. But for the 'regular way',
> I don't think users should have to do 'require "cmakelua"'. That
> should be built in and ready to go because by definition, they are
> already using CMakeLua. That's the point of using the fork.
>
> In the context of using CMakeLua as a package for what you are after
> in LuaDist/LuaRocks, I think that is reasonable. But I was bothered by
> the notion that you might do 'require "CMakeLua"' to import helper API
> functions that are not standard API functions. CMakeLua *is* the
> project. If you want helper, non-canonical functions, then I suggest
> something like 'require "CMakeLuaConvenienceAPIs"' or something else
> distinct from "CMakeLua".

Yes I agree, I only used that package name to illustrate the point. As
for LuaDist, I only need cmake as package rather than standalone
executable nothing else :P.

>
> LuaBind, lua++:
>
> Have you looked at the actual code yet? One of the really cool things
> about it is that Ken managed to create the entire C++ API interface
> glue in basically one function. We should be rolling with that instead
> of trying to reinvent the binding. I also think LuaBind would be a
> mistake because of the Boost dependency. If we make CMakeLua to hard
> to build/install or too bloated, nobody will want to use our version.

The way its done now is elegant. We should use it and only modify and
add on convenience behavior such as the mentioned tables as parameters
on the lua side. This is more maintainable and requires less changes
to the C++ source so keeping it in sync with cmake will be easier.

>
> Also, I know Gonzolo spoke of trying to refactor the code in such a
> way that it would be easier for somebody else to bind a different
> language to CMake as well.
>
> So I get the sense that hand bindings will be the best way to go.

I agree.

> Features that might be reasonable (aka, I'm not a total prude,
> really!):
>
>
> So I noticed there has been a lot of activity concerning full blown
> Perl style Regular Expressions. This is something I think would be a
> good feature to add to CMakeLua. This is not an API thing easily added
> to existing CMake. This seems to be a popular feature that the
> community is demanding and it sounds like CMake is going to eventually
> add it. For us, it should be a trivial matter of including lrexlib.
>
> I know Roberto has also been talking about LPEG. I don't know anything
> about PEGs and barely remember context free grammars from college, but
> it sounds like the 'correct' way of doing things over regular
> expressions, and also sounds like Roberto might be placing it as a
> future killer feature of Lua. So I'm wondering if we should also
> include this in. I do worry about bloating our distribution too much
> though, though this implementation doesn't seem too big. Another
> possibility is to use this in lieu of lrexlib, but I don't think the
> regex part of the module does Perl-compatible style regex which is the
> problem. (But I could be mistaken because the page doesn't explicitly
> say.)
>

As with OO experiments .. I would wait with this after we have a
working binding.

> Anyway, I feel bad about saying 'no' so repeatedly to you guys. (Not
> that I am any authority and can actually prevent you guys from doing
> it anyway.) But my proposal is we start on the basics that we can
> agree on and produce a fully capable/usable implementation that lacks
> many of the 'frills' we've been discussing. Then after that 1.0, we
> re-evaluate and look to the future. My mentality is that I have seen
> too many projects fail because they were too ambitious to start with.
> I say we focus on the basics, and we have a chance of actually
> succeeding if we keep it simple.
>
> I have a task list I've been working on which I would hope is the
> subset of things we can agree on.

Eric, thanks for taking the initiative. I'm back to study the code
more so I can actually help with something.

pd

[rod...@rodsoft.org]

Hi, sorry being away for that long, lack of free time due to work.

On Sun, Dec 30, 2007 at 09:03:22PM -0800, E. Wing wrote:
>
> Okay, a lot has been discussed here. Here's my attempt to respond and
> move to the next stage:
>

> So I repeat myself, but I am under the feeling that we should keep API
> parity with official CMake. This means that for every API CMake has in
> its documentation, we also have. But this doesn't mean that we can't
> support Lua-isms, like supporting tables, or using return values. But
> for an API function in CMake, we need to have a corresponding one in
> Lua by the same name.

Completely agreed, couldn't express myself better. Although we could
differentiate between a backwards compatible syntax using the current
cmake API, and the new one which would be made in terms of the former.
We'll strive to a cleaner, object-oriented syntax on this one, and use it
for our Find* scripts (or something like that). This syntax should be made
the preferred one, since it'll convey the real Lua approach. There's no
point in using lua to keep writing in cmake-ish style.

> New API functions:
> So in the context of things like "static_library", I think this is a mistake.

Having decided to make an lua api on top of a cmake-ish api, we should
postpone any new api decision because it's irrelevant if you don't have a
functional cmake-ish api at hand. I think new API's are better when
they're are created based on an actual experience of an existing API. I
wouldn't say we have already that experience because we haven't dealt with
CMakeLua's cmake-ish api (this is getting complicate to express...).

So, along the way, let's collect all ideas and suggestions we might have,
maybe present to an outsider, maybe Brandon Van Every, since he seems to
have good understanding about build systems, and see what he thinks.

> Object oriented:

> On the other hand, I can kind of see where you are going with this
> one. I admit, it might be interesting to see if this could work fully
> across CMake without problems and if there is indeed an overall
> benefit. One area that I've been thinking about is the SET function.
> Since we have the assignment operator, it seems like providing a way
> to use the assignment operator would be reasonable. But since SET has
> parameters that classify the variable, maybe something OO might help
> here. (But then again, variables are currently not bound to Lua in the
> fork, and the variables already push into the C++ core, so
> thought/investigation needs to be done here.)

That's where I think some kind of OO would be valuable. One of the
problems I see everyday is people using one tool for one problem, whereas
sometimes a mix of two or more would be better. C++ (and Lua as well)
excels in allowing us to use different programming paradigms to solve one
problem. It's up to the programmer to use it wisely. So, a pure OO
implementation would be wrong for our case because a CMakeLua script
should be a SCRIPT, i.e., a collection of commands that does something.
That's procedural programming for me. We could sprinkle OO to ease some
things, as those set_* commands you've mentioned, but the script should
have a 'procedural' look, as if it were a recipe. Let's use OO where it is
logical to do so, nothing more.

But, as I (and you) said before, it's a discussion for the future, we have
bigger issues right now.

> LuaBind, lua++:
> Have you looked at the actual code yet? One of the really cool things
> about it is that Ken managed to create the entire C++ API interface
> glue in basically one function. We should be rolling with that instead
> of trying to reinvent the binding. I also think LuaBind would be a
> mistake because of the Boost dependency. If we make CMakeLua to hard
> to build/install or too bloated, nobody will want to use our version.

I didn't have enough time to peruse the cmakelua's source code, you know,
hollidays with family, lack of free time at work, etc. But as soon as I
deliver the project I'm working on I'll leave some time to work on this.

> I know Roberto has also been talking about LPEG. I don't know anything
> about PEGs and barely remember context free grammars from college, but
> it sounds like the 'correct' way of doing things over regular
> expressions, and also sounds like Roberto might be placing it as a
> future killer feature of Lua. So I'm wondering if we should also

I don't know if you know, but I work with some researchers at PUC/RJ and
IMPA (Institute of Pure and Applied Mathematics), and that's where Roberto
and Luiz Henrique (main Lua developpers) work. In fact I've met the latter
these days. If we happen to have any difficulty in designing our
implementation of CMakeLua or an idea for Lua, I can get in contact with
those guys personally. It would be easier, at least for me, because I
could discuss in Portuguese.

> Anyway, I feel bad about saying 'no' so repeatedly to you guys. (Not
> that I am any authority and can actually prevent you guys from doing
> it anyway.) But my proposal is we start on the basics that we can
> agree on and produce a fully capable/usable implementation that lacks
> many of the 'frills' we've been discussing. Then after that 1.0, we
> re-evaluate and look to the future. My mentality is that I have seen
> too many projects fail because they were too ambitious to start with.
> I say we focus on the basics, and we have a chance of actually
> succeeding if we keep it simple.

Don't need to apologise. The discussion has been very fruitful up till
now. We can't rush into conclusions without considering all aspects, and
at least for me is always good to hear another opinion, be it for or
against what I've said. What can't happen is something like: "let's do it
because I think it's better, although I can't prove it", or an infinit
loop on our argumentations (that's never good). As long as we're
progressing, pro and against arguments will leads us to a common
denominator, hopefully a good one.

Regards,
Rodolfo Lima.