More Variable bridging feedback/discussion
E. Wing
Sorry, I got obscenely busy the past few days. Thanks for all the
effort you've been putting in. So I have some feedback and thoughts.
Since that thread was getting kind of long and the subject header
doesn't match, I've created a new thread, but some of my replies are
connected.
<quote>
I couldn't stop me ;)
The the sandbox branch supports now the lua command (cmLuaCommand.cxx).
LuaTest/Command/CMakeLists.txt:
set(A a b c)
set(B a2b2c2)
lua(
"
setmetatable(_G, {
__index = function (_, n) return GetDefinition(n) end,
__newindex = function (_, n1, n2) AddDefinition(n1, n2) end})
"
)
lua("A = B")
<end quote>
That is very cool. Nice feature. Just so we minimize potential name
conflicts, I'm wondering if we should rename it something like
'lua_dostring'.
<quote>
The actual functions for __index and __newindex are too simple.
My idea is to declare all CMake variables before using them, similar
to the example in the Lua book:
http://www.lua.org/pil/14.2.html
<end quote>
So to be clear though, you are not advocating the explicit variable
declaration technique in PiL 14.2, just the indirection technique so
metatables can be used right? I think end users should decide for
themselves if they want to require explicit variable declaration as in
PiL 14.2. I think by default, we shouldn't force it, because newbie
Lua programmers who copy some code from tutorials are going to wonder
why things don't work.
<quote>
Then we could check before a variable-access if it is a cmake variable,
and only if it was declared as a cmake variable we use the C++ interface,
if not it is a plain lua call.
<end quote>
So, yes, I've been thinking along these lines too. I have still been
thinking about inverting the mechanism so that it is done through the
native CMake-script and C++ side so the Lua side remains natural and
unhindered. But more on that below.
<quote>
Ken Martin wrote:
> Does this mean that ALL variables accessed in Lua call GetDefinition? Does
> this basically replace the standard Lua variable referencing? Hmm
The cmake variables could be moved into a lua table:
# move the cmake variables into the cmake table
lua("
setmetatable(_G, nil)
setmetatable(cmake, {
__index = function (_, n) return GetDefinition(n) end,
__newindex = function (_, n1, n2) AddDefinition(n1, n2) end})
")
set(C a b c)
set(D a2b2c2)
lua("cmake.C = cmake.D")
<end quote>
I'm not sure what Ken's thinking/concerns are here, but I have a
couple of thoughts.
First, off the top, I'm not a fan of placing non-official variables
that were in global space into the cmake namespace (cmake.c, cmake.d).
I think it's a little too non-obvious. I think there was already
controversy on placing offiicial cmake functions in the cmake
namespace.
Second, I might say 'Hmmm' in general because as you pointed out,
certain things don't work in a Lua-natural way, e.g.:
<quote>
emptyTable = {}
print(emptyTable)
does not wotk, it prints nil instead of the address of the table.
<end quote>
Funneling variables through AddDefinitions has the bad side effect of
losing the type information (number, boolean, string) since everything
in CMake is a string. It would be nice if we didn't have to lose this.
Looking towards the distance future, to maybe a day where Lua becomes
the dominant language, I'm also wondering if we will be hindering
certain things. By claiming the metatable, it makes it harder for 3rd
parties to enforce their own metatable rules like PiL 14.2 since they
will have to understand how not to interfere with our metatables.
<quote>
But I think we should play around a bit and see what would be best.
<end quote>
Hoping that maybe these things could be easily addressed, I created a
new branch called 'sandbox2' (I hope I did it right). It forked from
your code base, but I don't set the metatables. Instead, I attempted
to do the evil variable bridging from the C++ side and basically
inverted the approach you took. Instead of Lua-side calls changing the
CMake-state on Lua set/get events, I flipped it so CMake calls would
change the Lua state (set/getglobal).
So for example, when a CMake script sets a variable and calls
AddDefinition, it also copies the variable to the Lua state via
lua_getglobal. Then if the Lua side tries to read the variable, it
will have the correct value. And when CMake script gets a value,
GetDefinition retrieves the value from the Lua state so it gets any
recent changes. By doing this, I hoped to keep the Lua side using
native mechanisms. I also do some extra work to help preserve data
types when sensible so not everything is a "string" in Lua.
Unfortunately, it doesn't work fully yet. It's a bit ugly and there
are some complexities in the code that make this harder to write
correctly. Some things are working, but it is not as far along as your
metatable solution.
Some of the problems I encountered:
- C++ initialization order: There are some global cmake variables that
are set early on like CMAKE_C_COMPILER_ID (in AddDefaultDefinitions())
that seem to get called before I can easily access the LuaState
(though the state has already been created, it's just that all the
relationships are not set so I can't reach the pointer when I need
it). This means the Lua state is missing certain key variables.
- const member functions: In the cases where GetDefinition is called
and there is a synchronization difference between the CMake-internal
variable value and the Lua value, I currently have rules to determine
which value is correct and pick the correct value. it would be
convenient to set the wrong value (which is usually the
CMake-internal), but since the function is const, I can't change it.
- Since I can't push Lua values on demand in this inverted technique,
It appears I was missing sync points because there were other entry
points. I think one was AddCacheDefinition()).
All my new code bits have #ifdef CMAKE_BUILD_WITH_CMAKELUA
guards around them. So if you want to take a look, I would appreciate it.
But looking at the complexity, I am strongly favoring your approach
since it works and is much less complex (unless there is something
obvious I'm missing in my experiment that fixes everything).
But to leverage more Lua power, I've been thinking about what I just
tried doing and PiL 14.2, and I'm thinking that maybe we can try
something where we copy all globals into a private Lua table so every
value is stored within Lua and preserves type information. But like my
experiment, there is also a check against the CMake-internal
representation (via Add/GetDefinition) and a resolver/resync if there
is a mismatch.
Anyway, those are my thoughts for the moment. A couple of more benign
things before I sign off:
- I checked into two fixes into your sandbox. There is a compiler fix
in cmLuaUtils that adds a #include "lua.h" now that you added inline
functions. And I restored the fancy argument handling in
LuaPublicAPIHelper.lua that was inadvertently broken.
- Also, on the topic of LuaPublicAPIHelper.lua, I would suggest that
we move your setmetatable code into this file. The sole purpose of
this script is to be run first, behind the scenes, before the user
starts scripting, so we can setup a Lua environment that 'just works'.
The setmetatable code is perfect for this module.
Thanks,
Eric
Peter Kümmel
> Hi Peter,
> Sorry, I got obscenely busy the past few days. Thanks for all the
> effort you've been putting in. So I have some feedback and thoughts.
> Since that thread was getting kind of long and the subject header
> doesn't match, I've created a new thread, but some of my replies are
> connected.
Hi Eric,
no problem, I'm glad not to be left alone here ;)
> That is very cool. Nice feature. Just so we minimize potential name
> conflicts, I'm wondering if we should rename it something like
> 'lua_dostring'.
Instead of renaming I think we could introduce the flag FILE do execute
a file instead of a string.
> So to be clear though, you are not advocating the explicit variable
> declaration technique in PiL 14.2, just the indirection technique so
> metatables can be used right? I think end users should decide for
> themselves if they want to require explicit variable declaration as in
> PiL 14.2. I think by default, we shouldn't force it, because newbie
> Lua programmers who copy some code from tutorials are going to wonder
> why things don't work.
Atm I'm not advocating anything, I just wanna say that it is possible
> I'm not sure what Ken's thinking/concerns are here, but I have a
> couple of thoughts.
>
> First, off the top, I'm not a fan of placing non-official variables
> that were in global space into the cmake namespace (cmake.c, cmake.d).
> I think it's a little too non-obvious. I think there was already
> controversy on placing offiicial cmake functions in the cmake
> namespace.
I agree.
>
> Second, I might say 'Hmmm' in general because as you pointed out,
> certain things don't work in a Lua-natural way, e.g.:
>
> <quote>
> emptyTable = {}
> print(emptyTable)
>
> does not wotk, it prints nil instead of the address of the table.
> <end quote>
>
> Funneling variables through AddDefinitions has the bad side effect of
> losing the type information (number, boolean, string) since everything
> in CMake is a string. It would be nice if we didn't have to lose this.
I've uploaded a new example which introduuced the declaration of CMakeString,
so the Lua types are still available.
> Looking towards the distance future, to maybe a day where Lua becomes
> the dominant language, I'm also wondering if we will be hindering
> certain things. By claiming the metatable, it makes it harder for 3rd
> parties to enforce their own metatable rules like PiL 14.2 since they
> will have to understand how not to interfere with our metatables.
I'm not sure if this will be a problem. Maybe it will help to
code this in C.
>
> <quote>
> But I think we should play around a bit and see what would be best.
> <end quote>
>
>
> Hoping that maybe these things could be easily addressed, I created a
> new branch called 'sandbox2' (I hope I did it right). It forked from
> your code base, but I don't set the metatables. Instead, I attempted
> to do the evil variable bridging from the C++ side and basically
> inverted the approach you took. Instead of Lua-side calls changing the
> CMake-state on Lua set/get events, I flipped it so CMake calls would
> change the Lua state (set/getglobal).
When it is clear how we manage the variables we could hard code in C but
atm it is too much trouble.
Yes, it looks a bit complex especially the synchronization stuff which
could become very hairy. And couldn't the most be done on the Lua side?
>
> But to leverage more Lua power, I've been thinking about what I just
> tried doing and PiL 14.2, and I'm thinking that maybe we can try
> something where we copy all globals into a private Lua table so every
> value is stored within Lua and preserves type information. But like my
> experiment, there is also a check against the CMake-internal
> representation (via Add/GetDefinition) and a resolver/resync if there
> is a mismatch.
>
>
> Anyway, those are my thoughts for the moment. A couple of more benign
> things before I sign off:
>
> - I checked into two fixes into your sandbox. There is a compiler fix
> in cmLuaUtils that adds a #include "lua.h" now that you added inline
> functions. And I restored the fancy argument handling in
> LuaPublicAPIHelper.lua that was inadvertently broken.
>
> - Also, on the topic of LuaPublicAPIHelper.lua, I would suggest that
> we move your setmetatable code into this file. The sole purpose of
> this script is to be run first, behind the scenes, before the user
> starts scripting, so we can setup a Lua environment that 'just works'.
> The setmetatable code is perfect for this module.
Yes, when the code is stable it should be loaded on startup.
I've also updated the sandbox branch to the actual cvs code. I've
tried it with tailer/git but it failed. Therefore I've extracted
our Lua changes (lua.patch) and replaced all the files with original
cvs ones. Seems there were changes unrelated to lua in our repo,
I've removed them. Currently the sandbox is a cvs checkout with
patches from lua.patch. Hope this makes updating a easier.
(this way I've also done it in windbus)
Could you please have a look on the sandbox code again if it contains
your LuaPublicAPIHelper.lua fixes.
Peter
Peter Kümmel
how the variables could be handled.
CMake's output is:
--
-- t1 : value from CMake's str
-- t2 : value from Lua's str
-- t3 : value from CMake's str visibale again
-- t4 : value from Lua's str overwritten
-- ref : table: 01F10718
-- T : unknown in cmake.
-- Configuring done
-- Generating done
And the relevant CMake/Lua code looks like this:
# use CMake values as fall back
lua("
CMAKE_VARIABLES = {}
CMakeString = function (n)
local val = rawget(_G, n)
if val ~= nil then
AddDefinition(n, tostring(val) ) -- init with Lua value
rawset(_G, n, nil) -- overwrite already exising names
end
CMAKE_VARIABLES[n] = 1
end
setmetatable(_G, {
__index = function (_, n)
local val = GetDefinition(n)
if val == nil then
return _G[n]
else
return val
end
end,
__newindex = function (_, n1, n2)
if CMAKE_VARIABLES[n1] == 1 then
AddDefinition(n1, n2)
else
rawset(_G, n1, n2)
end
end
})
")
set(str "value from CMake's str")
lua("
CMakeString \"t1\"
t1 = str -- str not known in Lua use CMake value
str = \"value from Lua's str\" -- now there is str in Lua
CMakeString \"t2\"
t2 = str -- use value of Lua's str
")
set(str "value from CMake's str visibale again")
lua("
str = nil -- make CMake's str visibile again without using CMakeString
CMakeString \"t3\"
t3 = str
")
lua("
str = \"value from Lua's str overwritten\"
CMakeString \"str\" -- now Lua's str overwrites CMakes's variable str
CMakeString \"t4\"
t4 = str
")
lua("
T = {}
ref = T
CMakeString \"ref\" -- ref = tostring(T)
")
message(STATUS "")
message(STATUS "t1 : ${t1}")
message(STATUS "t2 : ${t2}")
message(STATUS "t3 : ${t3}")
message(STATUS "t4 : ${t4}")
message(STATUS "ref : ${ref}")
message(STATUS "T : unknown in cmake. ${T}")
It shows that we always have two "worlds" Lua's and CMake's. And
we have to find the best way of using the variables of the other
world in the actual.
Peter
Peter Kümmel
> we have to find the best way of using the variables of the other
> world in the actual.
We need one data type in both worlds: not strings but lists.
Setting the value of a list in one world should be immediately
be visible in the other under the same name. I think this
is a must, if not one gets lost in all the visibility handling.
In CMake lists are space separated strings quoted or not.
In C++ the items are separated by ; but this is in
principle not visible in the CMake script (or I'm wrong?):
In:
set(list1 a v b h t g)
set(list2 "a v" "b h" t g)
message(STATUS "list1 ${list1}")
message(STATUS "list2 ${list2}")
Out:
-- list1 a;v;b;h;t;g
-- list2 a v;b h;t;g
I would say in this output we see an implementation detail,
not the way how the cmake script sees the list.
On the Lua side we should use integer indexed tables (arrays) as lists
(Chapter 19 The Table Library: http://www.lua.org/pil/index.html#19 )
And strings are tables with just one entry.
Then we need a mapping from C++ to Lua. This is do problem because
in C++ we only have ; separated lists.
The other direction is more difficult, because of all the possible tables,
which could be arrays but have also non-string as values. Before we push
a list to C++/CMake we have to check
1. if the table is a array and
2. if all values are strings,
only then it could be pushed to C++/CMake. In all other cases it is just
plain Lua, invisible in CMake.
We could still think about if it makes sense to declare shared lists.
For the first implementation we could use Lua helper functions. I assume
Eric already has done a lot for the (nested) list conversion. Then we need
some tests to see if it really works. And at the end we could implement
the Lua function in C and make it inaccessible from Lua.
OK, this is my idea about the variable handling.
Hope I've not overseen something.
Peter
E. Wing
> > guards around them. So if you want to take a look, I would appreciate it.
> >
> >
> > But looking at the complexity, I am strongly favoring your approach
> > since it works and is much less complex (unless there is something
> > obvious I'm missing in my experiment that fixes everything).
>
> Yes, it looks a bit complex especially the synchronization stuff which
> could become very hairy. And couldn't the most be done on the Lua side?
So the approach this took was not to impact the native Lua side.
Notice that no metatables are in use and CMake global variables appear
in Lua's native global space (_G).
You've already proved we can do this from the Lua side and yours works
much better. But I was hoping we could achieve the same without
altering the native Lua behavior with the expectation that in the
future, people may try to leverage Lua even more, and our bindings
won't interfere with that.
>
> I've also updated the sandbox branch to the actual cvs code. I've
> tried it with tailer/git but it failed. Therefore I've extracted
> our Lua changes (lua.patch) and replaced all the files with original
> cvs ones. Seems there were changes unrelated to lua in our repo,
> I've removed them. Currently the sandbox is a cvs checkout with
> patches from lua.patch. Hope this makes updating a easier.
> (this way I've also done it in windbus)
>
So I suck at patch and I have a lot of problems with patch working
correctly in heterogeneous environments. I would much rather continue
using the tools that distributed SCMs provide as this is what they
were designed to do.
But I think we screwed up the repository with the branches which is
giving me a hard time with merging/pushing. As I said, Mercurial
doesn't have Git's notion of lightweight branches. From what I can
tell, Mercurial branches are currently permanent and cannot be
deleted. This seems to be the number one complaint of Mercurial users
who compare to Git as branching is one area where Mercurial is
horribly complex and Git is intuitive.
So two administrative things:
1) Can we see what we can do to 'fix' the repository so it is only one branch?
2) Let's just create new repos at Assembla or whereever for sandbox
branches. They can be children of the main repo so merging will work.
Also, FYI, I boosted your main repo access to Owner from Member. You
should have total control over everything now.
Thanks,
Eric
E. Wing
>
> > It shows that we always have two "worlds" Lua's and CMake's. And
> > we have to find the best way of using the variables of the other
> > world in the actual.
>
> We need one data type in both worlds: not strings but lists.
>
> Setting the value of a list in one world should be immediately
> be visible in the other under the same name. I think this
> is a must, if not one gets lost in all the visibility handling.
>
> In CMake lists are space separated strings quoted or not.
> In C++ the items are separated by ; but this is in
> principle not visible in the CMake script (or I'm wrong?):
>
> In:
> set(list1 a v b h t g)
> set(list2 "a v" "b h" t g)
> message(STATUS "list1 ${list1}")
> message(STATUS "list2 ${list2}")
>
> Out:
> -- list1 a;v;b;h;t;g
> -- list2 a v;b h;t;g
>
> I would say in this output we see an implementation detail,
> not the way how the cmake script sees the list.
So I think it *should* be an implementation detail, but I think the
reality is that it is not just an implementation detail and there are
real world noticeable effects due to this. In CMake-script, I think
lists can either be separated by semi-colons or spaces, but there are
actually differences in behaviors depending on which you use in
certain cases (bugs of a sort). I know others have complained about
this too. I think this is one of the motivations for wanting to switch
to a language with 'real' data structures. But because of these
problems, I don't know if it is safe to automatically coerce the data
types across the bridge. There is an additional ambiguity problem
crossing from CMake->Lua where we have a string which contains
semi-colons and/or spaces, but the user intended it's use as a string.
For tables/arrays, I'm thinking of backing off on the automatic
coercion and maybe opting for a utility function (I've been planning
on adding a set of helper functions in a "cmakelua." namespace) that
let's the user specify if they want to convert.
Maybe the Lua->CMake for tables direction is not as bad in the table
case since there are fewer ambiguity problems. But I claim there are
subtle differences between spaces and semicolons. (I think semi-colons
are the 'safer' ones in my experience.) But if the table contains
strings that have semi-colons, (or spaces if that is our delimiter), I
think an ambiguity problem still may arise. (Item is split when it
shouldn't have been.) And then if we do a full-round trip,
Lua->CMake->Lua, depending if we coerce again (instead of using a
native Lua access), we could end up a table that doesn't look like the
original.
So this is why I'm starting to think about a helper function for
manual coercion for table/lists. Going from CMake->Lua, Lua can just
get the full string and the user can then decide to split the string
into a native table or not. Since no Lua modules exist yet, we can
probably document that users planning to export lists should provide a
CMake-stringified list for the time being. Maybe there is something we
can do to allow users to enable/disable automatic coercion, and also
specify the CMake->Lua and Lua->CMake directions independently.
But for strings, numbers, booleans, I think automatic coercion is
mostly doable and desirable.
Thanks,
Eric
Peter Kümmel
> So I think it *should* be an implementation detail, but I think the
> reality is that it is not just an implementation detail and there are
> real world noticeable effects due to this. In CMake-script, I think
> lists can either be separated by semi-colons or spaces, but there are
> actually differences in behaviors depending on which you use in
> certain cases (bugs of a sort). I know others have complained about
> this too. I think this is one of the motivations for wanting to switch
> to a language with 'real' data structures. But because of these
> problems, I don't know if it is safe to automatically coerce the data
> types across the bridge. There is an additional ambiguity problem
> crossing from CMake->Lua where we have a string which contains
> semi-colons and/or spaces, but the user intended it's use as a string.
On the C++ side space isn't used as separator, or I'm wrong?
And aren't semi-colons encoded when they are not a separator? Or maybe
cmake just does not support semi-colons in lists. Splitting the string
at the semi-colons should be enough.
> For tables/arrays, I'm thinking of backing off on the automatic
> coercion and maybe opting for a utility function (I've been planning
> on adding a set of helper functions in a "cmakelua." namespace) that
> let's the user specify if they want to convert.
Isn't it better when the user is not forced to specify anything?
I prefer a simple rule:
"All string lists live in both world. Nothing else."
Means all changes are is visible to Lua and CMake.
With good metafunctions it should be possible to guarantee this.
> Maybe the Lua->CMake for tables direction is not as bad in the table
> case since there are fewer ambiguity problems. But I claim there are
> subtle differences between spaces and semicolons. (I think semi-colons
> are the 'safer' ones in my experience.) But if the table contains
> strings that have semi-colons, (or spaces if that is our delimiter), I
> think an ambiguity problem still may arise. (Item is split when it
> shouldn't have been.) And then if we do a full-round trip,
> Lua->CMake->Lua, depending if we coerce again (instead of using a
> native Lua access), we could end up a table that doesn't look like the
> original.
We just have to check how it is done currently.
> So this is why I'm starting to think about a helper function for
> manual coercion for table/lists. Going from CMake->Lua, Lua can just
> get the full string and the user can then decide to split the string
> into a native table or not.
Again, I propose to only provide lists/tables in Lua, the spitting/joining
should always be done by our binding.
> Since no Lua modules exist yet, we can
> probably document that users planning to export lists should provide a
> CMake-stringified list for the time being. Maybe there is something we
> can do to allow users to enable/disable automatic coercion, and also
> specify the CMake->Lua and Lua->CMake directions independently.
>
>
> But for strings, numbers, booleans, I think automatic coercion is
> mostly doable and desirable.
On LuaForge is the stdlib module which already provides the basic, split,
join functions we need in our metatables. I've uploaded a simple example
which shows the basic conversion functionalities:
First the CMake output:
-- list examples
-- -------------
-- number of entries : 3
-- joined with "-" : a1-a2-a3
-- joined with " , " : b 1 , b 2 , b 3
-- join(";", reverse(concat): b 3;b 2;b 1;a3;a2;a1
and here the code:
set(cml1 a1 a2 a3)
set(cml2 "b 1" "b 2" "b 3")
lua("
require \"list\"
require \"string_ext\"
")
lua("
CMakeString \"cml1\"
CMakeString \"num\"
l1 = string.split(\";\", cml1)
num = #l1
")
lua("
CMakeString \"joined_list1\"
joined_list1 = string.join(\"-\", l1)
")
lua("
CMakeString \"joined_list2\"
l2 = string.split(\";\", cml2)
joined_list2 = string.join(\" , \", l2)
")
lua("
CMakeString \"joined_list3\"
joined_list3 = string.join(\";\", list.reverse(list.concat(l1, l2)) )
")
message(STATUS "")
message(STATUS "list exmaples")
message(STATUS "-------------")
message(STATUS "number of entries : ${num}")
message(STATUS "joined with \"-\" : ${joined_list1}")
message(STATUS "joined with \" , \" : ${joined_list2}")
message(STATUS "join(\";\", reverse(concat): ${joined_list3}")
message(STATUS "")
To make the module loading simple I just copied the std files from
CMakeLuaSandbox1/Utilities/lua/stdlib/modules
into the folder containing the cmake binary.
Peter
E. Wing
I haven't forgotten about this. My time just got sucked up with other
things and the Google Summer of Code stuff. I'm hoping to get back to
you soon on this and give you a real answer.
> On the C++ side space isn't used as separator, or I'm wrong?
I need to check on this.
> And aren't semi-colons encoded when they are not a separator? Or maybe
> cmake just does not support semi-colons in lists. Splitting the string
> at the semi-colons should be enough.
All I remember are that there are some breakdowns. I think I have a
comment in the FindSDL_sound.cmake module that got bit by the
difference between semi-colons and spaces. The FindSDL.cmake module
returned something with a semi-colon and something broke the way I
originally did something. There was some workaround I had to do which
might be in the comments.
> > For tables/arrays, I'm thinking of backing off on the automatic
> > coercion and maybe opting for a utility function (I've been planning
> > on adding a set of helper functions in a "cmakelua." namespace) that
> > let's the user specify if they want to convert.
>
> Isn't it better when the user is not forced to specify anything?
Yes, it is better, but only if it is 100% correct. If there are corner
cases where the conversion does something unexpected, then people get
into trouble and curse us.
So for stupid, trivial example, if I set a string in CMake such as:
SET(MYVAR "Good morning; it is a nice day.")
Then cross into Lua and try to do something like:
print(MYVAR)
I'm expecting a string to be printed. But instead I may see something like:
table: 0x1020e0
In this case, a string was just a string, not a list.
Anyway, I need to test your patch still, so I should hopefully have
more intelligent feedback soon.
But I did have one random thought recently concerning scoping. Does
CMake have variable scoping related to the depth of the
subdirectories? So if you declare a variable in a subdirectory, the
parent directory script doesn't see the variable? If so, does your
metatable system inherit this, or did we break it by trying to keep
Lua variables native? And if it does break it, what should we do about
it?
Thanks,
Eric
Peter Kümmel
> Yes, it is better, but only if it is 100% correct. If there are corner
> cases where the conversion does something unexpected, then people get
> into trouble and curse us.
>
> So for stupid, trivial example, if I set a string in CMake such as:
> SET(MYVAR "Good morning; it is a nice day.")
> Then cross into Lua and try to do something like:
> print(MYVAR)
> I'm expecting a string to be printed. But instead I may see something like:
> table: 0x1020e0
All my first examples make the cmake vars a string in lua, but I don't
think we should go this route.
cmake-lists are lists not strings, therefore in Lua we also should have lists
(tables) and we have to make a string out of the list when we wanna print it,
explicit or implicit (using the metamethod __tostring).
>
> In this case, a string was just a string, not a list.
A list with one entry. When we decide to only use lists then we
have never to check if it is a string or a list.
>
> Anyway, I need to test your patch still, so I should hopefully have
> more intelligent feedback soon.
>
> But I did have one random thought recently concerning scoping. Does
> CMake have variable scoping related to the depth of the
> subdirectories? So if you declare a variable in a subdirectory, the
> parent directory script doesn't see the variable? If so, does your
> metatable system inherit this, or did we break it by trying to keep
> Lua variables native? And if it does break it, what should we do about
> it?
I assume the scoping is done on the C++ map, and we use the same C++
getter/setter functions on this map as cmake, so I don't think we will
have problem there.
Peter
Ken Martin
> think we should go this route.
> cmake-lists are lists not strings, therefore in Lua we also should have
> lists
> (tables) and we have to make a string out of the list when we wanna print
> it,
> explicit or implicit (using the metamethod __tostring).
All variables in CMake are stored strings. There is no such thing as a first
class list on the C++ side for a variable. A list is just a semicolon
separated string. See cmMakefile::ExpandArguments (below) for how the
arguments to a CMake command are typically processed. Basically it expands
any variable references via cmMakefile::ExpandVariablesInString and then it
calls cmSystemTools::ExpandListArguments to break the string into multiple
arguments to be passed into the command.
Ken
void cmMakefile::ExpandArguments(
std::vector<cmListFileArgument> const& inArgs,
std::vector<std::string>& outArgs)
{
std::vector<cmListFileArgument>::const_iterator i;
std::string value;
outArgs.reserve(inArgs.size());
for(i = inArgs.begin(); i != inArgs.end(); ++i)
{
// Expand the variables in the argument.
value = i->Value;
this->ExpandVariablesInString(value, false, false, false,
i->FilePath, i->Line,
false, true);
// If the argument is quoted, it should be one argument.
// Otherwise, it may be a list of arguments.
if(i->Quoted)
{
outArgs.push_back(value);
}
else
{
cmSystemTools::ExpandListArgument(value, outArgs);
}
}
}
Peter Kümmel
Here the metamethods to handle the lists, now also print() works.
Peter
Input:
message(STATUS "")
message(STATUS "list metatable examples")
message(STATUS "-------------")
set(cmlist x1 x2 x3 x4)
lua("
print(\"-- table in lua after set(cmlist x1 x2 x3 x4): \")
print(cmlist)
print()
items = {tostring(#cmlist)}
lualist = {\"l1\", \"l2\", \"l3\", \"l4\", \"l5\"}
itemslua = {tostring(#lualist)}
")
lua("
lualist2 = lualist
lualist3 = cmlist
")
message(STATUS "items in cmlist : ${items}")
message(STATUS "items in lualist : ${itemslua}")
message(STATUS "set(cmlist ...) : ${cmlist}")
message(STATUS "lualist = {...} : ${lualist}")
message(STATUS "lualist2 = lualist : ${lualist2}")
message(STATUS "lualist3 = cmlist : ${lualist3}")
message(STATUS "")
Output:
-- list metatable examples
-- -------------
-- table in lua after set(cmlist x1 x2 x3 x4):
{1=x1,2=x2,3=x3,4=x4}
-- items in cmlist : 4
-- items in lualist : 5
-- set(cmlist ...) : x1;x2;x3;x4
-- lualist = {...} : l1;l2;l3;l4;l5
-- lualist2 = lualist : l1;l2;l3;l4;l5
-- lualist3 = cmlist : x1;x2;x3;x4
--
-- Configuring done
Peter Kümmel
>> All my first examples make the cmake vars a string in lua, but I don't
>> think we should go this route.
>> cmake-lists are lists not strings, therefore in Lua we also should have
>> lists
>> (tables) and we have to make a string out of the list when we wanna print
>> it,
>> explicit or implicit (using the metamethod __tostring).
>
> All variables in CMake are stored strings. There is no such thing as a first
> class list on the C++ side for a variable. A list is just a semicolon
> separated string.
And there are never semicolons in the list items?
> See cmMakefile::ExpandArguments (below) for how the
> arguments to a CMake command are typically processed. Basically it expands
> any variable references via cmMakefile::ExpandVariablesInString and then it
> calls cmSystemTools::ExpandListArguments to break the string into multiple
> arguments to be passed into the command.
>
> Ken
>
>
> void cmMakefile::ExpandArguments(
> std::vector<cmListFileArgument> const& inArgs,
> std::vector<std::string>& outArgs)
> {
> std::vector<cmListFileArgument>::const_iterator i;
> std::string value;
> outArgs.reserve(inArgs.size());
> for(i = inArgs.begin(); i != inArgs.end(); ++i)
> {
> // Expand the variables in the argument.
> value = i->Value;
> this->ExpandVariablesInString(value, false, false, false,
> i->FilePath, i->Line,
> false, true);
>
> // If the argument is quoted, it should be one argument.
> // Otherwise, it may be a list of arguments.
> if(i->Quoted)
> {
> outArgs.push_back(value);
> }
> else
> {
> cmSystemTools::ExpandListArgument(value, outArgs);
> }
> }
> }
Thanks for the pointer Ken, I will have a closer lock at this
code it in the next days.
Peter
E. Wing
I finally played a little more with the code. My big problem though is
I don't have stdlib or any its dependencies installed on my system so
I'm having problems running it. How difficult is it to embed
everything within CMake? Is this something we want to do? Will these
dependencies build on all platforms that CMake currently builds on?
Are the licenses compatible with distribution? How much bloat and
increase in build time will these incur?
Also, in your new code, can you explain what the "CMakeString" stuff
is doing. I think I am misinterpreting it. But I want to make sure
that global variables declared on the Lua-side are automatically
visible from the CMake-side too. (Two-way bridging.)
> > So for stupid, trivial example, if I set a string in CMake such as:
> > SET(MYVAR "Good morning; it is a nice day.")
> > Then cross into Lua and try to do something like:
> > print(MYVAR)
> > I'm expecting a string to be printed. But instead I may see something
> like:
> > table: 0x1020e0
>
> All my first examples make the cmake vars a string in lua, but I don't
> think we should go this route.
> cmake-lists are lists not strings, therefore in Lua we also should have
> lists
> (tables) and we have to make a string out of the list when we wanna print
> it,
> explicit or implicit (using the metamethod __tostring).
>
> >
> > In this case, a string was just a string, not a list.
>
> A list with one entry. When we decide to only use lists then we
> have never to check if it is a string or a list.
>
I think our thinking is reversed. So as Ken pointed out, everything is
a string in CMake not a list. There is no formal list type in CMake. I
think CMake just tries to automatically coerce the data into lists on
demand as needed (like FOREACH) by looking for semi-colons.
But in my example, "Good morning; it is a nice day.", this is not
supposed to be a list. It is just a sentence that happens to contains
semi-colons. My concern is that with automatic coercion on the Lua
side would be considered unexpected and undesirable behavior. For this
sentence (string), I will get a table with two entries which is not
what I want. I'm concerned there will be a whole bunch of areas that
we cannot patch. For example, what if the user uses the concatenation
operator:
joined_string = MYVAR .. ", good bye"
Or what if the user uses the string with the string library functions
that expect a string instead of a table?
My concern is that automatic coercion may produce the wrong results.
My gut feeling is that if the result can be wrong, we probably
shouldn't convert automatically and the user should manually opt-in to
do the conversion.
> > But I did have one random thought recently concerning scoping. Does
> > CMake have variable scoping related to the depth of the
> > subdirectories? So if you declare a variable in a subdirectory, the
> > parent directory script doesn't see the variable? If so, does your
> > metatable system inherit this, or did we break it by trying to keep
> > Lua variables native? And if it does break it, what should we do about
> > it?
>
> I assume the scoping is done on the C++ map, and we use the same C++
> getter/setter functions on this map as cmake, so I don't think we will
> have problem there.
>
Since functions are first class values like variables in Lua, this got
me thinking that functions will have different scoping behavior than
variables. I'm wondering what we should do about this.
Thanks,
Eric
E. Wing
Neat. Can you explain how you implemented this? I don't see it.
Thanks,
Eric
E. Wing
>
> Ken Martin wrote:
> >> All my first examples make the cmake vars a string in lua, but I don't
> >> think we should go this route.
> >> cmake-lists are lists not strings, therefore in Lua we also should have
> >> lists
> >> (tables) and we have to make a string out of the list when we wanna print
> >> it,
> >> explicit or implicit (using the metamethod __tostring).
> >
> > All variables in CMake are stored strings. There is no such thing as a
> first
> > class list on the C++ side for a variable. A list is just a semicolon
> > separated string.
>
> And there are never semicolons in the list items?
I'm a little confused by this question. My interpretation is that
there are no list items. Everything is just a string. Strings may
contain many semi-colons. Looking at AddDefintion and GetDefinition in
the C++ implementation, I think the strings are stored as-is, and
don't necessarily go through any processing to split strings by
semi-colons. I dumped a bunch of printf statements in the code and I
see my strings with the semi-colons still in the string; everything is
intact.
But I think when a piece of code is hit that expects a list (like
FOREACH), then those semi-colon splitter functions are called and the
strings are chopped up into lists for immediate use.
Thanks,
Eric
Peter Kümmel
> Hi Peter,
> I finally played a little more with the code. My big problem though is
> I don't have stdlib or any its dependencies installed on my system so
> I'm having problems running it. How difficult is it to embed
> everything within CMake? Is this something we want to do? Will these
> dependencies build on all platforms that CMake currently builds on?
> Are the licenses compatible with distribution? How much bloat and
> increase in build time will these incur?
I don't think this is problematic: MIT, copying and shipping some small
files, same including strategy like for your lua helper file.
> Also, in your new code, can you explain what the "CMakeString" stuff
> is doing. I think I am misinterpreting it. But I want to make sure
> that global variables declared on the Lua-side are automatically
> visible from the CMake-side too. (Two-way bridging.)
CMakeString checks if there is already a global symbol , and if not
it introduces the symbol as cmake variable, but currently the 'else'
is missing. But in my opinion CMakeString is obsolete, the importand
part starts at # list metafunctions.
>
>
>>> So for stupid, trivial example, if I set a string in CMake such as:
>>> SET(MYVAR "Good morning; it is a nice day.")
>>> Then cross into Lua and try to do something like:
>>> print(MYVAR)
>>> I'm expecting a string to be printed. But instead I may see something
>> like:
>>> table: 0x1020e0
>> All my first examples make the cmake vars a string in lua, but I don't
>> think we should go this route.
>> cmake-lists are lists not strings, therefore in Lua we also should have
>> lists
>> (tables) and we have to make a string out of the list when we wanna print
>> it,
>> explicit or implicit (using the metamethod __tostring).
>>
>>> In this case, a string was just a string, not a list.
>> A list with one entry. When we decide to only use lists then we
>> have never to check if it is a string or a list.
>>
>
> I think our thinking is reversed. So as Ken pointed out, everything is
> a string in CMake not a list. There is no formal list type in CMake. I
> think CMake just tries to automatically coerce the data into lists on
> demand as needed (like FOREACH) by looking for semi-colons.
Aren't there CMakeLists.txt files and not CMakeString.txt files.
I think the best is to support only one data type, like Lua only supports table.
And when you write CMake code you think about lists not about strings.
You mostly implicit manipulate lists not strings, the fact that lists
are stored as string is an implementation detail, which should not be
visible to cmake/lua, at least this is my idea about the design of CMake.
>
> But in my example, "Good morning; it is a nice day.", this is not
> supposed to be a list. It is just a sentence that happens to contains
> semi-colons. My concern is that with automatic coercion on the Lua
> side would be considered unexpected and undesirable behavior. For this
> sentence (string), I will get a table with two entries which is not
Yes, when it is necessary to support semicolons in strings then my example
code doesn't work. But do we need it? If yes we could encode the semicolon.
> what I want. I'm concerned there will be a whole bunch of areas that
> we cannot patch. For example, what if the user uses the concatenation
> operator:
> joined_string = MYVAR .. ", good bye"
We could eventually overload the .. operator, or force the user to use concat.
>
> Or what if the user uses the string with the string library functions
> that expect a string instead of a table?
Why should he use the string library when he wanna manipulate a list?
>
> My concern is that automatic coercion may produce the wrong results.
> My gut feeling is that if the result can be wrong, we probably
> shouldn't convert automatically and the user should manually opt-in to
> do the conversion.
I only see problems when converting CMake scripts to Lua, because there
is maybe some acrobatic string manipulation which was not intended by the
original cmake language. Again, I understand cmake as list-manipulation
language, with the ugly possibility to work on the data type representation
directly which should in principle not be visible to the user.
>>> But I did have one random thought recently concerning scoping. Does
>>> CMake have variable scoping related to the depth of the
>>> subdirectories? So if you declare a variable in a subdirectory, the
>>> parent directory script doesn't see the variable? If so, does your
>>> metatable system inherit this, or did we break it by trying to keep
>>> Lua variables native? And if it does break it, what should we do about
>>> it?
>> I assume the scoping is done on the C++ map, and we use the same C++
>> getter/setter functions on this map as cmake, so I don't think we will
>> have problem there.
>>
>
> Since functions are first class values like variables in Lua, this got
> me thinking that functions will have different scoping behavior than
> variables. I'm wondering what we should do about this.
Currently it is not clear to me if we get problems because of the scoping.
We will see when we have more complex examples.
Peter
Peter Kümmel
>> Here the metamethods to handle the lists, now also print() works.
>
> Neat. Can you explain how you implemented this? I don't see it.
Yes i forgot to also post the meta functions, here the code
with the important part:
# list metafunctions
# use CMake values as fall back
lua("
isStringList = function(x)
-- TODO
return true
end
setmetatable(_G, {
__index = function (_, n)
local val = GetDefinition(n)
if val == nil then
return _G[n]
else
-- return the CMake list as indexed table
return string.split(\";\", val)
end
end,
__newindex = function (_, n1, n2)
if isStringList(n1) then
-- only string lists are visible in CMake
AddDefinition(n1, string.join(\";\", n2))
else
rawset(_G, n1, n2)
end
end
})
")
Peter Kümmel
> On 3/18/08, Peter Kümmel <synth...@gmx.net> wrote:
>> Ken Martin wrote:
>>>> All my first examples make the cmake vars a string in lua, but I don't
>>>> think we should go this route.
>>>> cmake-lists are lists not strings, therefore in Lua we also should have
>>>> lists
>>>> (tables) and we have to make a string out of the list when we wanna print
>>>> it,
>>>> explicit or implicit (using the metamethod __tostring).
>>> All variables in CMake are stored strings. There is no such thing as a
>> first
>>> class list on the C++ side for a variable. A list is just a semicolon
>>> separated string.
>> And there are never semicolons in the list items?
>
> I'm a little confused by this question. My interpretation is that
> there are no list items. Everything is just a string. Strings may
> contain many semi-colons. Looking at AddDefintion and GetDefinition in
> the C++ implementation, I think the strings are stored as-is, and
> don't necessarily go through any processing to split strings by
> semi-colons. I dumped a bunch of printf statements in the code and I
> see my strings with the semi-colons still in the string; everything is
> intact.
Again, I think the basic data type is a list not a string. CMake isn't
that clear on this, and that is one of the weak points of the current
cmake macro language.
I assume, the map storing the values of all cmake values could also
be a map to lists
map<string, list<string> >
instead of a map to strings
map<string, string>
I wonder why the original code doesn't use a map to lists, it would make
things clearer. And I currently don't know any argument why we need
strings as basic data type.
>
> But I think when a piece of code is hit that expects a list (like
> FOREACH), then those semi-colon splitter functions are called and the
> strings are chopped up into lists for immediate use.
Yes, currently cmake mixes to different data types into one, and implicit
converts them.
>
> Thanks,
> Eric
>
> >
--
Peter Kümmel
Ken Martin
> there are no list items. Everything is just a string. Strings may
> contain many semi-colons. Looking at AddDefintion and GetDefinition in
> the C++ implementation, I think the strings are stored as-is, and
> don't necessarily go through any processing to split strings by
> semi-colons. I dumped a bunch of printf statements in the code and I
> see my strings with the semi-colons still in the string; everything is
> intact.
The basic data type is a string. When a function is invoked all of its
arguments are passed to it as a std::vector of strings. The vector then gets
expanded. Some examples
Foo (a;b;c d;e;f)
Has two arguments, almost all functions expand their arguments before
actually doing the work (there is generic code in cmCommand.h line 63 that
does this, most functions use this, a few override it and handle expansion
themselves). After expansion there will be six arguments.
Set (bar a b c) # four arguments
Foo (${bar}) # one argument before expansion, 3 after it
Foo ("a b c") # one argument both before and after expansion
Foo ("${bar}") # one argument both before and after expansion
Foo ("a b" c d) three arguments both before and after expansion
Foo (${bar};d;e) one argument before expansion, five after it
Foo ( a\;b ) Uhh, I've got nothing :) I think this is one argument both
before and after, I would have to look at the code to be sure
Foo ("a;b" c;d) two arguments before expansion, three after it
Hope that helps some
Ken
Peter Kümmel
>> I'm a little confused by this question. My interpretation is that
>> there are no list items. Everything is just a string. Strings may
>> contain many semi-colons. Looking at AddDefintion and GetDefinition in
>> the C++ implementation, I think the strings are stored as-is, and
>> don't necessarily go through any processing to split strings by
>> semi-colons. I dumped a bunch of printf statements in the code and I
>> see my strings with the semi-colons still in the string; everything is
>> intact.
>
> The basic data type is a string. When a function is invoked all of its
> arguments are passed to it as a std::vector of strings. The vector then gets
> expanded. Some examples
>
> Foo (a;b;c d;e;f)
>
> Has two arguments, almost all functions expand their arguments before
> actually doing the work (there is generic code in cmCommand.h line 63 that
> does this, most functions use this, a few override it and handle expansion
> themselves). After expansion there will be six arguments.
>
In Lua, with the list-only approach it would look like this,
assuming Foo could only take an arbitrary number of lists.
Before calling C++ the lists would be expanded into one string:
> Set (bar a b c) # four arguments
bar = {"a","b","c"}
> Foo (${bar}) # one argument before expansion, 3 after it
Foo (bar) -- pass one list as argument
> Foo ("a b c") # one argument both before and after expansion
Foo {"a b c"} -- one list with on item
> Foo ("${bar}") # one argument both before and after expansion
Foo {join(bar)} -- one list with one item, the joined bar
> Foo ("a b" c d) three arguments both before and after expansion
Foo ({"a b"}, {"c"}, {"d"}} -- three list with one items
> Foo (${bar};d;e) one argument before expansion, five after it
Foo (bar .. {"d", "e"}) -- one list as argument, concatenated by operator ".."
> Foo ( a\;b ) Uhh, I've got nothing :) I think this is one argument both
> before and after, I would have to look at the code to be sure
Foo {"a;b"} -- one list, semicolon could be encoded by the metafunctions
> Foo ("a;b" c;d) two arguments before expansion, three after it
Foo ({"a;b"}, {"c","d"}) -- two lists
With this examples it becomes clear that one big disadvantage using Lua
is the requirement for quotation marks.
Lua has only an advantage over the cmake syntax when calling a function
with a variable: Foo(bar) <-> Foo(${bar})
But this is one of the most used code, so it is maybe worth the effort.
Peter
Peter Kümmel
>> I'm a little confused by this question. My interpretation is that
>> there are no list items. Everything is just a string. Strings may
>> contain many semi-colons. Looking at AddDefintion and GetDefinition in
>> the C++ implementation, I think the strings are stored as-is, and
>> don't necessarily go through any processing to split strings by
>> semi-colons. I dumped a bunch of printf statements in the code and I
>> see my strings with the semi-colons still in the string; everything is
>> intact.
>
> The basic data type is a string.
I rather would describe it as
"The basic data type is a list, represented by a string"
And when using Lua we could take the chance to improve the type system.
> When a function is invoked all of its
> arguments are passed to it as a std::vector of strings. The vector then gets
> expanded. Some examples
>
> Foo (a;b;c d;e;f)
>
> Has two arguments, almost all functions expand their arguments before
> actually doing the work (there is generic code in cmCommand.h line 63 that
> does this, most functions use this, a few override it and handle expansion
> themselves). After expansion there will be six arguments.
>
> Set (bar a b c) # four arguments
> Foo (${bar}) # one argument before expansion, 3 after it
> Foo ("a b c") # one argument both before and after expansion
> Foo ("${bar}") # one argument both before and after expansion
> Foo ("a b" c d) three arguments both before and after expansion
> Foo (${bar};d;e) one argument before expansion, five after it
> Foo ( a\;b ) Uhh, I've got nothing :) I think this is one argument both
> before and after, I would have to look at the code to be sure
> Foo ("a;b" c;d) two arguments before expansion, three after it
>
> Hope that helps some
>
> Ken
>
>
> >
--
Peter Kümmel
Ryan C. Gordon
> The basic data type is a string. When a function is invoked all of its
> arguments are passed to it as a std::vector of strings. The vector then gets
> expanded. Some examples
lists of lists, not lists of strings (or rather, tables of various
sorts). As far as the end user goes, the CMake syntax is the same...
INSTALL(TARGETS mylib;mylib-static;mylib-testapp
RUNTIME DESTINATION bin
LIBRARY DESTINATION lib
ARCHIVE DESTINATION lib)
...but it looks more like this Lua representation internally...
install = {
targets = {
names = { "mylib", "mylib-static", "mylib-testapp" },
runtime_destination = "bin",
library_destination = "lib",
archive_destination = "lib",
},
}
(or something like that.)
I think that, as Erik noted, the semicolon could theoretically end up
in a string legitimately, so it's risking loss of information to
flatten to a string and then expand back to a list.
That being said, I think that's not something we should be changing
right now...it's much more valuable to focus on getting the Lua stuff
up and working well without touching unnecessary bits of CMake's
internals. If the Lua stuff is well-received, we can start talking
about getting things into tables, but I'm less interested in this
project collecting dust because we changed a lot of code that was
previously in a known-good state, got overly ambitious, or focused on
trivia instead of the specific issue at hand. Hell, I'm a little
scared at how much energy is spent on revision control right out of
the gate. :)
--ryan.
E. Wing
will work, so I'm willing to go along with it.
To make Peter's changes work, we are going to have to embed PCRE,
lrexlib, and Lua-stdlib. I started embedding the first two. I haven't
gotten around to the third, but the stdlib looks like it's all raw lua
scripts, so there is nothing to compile.
I have pushed them to:
http://hg.assembla.com/CMakeLuaSandbox4
This is a branch off the current mainline.
Notes:
- I am not building the posix stuff in lrexlib because I don't expect
Visual Studio or other non-posix systems to have the correct
dependencies.
- I am adding these libraries so they are built after the bootstrap
process, unlike Lua, but like the other utility modules.
- I am building these libraries as static. This avoids the dynamic
library path finding issue.
- I currently don't correctly handle the 'USE_SYSTEM_LIBRARIES'
option. Find modules for PCRE and LREXLIB will need to be written.
- We need to change the lua search paths for module finding. We're
going to need this for the Lua-stdlib scripts. We need to invent CMake
paths to be searched first (maybe just reuse the Modules directory).
And it is probably wrong to look in standard lua places since it is
not reliable to expect different systems to have the same Lua modules
installed, so we should cripple this. We should also make sure it is
easy for users to include their own scripts from their own source
trees.
- I needed to disable code in the Lua build system that set the executable build
directory. It was breaking the other utilities because all the
following utilities installed to the same place.
I have never used PCRE and lrexlib so somebody should inspect my
integration work. Once everybody's happy, we can push this into the
main tree.
And Peter, can you start merging in your changes?
Thanks,
Eric
E. Wing
>
>
> > The basic data type is a string. When a function is invoked all of its
> > arguments are passed to it as a std::vector of strings. The vector then
> gets
> > expanded. Some examples
>
> It would seem to me that, eventually, CMake should be thinking of
> lists of lists, not lists of strings (or rather, tables of various
> sorts). As far as the end user goes, the CMake syntax is the same...
>
> INSTALL(TARGETS mylib;mylib-static;mylib-testapp
> RUNTIME DESTINATION bin
> LIBRARY DESTINATION lib
> ARCHIVE DESTINATION lib)
>
>
> ...but it looks more like this Lua representation internally...
>
> install = {
> targets = {
> names = { "mylib", "mylib-static", "mylib-testapp" },
> runtime_destination = "bin",
> library_destination = "lib",
> archive_destination = "lib",
> },
> }
Something like this was also a goal. We were thinking about eventually
extending the fancy argument parsing code (already added) to handle
something like named parameters for functions, though 'install' would
probably still be a function and not a table in the above example.
> That being said, I think that's not something we should be changing
> right now...it's much more valuable to focus on getting the Lua stuff
> up and working well without touching unnecessary bits of CMake's
> internals. If the Lua stuff is well-received, we can start talking
> about getting things into tables, but I'm less interested in this
> project collecting dust because we changed a lot of code that was
> previously in a known-good state, got overly ambitious, or focused on
> trivia instead of the specific issue at hand. Hell, I'm a little
> scared at how much energy is spent on revision control right out of
> the gate. :)
Since the CMake API functions have already been bound (already with
some fancy argument handling), the variable bridging seemed like the
most critical thing we could tackle, because if we can pull it off, it
mostly eliminates the immediate need to translate every single
CMake-script. Peter's metatable approach also has a nice benefit of
not having to change much C++ code. With both functions and variables
in place, I think we will have 80% of what we need, and also have a
system people may be willing to try/adopt since they can opt-in to Lua
as gradually as they desire. The only trick is that it may impact the
syntax of how we do things which is why I preferred settling this
matter before writing too many scripts that depended on it, otherwise
if something needs to change, we need to rewrite everything.
As I said, I'm willing to go along with Peter's approach. So at this
point, he just needs to finalize the details and merge, and I think we
can start writing some real (test) build systems.
Thanks,
Eric
Peter Kümmel
> I've stated my concerns. Peter seems convinced that the table approach
> will work, so I'm willing to go along with it.
Great, then we could leave the example status!
>
> To make Peter's changes work, we are going to have to embed PCRE,
> lrexlib, and Lua-stdlib. I started embedding the first two. I haven't
> gotten around to the third, but the stdlib looks like it's all raw lua
> scripts, so there is nothing to compile.
We don't need all functionality from stdlib. All the examples work without
PCRE and rlrexlib, we only need base, table_ext, string_ext, and list.
So I'm not sure if we should start shipping it (camke uses expat); but if you
already done it, why not.
> I have pushed them to:
> http://hg.assembla.com/CMakeLuaSandbox4
> This is a branch off the current mainline.
>
>
> Notes:
> - I am not building the posix stuff in lrexlib because I don't expect
> Visual Studio or other non-posix systems to have the correct
> dependencies.
>
> - I am adding these libraries so they are built after the bootstrap
> process, unlike Lua, but like the other utility modules.
>
> - I am building these libraries as static. This avoids the dynamic
> library path finding issue.
Yes, we should not depend on the target system installations.
>
> - I currently don't correctly handle the 'USE_SYSTEM_LIBRARIES'
> option. Find modules for PCRE and LREXLIB will need to be written.
>
> - We need to change the lua search paths for module finding. We're
> going to need this for the Lua-stdlib scripts. We need to invent CMake
> paths to be searched first (maybe just reuse the Modules directory).
> And it is probably wrong to look in standard lua places since it is
> not reliable to expect different systems to have the same Lua modules
> installed, so we should cripple this. We should also make sure it is
> easy for users to include their own scripts from their own source
> trees.
We should just hard code the path to some where in the cmake installation.
We could also rename stdlib which ships with our CMake-Lua version.
>
> - I needed to disable code in the Lua build system that set the executable build
> directory. It was breaking the other utilities because all the
> following utilities installed to the same place.
>
>
> I have never used PCRE and lrexlib so somebody should inspect my
> integration work. Once everybody's happy, we can push this into the
> main tree.
>
> And Peter, can you start merging in your changes?
Into CMakeLuaSandbox4? Wy not working directly on CMakeLua?
Peter Kümmel
> I have never used PCRE and lrexlib so somebody should inspect my
> integration work. Once everybody's happy, we can push this into the
> main tree.
>
> And Peter, can you start merging in your changes?
OK, read again, then I push my changes into CMakeLua.
What was the status of cmLuaUtils.cxx and LuaUtils_RegisterFunc?
Could I apply my changes? What is with CMAKELUA_NO_NAMESPACE in
cmake.cxx? Do we need a cmake a cm_ table? Later all cmake functions
should be global.
Peter
Peter Kümmel
> I've stated my concerns. Peter seems convinced that the table approach
> will work, so I'm willing to go along with it.
>
> To make Peter's changes work, we are going to have to embed PCRE,
> lrexlib, and Lua-stdlib. I started embedding the first two. I haven't
> gotten around to the third, but the stdlib looks like it's all raw lua
> scripts, so there is nothing to compile.
>
> I have pushed them to:
> http://hg.assembla.com/CMakeLuaSandbox4
> This is a branch off the current mainline.
>
>
> Notes:
> - I am not building the posix stuff in lrexlib because I don't expect
> Visual Studio or other non-posix systems to have the correct
> dependencies.
>
> - I am adding these libraries so they are built after the bootstrap
> process, unlike Lua, but like the other utility modules.
>
> - I am building these libraries as static. This avoids the dynamic
> library path finding issue.
On Windows we need some where a
add_definition(-DPCRE_STATIC)
>
> - I currently don't correctly handle the 'USE_SYSTEM_LIBRARIES'
> option. Find modules for PCRE and LREXLIB will need to be written.
>
> - We need to change the lua search paths for module finding. We're
> going to need this for the Lua-stdlib scripts. We need to invent CMake
> paths to be searched first (maybe just reuse the Modules directory).
> And it is probably wrong to look in standard lua places since it is
> not reliable to expect different systems to have the same Lua modules
> installed, so we should cripple this. We should also make sure it is
> easy for users to include their own scripts from their own source
> trees.
>
> - I needed to disable code in the Lua build system that set the executable build
> directory. It was breaking the other utilities because all the
> following utilities installed to the same place.
>
>
> I have never used PCRE and lrexlib so somebody should inspect my
> integration work. Once everybody's happy, we can push this into the
> main tree.
>
> And Peter, can you start merging in your changes?
>
> Thanks,
> Eric
>
> >
--
Peter Kümmel
E. Wing
>
> E. Wing wrote:
> > I have never used PCRE and lrexlib so somebody should inspect my
> > integration work. Once everybody's happy, we can push this into the
> > main tree.
> >
> > And Peter, can you start merging in your changes?
>
> OK, read again, then I push my changes into CMakeLua.
Yes, that's fine. I made Sandbox4 because I wanted to be conservative
with the PCRE/lrexlib stuff to begin with. It's a lot of new files.
Once it gets absorbed into the mainline, it's there forever. (Even if
we delete the files, the revision history is there so the files are
effectively there forever.) If it's fine, and works for everybody, we
can push them into mainline.
> What was the status of cmLuaUtils.cxx and LuaUtils_RegisterFunc?
> Could I apply my changes?
Yes, I think you can apply. You had good fixes. I thought I had
already merged those though. But I could be wrong.
What is with CMAKELUA_NO_NAMESPACE in
> cmake.cxx? Do we need a cmake a cm_ table? Later all cmake functions
> should be global.
I think I added that because I didn't know how to fix the bug in
LuaUtils_RegisterFunc, which you ended up fixing. Ken's original code
put everything into cm_* instead of cmake.*. I wanted to change Ken's
implementation to do cmake.*, but because of the bug, it didn't work
right. So I added the CMAKELUA_NO_NAMESPACE so I could leave both
versions of the code in and wait for a fix (which you provided). Since
the cmake.* works now, we probably can gut the old cm_* stuff.
Thanks,
Eric
Peter Kümmel
> E. Wing wrote:
>> I have never used PCRE and lrexlib so somebody should inspect my
>> integration work. Once everybody's happy, we can push this into the
>> main tree.
>>
>> And Peter, can you start merging in your changes?
>
> OK, read again, then I push my changes into CMakeLua.
>
I've pushes attached changes, needs some fixes, because the examples
of my sandbox1 don't work any more.
Peter
E. Wing
> PCRE and rlrexlib, we only need base, table_ext, string_ext, and list.
> So I'm not sure if we should start shipping it (camke uses expat); but if
> you
> already done it, why not.
>
That's why I wanted you to check my work. All I did was see the errors
when I ran and the documentation of stdlib stating that it depends on
lrexlib, which in turn depends on PCRE.
But in defense of keeping the changes, we were talking about embedding
PCRE/lrexlib anyway because people have already been asking for PCRE
in regular CMake. This gives us a chance to leapfrog the official
distribution by leveraging an implementation that has been proven.
Thanks,
Eric
Peter Kümmel
> On 3/25/08, Peter Kümmel <synth...@gmx.net> wrote:
>> E. Wing wrote:
>>> I have never used PCRE and lrexlib so somebody should inspect my
>>> integration work. Once everybody's happy, we can push this into the
>>> main tree.
>>>
>>> And Peter, can you start merging in your changes?
>> OK, read again, then I push my changes into CMakeLua.
>
> Yes, that's fine. I made Sandbox4 because I wanted to be conservative
> with the PCRE/lrexlib stuff to begin with. It's a lot of new files.
> Once it gets absorbed into the mainline, it's there forever. (Even if
> we delete the files, the revision history is there so the files are
> effectively there forever.) If it's fine, and works for everybody, we
> can push them into mainline.
As already said, I'm not sure if we really need pcre. Therefore I would
wait before we add it.
>
>> What was the status of cmLuaUtils.cxx and LuaUtils_RegisterFunc?
>> Could I apply my changes?
>
> Yes, I think you can apply. You had good fixes. I thought I had
> already merged those though. But I could be wrong.
Working on one repository makes things easier.
>
> What is with CMAKELUA_NO_NAMESPACE in
>> cmake.cxx? Do we need a cmake a cm_ table? Later all cmake functions
>> should be global.
>
> I think I added that because I didn't know how to fix the bug in
> LuaUtils_RegisterFunc, which you ended up fixing. Ken's original code
> put everything into cm_* instead of cmake.*. I wanted to change Ken's
> implementation to do cmake.*, but because of the bug, it didn't work
> right. So I added the CMAKELUA_NO_NAMESPACE so I could leave both
> versions of the code in and wait for a fix (which you provided). Since
> the cmake.* works now, we probably can gut the old cm_* stuff.
Even when I undef CMAKELUA_NO_NAMESPACE my examples don't work any more,
I'll have to look at this in the next days.
But I'm glad to see progress. ;)
Cheers,
Peter
Peter Kümmel
Yes, a good regex support could be a nice "unique selling point" :)
--
Peter Kümmel
E. Wing
> I've pushes attached changes, needs some fixes, because the examples
> of my sandbox1 don't work any more.
I don't have time to really look at it at the moment, but the fancy
argument helpers must be flipped to match whether we are using cm_* or
cmake.*. The code you checked in was still on cm_*. I'm assuming you
flipped it to cmake.*, so I made the change in LuaPublicAPIHelper.lua
to match this and just pushed in the result. You might try it now and
see if it works.
Thanks,
Eric
E. Wing
> > PCRE/lrexlib anyway because people have already been asking for PCRE
> > in regular CMake. This gives us a chance to leapfrog the official
> > distribution by leveraging an implementation that has been proven.
>
> Yes, a good regex support could be a nice "unique selling point" :)
>
So should I merge this in, or should I hold off?
Thanks,
Eric
Peter Kümmel
Let us at least wait until the other stuff has a bit stabilized.
But I still think this is very nice extension to cmake.
Peter
Peter Kümmel
I also tried with cmake.*, but no success. But it is not a big problem,
time is.
Peter
Peter Kümmel
> We don't need all functionality from stdlib. All the examples work without
> PCRE and rlrexlib, we only need base, table_ext, string_ext, and list.
> So I'm not sure if we should start shipping it (cmake uses expat); but if you
> already done it, why not.
expat is for xml so it's not redundant to add pcre.
Peter
E. Wing
> I also tried with cmake.*, but no success. But it is not a big problem,
> time is.
>
isn't going to work with the way I flipped the helper module. So I
flipped the C++ to match. Everything should use the cmake.* namespace
now.
Testing against the original Tests/Lua/CMakeLists.lua script,
everything is working now.
Testing against your stuff in Tests/Lua/Command shows breakage.
Looking at the line numbers, it appears that your inline "lua()"
commands are not working. I don't actually know how you implemented
this originally, but I would guess that this block of code did not get
merged in. This only existed in your branch, so you'll need to look at
it.
The remaining errors I get are due to the lack of having stdlib
installed. You'll have to verify these.
Thanks,
Eric
E. Wing
> Looking at the line numbers, it appears that your inline "lua()"
> commands are not working. I don't actually know how you implemented
> this originally, but I would guess that this block of code did not get
> merged in. This only existed in your branch, so you'll need to look at
> it.
> The remaining errors I get are due to the lack of having stdlib
> installed. You'll have to verify these.
Oh, wait, I just remembered that the first pass never worked for me. I
have to run the generation twice before it works. (I reported this
before.) After doing this, I only seem to get errors about the missing
stdlib. So the mainline branch now behaves like your branch has always
behaved for me. So I think the merge was successful.
I'm speculating that the registerMemberFunction()'s aren't getting
called early enough or at all in the first pass. I recall when adding
the LuaHelper code, I had some kind of initialization order problem so
I think I had to move it to where it is now (cmake.cxx) which was
earlier in the start-up process. Maybe the registerMemberFunction is
a similar problem. Can you look into this?
Thanks,
Eric
Peter Kümmel
hg push ;)
> >
--
Peter Kümmel
Peter Kümmel
I mean pull.
Now stdlib is in Modules/lua, also LuaPublicAPIHelper.lua.
In LuaPublicAPIHelper.lua I set the correct search path and load
the required stdlibs.
I've also fixed the generators for Studio.
>
>
>
--
Peter Kümmel
Peter Kümmel
> Now stdlib is in Modules/lua, also LuaPublicAPIHelper.lua.
> In LuaPublicAPIHelper.lua I set the correct search path and load
> the required stdlibs.
Now we can start to add the metatable stuff to LuaPublicAPIHelper.lua.
This file is always loaded first.
Peter
Peter Kümmel
Done. We can start with converting real world examples.
I've also changed cmakelua to look first for CMakeLists.lua.
This way we can have .lua and .txt CMakeLists in the same folder.
And converting the tests would become very simple.
For the most simple test (Test/simple/CMakeLists.lua) we will have:
-- a simple test case
project {"Simple"}
add_executable {"Simple", "simple.cxx"}
add_library {"simpleLib", "STATIC",
"simpleLib.cxx",
"simpleCLib.c",
"simpleWe.cpp"
}
target_link_libraries {"Simple", "simpleLib"}
-- make sure optimized libs are not used by debug builds
if tostr(CMAKE_BUILD_TYPE) == "Debug" then
target_link_libraries{"Simple", "optimized", "c:/not/here.lib"}
end
Maybe we could convert automatically and see how far this approach takes us.
Happy converting,
Peter
E. Wing
>
> Peter Kümmel wrote:
> > Peter Kümmel wrote:
> > > Now stdlib is in Modules/lua, also LuaPublicAPIHelper.lua.
> >> In LuaPublicAPIHelper.lua I set the correct search path and load
> >> the required stdlibs.
> >
> > Now we can start to add the metatable stuff to LuaPublicAPIHelper.lua.
> > This file is always loaded first.
>
> Done. We can start with converting real world examples.
> I've also changed cmakelua to look first for CMakeLists.lua.
> This way we can have .lua and .txt CMakeLists in the same folder.
> And converting the tests would become very simple.
Very nice. I'm wondering though, how hard would it be to provide a
runtime switch (or even CMake menu option) that allows toggling the
preference order. I'm thinking that if a large project that is in-use
tries porting, the project may try to keep the existing the existing
build scripts side-by-side with the new experimental stuff. Being able
to swap the preference order would be handy.
> add_library {"simpleLib", "STATIC",
So we should probably finalize what we want to do for the CMake
keyword constants. We were talking about providing variables so users
don't have to type the quotes. (Compiler might more easily catch typos
too.)
The problem is that our keyword variables will be global and users
can't reuse these variable names. I had suggested namespacing them
like:
cmake.STATIC,
but maybe it's too verbose? Also the casing mismatch between the
namespace and constant looks a little funny.
Thanks,
Eric
E. Wing
not being built by bootstrap and thus creates undefined symbol errors.
Any idea what caused this to break? I believe I had successfully
bootstrapped earlier today.
Thanks,
Eric
E. Wing
Okay, I added cmLuaUtils to the bootstrap file and it bootstraps
again. I still don't know why it suddenly broke.
Thanks,
Eric
Peter Kümmel
Do we need bootstraping at this satge?
How do I check if bootstraping works?
--
Peter Kümmel
Peter Kümmel
> On 3/26/08, Peter Kümmel <synth...@gmx.net> wrote:
>> Peter Kümmel wrote:
>>> Peter Kümmel wrote:
>>>> Now stdlib is in Modules/lua, also LuaPublicAPIHelper.lua.
>>>> In LuaPublicAPIHelper.lua I set the correct search path and load
>>>> the required stdlibs.
>>> Now we can start to add the metatable stuff to LuaPublicAPIHelper.lua.
>>> This file is always loaded first.
>> Done. We can start with converting real world examples.
>
>> I've also changed cmakelua to look first for CMakeLists.lua.
>> This way we can have .lua and .txt CMakeLists in the same folder.
>> And converting the tests would become very simple.
>
> Very nice. I'm wondering though, how hard would it be to provide a
> runtime switch (or even CMake menu option) that allows toggling the
> preference order. I'm thinking that if a large project that is in-use
> tries porting, the project may try to keep the existing the existing
> build scripts side-by-side with the new experimental stuff. Being able
> to swap the preference order would be handy.
Shouldn't be that hard.
>
>> add_library {"simpleLib", "STATIC",
>
> So we should probably finalize what we want to do for the CMake
> keyword constants. We were talking about providing variables so users
> don't have to type the quotes. (Compiler might more easily catch typos
> too.)
>
> The problem is that our keyword variables will be global and users
> can't reuse these variable names. I had suggested namespacing them
> like:
> cmake.STATIC,
> but maybe it's too verbose? Also the casing mismatch between the
> namespace and constant looks a little funny.
I think about some helper tables
- ts, short for toString. type(ts.X) == "string"
It looks for a value in cmake and returns its string:
ts.STATIC -- if there is a STATIC variable in cmake then return its value
-- otherwise return "STATIC"
- tb, short for toBoolean, type(tb.X) == "boolean"
tb.HAS -- return true if the cmake variable HAS == "TRUE" or HAS=="1"
-- return false if the cmake variable HAS == "FALSE" or HAS==""
-- otherwise return nil
- tn, short for toNumber, type(tn.X) == number
tn.COUNT -- if COUNT is a number return the value otherwise nil
With this we could write:
add_library {"simpleLib", ts.STATIC,
"simpleLib.cxx",
"simpleCLib.c",
"simpleWe.cpp"
}
if ts.CMAKE_BUILD_TYPE == "Debug") then
end
if tb.CMAKE_HAS_VTK then
end
--
Peter Kümmel
E. Wing
>
> E. Wing wrote:
> > Odd, bootstrap is no longer working for me. It appears cmLuaUtils is
> > not being built by bootstrap and thus creates undefined symbol errors.
> > Any idea what caused this to break? I believe I had successfully
> > bootstrapped earlier today.
I've come up with a rationalization. Since the cmake.* namespace stuff
was disabled before, I think the RegisterFunction stuff was unused. I
may have never bootstrapped under the condition where it was enabled
before.
> Do we need bootstraping at this satge?
> How do I check if bootstraping works?
We always need bootstrapping. It is for people that have downloaded
the CMake source and don't have a (previous) version of CMake
installed. CMake needs to first build a subset of itself so it can be
used to build the entire CMake source.
I don't know about Visual Studio, but on Unixy platforms, you run the
'bootstrap' script first. Then you run 'make'. It feels very
autoconfy.
Thanks,
Eric
E. Wing
The relocation of LuaPublicAPIHelper.lua and the Lua stdlib
(extension) were not being installed by the CMake 'make install'. This
requires explicit opt-in from CMakeLists.txt. I've added the
appropriate CMakeLists.txt files and instruct the system to install
*.lua files. I do not (yet) install the other files (like the README
and *.html files, etc). I'm not sure if I should bother or not.
Some changes that should be made that revealed themselves to me while
trying to track down the above problem:
- The failure to find LuaPublicAPIHelper.lua should probably result in
an explicit error message, and probably failure/abort. (Something I
probably should have done earlier on when I added the code.)
- Error messages on the argument parsing ('must require a single
parameter of a table') should probably use the lua debug API to
specify the file and line number of the error. The current message is
not specific enoughl. (I think this was also my doing :P)
- New support functions like 'isStringList' should be encapsulated in
its own namespace (I suggest cmakelua.*) and follow the CMake coding
style of all_lower_case_with_underscores.
Thanks,
Eric
Peter Kümmel
> add_library {"simpleLib", "STATIC",
>
> So we should probably finalize what we want to do for the CMake
> keyword constants. We were talking about providing variables so users
> don't have to type the quotes. (Compiler might more easily catch typos
> too.)
>
> The problem is that our keyword variables will be global and users
> can't reuse these variable names. I had suggested namespacing them
> like:
> cmake.STATIC,
> but maybe it's too verbose? Also the casing mismatch between the
> namespace and constant looks a little funny.
When someone writes add_library {"simpleLib", "STATIC", "file.cpp")
he definitely wanna add a static lib, then why not give him a
special function:
add_static_library("simpleLib", "file.cpp")
The extra argument is only needed for generic usage:
add_library {simpleLib ${lib_type} code.cpp)
or
add_library {"simpleLib", lib_type, "file.cpp")
but then there is no quoted argument for the lib type argument.
Peter
E. Wing
> > add_library {"simpleLib", "STATIC",
> >
> > So we should probably finalize what we want to do for the CMake
> > keyword constants. We were talking about providing variables so users
> > don't have to type the quotes. (Compiler might more easily catch typos
> > too.)
> >
> > The problem is that our keyword variables will be global and users
> > can't reuse these variable names. I had suggested namespacing them
> > like:
> > cmake.STATIC,
> > but maybe it's too verbose? Also the casing mismatch between the
> > namespace and constant looks a little funny.
>
so much. This is Lua, so the user can easily create their own shorter
name alias if they choose.
> When someone writes add_library {"simpleLib", "STATIC", "file.cpp")
> he definitely wanna add a static lib, then why not give him a
> special function:
>
> add_static_library("simpleLib", "file.cpp")
>
>
> The extra argument is only needed for generic usage:
>
> add_library {simpleLib ${lib_type} code.cpp)
> or
> add_library {"simpleLib", lib_type, "file.cpp")
>
> but then there is no quoted argument for the lib type argument.
>
So I know this discussion has appeared on this list before. I don't
remember if it was you or somebody else. But my feeling is that this
type of thing we shouldn't change. This is a general CMake API issue,
not really a CMakeLua issue. I suggest pitching the idea to the
mainstream CMake users. If they adopt it, then we inherit it. But
otherwise, we should focus on CMakeLua-related issues.
That aside, my personal instinct is that too many API functions that
are similar and overlap functionality make things more confusing and
not easier. There is already enough confusion over existing CMake APIs
that seem redundant, even with those where one is marked deprecated
(such as SUBDIRS and ADD_SUBDIRECTORY).
Furthermore, in the ADD_LIBRARY case, I find that the real-world usage
tends to use a GUI variable for STATIC|DYNAMIC|MODULE because
end-users like/need to change things. So I don't think adding this
specific alias is much of a win since most real-world cases need to
use the general form anyway.
Thanks,
Eric
E. Wing
> It looks for a value in cmake and returns its string:
> ts.STATIC -- if there is a STATIC variable in cmake then return its value
> -- otherwise return "STATIC"
>
> - tb, short for toBoolean, type(tb.X) == "boolean"
> tb.HAS -- return true if the cmake variable HAS == "TRUE" or HAS=="1"
> -- return false if the cmake variable HAS == "FALSE" or HAS==""
> -- otherwise return nil
>
> - tn, short for toNumber, type(tn.X) == number
> tn.COUNT -- if COUNT is a number return the value otherwise nil
Interesting idea. I need to think about it some more, but I think it
is a good idea. My inclination is to spell out the functions though
instead of the abbreviations. Also, my inclination is to embed this
within the cmakelua.* namespace as well.
> if ts.CMAKE_BUILD_TYPE == "Debug") then
> end
>
> if tb.CMAKE_HAS_VTK then
> end
Hmmm, is this one of those areas where treating everything as tables
instead of strings gets us into trouble? Were we going to do something
special in the case of a table with one item, i.e. make it string? I
was originally thinking that the scripter should just be able to
write:
if CMAKE_BUILD_TYPE == "Debug" then end
(Note, we may need to document case-sensitivity issues about Debug vs
debug vs DEBUG.)
or maybe:
if cmake.CMAKE_BUILD_TYPE == "Debug" then end
Maybe we can/should make an exception for all official/special CMake
variables like this and make sure they are automatically bridged as
Lua strings.
And for the boolean type, how bad would automatic coercion be instead
of manual opt-in. (I expect the tables to be more dangerous than the
booleans.)
Finally, I saw this posting on the main mailing list (a way to iterate
through all variables):
get_cmake_property( P VARIABLES )
foreach( VAR in ${P} )
message( STATUS " ${VAR}=${${VAR}}" )
endforeach()
I was actually already thinking about this so it was a coincidence of
sort. But I'm thinking the ability to iterate through all variables is
a nice thing for debugging and potential things we may do. (For
example, I iterate through all CMake API functions to enable fancy
argument parsing.)
So two thoughts. First, do you think you would be able to supply an
iterator function that works with the current metatable magic (calls
to GetDefinition) to allow us to iterate through all variables?
Second, I'm wondering if special CMake variables should be special (as
I allude to in the previous section). Maybe these reside in the
cmake.* table so the user can iterate through that table via pairs()
to find those.
Thanks,
Eric
Peter Kümmel
Yes, this is much better that being forced to use a helper function/table.
And my "everthing is a list" approach is mabye not the best solution to the problem.
Lua is dynamically typed, why not also the values we get from cmake.
I've changed the metafunction to return a string when there is only one item
in the cmake list, by this we could, as you suggested, write
if CMAKE_BUILD_TYPE == "Debug" then end
Could be tested with
print(CMAKE_ROOT)
in
/CMakeLua/Tests/Simple/CMakeLists.lua
--
Peter Kümmel
E. Wing
I'm still here too, just busy also. Your change looks good. I think
this is the way to go.
I started moving some directories around. I'm creating a new test
directory call LuaTests that should roughtly mirror the Tests
directory. The idea is we will have a Lua test suite that is separate
from the regular one. We want to make sure we preserve existing
functionality, so we don't want to break the original tests. I started
writing the CMakeLists.lua script for that directory, but it is
untested and incomplete. But I checked it in anyway.
I also renamed some things in the LuaPublicHelperAPI to avoid
potential namespace clashes. I placed things either in the cmake or
cmakelua namespace. The former is for cmake official things. The
latter is for helper things that are only relevant to CMakeLua. I also
renamed variables/functions to conform to the cmake_syntax_style
(underscores instead of camelCase). I think we probably should do
something to obscure the AddDefinitions and GetDefinitions functions
so people don't accidentally clash with these.
So on my immediate to do list:
- I think we still need an iterator so we can iterate through all
CMake built-in variables like the example I posted before. And maybe
more generally, a way to iterate through all variables in the CMake
definitions list. What do you think?
- Need to fill out CMake convenience constants such as:
cmake.STATIC = "STATIC"
cmake.DYNAMIC = "DYNAMIC"
cmake.MODULE = "MODULE"
cmake.DEBUG = "Debug"
cmake.RELEASE = "Release"
(Note, these are the ones in the helper api right now.)
- Need to start looking at porting some of the common MACRO functions
in CMake unless there is a clever way we can bridge these too. I
started looking at CheckSizeType. It was more involved than what I had
time for unfortunately.
Incidentally, I'm wondering if we need to worry about the case where
we start in a classic CMake-script CMakeLists.txt and INCLUDE a macro
from a file that has both a classic and Lua implementation, but the
Lua one gets invoked instead of the classic. This is a problem since
we can't bridge functions/macros.
Thanks,
Eric
E. Wing
cmakelua.toboolean
cmakelua.isboolean
cmakelua.isnumber
cmakelua.tonumber
cmakelua.totable
The number functions are just aliases to the standard tonumber, but,
the boolean functions take into account the different values for
true/false such as true/false, yes/no, 1/0, y/n, notfound, on/off.
While trying to port the CheckSizeTypes macro, I noticed that it was a
consumer of automatic coercion between strings and tables (liked to
use FOREACH). I realized that our code that decides between string or
table based on number of entries will require extra checking if you
want to indiscriminately use a Lua for-loop because for some
variables, it is impossible to distinguish between a list of one and a
string. So I added cmakelua.totable which will automatically convert a
single string into a table or return the untouched table if already a
table so we can more easily support this paradigm.
I also decided to get more aggressive, so in the __index metamethod, I
try to automatically coerce to boolean or number as well as string.
There is also code to traverse through a table to coerce its
individual items, though I haven't tested this code yet.
This also got me thinking/wondering if we need a __tostring
metamethod, plus maybe the others (e.g. __tonumber, __toboolean,
__totable), not that they will probably be respected.
Finally, while going through CheckSizeTypes, I remembered that we
still need to handle return values. TRY_COMPILE returns a RESULT
through the first variable and also has an OUTPUT_VARIABLE parameter
which returns the build process output. I still don't what Ken's
original binding does in this case.
Thanks,
Eric
Ken Martin
> still need to handle return values. TRY_COMPILE returns a RESULT
> through the first variable and also has an OUTPUT_VARIABLE parameter
> which returns the build process output. I still don't what Ken's
> original binding does in this case.
I override the default LuaFunction for the command. Basically I add a
temporary variable argument to the command invocation and then retrieve the
value from it after the invocation and return it to Lua. I only did this for
one command as an example to verify it would work, specifically from
cmGetpropertyCommand.cxx
static int cmGetPropertyLua(lua_State *L)
{
// build a list file function
cmListFileFunction lff;
lff.Name = lua_tostring(L, lua_upvalueindex(1));
// stick in a temp var
lff.Arguments.push_back
(cmListFileArgument("__GET_PROPERTY_LUA_TEMP", false, 0, 0));
int i;
int top = lua_gettop(L);
for (i = 1; i <= top; i++)
{
const char *arg = lua_tostring(L, i);
lff.Arguments.push_back(cmListFileArgument(arg, false,
0, 0));
}
// get the current makefile
lua_pushstring(L,"cmCurrentMakefile");
lua_gettable(L, LUA_REGISTRYINDEX);
cmMakefile *mf = static_cast<cmMakefile *>(lua_touserdata(L,-1));
// pass it to ExecuteCommand
mf->ExecuteCommand(lff);
// get the return value
const char *result = mf->GetDefinition("__GET_PROPERTY_LUA_TEMP");
lua_pushstring(L, result);
return 1; /* number of results */
}
// special lua code
cmGetPropertyCommand::cmGetPropertyCommand()
{
this->LuaFunction = cmGetPropertyLua;
}