Trying to develop some way to share work between our projects
Jon Akhtar
of the same challenges.
I would just like to open up a dialog on this to see if a meeting of
the minds is possible.
I have considered Metalua before - but I am not quite sure what
direction I want to explore that in yet. I' like the idea of our
projects developing a way for Lua users to write their own code
analysis extensions using Lua - I think that would be good for
everyone overall, and it could be something that could evolve on its
own.
Beyond that. There are still some IDE features that I support, but
don't really want to be part of the problem so to speak by having a
NIH type attitude.
(Fair warning - these developer tool problems are not only time
consuming,. I just really enjoy working them.)
Ok, back to the example.
In the list of common IDE features you have: Code Completion,
Documentation, and Parameter Information.
For example. I will show you how I supply these things for the common
Lua libraries.
My main goals were 1) No special syntax. This means the API's needed
to be fully describable using common Lua.
This is my current direction to improve completion via type inference
[1] , it is working out pretty well, but as you can see I have covered
only part of the problem. What about documentation and ParameterInfo.
The IntelliJ Lua plugin supports LuaDoc comments, so that I one way to
gather information. It supports using the HTML Lua user manual to
document the existing API, I'll show you the implementation in just
one sec.
Since I already needed a way to fetch and display the correct HTML
documentation arbitrary Lua, I decided to make this extensible via a
sandboxed Lua interpreter and some rules on how to load an executable
Lua script to look after the details in [2].
If you look, I am just using a Lua script to provide information. My
thoughts were that why write a special config file - we already have
nice tool to write Lua - so use Lua
Anyhow. Comments are appreciated, and let me know if there is anything
you would be interested in working on together.
Thanks,
Jon
David Manura
> In the list of common IDE features you have: Code Completion,
> Documentation, and Parameter Information.
> For example. I will show you how I supply these things for the common
LuaInspect has a module for something like that too: a database of
metadata on standard Lua functions [3]. On one hand it has the Lua
Reference Manual style function signatures like you have for tooltip
display. It should probably likewise derive URLs to the manual.
On the other hand, there is also information in this module for value
inferencing across calls to these standard functions. This is why in
a script like
local xx = math.sqrt(4)
local yy = math.sqrt(math.random())
local zz = math.sqrt("abc")
local ww = yy .. yy
double clicking on `xx` will display 2,
double clicking on `yy` will display `number`,
double clicking on `zz` will display an error about type [4] mismatch, and
double clicking on `ww` will display `string` since the concatenation
of two numbers is a string (the latter part about operator behavior is
actually in init.lua rather than signatures.lua).
Nothing in this module is dependent on Metalua.
Now in terms of documenting signatures for modules outside of the
standard library, I don't currently infer from LuaDoc @param tags but
maybe should if it is useful. (One reason I haven't bothered is that
personally I prefer a more free-form function documentation style like
the Lua Reference manual API or POD-like structure but in
Markdown-esque syntax.)
LuaInspect also supports a form of type definition based on Hungarian
notation. For example, in LuaInspect's own sources, there are ample
uses of AST nodes. By convention, I always postfix names of these
variables by 'ast'. Now, in a form of metacode (embedded in special
--! comments), I can tell LuaInspect that all variables following that
naming convention within a lexical scope have an AST node structure.
It's a bit ugly on first write, but here it is [5]:
-- i.e. an AST node is a table with a field 'tag' of type string, a
field `lineinfo`
-- with a complicated structure, and some number of other AST nodes
recursively inside.
local ast = T.table {
tag = T.string,
lineinfo=T.table{first=T.table{comments=T.table{
T.table{T.string,T.number,T.number}},T.number,T.number,T.number,T.string},
ast=T.table{comments=T.table{T.table{T.string,T.number,T.number}},
T.number,T.number,T.number,T.string}}, . . .}
ast[1] = ast; ast[2] = ast -- approximately
setmetatable(ast, ast_mt)
context.apply_value('ast$', ast) -- map type to hungarian notation.
This is why LuaInspect upon loading that [5] will infer (unless
overridden) that the following function returns a variable of type
string:
function foo(bast)
local xx = bast[1].tag
return xx
end
This inferencing is not necessarily dependent on the IDE (in fact,
it's localized in init.lua, which has no SciTE specific code).
Personally, I'd prefer is someone with more time and experience in
this area would maintain the inferencing module (or rewrite a better
one if they wish).
[3] https://github.com/davidm/lua-inspect/blob/master/luainspectlib/luainspect/signatures.lua
[4] https://github.com/davidm/lua-inspect/blob/master/luainspectlib/luainspect/types.lua
[5] https://github.com/davidm/lua-inspect/blob/master/luainspectlib/luainspect/typecheck.lua
Fabien Fleutot
Jaco
David Manura
Looking briefly through http://git.eclipse.org/c/koneki/ , I see a
copy of Metalua, a bunch of Java sources, and a few .lua/.mlua files.
Maybe there are opportunities for sharing Java parts between the
IntelliJ and Eclipse efforts for semantic Lua editors, but Java is not
that practical for the SciTE/Scintilla and VIM efforts. I see a
Scintilla plugin for Eclipse [1], but it looks dormant. LuaInspect
and Eclipse LDT already reuse Metalua, and the VIM plugin reuses
LuaInspect, but what other parts may be shared remains to be seen.
One thing all our projects will need to do is upgrade the
lexer/parser/analyzer support for 5.2, and a lot of that depends on
Metalua upgrading. When loading a ".lua" file, we'll also somehow
need to determine whether that's interpreted in 5.1 or 5.2. If code
compiles in both or neither, then it's not as straightforward.
There also may be a possibility of reusing the debugger between these
projects. I haven't used Lua debuggers that much. LuaForWindows has
a debugger component it installs in SciTE (scite-debug). Remote
debugging support (e.g. like RemDebug) will be necessary in general.
There is, I think, a blurry line between debuggers and static analysis
in Lua. For example, I optionally run code in the Lua state to assist
in the static analysis (e.g. require some module and inspect its table
values), and being able to do that runtime evaluation in the "actual"
Lua state accessed through the mechanism of a remote debugger session
would be helpful. An interesting possibility is to run the static
analysis during breakpoints in the debugging, using the current
runtime values obtained from the debugger to inform the static
analysis at that breakpoint.
[1] http://www.mail-archive.com/scintilla...@lyra.org/msg00177.html
Fabien Fleutot
All these are very exciting developments. I'm not sure what editor is being used for the Eclipse plugin, but would it be possible to take the work being done for that project and integrate it into the Scintilla editor?
Fabien Fleutot
+---
| 33 chemin de Mange-Pommes
| 31520 Ramonville Saint-Agne -- France
| mobile: +33 6 28 06 09 97
| office: +33 5 61 00 06 49
| home: +33 5 61 75 05 67
Jon Akhtar
performed, there are opportunities for developing common formats for
how to provide documentation and tab completion information. Fabien
already mentioned one extension to LuaDoc he is working on. And it
sounds to me that we both had a similar idea of using mock
implementations as a means to supply type information.
I see this would be generally useful for any development tool. I think
one possible way to go would be to make a shared repository for these
"mock api implementation". I currently use mock lua code combined with
a Lua script that can provide more detail and
Your point about the debugger is spot on. I already use RemDebug as my
remote debugger, so that is certainly something of mutual usefulness
though I like the concept of dynamic introspection of the Lua state.
You can take it further and say why not just leave it loaded while you
are editing it in cases where that is actually practical.
I think the most useful thing though was is the shared format for
supplying API information - the typical basic need for any developer
using embedded Lua.