Getting the Proto of a given filename and linenumber

[Thomas Jericke]

Hello List

I am currently porting my patches from Lua 5.2 to Lua 5.4

One the main patches is my "breakpoint" patch. What this patch does, is adding a breakpoint OP_Code which can be "installed" into the bytecode. When the breakpoint opcode is hit, the debug hook will be called (if one is set with the according mask).

Now in the Lua 5.2 version I had to made quite a lot of changes to the Lua code to find the the bytecode for a given filename and linenumber. The main problem is finding the correct "Proto" structure for the file and line.
Once that is found it is actually quite easy to get the pointer to the first OP_Code of the requested line of code and replace the original OP_Code with the breakpoint OP_Code. All information is stored in the Proto structure.

But there seems to be no function to get the right Proto. What I need is kind of the reverse function to luaG_getfuncline.

As I understand the Lua source, there is no place where all the Protos are stored together. A Proto can be part of another Proto, it can be part if a function that is stored in some place like a table or a local variable and it can be on the stack as the function that is executed.

My solution in Lua 5.2 is to put all the Proto structures into a linked list, this way I can traverse them when I need to install a breakpoint. That solution would work for Lua 5.4 as well, but I would like to check first if I miss some easy way to find the correct Proto in another way. Basically my goal is to make a little changes to the Lua code as possible. 

What I ruled out (yet) is to go through the GC list, because it would mean that in worst case the function needs to traverse all Lua objects just to install a single breakpoint.

Any idea is welcome

--

Thomas

[Roberto Ierusalimschy]

> I am currently porting my patches from Lua 5.2 to Lua 5.4
>

> [...]
>
> Any idea is welcome

(Just brainstorming...)

Do you use a call hook? If so, maybe you could do something on
demand. Each time a function is called, you check its proto. If it's
not registered yet, then you register it and add to it any pending
breakpoints. A downside is that you cannot give a proper feedback when
someone sets the breakpoint, such as "function not found".

-- Roberto

[Thomas Jericke]

That would surely work too. I just don't see that much of a benefit. In my "old" solution,
I patched luaF_newproto function to register all protos into a list which is very efficient.
In luaF_freeproto I have to remove the proto from the list which is O(N) to the number of Protos,
but I don't care that much as this is during GC.
Installing a breakpoint is also O(N) to the number of protos, which is OK.
Only registering the Protos that are called, would reduce the number N,
but that number would also grow as the program runs and more and more functions are reached.

What I could change is to use a different data structure.
Instead of a list I could register the Protos in Lua weak table.
That would make the lookup much faster but I think the added complexity isn't worth it.

I think I try my old approach again. It may not be the optimal possible,
but good enough when considering the simplicity of the change.

--

Thomas

[Tom Sutcliffe]

I've tried something similar to this in the past, and found that the overhead of all the extra debug calls to look up the filename on every function call was prohibitively expensive for what I was trying to do. Although this was with a line hook rather than a call hook.

(What I did instead was a hack where I added 100000 newlines to the start of the file I was interested in when it was being loaded, so that I could immediately check in the hook if the line number was > 100000 then it was a function I was interested in. Sounds horrible but I could be sure I'd never be parsing files so big as to get false positives, and the order of magnitude speed up was worth the hack.)

The last time I experimented with breakpoints I "solved" it by requiring that the caller supply the actual function where the breakpoint was, and didn't support specifying them by filename. It wasn't a very good debugger :D

If you control the module loading process in your environment, perhaps you could compare the gc list before and after a module is loaded, so you only end up iterating the bit of the gc list that's new? Assuming things are added onto the gclist in a deterministic fashion of course, not sure of the details of that. Just a thought.

Cheers,

Tom

[Thomas Jericke]

As I am patching the code, I can add additional code anywhere.
Therefore I don't have any problem adding breakpoints when new Protos are loaded/created.
The more complicated part is to find a Proto when the breakpoint is created one a running system.

--

Thomas