[llvm-dev] Debugging JIT'ed code with ORC JIT?

[Connor Gray via llvm-dev]

Hi,

I’m wondering if it’s possible to debug code JIT’ed with the newer ORC JIT. The LLVM documentation has a page at

    llvm.org/docs/DebuggingJITedCode.html

showing an example of using gdb to debug MCJIT’ed code, but has no mention of ORC JIT.

From searching around online I’ve gotten the impression that ORC JIT does *not* support providing debugging information to attached debuggers, but no definitive source. Is it the case that ORC JIT in fact does support debugging, and if so, are there examples available; or is my initial impression correct?

Thanks,

Connor

[Jameson Nash via llvm-dev]

That is correct – there is no built-in support. You have to provide your own Registrar to the ObjectLinkingLayer to provide this sort of functionality. (For example JuliaLang does that at https://github.com/JuliaLang/julia/blob/1216e5f60cd2b23e29856b5227399ab0f3abef76/src/jitlayers.cpp#L437, in addition to registering the function pointer and object file info in several other places).

_______________________________________________
LLVM Developers mailing list
llvm...@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

[Lang Hames via llvm-dev]

Hi Connor,

...The LLVM documentation has a page at

    llvm.org/docs/DebuggingJITedCode.html
showing an example of using gdb to debug MCJIT’ed code, but has no mention of ORC JIT.

What debugging support MCJIT has is provided by the RuntimeDyld utility, which ORC shares. I would expect anything in that document to apply to ORC as well, though I haven't tested it personally.

For what it's worth, this is something I'm interested in and hope to make some progress on in tree in the not too distant future. 

-- Lang.

[Yichao Yu via llvm-dev]

> What debugging support MCJIT has is provided by the RuntimeDyld utility,
> which ORC shares. I would expect anything in that document to apply to ORC
> as well, though I haven't tested it personally.

I'm pretty sure that's not the case, at least not with any simple
orcjit tutorials I've seen.

The GDB support is provided by `GDBJITRegistrationListener` which is
registered to MCJIT directly in the constructor and AFAICT no where
else. Also, since there doesn't seem to be a default (non-noop)
implementation of `ExecutionEngine::RegisterJITEventListener` I
believe you can't just call `RegisterJITEventListener` on non-MCJIT
and expect it to work.

So while I totally believe one can use
`JITEventListener::createGDBRegistrationListener()` to hook the JIT
with the the gdb registration function, it won't work without writing
code to explicitly do that (in additional to actually writing code to
interface with the event listener directly). OTOH though when using
MCJIT, this is done by default and one don't need to write any code
for it to work.

[Lang Hames via llvm-dev]

HI Yichao,

RTDyldObjectLinkingLayer has a NotifyObjectLoaded hook that you can use to call NotifyObjectEmitted on your GDBRegistrationListener.

If code is going to be unloaded we would have to add an extra hook to call NotifyFreeingObject -- that seems totally reasonable to add.

-- Lang.

[Lang Hames via llvm-dev]

Oh, you're right though: The possibility of doing this is all in my head, but not well documented anywhere yet. :/

-- Lang.

[Yichao Yu via llvm-dev]

> RTDyldObjectLinkingLayer has a NotifyObjectLoaded hook that you can use to
>> call NotifyObjectEmitted on your GDBRegistrationListener.

> but not well documented anywhere yet. :/

Right. That's exactly what we (julia) do although we don't use
`GDBRegistrationListener` directly. I just mean that this has to be
done manually (so no "builtin" support for some meaning of "builtin")
and it can't be accomplished just by reading the mcjit doc. :/

[Stefan Gränitz via llvm-dev]

Am 11.10.17 um 19:44 schrieb Yichao Yu via llvm-dev:

So while I totally believe one can use `JITEventListener::createGDBRegistrationListener()` to hook the JIT with the the gdb registration function, it won't work without writing code to explicitly do that

Implementing the GDB JIT Interface for an ORC-based JIT is pretty straightforward. For 5.0 you need that tiny workaround, fixed on trunk:
https://github.com/weliveindetail/JitFromScratch/commit/d4b6778d8d462299674e103d8ecdec1140a45cfe

It will work out-of-the box on Linux, as the native object format is ELF. It does not work with other object formats, because no one else implements LoadedObjectInfo::getObjectForDebug():
https://github.com/llvm-mirror/llvm/blob/a5204d048729eef09b162dd12622c4afefb4978a/include/llvm/ExecutionEngine/RuntimeDyld.h#L79

Am 07.10.17 um 23:12 schrieb Connor Gray via llvm-dev:

I’m wondering if it’s possible to debug code JIT’ed with the newer ORC JIT. The LLVM documentation has a page at llvm.org/docs/DebuggingJITedCode.html

So bascially: yes. I think you could use ELF on any OS (like Julia does), but depending on your use-case you will need to workaround other issues. If you have requirements that restrict you to your OS's native object format, you need to implement getObjectForDebug(). I guess with COFF on Windows it will be complicated, but possible with the newly added PDB support. I'd be surprised about major issues for MachO on Mac.

I heard that request from other parties, so at some point it's gonna be worth a "common endeavor" :)

Cheers
Stefan

Am 11.10.17 um 21:34 schrieb Yichao Yu via llvm-dev:

-- 
https://weliveindetail.github.io/blog/
https://cryptup.org/pub/stefan....@gmail.com