[LLVMdev] Old JIT Status (i.e., can we delete it?)

[Rafael Espíndola]

Writing some patches for MC I noticed that the old (non MC) JIT still
has a lot of duplicated code for producing EH frames which makes it a
bit harder to refactor code used by both implementations.

Given that MCJIT is making good progress and that we just branched
3.2, can we delete the old JIT or at least its EH support? The
attached patch removes just the EH bits.

Cheers,
Rafael

[Albert Graef]

On 11/24/2012 02:18 AM, Rafael Espíndola wrote:
> Given that MCJIT is making good progress and that we just branched
> 3.2, can we delete the old JIT or at least its EH support? The
> attached patch removes just the EH bits.

The "old" JIT is still in heavy use by 3rd party dynamic language
projects, at least that's the case for my own (Pure). For these kinds of
projects the facilities of the old JIT are an essential component of
LLVM; you need to be able to replace the definition of any function at
any time, having to recompile an entire module doesn't cut it.

I'd really like to give MCJIT a whirl and see whether I could use it for
Pure, but apart from some past developer meeting slides there hasn't
been a progress report on MCJIT in a while, and the available blog posts
are from 2010. Is MCJIT working on Windows yet? PPC? ARM? Does it
support lazy compilation now? Module linkage? Is there documentation on
how to migrate from the "old" JIT somewhere?

Albert

--
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email: Dr.G...@t-online.de, a...@muwiinfa.geschichte.uni-mainz.de
WWW: http://www.musikinformatik.uni-mainz.de/ag
_______________________________________________
LLVM Developers mailing list
LLV...@cs.uiuc.edu http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

[Benjamin Kramer]

The last major roadblock is lazy compilation. I don't think we can remove the old JIT until MCJIT supports that.

- Ben

[Rafael Espíndola]

> The "old" JIT is still in heavy use by 3rd party dynamic language
> projects, at least that's the case for my own (Pure). For these kinds of
> projects the facilities of the old JIT are an essential component of
> LLVM; you need to be able to replace the definition of any function at
> any time, having to recompile an entire module doesn't cut it.
>
> I'd really like to give MCJIT a whirl and see whether I could use it for
> Pure, but apart from some past developer meeting slides there hasn't
> been a progress report on MCJIT in a while, and the available blog posts
> are from 2010. Is MCJIT working on Windows yet? PPC? ARM? Does it
> support lazy compilation now? Module linkage? Is there documentation on
> how to migrate from the "old" JIT somewhere?

I think ARM and PPC64 work. Windows work (using ELF in memory). As
Benjamin points out lazy compilation is missing.

Do you need the EH bits I remove in the patch in the original message?

> Albert


Cheers,
Rafael

[Albert Graef]

On 11/24/2012 01:13 PM, Rafael Espíndola wrote:
> Do you need the EH bits I remove in the patch in the original message?

No.

--
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email: Dr.G...@t-online.de, a...@muwiinfa.geschichte.uni-mainz.de
WWW: http://www.musikinformatik.uni-mainz.de/ag

[Eli Bendersky]

Rafael, could you mention in a few words the role these "EH bits"
serve for the legacy JIT? What is the meaning/impact of removing them?
- i.e. what will stop working? Given that there's still need of the
legacy JIT to be around, what is the advantage of removing this code,
at the expense of losing existing functionality?

Eli

[Albert Graef]

On 11/24/2012 11:05 AM, Benjamin Kramer wrote:
> The last major roadblock is lazy compilation. I don't think we can remove the old JIT until MCJIT supports that.

I agree. But that alone may not be enough to support dynamic languages.
In the Pure JIT I often need to be able to replace the IR of the body of
an already lowered function and generate new code for it. This also
needs to be fast, so recompiling the entire module when this happens is
not an option. Will this be possible with MCJIT?

--
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email: Dr.G...@t-online.de, a...@muwiinfa.geschichte.uni-mainz.de
WWW: http://www.musikinformatik.uni-mainz.de/ag

[Óscar Fuentes]

Albert Graef <Dr.G...@t-online.de> writes:

> On 11/24/2012 02:18 AM, Rafael Espíndola wrote:
>> Given that MCJIT is making good progress and that we just branched
>> 3.2, can we delete the old JIT or at least its EH support? The
>> attached patch removes just the EH bits.

What Albert said.

As a more specific note, I have no problem with removing the EH support,
but the "old" JIT should stay until MCJIT is on par with it.

And now that we are talking about the JITs, is the MCJIT faster than the
"old" JIT at generating code with the same optimization level? On
medium-sized modules (hundreds of KB of native code)?

[Chris Lattner]

As others have said, unfortunately it is premature to remove the old jit. I don't know of anyone using the EH code though... It can almost certainly be removed!

-Chris

> <t.patch>

[Rafael Espíndola]

> Rafael, could you mention in a few words the role these "EH bits"
> serve for the legacy JIT? What is the meaning/impact of removing them?
> - i.e. what will stop working? Given that there's still need of the
> legacy JIT to be around, what is the advantage of removing this code,
> at the expense of losing existing functionality?

I have never used them myself, so I can talk about what they do, not
what they are used for. They produce exception handling tables that
look similar to (are?) the ones specified by the x86_64 abi. This
tables are used by stack unwinders and could probably be used to JIT
code that uses "zero cost" exceptions.

> Eli

Cheers,
Rafael

[Rafael Espíndola]

On 24 November 2012 17:36, Chris Lattner <clat...@apple.com> wrote:
> As others have said, unfortunately it is premature to remove the old jit. I don't know of anyone using the EH code though... It can almost certainly be removed!

Cool. I will remove the EH bits by the end of the week if no one complains.

> -Chris

Cheers,
Rafael

[Kaylor, Andrew]

So when you say 'lazy compilation' you mean 'deferred compilation of individual functions within a module until that function is needed,' right? That's certainly what 'lazy compilation' meant in the legacy JIT.

I don't think that will ever be possible in MCJIT. If you look at the MCJIT engine you'll see that it delegates absolutely everything. In particular, compilation is done by the MC code, and it is done in exactly the same way that static compilation of objects is done. Rather than producing a collection of loosely coupled pieces of executable memory containing machine code for functions, MCJIT produces executable object images and prepares those for execution. I don't see any way for a 'partial' object to be produced without drastically changing the design of the MCJIT engine.

I'd like to suggest that instead of talking about how we can get MCJIT to do 'lazy compilation' in the old way, we should be talking about how to solve the same problem within the constraints of what MCJIT does. I'm not sure I understand the constraints of the client side well enough to address this problem thoroughly, but I'll take a stab at first steps.

I think the way to approach this problem is to deconstruct large modules into smaller modules which we can afford to re-JIT, and then optimize the process of linking modules together. Currently, a single instance of MCJIT only supports handling of a single module, but I did recently add a mechanism to delay compilation of the module so that it doesn't happen when the MCJIT engine is constructed.

Once MCJIT supports multiple modules, if it had a mechanism for determining dependencies between modules and managing lazy compilation of those dependencies would that be acceptable?

This still leaves a bit of a problem with regard to function replacement, but I think it suggests a direction for making that happen too.

-Andy

[Albert Graef]

On 11/26/2012 10:35 PM, Kaylor, Andrew wrote:
> So when you say 'lazy compilation' you mean 'deferred compilation of individual functions within a module until that function is needed,' right? That's certainly what 'lazy compilation' meant in the legacy JIT.

Yes. Let's not forget what JIT actually means; this *is* what makes a
compiler a JIT compiler. In Pure you can add new functions and change
existing ones at any time, by just typing an equation at the command
prompt of Pure's interactive frontend. Other dynamic languages work in a
similar fashion. The old JIT makes this very easy to do; you just
(re)compile the function to IR on the fly, and the JIT will then take
care of lowering the function to native code when it's first needed. For
me this has always been one of the coolest features of LLVM, a feature
which really sets it apart from traditional compiler technology.

If the MCJIT lacks this feature then it's not a real JIT compiler for
me, it's an in-memory compiler/linker/loader. That's sufficient for many
applications, but if you're striving to eventually replace the old JIT
then you'll have to find a way to replicate the lazy compilation bit in
some way. Otherwise we'd need the old JIT to stick around forever to
support this kind of application (and then we'd also need a working ARM
backend for it).

> I don't think that will ever be possible in MCJIT.

That's bad news. :(

> I think the way to approach this problem is to deconstruct large modules into smaller modules which we can afford to re-JIT, and then optimize the process of linking modules together. Currently, a single instance of MCJIT only supports handling of a single module, but I did recently add a mechanism to delay compilation of the module so that it doesn't happen when the MCJIT engine is constructed.

In Pure this would basically mean that each function would have to be in
its own module. (More precisely, each global Pure function. Pure also
has lexical closures, these could of course be in the same module as the
global functions they belong to.)

> Once MCJIT supports multiple modules, if it had a mechanism for determining dependencies between modules and managing lazy compilation of those dependencies would that be acceptable?

Yes, this might be an acceptable (if somewhat awkward) solution. But I'm
not really convinced that it's feasible. It would need a linker that's
able to cope with thousands of modules, add new modules and replace
existing ones at any time, and do this *fast*. Do you think that this
will be possible with MCJIT? Maintaining all these modules and the
linker tables in memory might add significant overhead compared to the
old JIT, right? Will it be at least as fast as the old JIT?

Albert

[David Chisnall]

On 27 Nov 2012, at 00:23, Albert Graef wrote:

> Yes. Let's not forget what JIT actually means; this *is* what makes a
> compiler a JIT compiler. In Pure you can add new functions and change
> existing ones at any time, by just typing an equation at the command
> prompt of Pure's interactive frontend. Other dynamic languages work in a
> similar fashion. The old JIT makes this very easy to do; you just
> (re)compile the function to IR on the fly, and the JIT will then take
> care of lowering the function to native code when it's first needed. For
> me this has always been one of the coolest features of LLVM, a feature
> which really sets it apart from traditional compiler technology.

Have you done any profiling of this? I admit that I haven't since 2.9, but when I did I found that the time spent in the JIT itself was negligible compared to the time spent in optimisation, even with a fairly reduced set of passes. And even that was pretty small if you are compiling a small module (less than a few dozen functions). It's also relatively easy to implement this yourself on top of the existing JIT: just install a stub function that calls out to something that triggers the JIT to compile that module and then jumps to the real function when it's called.

The biggest limitation of the JIT for this kind of use (dynamic languages with lots of short-lived functions / closures) is that it doesn't provide any way of detecting that a function is in use. This really requires stopping the world and walking all of the threads' stacks to ensure that an old function is not being called (you can do it lazily by doing a periodic trace, if you can have a barrier somewhere in every function. Things like libdispatch complicate this significantly because then you have threads that are harder to find). It would be nice if we could have some generic infrastructure for doing this: notify the JIT that you have removed all public references to a function and want it to be deleted when it is no longer on any stack. Doing this in a portable way is impossible (POSIX doesn't provide the required functionality, although FreeBSD and OS X do provide everything needed to do it via non-portable extensions and presumably other systems do as well) so it !
would need a lot of code in the support library.

David

[Solomon Boulos]

On Nov 24, 2012, at 3:50 PM, Rafael Espíndola wrote:

> On 24 November 2012 17:36, Chris Lattner <clattner at apple.com> wrote:
>> As others have said, unfortunately it is premature to remove the old jit. I don't know of anyone using the EH code though... It can almost certainly be removed!
>
> Cool. I will remove the EH bits by the end of the week if no one complains.

Do you mean you're going to remove EH entirely from the old JIT? If so, I'll complain ;).

As you pointed out earlier the EH bits are useful for stack frames on x86_64. In particular, to get useful stack traces from gdb with JIT'ed code those EH frames need to be there. So even if other people don't currently set JITExceptionHandling for this purpose (I do in a few projects), they probably will when they see crashes.

Solomon

[Kaylor, Andrew]

I don't think that GDB support is even present in the latest version of the old JIT in trunk. It used to have code to register emitted functions with GDB in a way that let GDB find the function name and EH frame information, but that got taken out back in January (r147615). As it stands now, I don't think GDB would provide any useful information at all about code emitted by the old JIT.

MCJIT, on ELF platforms at least, registers emitted code with GDB for full source level debugging (when debug info is available).

-Andy

-----Original Message-----
From: llvmdev...@cs.uiuc.edu [mailto:llvmdev...@cs.uiuc.edu] On Behalf Of Solomon Boulos
Sent: Thursday, November 29, 2012 1:01 PM
To: llv...@cs.uiuc.edu
Subject: Re: [LLVMdev] Old JIT Status (i.e., can we delete it?)

[Solomon Boulos]

Yes, the GDB-jit registration stuff got pulled out (perhaps prematurely). That said, even without function names, it's still useful to have the EH bits so that gdb can unwind through the JIT'ed frames. Otherwise gdb (and glibc's backtrace() function, valgrind, etc) end up stopping the unwind prematurely, usually at the first JIT'ed frame. Combined with trivial use of the JITEventListener, the old JIT lets you pretty easily map 'confused' frames from gdb and such to useful names.

Don't get me wrong, I'm quite excited about seeing MCJIT nuke the old JIT. I'd just prefer not to keep losing features from the old JIT before MCJIT is ready for prime time.

Solomon