Embedding Julia in an app / calling from C++

[Dibyendu Majumdar]

Hi,

I am working on a financial app where I would very much like to use Julia. I am currently using Lua as an embedded scripting language, but Lua doesn't perform very well with numeric computing. I have tried Luajit but no luck as Luajit doesn't optimise very well when called as callbacks from C++. 

Julia seems to be the language I was looking for. I tried a small experiment of calling Julia from C++ and noticed a few things:

1. The jl_init() function called exit() because it did not like the path to Julia home directory.
2. There doesn't seem to be a way to control instances of Julia - it seems to rely on global state management. There seems to be an assumption that there is only one global Julia environment in an app.
3. The startup appears to take time.

These are kind of show stopper issues for my use case so I was wondering if:

a) There are plans to make Julia more embeddable ... or is that a non-goal or low priority?
b) How hard is it to change Julia to encapsulate all state in a Struct of some sort? Is this impossible - or can it be done?

One other thing that I have been trying to work out is whether Julia JITs code when it is called in an embedded manner - this point doesn't seem to be covered in the Julia documentation.

Thanks and Regards

Dibyendu
 

[Stefan Karpinski]

We've been making some strides towards Julia being easier to embed lately. Calling exit in jl_init is definitely no good – that should be changed to return an error code that can be trapped and handled appropriately. It would be helpful to know what error message you're getting.

The use of global state for the Julia interpreter is unlikely to go away, although it is certainly possible. It may, however, be easier to have multiple independent Julia interpreters in a single process with separate Julia Core/Base/Main modules. If the interpreters are going to be in different threads, there would have to be locks around access to mutable global structures, but that might be ok. Unlike, say Python, there's no interpreter, so there would be no GIL – once code is generated, it can freely execute without different threads blocking each other. That would actually be a rather interesting project. You also have to be aware that many of the libraries that Julia uses are not threadsafe.

Regarding startup time, we do have the ability to load a pre-compiled system image from a .so file these days, which takes about 0.3 seconds on my system – that's not super fast, but it's ok. If you're giving the wrong path to Julia, it's possible that it's not finding a system image, then again, if it's not finding the system image, it's probably not JITing anything.

Embedded Julia code may still JIT.

[Dibyendu Majumdar]

The error message I got was:

System image file "C:\project\Debug\../lib/julia/sys.ji" not found

This occurred when I called jl_init() with NULL. It seems that it is trying to find files relative to the executable file location.
I tried setting JULIA_HOME environment variable but that did not help.
I then changed the argument to "c:\julia\bin" which worked.

It takes about 2 seconds to startup on my laptop (2009 Intel duo Mac).

I was wondering if I could modify Julia to only be a language - i.e. remove all the standard library and therefore most (all?) of the dependencies. As an embedded language I don't really want all these external dependencies.
Is this possible? Where should I start? Are any of the dependencies must have?
Is there a developer's guide for Julia?

Thanks and Regards
Dibyendu

[Patrick O'Leary]

This isn't really possible right now--large, essential parts of Julia are written in Julia, and there's not full differentiation between the core needed just to get the language running and the other components included in Base. Please see https://github.com/JuliaLang/julia/issues/5155 and recent comments on https://github.com/JuliaLang/julia/issues/1906 for a bit more of the current discussion on this topic.