Embedding julia?

654 views
Skip to first unread message

Tim Holy

unread,
Mar 20, 2012, 9:24:10 AM3/20/12
to juli...@googlegroups.com
Hi,

This is surely a naive question, but: how hard would it be to embed julia
inside a C/C++ application? Choose one: medium, hard, very hard, very very
hard, or downright impossible. :-)

I took an idle gander at the code in julia_init (init.c), and I was wondering
if in some miraculous world one could snip out some of those lines, and then
have some mechanism for feeding commands to julia and receiving return values.

Two applications occur to me:
1. I could call julia from Matlab. It would be sort of like a MEX syntax,
except that over time I'd be building up a library of julia code that, once
julia has everything I need, would make it easier to switch to full-julia.
(Right now julia is a "side project" indepdent of my real work.)

2. GUIs. Mike, I only dimly understand you current work on the graphics
system, but it seems you're writing native-julia plotting. This is great;
within Matlab, I've long wished for the ability to tweak their default
plotting capabilities, and you're setting up just such a system. From peeking
at your code, this seems to involve lots of ccalls to draw on a cairo canvas.
What about communication in the other direction, from the canvas to julia? Is
there a clear mechanism for setting up callbacks from graphical objects back
to julia functions? If the answer is "no," then I wonder if embedding julia
inside a larger Qt/GTK/whatever GUI would make that more straightforward?

Since I know nothing about these issues, it would be nice to hear back from
core developers, even if the answer is "not without an incredible amount of
work."

Best,
--Tim

Stefan Karpinski

unread,
Mar 20, 2012, 9:29:59 AM3/20/12
to juli...@googlegroups.com
Jeff has contemplated this a lot, so I'll let him answer at greater length, but in short the answer is not that hard, but it does require some work still. In "the fullness of time", it will become completely trivial (like as easy as calling a C library from C code).

Tim Hoffmann

unread,
Mar 21, 2012, 5:39:59 AM3/21/12
to julia-dev
I was wondering this too. I am playing around with making a small
interactive working environment (especially with 3d graphics) in java
and for now I've settled to communicate with a julia instance via
socket - in fact mimicking the way the scgi-julia server talks to it
(see ui/webserver/...). This seem to work for me until "in fullness of
time" I can just wrap a c library :), but I would like to see some
introspective features to be doable from within julia. Especially is
there a way to get something like the tab_completion from inside
julia? repl-readline does this by traversing a symbol tree, but I have
not found a way to get a handle on that tree from within the language.
(I even played around with calling the c methods via ccall (fun
calling julia implementation methods form within julia) which should
work but I've not yet figured what to give it for the **char
argument... a hint for that anyone?)
Reason I mention that here is, that many application that embed a
julia will need a way to get information about the current session
(like defined symbols, etc...) and I don't know how to get them right
now.
Sorry for jumping on the wagon here...

best
Tim.

Jameson Nash

unread,
Mar 21, 2012, 10:58:40 AM3/21/12
to juli...@googlegroups.com
I did some demo language introspection last night (using the tab_completion callback actually). I'll find some place to make that code accessible later tonight. However, I was thinking of writing some more direct methods, to make access to this from Julia easier, if nobody is opposed. Depending on what you are looking for, the function call whos() gets some of that information.

I'm guessing some of libuv stuff will be helpful for doing the sockets directly in Julia, once that branch is stable.

-Jameson

Stefan Karpinski

unread,
Mar 21, 2012, 11:14:42 AM3/21/12
to juli...@googlegroups.com
Moving all of that introspection machinery and even all the repl code into Julia should be doable, and in fact, should ultimately be done. It was all originally in C and gradually more and more of it has migrated to Julia, which is why it's half and half now. The three things that are necessary to make it happen fully are:
  1. Symbol table introspection accessible from Julia
  2. Readline bindings from Julia, including callback support
  3. A proper option-parsing library in Julia (optional, really)
None of these is *too* hard, but they are a fair amount of work put together, and not, at the moment strictly necessary, which is the main reason it hasn't happened yet. I personally would love to see all of them.

Jeff Bezanson

unread,
Mar 21, 2012, 4:12:41 PM3/21/12
to juli...@googlegroups.com
Yes, julia is embeddable; in effect ui/repl.c is a C program that
embeds julia. It uses jl_get_global and jl_apply to call the julia
"_start" function, which takes control from there. This is needed
since julia manages its event loop (which includes stdin and sockets
connected to worker nodes). If you don't want/need the event loop
though, it's easy. You can just call a different function than _start.
The "embedding API" is not fully worked out though. There are only a
few entry points, like jl_alloc_array_2d, etc.

Jameson is right, we need some more hooks like getting the list of
defined symbols from a module.

On Tue, Mar 20, 2012 at 9:24 AM, Tim Holy <tim....@gmail.com> wrote:

Jameson Nash

unread,
Mar 22, 2012, 12:02:17 AM3/22/12
to juli...@googlegroups.com
I started to add some hooks to allow more introspection and some example code using it (inspect.c and autodoc.jl, respectively). Since it isn't quite complete, I've pushed it only to my fork on GitHub (https://github.com/vtjnash/julia)

Does anyone know how to force unreferenced functions to still be included in the binary? They are in the library, but I can't ccall them without explicitly dlopen'ing libjulia-release.dylib first.

@Jeff: I added some code to ccall to turn null pointers to pointers into Julia exceptions instead of segfaults when I try to use them. However, this seems to reveal that compile_hint in sysimg.jl fails on more or less everything. It this intended/expected behavior? 

-Jameson

Keno Fischer

unread,
Mar 22, 2012, 12:05:42 AM3/22/12
to juli...@googlegroups.com
> Does anyone know how to force unreferenced functions to still be included
in
> the binary? They are in the library, but I can't ccall them without
explicitly
> dlopen'ing libjulia-release.dylib first.

Add them to src/julia.expmap

-Keno

Jeff Bezanson

unread,
Mar 22, 2012, 1:35:46 AM3/22/12
to juli...@googlegroups.com
I never intended to allow returning NULL from a c function declared to
return a julia value. I think it should just not be allowed. Re-using
jl_get_specialization for the purposes of compile_hint was kind of a
hack; compile_hint should actually call a void C function that calls
jl_get_specialization and ignores its value.

I think a better interface would be to write a C function that returns
a cell array of the symbols bound in a given Module (jl_module_t*).
That will be much more forwards-compatible, plus you don't have to
check isbound() on everything.

Reply all
Reply to author
Forward
0 new messages