Embedding Julia

[Bill Hart]

I tried the example [1] for embedding Julia in C, using the latest Julia-0.5 nightly, but have run into problems.

One trivial problem I had was that jl_atexit_hook(); seems to now require a return argument. That was easy to fix.

However the C program given at [1] didn't link. It claims to not be able to find libjulia. Yet Julia is working fine on the system.

I did a search to see if I could find libjulia and I have /usr/lib/x86_64-linux-gnu/julia/libjulia.so. But linking against this causes undefined symbol errors, and indeed nm seems to indicate that this library contains no symbols (it may be just a leftover from an old version of Julia I had installed or something).

Is embedding currently supported with the nightlies? Do I need to install a different Ubuntu package to get libjulia? (I'm using Ubuntu 14.04).

On a different, but related note, does anyone know what the current plans are for producing .o files or executables from Julia, rather than using it via the REPL? I know there was at one point some kind of compiler available. Has that been abandoned, or is it likely to resurface again in some form? I realise it's probably technically feasible. I'm really just asking whether it sits in the currently forseeable roadmap or not.

Bill.

[1] http://docs.julialang.org/en/latest/manual/embedding/

[Kristoffer Carlsson]

For source build I link with libjulia.so in julia/usr/lib.

For binary builds I link with /lib/julia/libjulia.so

Here is a little example of my Cmake setup and c++ file: https://github.com/KristofferC/exploration/tree/master/embed_julia

[Bill Hart]

I don't have this file on either of the systems I have Julia installed on, either Julia-0.4 or the Julia-0.5 nightly.

I did try to link against a libjulia.so in a source build of Julia-0.4 but then it complains:

ERROR: system image file "/home/wbhart/../lib/julia/sys.so" not found

I imagine it needs some environment variable set to find this file? I thought maybe JL_SHARE, but setting this didn't seem to help.

Bill.

[Bill Hart]

I figured out how to get it to run against a source build of Julia. That directory name comes from the directory the binary is installed, so the only way to make it work is to put the binary (./test) into julia/usr/bin. Then that relative directory is correct for sys.so.

So there seem to be three issues here:

1) jl_atexit_hook() requires a return argument (this is also true in 0.4, not just 0.5). This is just a documentation issue.

2) It should be possible to allow Julia to find its sys.so in alternative locations. In particular it should be possible to set this location when embedding Julia, e.g. by setting an environment variable, etc. The example in the documentation should say how to do this, since it is not at all obvious.

3) The Ubuntu packages should install libjulia.so and julia.h so that it is possible for users to make use of embedding.

I can make a pull request to fix 1. Should I make tickets for the other two issues?

Bill.

[Bill Hart]

Apparently there is already a pull request for 1, namely [1]. This leaves the issues 2 and 3 below.

Bill.

[1] https://github.com/JuliaLang/julia/pull/14262

[Ales Tsurko]

I have exactly the same issue on mac: https://groups.google.com/forum/#!topic/julia-users/tpG0ZTlJObA

вторник, 8 декабря 2015 г., 20:44:28 UTC+3 пользователь Bill Hart написал:

[Kristoffer Carlsson]

Strange, I don't have any special paths (except the ones I add in Cmake for header and library directories). Maybe you can try something similar to my cmake file above?

[Bill Hart]

@Kristoffer That's because you have libjulia.so installed on your system. I've only installed Julia using an Ubuntu ppa, which is missing libjulia.so apparently.

It obviously looks in the standard system library location for the libraries it needs (as it should).

The problem here is that we cannot ask our users to install Julia from source just so they can make use of embedding.

Also, some of our developers cannot install Julia systemwide. They install it in their home directories, and then the only place you can put the binary (from the C compiler) is in julia/usr/bin, which is clearly problematic.

Bill.

[Tony Kelman]

That sounds like a serious bug in the PPA packaging, or it's just putting libjulia somewhere you haven't found it. The PPA is not very actively maintained at the moment, the generic tarball binaries are the main binary install recommendation on Linux right now.

[Bill Hart]

I've searched my machine and really haven't found libjulia.so, except the copy I mentioned, which has no symbols.

The PPA's seemed to be very up-to-date with v0.4.1 being available the day it was released. They also work just fine. I think they are just missing something.

Where would I even report that issue? Is it a Julia developer who maintains the staticfloat ppa's?

Bill.

[Bill Hart]

By the way, I haven't used the generic Linux binaries because I couldn't figure out how to install them. There's no Makefile and no instructions. The last time I tried they didn't work when just placed in my home directory. They seem to need installation somewhere.

I'm pretty sure our Ubuntu users are going to prefer the ppa magic anyway.

[Tony Kelman]

The PPA is maintained by staticfloat, aka Elliot Saba. He's had very little time for Julia lately and no one has stepped up to take over the PPA maintenance from him.

You just extract the Linux tarballs, then run bin/julia. There's nothing to install. If you want to have julia on your path, you can add it in your bashrc.

[Bill Hart]

On 9 December 2015 at 01:54, Tony Kelman <to...@kelman.net> wrote:

The PPA is maintained by staticfloat, aka Elliot Saba. He's had very little time for Julia lately and no one has stepped up to take over the PPA maintenance from him.

Thanks for letting me know. I'll circulate that and see if anyone locally wants to step up. You never know.

 

You just extract the Linux tarballs, then run bin/julia. There's nothing to install. If you want to have julia on your path, you can add it in your bashrc.

Oh, that works for me now. Last time I tried it (quite some time ago now) it didn't. I figured the Ubuntu ppa's were there for a reason at the time, so didn't think much more of it.

Bill.

[Bill Hart]

I should add that it would still be useful if there was a Makefile to install Julia system wide after extracting the tarball.

Bill.

[Tony Kelman]

I'd think a simple shell script, install-julia.sh or something, would be better than a Makefile - you don't always have build-essential installed. Putting something in contrib (along with a corresponding uninstall-julia.sh script?) and adding it to the `make binary-dist` tarball generation rules for Linux would be okay by me.

[Bill Hart]

On 9 December 2015 at 04:47, Tony Kelman <to...@kelman.net> wrote:

I'd think a simple shell script, install-julia.sh or something, would be better than a Makefile - you don't always have build-essential installed. Putting something in contrib (along with a corresponding uninstall-julia.sh script?) and adding it to the `make binary-dist` tarball generation rules for Linux would be okay by me.

Yes a shell script ought to do it. I'll add it to my todo list (which is quite long).

Bill.

[Bill Hart]

One mystery is solved. Elliot Saba pointed out that I was issuing the wrong command to look for symbols in libjulia.so. And in fact libjulia.so is expected to be in the "odd" location that I reported, on Ubuntu.

And linking against it does in fact work.

So that leaves one remaining issue, namely that there is a hard coded relative path for sys.so somewhere so that embedding doesn't work unless the compiled binary is in a specific location relative to sys.so.

This means that a user trying to do embedding with say an Ubuntu installed libjulia.so would need sudo privileges to put the binary in the right location. I'm not sure where exactly Julia looks for its sys.so, but on Ubuntu at least, it looks in the wrong place.

Bill.

[Tony Kelman]

I'm pretty sure there's an API for specifying the system image path in julia_init. Maybe not documented in enough detail yet.

[Tracy Wadleigh]

Yes, Tony, you don't need compiled-in paths. You can call jl_init_with_image(<JULIA_HOME>,<PATH_TO_JULIA_IMAGE>).

[Sebastian Wiesendahl]

For me it worked that way:

1. unzipped julia to a destination folder. In my case:

/opt/julia/julia-2ac304dfba/bin

2. put $JULIA_HOME into ".bashrc":

export JULIA_HOME=/opt/julia/julia-2ac304dfba/bin
export PATH=$PATH:$JULIA_HOME

NOTICE: the $JULIA_HOME variable already has to contatin /bin in its path to be able to resolve the relative paths, which julia uses the variable internally. It will not work, if /bin is only added to the $PATH

[waTeim]

Well sorry to have missed this all from back in Decepber.  These are mostly non-issues and misunderstandings.