"statically" compiled julia

[Jeff Bezanson]

Yesterday I did a small experiment in using julia to generate native
libraries. After all the work by Keno and Jameson, the only new
feature I needed was technically quite trivial (about 1 line of real
code), but still *feels* really cool. Here is a brief recipe:

1. Add code to userimg.jl (or rewrite sysimg.jl entirely), including this:

@Base.ccallable foo(x::Int) = x + 1

2. Run

cd base
../julia --build ../usr/lib/julia/stuff -J ../usr/lib/julia/sys.ji sysimg.jl
cd ..
g++ -shared -fPIC -Lusr/lib/julia -Lusr/lib -o usr/lib/julia/stuff.so
usr/lib/julia/stuff.o

3. Compile a C program such as

#include <stdio.h>
#include <julia.h>

extern int64_t foo(int64_t x);

int main()
{
jl_init_with_image(NULL, "../lib/julia/stuff.ji");

printf("foo(1) = %d\n", foo(1));
return 0;
}

Link it against -ljulia and stuff.so. For now the binary has to be
placed (with -o) in usr/bin since the rpath is set based on that.
examples/Makefile works just with adding usr/lib/julia/stuff.so to the
link command.

Run the program, and it will directly call the julia foo() function.

The next steps will be to include the system image in some section of
the .so file, and generate an init section to call jl_init, and then
you won't even need julia.h.

Again, since we already had sys.so everything was basically in place
already, but it was just fun to see this work with a direct call.

[Jameson Nash]

very cool to see that this works now

i had contemplated moving the sys.ji file inside the sys.so file, but
initially I rejected the idea so that the sys.ji file could be used
independently

I think `@ccallable` should be replaced by a language keyword
(`cexport`?), to make it clear that this is a static declaration,
similar to other the other export statement

[Jake Bolewski]

Hey this is really cool!  Following your steps I was able to get this to work.
Is it possible to build a static binary so you would not have to have a dynamic link to libjulia?

[Tobi]

This is sooo impressive. This kind of cries for a minimal Julia installation (i.e shrinking Base and introducing some kind of Standard library).

Imagine a high quality "C library" that is entirely written in Julia 

[Stefan Karpinski]

Once all the details get ironed out and this becomes standard thing to do, this seems to me like it is going to be kind of a big deal. We definitely do need to work on having a more minimal Julia installation with standard packages that just happen to be loaded for you by default.

[Tony Kelman]

Awesome that the infrastructure is there for this to work so soon. In case there was any reason to think it wouldn't, I can confirm this toy test also works in MinGW.

I get a feeling this will shape up to be a much more satisfying solution in every possible way than either py2exe or deploying the whole Matlab Compiler Runtime.

[Carlos Mundi]

Oooh!  I just tried this and it is sooo cool!  :)

And we already have a winner (T.Kelman) for this month's Sly Understatement Award!  I'm not subtle: friends don't let friends use Matlab Compiler Runtime. At the rate things are advancing, within a year Julia may make me say "Matlab who?"  

[Fernando Perez]

On Sat, Apr 5, 2014 at 6:02 AM, Stefan Karpinski <ste...@karpinski.org> wrote:

Once all the details get ironed out and this becomes standard thing to do, this seems to me like it is going to be kind of a big deal

ya think? This is *awesome* in so many ways... Thanks so much for all the amazing work!

f

--
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail

[Ondřej Čertík]

On Sat, Apr 5, 2014 at 6:03 PM, Fernando Perez <fpere...@gmail.com> wrote:
>
> On Sat, Apr 5, 2014 at 6:02 AM, Stefan Karpinski <ste...@karpinski.org>
> wrote:
>>
>> Once all the details get ironed out and this becomes standard thing to do,
>> this seems to me like it is going to be kind of a big deal
>
>
> ya think? This is *awesome* in so many ways... Thanks so much for all the
> amazing work!

Indeed, this is way more than "kind of a big deal".

This is a necessary step in order to be able to write reusable
libraries in Julia, that can be called from other languages.
One can in principle call Python from C++ as well, I've done it myself
using Cython many times, but it's not very robust in practice.
So these days I prefer to just write things in C++ or Fortran directly
and only create high level wrappers in Python. Julia might change this
a lot.

Ondrej

[Patrick O'Leary]

On Saturday, April 5, 2014 9:33:51 AM UTC-5, Tony Kelman wrote:

Awesome that the infrastructure is there for this to work so soon. In case there was any reason to think it wouldn't, I can confirm this toy test also works in MinGW.

I get a feeling this will shape up to be a much more satisfying solution in every possible way than either py2exe or deploying the whole Matlab Compiler Runtime.

Not to be left out, if we have debugger support back to Julia source it will also be more satisfying than Embedded Coder. For me that's a huge win.

[Juha Heiskala]

How did you do the MinGW build?

When I use the mingw-builds and MSYS2 environment as in the current Windows Julia build instructions, I get a long list of undefined symbol errors from the linker in the g++ build stuff.dll step.

Juha Heiskala

[Tony Kelman]

I guess I had to tweak the link line a tiny bit, since you can't make a DLL with undefined references and DLL's are located in bin instead of lib. Try

g++ -shared -Lusr/lib/julia -Lusr/bin -ljulia -o usr/lib/julia/stuff.dll usr/lib/julia/stuff.o

[Juha Heiskala]

That worked, great.

Thanks

[Eric Jang]

This is cool. Are there any updates to methods for compiling julia code to native libraries (i.e. for distribution) or ongoing projects that encompass this functionality?