Julia on OpenBSD

[Mike H]

Just curiious, has anyone out there tried building Julia on OpenBSD?

Cheers,
Mike

[Antoine Jardin]

Hi, I have thought about it

I'm currently running linux on my day to day laptop workstation but i used to run openbsd.
I'll try to find the time to run a test build.

llvm is v3.5 in OpenBSD 5.6
ggc is 4.9
gmake is 4.0
gfortran is 4.2
git is 1.9
perl is in base
wget is in ports
m4 is 1.4.17
cmake is ok (2.8)

patch I guess is patchutils

so on paper it should work, I'm very interested in the matter

[Mike H]

Thanks Antoine,

It's something I might work on in the future when I get some more time. It looks like it would take a decent amount of work patching things, but possible.

Cheers,
Mike

[Viral Shah]

I do believe julia builds on freebsd and Mike Nolta has generally maintained it over time. I would expect it may not be too much work to get it to work on openbsd.

-viral

[Maurizio Tomasi]

Just not to leave this thread silent for too long, I am trying to compile latest Julia on OpenBSD 5.7 (64-bit), but so far with little success. A few trivial problems lie in the usage of Linux-isms in the Makefiles (e.g., "cp -a"), but the most problem I'm experiencing are in the compilation of openlibm.

Anybody else has had the chance of compiling it successfully under OpenBSD?

Maurizio.

[Yichao Yu]

On Mon, Aug 31, 2015 at 5:27 PM, Maurizio Tomasi <ziot...@gmail.com> wrote:
> Just not to leave this thread silent for too long, I am trying to compile
> latest Julia on OpenBSD 5.7 (64-bit), but so far with little success. A few
> trivial problems lie in the usage of Linux-isms in the Makefiles (e.g., "cp
> -a"), but the most problem I'm experiencing are in the compilation of
> openlibm.

No idea about OpenBSD but you can use `USE_SYSTEM_LIBM=1` to avoid
compiling openlibm.

[Viral Shah]

I believe openlibm received some patches to build on openbsd recently. However, I have not tagged a new version.

-viral

[Maurizio Tomasi]

@Viral, yes, indeed I was the submitter of those patches. Unfortunately, I realized that part of the C code has a number of #ifdefs that do not cover OpenBSD properly. Maybe I should create a bugreport in openlibm to keep track of this?

@Yychao, thank you very much for this tip! It has saved me a lot of time, as I have found problems with other dependencies as well (OpenBLAS, PCRE, and FFTW, at the moment). With your trick, I have been able to solve them all.

At the moment I am struck with Rmath-julia, which shows similar problems as openlibm. In this case, the code seems simpler, and I have already made a pull request (https://github.com/JuliaLang/Rmath-julia/pull/9). @Viral, is there an easy workaround to force the build system to use the HEAD of a dependency instead of a tagged commit? I have tried to understand how julia/deps/Makefile works, but it is full of GNU Make constructs which I do not grasp...

Thanks a lot,

  Maurizio.

 

[Maurizio Tomasi]

Update: I made a lot of progress, but now it seems I've got a quite serious stopper. Julia depends on libunwind, which seems not to be supported under OpenBSD. This seems in turn to be due to a 9-year-old OpenBSD bug (http://marc.info/?l=openbsd-bugs&m=114670090311718).

Unless there is a simple way to remove the libunwind dependency, I guess this is the best I can do at the moment.

If anybody volunteers to continue this effort, the (possibly incomplete) list of packages I had to install before compiling Julia is: bash, blas, cblas, coreutils, fftw, g++, g95, gmake, gmp, lapack, libf2c, pcre. After a bit of tweaking (mainly to force the Makefile to use the latest commit from Rjulia-math and to extend some FreeBSD-related cases to OpenBSD as well), the following command compiles the most of Julia (devel branch):

    gmake CC=egcc CXX=eg++ FC=egfortran LDFLAGS="-lm -lg2c" \

        USE_SYSTEM_BLAS=1 \

        USE_SYSTEM_FFTW=1 \

        USE_SYSTEM_GMP=1 \

        USE_SYSTEM_LAPACK=1 \

        USE_SYSTEM_LIBM=1 \

        USE_SYSTEM_PCRE=1

Cheers,

  Maurizio.

[Jameson Nash]

The libunwind dependency is very shallow, but you would lose backtraces. However, I'm surprised nobody has created that header yet, since the struct that I would expect to be the hard part is required for the sigaction libc function and is defined at http://fxr.watson.org/fxr/source/sys/signal.h?v=OPENBSD#L200.

[Maurizio Tomasi]

@Jameson, I was able to go a little further by modifying libunwind's source code in order to include <signal.h> instead of <ucontext.h>. However, there is some point where non-trivial assembly code needs to be ported:

https://github.com/pathscale/libunwind/blob/vanilla_pathscale/src/x86_64/setcontext.S#L95

I haven't the slightest idea of what this code does, so I'm stuck on this point.

  Maurizio.

[Maurizio Tomasi]

Update: I submitted a PR for libunwind and got the following answer from cbergstrom (https://github.com/pathscale/libunwind/pull/5#issuecomment-138237585):

I don't want to turn you away from pathscale/libunwind, but do you know if Julia can use the Apple libunwind?

I really don't know what to answer here… However, it seems that neither Apple's library is supported on OpenBSD.

  Maurizio.

[Jameson Nash]

It is essentially a copy of siglongjump (link to OpenBSD source for this funciton: http://ftp.asia.edu.tw/OS/OpenBSD/src/lib/libc/arch/amd64/gen/sigsetjmp.S) with slightly different arguments plus some extra work to restore the floating-point state. The purpose of this function is to replace all callee-saved registers with their values at the time they executed the function call from that stack frame, and ending by replacing the instruction pointer (eip/rip). This approximately simulates a direct return from that function. The OS-specific part of it is the execution of the syscall equivalent to sigprocmask and the restoration of the state for the floating-point unit. Julia doesn't actually use or need this functionality, so you can also safely remove it from the final library for your current purposes.

You can try the OSX unwinder by doing `make -C deps install-osxunwind` (source code from https://github.com/JuliaLang/libosxunwind) and see if that gets you any further. I don't know what to expect from that.

[Antoine Jardin]

I intend to install openbsd on my x230 for the purpose of trying to port julia.

Maurizio, have you documented the steps you took to get where you are ? 

Were you using -release or -current ? 

Thanks for your work ! 

Best, 

Antoine

[Maurizio Tomasi]

Hi Antoine,

  Currently I'm struck with libunwind's dependency, which is needed to produce backtraces. I have forked the repository and am working towards making the code compile, but the task is hard because of the many low-level processing the library has to do. Have a look here: https://github.com/ziotom78/libunwind/tree/openbsd

Regarding julia, there are a number of changes I've not put on GitHub yet, as they are not polished yet. I am attaching a patch file which shows my changes with respect to the current state of Julia's master on GitHub (commit 081170435dad70bcaa6a5a80ef1d926ca2ef987d).

If you look above, you'll find a post of mine where I detail which ports you need and which flag to pass to "gmake".

Hope my bits are clear, if you need help just ask.

  Maurizio.

[Antoine Jardin]

Thanks a lot, I will look at it as soon as I find some time, even if I may not be qualified enough to help you on that.

I found that discussion that may be interesting : 

http://comments.gmane.org/gmane.os.openbsd.ports/72423

" libunwind depends on a libc extension called Setcontext, which i quote

wikipedia <is somewhat cumbersome to use effectively>, this libc
extension is present on NETBSD and FREEBSD but having this entering
into openBSD libc may take a long time, even bikeshed are taking time
here."

Best,

Antoine