Collaboration with Octave?

[Lachlan Andrew]

Greetings all,

I currently use (and develop) Octave to develop code that I run on both Matlab and Octave.  For that, I need Octave's matlab compatibility, rather than than Julia's choice of just being easy for a Matlab programmer to learn.

However, I'm frustrated by Octave's lack of JIT compiler, and hence painful performance on non-vector code.  I'd like the benefits of both Julia and Octave.  It seems silly that the open source community doubles up so much, with Julia, Octave, SciLab and others, all developing their own GUI, their own JIT compiler etc., for languages that are so similar.

Can anyone comment on how feasible it would be to have a "Matlab compatibility" mode in Julia?
Can anyone comment on how feasible it would be to adapt Julia's JIT compiler to run Octave / Matlab code?
Would anyone be interested in forging closer links between Julia and Octave to reduce the duplication of effort for the GUI and anything else that may be common?

Perhaps this is just wishful thinking, but I thought I'd ask.

Cheers,
Lachlan

[Tony Kelman]

The Matlab language (and hence Octave due to its desire for compatibility) makes a number of decisions that are difficult to semantically analyze and optimize - Matlab puts a lot of resources into their JIT and it still doesn't do as well as Julia in a lot of cases. I don't think developing a compatibility mode within Julia would be that great of a use of resources. Why not a Python compatibility mode or an R compatibility mode at that point? If you could automatically convert code written in those languages into JIT-compilable Julia, then you would just do that and Julia would be an implementation of those languages rather than need to exist as its own language. There's a quote from Jeff floating around somewhere, roughly to the effect of if all he wanted was a faster Matlab, he would have written a bunch of higher-performance algorithms for Octave.

Octave's license is GPL which is more restrictive than Julia's MIT license, and less desirable in some use cases due to its copyleft nature. We couldn't reuse Octave code in base Julia unless it were relicensed, or behind a build flag that you could disable if you wanted to remove GPL components. The Octave developers are nice people though so if there are areas where we could meaningfully collaborate despite the differences in licenses and language semantics, maybe we should.

If Octave has a C API, you could write the equivalent of PyCall or RCall to run octave code from within Julia by invoking the Octave interpreter, and transfer data back and forth in-memory between Julia and Octave. You could probably do the same in reverse via the Octave equivalent of mex files.

There's also a MatlabCompat.jl package that introduces several additional routines and API's that are more in the Matlab style than the idiomatic Julia style.

[Lachlan Andrew]

Tony,

Thanks for your prompt and constructive reply.

Could you please be more specific about which language decisions in Matlab are difficult to analyze and optimize?  (No worries if you don't have an specific examples.)  I haven't yet given up hope of writing a JIT compiler for Octave based on Julia's, and would be glad to know what challenges there will be.

Regarding the licence, I don't think there would be a problem in having some files in Octave that are MIT licensed.  In particular, the console/command window in Octave's GUI is completely different for Windows and Posix systems, and I'd be tempted to investigate using Julia's command window.  Any improvements to that that were made by Octave developers would then be able to be merged back into Julia.  I haven't yet checked out Julia's GUI -- perhaps its already too mature to need help.

One note where I disagree with you: you said that if Matlab etc. could be converted to JIT-compatible Julia there would be no need for Julia as a distinct language.  Julia could still have extensions that are not in the other languages.  Julia is only redundant if it can be trivially translated into the other languages, not the other way around.

Thanks again,
Lachlan

[Mauro]

On Sat, 2016-08-06 at 00:08, Tony Kelman <to...@kelman.net> wrote:
> If Octave has a C API, you could write the equivalent of PyCall or RCall to
> run octave code from within Julia by invoking the Octave interpreter, and
> transfer data back and forth in-memory between Julia and Octave. You could
> probably do the same in reverse via the Octave equivalent of mex files.

There is MATLAB.jl which allows to call into Matlab, similar to PyCall
and RCall. There, Octave was mentioned briefly
https://github.com/JuliaLang/MATLAB.jl/issues/4

[Tony Kelman]

Things like assigning to an element past the end of an array causing the array to grow instead of being an error. You could probably implement some simple optimizations for Octave, like not expanding a dense vector when a range such as "1:n" appears in a for loop. It's hard to tell nowadays how much of the Matlab JIT is using LLVM vs their custom tooling running on the JVM, but any techniques they use to optimize Matlab code you could likely use to optimize Octave code too.

You also have to deal with variables possibly changing type at any point in your code. You could write a type inference algorithm similar to Julia's, but the Octave/Matlab standard library functions aren't designed with type stability in mind so you'd have to be more conservative and more variables would need to account for the possibility of dynamically changing type since you can't prove they don't. Auto-promotion to complex numbers for square roots is the canonical example here.

[Isaiah Norton]

Could you please be more specific about which language decisions in Matlab are difficult to analyze and optimize?  (No worries if you don't have an specific examples.)  I haven't yet given up hope of writing a JIT compiler for Octave based on Julia's, and would be glad to know what challenges there will be.

If you are going down this road, read up on those who have gone before you! Maxime Chevalier-Boisvert wrote a type-specializing JIT compiler for the M(atlab) language using LLVM a few years ago. Not Octave-specific, but most of the same issues apply. In particular, see the technical discussion, as well as related-work reviews, in the journal paper and thesis at [1].  I don't think McVM is actively developed, but there is a BSD-licensed code drop [2] linked from [3].

 

Regarding the licence, I don't think there would be a problem in having some files in Octave that are MIT licensed.  In particular, the console/command window in Octave's GUI is completely different for Windows and Posix systems, and I'd be tempted to investigate using Julia's command window.

FYI, plain julia (not IJulia/Juno etc.) runs in the OS-provided terminal, which is suboptimal on Windows, to say the least.

[1] https://pointersgonewild.com/about/

[2] https://github.com/Sable/mcvm

[3] http://www.sable.mcgill.ca/mclab/projects/mcvm/

[c.d. mclean]

potentially useful and/or of interest, an Octave kernel exists for Jupyter, so in addition to a Julia kernel there may be some Jupyter magics for swapping kernels, etc.

good luck !

[c.d. mclean]

had intended to point to this URL:

https://github.com/calysto/octave_kernel

[Steven G. Johnson]

On Saturday, August 6, 2016 at 1:29:33 AM UTC-4, Lachlan Andrew wrote:\

Could you please be more specific about which language decisions in Matlab are difficult to analyze and optimize?  (No worries if you don't have an specific examples.)  I haven't yet given up hope of writing a JIT compiler for Octave based on Julia's, and would be glad to know what challenges there will be.

Probably a good starting point would be to read Jeff's PhD (https://github.com/JeffBezanson/phdthesis).   "Type stability," for example, is a key concept and is hard to retrofit onto an existing language and its standard library.

[Yichao Yu]

Another good talk is Jeff's talk at the india juliacon and in particular Oscar's quote in it. In short, julia is fast not because of it's jit (in fact it has a pretty stupid jit, other than the optimizations we get from LLVM for "free"). What makes it fast is the design of the language which makes it easier to analysis in most cases despite being a "dynamic"/scripting language.

 

[Henri Girard]

Correcting my first message :
In fact I still had it install, and I notices I installed it with pip3 which means I used python3, I upraded it and run the notebook example and the problem is a black graphic, I investigating where this come from ? (I already read few months ago it was a problem of version on ubuntu.)

[Henri Girard]

I forget the link :
https://github.com/Calysto/octave_kernel/issues/6#issuecomment-104786494

Le vendredi 5 août 2016 22:36:13 UTC+2, Lachlan Andrew a écrit :

[Henri Girard]

Googling I am going further :

%plot gnuplot
t = linspace(0,6*pi,100);
plot(sin(t))
grid on
hold on
plot(cos(t), 'r')
%plot --format svg (makes it)

Le vendredi 5 août 2016 22:36:13 UTC+2, Lachlan Andrew a écrit :

[Henri Girard]

I installed jupyter notebook with octave_kernel (calypso) python3 on ubuntu 16.04 and it works fine. I don't know how to use it with julia.

Le vendredi 5 août 2016 22:36:13 UTC+2, Lachlan Andrew a écrit :

[Henri Girard]

I tried to use octave kernel, once it worked in jupyter python, but the problem is the octave version 4 or 5 (5 didn't work for me, blanck outpout). I tried both octave kernel and calypso. I mean magic and kernel. I wasn't satisfied because very unstable I will give a try again because I need matlab like. Is there any chance to have a julia/maxima ?
It seems to work on jupyter but I didn't success too in installing it help how to config it it quiet difficult to understand for me as english is not my mother language.

Le vendredi 5 août 2016 22:36:13 UTC+2, Lachlan Andrew a écrit :

[Stefan Karpinski]

Note that Henri's messages came through out of order due to the first-time-post moderation queue.

[Henri Girard]

I thaught they had been refused ... That's what i did today...
Sorry it's not in julia but can maybe give some ideas ?
Regards,
Henri

[c.d. mclean]

these developments may prove relevant ...

https://twitter.com/johnmyleswhite/status/767516402802577408:

   https://github.com/stevengj/NBInclude.jl

~ cdm