How to get even better performance

[Paul Wendler]

On Mon, Jun 15, 2012 at 23:10:50 UTC+2, Stefan Karpinski wrote:

More generally, the .NET languages, like JVM languages (and all managed runtime languages, really), are not, imo, a great choice for numerical computing. While you don't need as much control for numerical computing as you do for systems programming, you still need more than any of the managed runtime platforms provide. Not to mention issues like 32-bit pointer limits, which, afaik, both the JVM and CLR are saddled with.

I am pretty new to Julia and came across the project while looking for some way to improve our transition experiences - converting large amounts of Matlab code to desktop applications. I was pleased with the convenient syntax of Julia but struggled with the integration capabilities. In the meantime I found two .NET libraries I am very satisfied with; we use C# with about the same syntax as Matlab does and get a large factor of speedup and do not have to use another language as our application runs on. So I tend to stay on .NET until now, but ... 

Since performance is one key factor for us: in that thread about F# / Julia comparison it sounds, as if Julia could even bring more speed, because of more options to better utilize HW specific features ? Which are they and how are they applied in Julia? And which issues is Stefan referring to regarding 32 bit pointers ? Any link to a source? 

I wonder if there are any experiences from other Julia users for platform independent algorithm development? Is Julia easy to incorporate into own applications? Please point me to some documentation, if possible. 

Thanks a lot! Paul

[Stefan Karpinski]

What I was referring to, although I wasn't super clear, was the issue of creating and indexing into large arrays – ones where the indices get bigger than 2^31-1 ≈ 2 billion, which I don't believe you can do on the JVM because you index into arrays using 32-bit signed integers. I suspect the CLR has the same issue. A recent batch of work by Viral was precisely to allow doing BLAS, LAPACK, FFTW and DSFMT operations on arrays larger than this. This isn't really a performance issue, but rather a scale issue.

At the moment Julia is not particularly well suited creating desktop applications, but once there's a Julia compiler, that will probably change quite a bit (although you'll still need a runtime, but that's true of JVM and CLR as well – people are just more likely to already have that runtime).

--
 
 
 

[Paul Wendler]

Thanks, Stefan. We are making good use of C# pointer arithmetic. So this should not be an issue - at least on the CLR. Your statement suggested, Julia would allow "more control than any managed runtime could provide". Is this based on some future perspective or do  there exist any concrete examples for this already ?   Like I said, anything which would allow for better execution speed would be potentially interesting for us. Platform compatibility is also important, of course. If there was any magic feature added recently, I would love to give it a try ? 

[Stefan Karpinski]

The ability to just call C libraries so easily might qualify and creating mmap-backed arrays might also qualify. Defining your own bits types and using LLVM primitives to operate on them might also qualify. None of these are about performance, however, they're about expressing things that managed runtimes don't let you express – or if they do, they don't let you express them easily.

--