Does Julia really solve the two-language problem?

[Sisyphuss]

The two-language problem refers to prototyping with one slow dynamic language and rewrite it with a fast static language for the final product.

If Julia really solves the two-language problem, it should meet the following criteria:

Let A be the code written during prototyping, B be the code written for the final product, with a small net increment $\Delta$, A+\Delta=B.

If Julia uses one code style to do prototyping, and then uses a completely different style to write final product, then it can't be called the same language. At best, Julia turns the 2-language problem to a 1.5-language problem.

[Christoph Ortner]

Just speaking from my own experience in scientific computing (PDEs and molecular simulation): A+\Delta=B. with \Delta small is *not* necessary for Julia to "solve the two-language problem". In fact, I tend to write three iterations: (i) a crappy piece of code that solves the core problem but is neither fast nor elegant; (ii) a complete rewrite, using Julia's type-system to make sure that the code is readable and easily generalisable in the future; (iii) find the bottlenecks and optimise the performance. All three steps work brilliantly for me; there are usually very small parts that I need to optimise for performance, but even if they were bigger, it is extremely useful for productivity to have a working code in the same language to incrementally optimise it. The only "problem" I have, is that I have to be very disciplined in step (i) to not try to incorporate aspects of (ii) or (iii) too early. 

Christoph

[Christof Stocker]

I agree with you in that most of the code I start out with usually
contains some dirty hacks that I later refactor. But that is just my
personal approach to programming -> get something running quickly and
iterate on it

That being said, based on my personal experience with Julia (which is
about 2 months now) one can write simple readable code that is also
efficient. To me it was just a matter of getting used to the language.
Coming from C++ and Scala that "getting used" took me about 1 month.

To me the most important aspect to at least somewhat consider the
two-language problem addressed is that non-programmers should be able to
read and understand an underlying implemented algorithm if he/she has
the domain knowledge. With underlying I mean the algorithm that really
does the heavy lifting. I am pretty happy with the state of Julia in
that regard

[Kristoffer Carlsson]

If Julia uses one code style to do prototyping, and then uses a completely different style to write final product, then it can't be called the same language. At best, Julia turns the 2-language problem to a 1.5-language problem.

 What is it with Julia that requires you to write in two completely different styles? If what you are saying is that you can't just copy paste code from Matlab and get 10x speed up then that is not really fair. 

For all the codes I am writing, sticking to good idiomatic julia code, it is very rarely needed to rewrite large portions of the good for performance. Tweaking some stuff in hot loops maybe but not starting from scratch.

[Stefan Karpinski]

There are different styles and levels of writing code in every language. There's highly abstract C++ and there's low-level pointer-chasing C++ that's basically C. Even in C there's the void*-style of programming which is effectively dynamically typed without the safety. Given this fact, I'm not sure what solving the two language problem would look like from the perspective this post is posing. Enforcing only one style of programming sounds like a problem to me, not a solution. On the contrary, I think one of Julia's greatest strengths is its ability to accommodate a very broad range of programming styles and levels.

[LarryD]

How does this compare to the "old standard?"

1.  Fortran requires every variable to have a defined type, but automatically defaults to floating or integer type based on the first letter of the variable name. Or, you can pre-define your own sets of first-letter defaults with the Implicit command. Once you get used to this, writing new code goes just as fast as if you didn't need to define types.

2.  Optimizing compilers take care of most of the clean-up phase. No work required.

Larry

[Stefan Karpinski]

Fortran is an awesome language, but clearly there's demand for something easier. Keep in mind that every single popular technical computing environment is dynamically typed.

[Kristoffer Carlsson]

Having the type of a variable be determined by the variable name is craziness. Which is why you always run with implicit none.

[Páll Haraldsson]

On Sunday, October 18, 2015 at 12:51:21 PM UTC, Sisyphuss wrote:

The two-language problem refers to prototyping with one slow dynamic language and rewrite it with a fast static language for the final product.

Yes, but there is one other two (or more..) language "thing" (I wouldn't call it a "problem", would you?):

People using Python/Numpy, MATLAB, etc. are used to having to reimplement a tiny part of their code (or not so..?) in C. [Often you reuse those solutions (or pure ones) and to not even see a two language problem.]

In Julia (and new languages in general), theoretically, most code could be only in Julia, but practically you reuse slow (or fast..) libraries in other languages. That seems to be a great thing to me! Maybe I'm in minority, as I've found it difficult to "convert" people, but I can say, that seems to be changing..

Do you people consider it a hurdle to call, say using PyCall or whatever (do usres need to know (too..) much else than "Python and the relevant to me library" exists?)? Are wrappers (using PyCall) preferred or even reimplementiong stuff in pure Julia?

If you are only reusing libraries, you do not have to know (much about) the other languages, but still, you might have to/want to (end up) maintain those libraries.. I forsee that, in many cases a permanent "solution" (or "problem" if you will). Do you think for most things, calling other languages, to be a stop-gap solution?

-- 

Palli.

[Art Kuo]

Having the type of a variable be determined by the variable name is craziness. Which is why you always run with implicit none. 

It had its reason back in its day. For math it is/was typical to choose counter-like integers for things like a series, with variables like i, j, k, l, m, or n. It wasn't unreasonable to carry that convention into Fortran 66. Back in the days of punch cards and 1024-byte memory, variable names were often one letter and one alphanumeric, so the easier/shorter the declaration, the better. In Fortran 77, "implicit" was introduced as a way to maintain backwards compatibility and also optionally break from the i-n convention and be clear about it. Nowadays we can be thankful for upper and lower case, and the backspace key makes long variable names trivial. 

[Steven G. Johnson]

On Sunday, October 18, 2015 at 10:01:20 AM UTC-4, Stefan Karpinski wrote:

There are different styles and levels of writing code in every language. There's highly abstract C++ and there's low-level pointer-chasing C++ that's basically C. Even in C there's the void*-style of programming which is effectively dynamically typed without the safety. Given this fact, I'm not sure what solving the two language problem would look like from the perspective this post is posing. Enforcing only one style of programming sounds like a problem to me, not a solution. On the contrary, I think one of Julia's greatest strengths is its ability to accommodate a very broad range of programming styles and levels.

Even forgetting about pointers etc, the fact is that highly optimized code often looks very different from unoptimized code in any language.   e.g. the simplest way to implement a matrix multiplication in C is three loops.   Getting decent performance (compared to peak flops) requires 100+ lines of code.  Super-optimized implementations require tens of thousands of lines of code.  This has nothing to do with safety and pointers, and everything to do with locality (for caches) and unrolling (for registers).

There is no language in which optimization doesn't typically involve rewriting critical code.

[Tamas Papp]

Still, in a nice language the programming effort vs speed curve should
not be too steep. Eg you can get 80% of the speed with 20% of the
effort, both compared to the absolute super fastest solution.

Julia is a nice language: with a few simple rules that one pick ups up
after a bit of experience (= mistakes) [1], it does not take a whole lot
of extra effort to write reasonably fast code. Getting within a factor
of 2-5 of C with significantly less effort is enough for a lot of
applications, especially considering that in scientific computing a lot
of code is only used a few times (analyze a problem, estimate a model,
etc). For the rest, one can optimize further.

The "two language problem" is really about a discontinuity in the effort
vs speed curve. You hit the limits of Language A, you have to go to
Language B, which is significantly different. In Julia you can make a
lot of incremental optimizations.

Best,

Tamas

[1] http://docs.julialang.org/en/release-0.4/manual/performance-tips/

[Sisyphuss]

I define Language C := Union{A,B}. Then it solves the two language problem.

[Tamas Papp]

"I think you'll find it's a bit more complicated than that ..." :-)

Best,

Tamas

[lawrence dworsky]

I certainly am not arguing against developing new languages with improved programming techniques, but I think you're being too quick to consider liking an older technique as heresy. I programmed in Fortran for over 4 decades and never had any trouble with the variable name typing system:

1.  Once you're used to it, it becomes second nature and doesn't slow down program development at all. If anything, it speeds it up a bit.

2.  If you really hate it, you can turn it off - either selectively or completely (implicit none).

3.  It doesn't restrict program structure or development in any way.

4.  It doesn't interact with program execution speed in any way.

This last point, I think, is very important. I'm a Julia newbe so I'm just reading all the stuff about the interaction between variable typing and execution speed. I've gotta tell you, having a language where I don't worry about this at all and then the compiler handles any optimization issues has a lot to say for it. Why is this "craziness?"

Larry

[Stefan Karpinski]

On Monday, October 19, 2015, Tamas Papp <tkp...@gmail.com> wrote:

The "two language problem" is really about a discontinuity in the effort
vs speed curve. You hit the limits of Language A, you have to go to
Language B, which is significantly different. In Julia you can make a
lot of incremental optimizations.

Yes! This is very well put – I really like this way of explaining it.

[Steven G. Johnson]

On Monday, October 19, 2015 at 2:31:16 PM UTC-4, lawrence dworsky wrote:

This last point, I think, is very important. I'm a Julia newbe so I'm just reading all the stuff about the interaction between variable typing and execution speed. I've gotta tell you, having a language where I don't worry about this at all and then the compiler handles any optimization issues has a lot to say for it. Why is this "craziness?"

An untyped variable in Julia is a completely different beast from an untyped variable in Fortran 77 style with implicit typing.

If I write f(x) = x+1 in Julia, the code is totally polymorphic — the function f(x) works for any  x that supports addition (+) with an Int (1), including bigints, arrays, all the different floating-point types, complex numbers, external-package types like quaternions, etc.  Furthermore, f(x) is fast for all of those types, because the Julia compiler specializes f into a different compiled version for each argument type as needed.

If you use an implicitly typed variable "x" in Fortran (without "implicit none"), then it is not polymorphic.  x has one and only one type (real, i.e. floating-point, either single or double precision depending on a compiler switch), and your code only works with that type.

The advantage of the polymorphism and dynamism in Julia is that code is much more re-usable — we can implement a function like sum(x) once and it will work with any type.   (This is a generalization of the polymorphism that e.g. object-oriented programming or C++ templates give you.)    The price you pay for this is that if you want it to be fast, you need to think a little bit more about types than you would with code that worked with one and only one type.

(If you want code that works with one and only one type in Julia, of course you can do it, just by explicit type declarations.  But it is more powerful to get used to thinking about dynamic polymorphism, and learning to follow the simple rules that will make such code fast.)

[John Gibson]

Having a language where you don't worry about types and the compiler handles optimization is possible in Fortran because it has a dead-simple, nonextendable set of types. The downside is that you can't do abstraction in Fortran, and all your code ends up being a long sequence of low-level operations on the fundamental numeric types. 

From my experience having an extensible type system and well-designed libraries of high-level types (strings, matrices with no structure, special structure) is well worth the added complexity of types, if for no other reason than that you can represent high-level concepts with high-level expressions, making code compact and readable. 

In Julia you have to think about types harder than in Python, but the benefit is you get extendable high-level abstraction, compact, readable code, polymorphic code generation, and nearly the speed of C. It's a winning combination!

"craziness" would apply to a language with the rich type system of Julia but the implicit type system of Fortran. Yikes!

John

On Monday, October 19, 2015 at 2:31:16 PM UTC-4, lawrence dworsky wrote: