Julia allocates huge amounts of memory in for loops which should overwrite scalars

[Michael Wojnowicz]

Hi everyone, 

This very simple loop

m=0.0

@time for i=1:1000000

      r= rand()-.05

      m=max(m,abs(r))

end

allocates 32 MB of memory.

I do not have this problem when I write for loops over vector components, but how can I overwrite scalars in Julia?

Thanks a lot and looking forward to any possible tips or ideas.

-Mike

[Jake Bolewski]

http://julia.readthedocs.org/en/latest/manual/performance-tips/

[Michael Wojnowicz]

Thank you.  Which section are you referring to? (Sorry for my ignorance, I am trying to improve my programming from an R/MATLAB background.)

[Jake Bolewski]

m is a global (at least from what you posted).  So I would guess the first section.

If you could, please explain what is not clear about the performance tips section so we can improve it.

[Michael Wojnowicz]

The variables are both of the same type, and wrapping the code inside a function does not solve the problem.  

[Jake Bolewski]

julia> function test()

       m=0.0

       @time for i=1:1000000

             r= rand()-.05

             m=max(m,abs(r))

       end; m

       end

test (generic function with 1 method)

julia> test()

elapsed time: 0.004459659 seconds (0 bytes allocated)

0.9499997082184952

julia> begin

       m=0.0

       @time for i=1:1000000

             r= rand()-.05

             m=max(m,abs(r))

       end; m

       end

elapsed time: 0.040078291 seconds (32002568 bytes allocated)

0.9499986056323748

Are you sure?

[Steven G. Johnson]

function foo(n)
    m = 0.0
    for i=1:n
          r= rand()-.05
          m=max(m,abs(r))
    end
    return m
end
@time foo(1000000)
elapsed time: 0.005688064 seconds (96 bytes allocated)

[Michael Wojnowicz]

So I figured out the problem.   Yes, as you both showed below, wrapping my code snippet within a function seemed to get rid of memory allocation.  But my full code was still allocating memory, and I thought the problem came from iterating OVER (rather than within) the function.   I thought that the results with these simple functions were misleading.

 In particular, I was running an optimization algorithm on a sparse dataset of size 100,000 by 7.6 million, and, as I learned from the useful command 

julia --track-allocation=user yourscript.jl.

my seemingly simple maximum/absolute value function was allocating, by the time of convergence, about 20 GB of data !  

As it turned out, my problem was indeed type stability, which people describe as the #1 source of excess memory allocation.    I had not explicitly specified types in the form function(m::Float64,r::Float64); rather I wrote my function in the form function(m,r).   I assumed that wasn't a problem, as discussed in the performance manual's section on "Type Declarations," because my m values and r values were indeed always Float64's.  

But alas I had initialized 

m=0

instead of 

m=0.0

So I had initialized an integer, rather than a Float64.  I had READ about this mistake in the performance guide before!  (It is discussed in "Avoid changing the type of a variable").  But still, the "m=0" easily passed inspection by my R- and MATLAB-trained eyes.  And in the context of my code, that 0 vs. 0.0 was the difference between allocating 20 GB of data and allocating only 104 bytes!  Amazing.

Well, sorry to rehash the same old problems, but thanks for the help and for helping me to discover this interesting example which really, painfully emphasized an important difference between Julia and MATLAB/R for new users. 

[cdm]

Michael ...

many thanks for the excellent and informative follow up summary.

i too believe that these sorts of closing posts are very helpful

to the newly Julian ...

best,

cdm