where construct causing stack overflow for large arrays

[James Tursa]

I was recently attempting to compare timing of a MATLAB simple array
element replacement with similar Fortran code. The MATLAB code simply
takes an input array and sets all elements less than 0.5 to zero, ala:

b( b < 0.5 ) = 0;

The MATLAB code creates the logical array result of b<0.5, and then
uses that result (i.e., tests the result again) to do the appropriate
element assignments. The Fortran code I compared against was a simple
loop:

do i=1,n
if( b(i) < 0.5d0 ) b(i) = 0.d0
enddo

Not surprisingly it ran faster, about 4 times faster for a large
random array on [0,1], I presume because of the absence of the large
temporary logical array and the additional tests involved in the
MATLAB version. So then I decided to test against the Fortran where
construct:

where( x < 0.5d0 ) x = 0.d0

Here I got a stack overflow. Apparently the x < 0.5d0 temporary
logical array result was being created on the stack and it overflowed
the stack for the large array I was trying. MATLAB did not have the
same problem I presume because it was grabbing the memory for the
temporary logical array result from the heap.

This seems to be a serious drawback to using the where construct in
Fortran. You could have a perfectly fine program that works for
smaller size arrays, and plenty of memory for larger arrays, but the
program could bomb unexpectedly down the road when larger arrays are
used with the code.

Is there any way to force Fortran to use heap memory for these types
of cases to give the program a better chance or running with large
arrays? Or am I forced to go back and recompile with a larger stack
each time I run into this type of problem with where? I couldn't find
any specific info on this in the compiler doc. I suspect that this
would be an issue for all array constructs creating temporary arrays,
not just the where construct. This would be an incentive to always use
loops for this type of stuff and avoid the array constructs since you
would avoid potential stack problems later on.

I am using Intel 10.0 on WinXP.

James Tursa

[Jugoslav Dujic]

James Tursa wrote:
<snip>

> Is there any way to force Fortran to use heap memory for these types
> of cases to give the program a better chance or running with large
> arrays? Or am I forced to go back and recompile with a larger stack
> each time I run into this type of problem with where? I couldn't find
> any specific info on this in the compiler doc. I suspect that this
> would be an issue for all array constructs creating temporary arrays,
> not just the where construct. This would be an incentive to always use
> loops for this type of stuff and avoid the array constructs since you
> would avoid potential stack problems later on.
>
> I am using Intel 10.0 on WinXP.

For Ifort, the option is /heap-arrays. In Visual Studio IDE,
it's in Optimization/Heap arrays (and not, as documentation says,
in Data/Allocate Automatics to the Heap).

While that answers that question, I'm puzzled why the compiler
has generated a temporary on that particular WHERE.; evaluating
it as a do-loop ought to have been more efficient...

--
Jugoslav
www.xeffort.com
Please reply to the newsgroup.
You can find my real e-mail on my home page above.