Automatic (?) array bounds checking
Jared Ahern
the automatic (?) array result and the array I was assigning it to.
Here is a simplified example, with outputs in comments:
PROGRAM calls
IMPLICIT NONE
INTEGER :: a(2), b(3), c(6), n
c = myfunc(a,b)
WRITE(*,*) "c:",c !! gives "c: 1 2 3 4 5 6"
n = 5
c = 0
c(1:n) = myfunc(a,b)
WRITE(*,*) "c:",c !! gives "c: 1 2 3 4 5 0"
CONTAINS
FUNCTION myfunc(v1,v2) RESULT(v3)
IMPLICIT NONE
INTEGER, INTENT(IN) :: v1(:), v2(:)
INTEGER :: v3(SIZE(v1,1)*SIZE(v2,1))
INTEGER :: i, j, k
DO i=1,SIZE(v1,1)
DO j=1,SIZE(v2,1)
k = (i-1)*SIZE(v2,1) + j
v3(k) = k
ENDDO
ENDDO
WRITE(*,*) "myfunc: v3:",v3 !! always gives "v3: 1 2 3 4 5 6"
END FUNCTION myfunc
END PROGRAM calls
This was compiled with: g95 -g -fbounds-check -Wall -Wextra -o calls
calls.F90
The second result should be a runtime bounds-check error in some way,
no? What would be a good way to catch this error?
If it were a subroutine, I think that I would try to change
occurrences like that above to just use a standard assumed-shape array
for the INTENT(OUT) array v3, and check the size of v3 explicitly, but
I don't think that you can have a deferred-shape function result, so
that wouldn't work here. I could also in this case force it to be a
compile time error (with v3(6)), but not in the general case.
In all honestly, I'm also having problems with this because I guess
I'm just not sure how functions return results, and how that relates
to automatic arrays. My understanding is that in this case, the
function allocates a temporary v3, and then the assignment statement
copies the values (or probably a reference?) to the array c. Or is it
possible for the compiler to see the other side of the assignment, and
pass c into the function? I would greatly appreciate any
clarification you can provide!
Thanks,
Jared
Richard Maine
> So I recently ran into a coding error wherein I had a mismatch between
> the automatic (?) array result and the array I was assigning it to.
...
> The second result should be a runtime bounds-check error in some way,
> no? What would be a good way to catch this error?
See below.
> If it were a subroutine, I think that I would try to change
> occurrences like that above to just use a standard assumed-shape array
> for the INTENT(OUT) array v3,
Yes, subroutines are a lot simpler. I often recommend using them. It
seems moderately common to find people using functions because they like
the style better, but then having to contort things around the problems
of using functions. That seems to be the price they pay for such styles.
I've regularly seen that with functions that return pointers, which I
fairly strongly disrecommend. (Yes, there are cases where you can do it
"safely", but I find it a lot simpler to just say "don't do that" rather
than to try to help people with the more complicated problem of
understanding what cases are safe and why.)
> In all honestly, I'm also having problems with this because I guess
> I'm just not sure how functions return results, and how that relates
> to automatic arrays. My understanding is that in this case, the
> function allocates a temporary v3, and then the assignment statement
> copies the values (or probably a reference?) to the array c. Or is it
> possible for the compiler to see the other side of the assignment, and
> pass c into the function? I would greatly appreciate any
> clarification you can provide!
From the standard's perspective, it is as you describe first above. The
function allocates a temporary result. Keep in mind that a function is
*NOT* necessarily invoked in a simple form like
variable = function(args)
Much of the point of a function is that it can be used in an expression.
If you are never going to invoke the function in any way other than the
above, I'd say there was little benefit to making it a function; one
might as well make it a subroutine, which has fewer complications.
A compiler might well optimize the simpler cases, but understand that
any such thing is an optimization rather than the basic definition of
what a function does. The details of such optimizations will vary among
compilers, although one could probably make generalizations about what
optimizations are more or less likely.
I would say that the main problem here is not the function reference,
but rather the assignment. Some people tend to think of the two as being
a single thing, but they really aren't. The function invocation is
(presumably) returning the right thing; it is just that what is returned
can't then be validly assigned to the variable.
Some compilers ought to be able to detect such things as shape mismatch
in an assignment, provided you ask for it. That is, in general a
run-time test and might need a specific switch to turn on. I don't have
a handy list of compiler switches; of course, it will vary among
compilers.
In f2003, one might argue for making variable an allocatable array. In
that case, it will automatically reallocate to the appropriate size when
something is assigned to it. I'm not sure how many compilers have yet
implemented that f2003 feature, though. In f95, assignment to an
allocatable array doesn't do anything special; if the shape is wrong,
that's just an error (and one that the standard doesn't require that
compilers be able to diagnose).
--
Richard Maine | Good judgment comes from experience;
email: last name at domain . net | experience comes from bad judgment.
domain: summertriangle | -- Mark Twain
dpb
> So I recently ran into a coding error wherein I had a mismatch between
> the automatic (?) array result and the array I was assigning it to.
> Here is a simplified example, with outputs in comments:
>
> PROGRAM calls
> IMPLICIT NONE
> INTEGER :: a(2), b(3), c(6), n
>
> c = myfunc(a,b)
> WRITE(*,*) "c:",c !! gives "c: 1 2 3 4 5 6"
> n = 5
> c = 0
> c(1:n) = myfunc(a,b)
> WRITE(*,*) "c:",c !! gives "c: 1 2 3 4 5 0"
>
> CONTAINS
>
> FUNCTION myfunc(v1,v2) RESULT(v3)
> IMPLICIT NONE
> INTEGER, INTENT(IN) :: v1(:), v2(:)
> INTEGER :: v3(SIZE(v1,1)*SIZE(v2,1))
> The second result should be a runtime bounds-check error in some way,
> no? What would be a good way to catch this error?
a) no
b) what error?
c is a fixed-size array of size 6, the second call only assigned values
for the first five (5) locations after the entire array had been set to
zero just above. The resulting contents of the array are as the write
shows.
In the subroutine, the size of v3 is also consonant w/ the size of the
resulting array as it was determined from the fixed sizes of v1 and v2
and those values happen to be consonant w/ the fixed size of the result.
--
Richard Maine
> Jared Ahern wrote:
[code elided]
> > The second result should be a runtime bounds-check error in some way,
> > no? What would be a good way to catch this error?
>
> a) no
>
> b) what error?
> [explains why the code has no error]
Oh. Oops. I hadn't noticed that. There is a class of errors that involve
assigning an array function result to an array of the wrong shape. So on
seeing that was the kind of thing being asked about, I jumped from there
to some discussion of that class of error... but I failed to notice that
this code doesn't actually have the error. If the code had a line like
c = myfunc(a,b)
instead of
c(1:n) = myfunc(a,b)
then it would havee the error in question.
mecej4
What you have is not a bounds-check error, since you declared the size
of c to be 6, and the bounds of 1 and 6 are not exceeded. Rather, you
have a nonconformant array assignment (assigning a size-6 array to a
size-5 section of a variable). Whether this error is caught or not
depends on the compiler. Salford FTN95 and Lahey LF95 catch this
error; G95 does not.
I do not think that there is any compiler/switch combination that can
be relied upon to catch all possible errors. Even if one were to
exist, the heavy consumption of resources that would occur with such
massive checking would preclude the use of such checking except for
modest-size programs.
-- mecej4
>
> Thanks,
> Jared
Jared Ahern
> b) what error?
Well, another thing you can do is replace c(1:n) with an array d of
declared length 5. Then, we are clearly assigning a 6-element array
to a 5-element one. While I agree with you that copying the first 5
values over and ignoring the rest are what I would expect to be most
likely to happen, I'm not convinced that that result is actually
"correct". I would really expect it to be an array-bounds error, as
noted previously (if the compiler can pick it up).
> c is a fixed-size array of size 6, the second call only assigned values
> for the first five (5) locations after the entire array had been set to
> zero just above. The resulting contents of the array are as the write
> shows.
Right, I'm not really saying that the result doesn't make sense. But
I'm basically doing c(1:5) = v3(1:6), though obscured by the function
call; this would be an error as written, I believe.
> In the subroutine, the size of v3 is also consonant w/ the size of the
> resulting array as it was determined from the fixed sizes of v1 and v2
> and those values happen to be consonant w/ the fixed size of the result.
Yes, the size of v3 is correct; as noted by Richard, the problem lies
mainly with the assignment.
Richard Maine
> dpb <no...@non.net> wrote:
>
> > Jared Ahern wrote:
> [code elided]
> > > The second result should be a runtime bounds-check error in some way,
> > > no? What would be a good way to catch this error?
> >
> > a) no
> >
> > b) what error?
> > [explains why the code has no error]
>
> Oh. Oops. I hadn't noticed that. There is a class of errors that involve
> assigning an array function result to an array of the wrong shape. So on
> seeing that was the kind of thing being asked about, I jumped from there
> to some discussion of that class of error... but I failed to notice that
> this code doesn't actually have the error.
Obviously not my day for studying things carefully. :-(
On third look, yes the original code does have the error as it assigns a
size 6 function result to a size 5 slice.
I can usually count on dpb getting things right, so I guess I jumped to
fast to believe his explanation here. :-)
Jared Ahern
> Yes, subroutines are a lot simpler.
I agree, and the change I proposed would seem to remove any problems.
However, note that if I use a subroutine with
INTEGER, INTENT(OUT) :: v3(SIZE(v1,1)*SIZE(v2,1))
I get the same result as with the function here. Why that is not
detected at runtime is perplexing to me. Maybe I should notify the
gg95 group.
> It seems moderately common to find people using functions because they like
> the style better, but then having to contort things around the problems
> of using functions.
Yes, this function was not originally used in this way; but re-use of
the results became required, and the function is fairly long to call
repeatedly. I just need to go though and replace it (and its cousins)
with subroutines.
> > My understanding is that in this case, the
> > function allocates a temporary v3, and then the assignment statement
> > copies the values (or probably a reference?) to the array c.
>
> From the standard's perspective, it is as you describe first above.
Did you mean that it always copies the values? Or will a reference be
used where practicable?
> Keep in mind that a function is
> *NOT* necessarily invoked in a simple form like
>
> variable = function(args)
> ...
Very true. I could see optimizing these assignments being quite
tricky.
> I would say that the main problem here is not the function reference,
> but rather the assignment.
That was my thought as well. I just wasn't sure if my understanding
of the process was correct!
> Some compilers ought to be able to detect such things as shape mismatch
> in an assignment, provided you ask for it. That is, in general a
> run-time test and might need a specific switch to turn on.
That's why I gave my option list - maybe someone around here knows if
I'm missing the critical one.
> In f2003, one might argue for making variable an allocatable array. In
> that case, it will automatically reallocate to the appropriate size when
> something is assigned to it.
Ha! That's a good point. In actuality, I was assigning "v3" to an
allocatable array "c"; but I don't think that it was changing size (my
real problem was that I had an input argument "v2" that was too large,
but my "c" was correct for my purposes; and I was surprised when this
wasn't caught at run-time). So I guess if "c" is allocatable, there
just isn't any runtime error to check - perhaps this could be why this
is not a checked for in general.
Thanks again!
Jared Ahern
> of c to be 6, and the bounds of 1 and 6 are not exceeded. Rather, you
> have a nonconformant array assignment (assigning a size-6 array to a
> size-5 section of a variable). Whether this error is caught or not
> depends on the compiler. Salford FTN95 and Lahey LF95 catch this
> error; G95 does not.
Ah, interesting. I was lumping the two together. I'll let the g95
maintainer know; never bad to have another check, right!
> I do not think that there is any compiler/switch combination that can
> be relied upon to catch all possible errors. Even if one were to
> exist, the heavy consumption of resources that would occur with such
> massive checking would preclude the use of such checking except for
> modest-size programs.
Well, you make a good point, but I wouldn't think that a "turn on all
the possible checking you can think of" compiler flag is a really
terrible idea. It seems to me that the more errors you can catch when
debugging, the better. Then just turn the flag off later. But you're
right, it's not a good plan to rely on this.
dpb
> Richard Maine <nos...@see.signature> wrote:
>
>> dpb <no...@non.net> wrote:
>>
>>> Jared Ahern wrote:
>> [code elided]
>>>> The second result should be a runtime bounds-check error in some way,
>>>> no? What would be a good way to catch this error?
>>> a) no
>>>
>>> b) what error?
>>> [explains why the code has no error]
>> Oh. Oops. I hadn't noticed that. There is a class of errors that involve
>> assigning an array function result to an array of the wrong shape. So on
>> seeing that was the kind of thing being asked about, I jumped from there
>> to some discussion of that class of error... but I failed to notice that
>> this code doesn't actually have the error.
>
> Obviously not my day for studying things carefully. :-(
>
> On third look, yes the original code does have the error as it assigns a
> size 6 function result to a size 5 slice.
>
> I can usually count on dpb getting things right, so I guess I jumped to
> fast to believe his explanation here. :-)
Oh, shucks! and oops for us both...I failed to catch it's an assignment
to the slice instead thinking of it as a subset of the array so that it
is, indeed an error. :(
--
dpb
>> a) no
>> b) what error?
>
> Well, another thing you can do is replace c(1:n) with an array d of
> declared length 5. Then, we are clearly assigning a 6-element array
Yeah, see Richard's comments--he caught my oversight on a
second/third(?) look. I apparently fooled him, too, on first pass. :(
Sorry...
--
dpb
...
> What you have is not a bounds-check error, since you declared the size
> of c to be 6, and the bounds of 1 and 6 are not exceeded. Rather, you
> have a nonconformant array assignment ...
bingo!!! I only wish I had been so observant. :(
--
Richard Maine
> > > My understanding is that in this case, the
> > > function allocates a temporary v3, and then the assignment statement
> > > copies the values (or probably a reference?) to the array c.
> >
> > From the standard's perspective, it is as you describe first above.
>
> Did you mean that it always copies the values? Or will a reference be
> used where practicable?
See the other comments about optimization. The standard describes it
as... well... it doesn't use the word "copy", but that's what the
description amounts to. Other things might happen, but they would be
optimizations. In order for them to be valid optimizations, they have to
have the same effect as copying (but might be faster). In many cases,
just making a reference isn't going to be able to do that; in some
special cases, it might.
For thinking about what ought to work, just consider it to be a copy,
insomuch as that is basically the mechanism described in the standard.
When worrying about things like speed (and maybe memory usage if the
array is really large), you can consider likely optimizations. But one
needs to first get the basic definition straight before getting into
optimizations.
> > In f2003, one might argue for making variable an allocatable array. In
> > that case, it will automatically reallocate to the appropriate size when
> > something is assigned to it.
>
> So I guess if "c" is allocatable, there
> just isn't any runtime error to check - perhaps this could be why this
> is not a checked for in general.
I doubt that explains it. Probably just a check got omitted. Note that
the reallocation thing for allocatables is new to f2003. F95 doesn't
specify that. Also note that, even if c is allocatable a slice like
c(1:n) is not. If you assign to c(1:n), no reallocation is going to
happen in any case.
Jared Ahern
Oh not a problem. Heck, I coded the thing wrong to begin with!
paul.rich...@gmail.com
paul.rich...@gmail.com
gfortran catches the error in runtime but it requires the bounds check
option:
/irun4.3/bin/gfortran -fbounds-check jahed_1.f90
[root@localhost tmp]# ./a.out
myfunc: v3: 1 2 3 4
5 6
c: 1 2 3 4
5 6
At line 25 of file jahed_1.f90
Fortran runtime error: Array bound mismatch for dimension 1 of array
'v3'
I believe but am not sure that g95 has something similar.
Paul
mecej4
I wonder if GFortran is misrepresenting the error. The error message
suggests that GFortran is inferring the size of the assignee from the
LHS of the assignment statement on Line-9, rather than from the RHS
expression on the same line. The former has dimension 1:5, the latter
has 1:6. MRC, Sec. 3.11 says "..if the expression includes a reference
to the array variable or to a part of it, the expression is
interpreted as being fully evaluated before the assignment commences".
The extent of the expression is 6; the extent of the lvalue is 5.
Following Oliver Hardy's advice, "let's really make it dirty", I added
to the RHS the term
+ (/(i+2,i=0,n)/)
and added i to the declaration of integers on Line-3. After this
change, GFortran gave this spurious message at compile time
err.f90:9: warning: 'i' may be used uninitialized in this function
and, despite the bounds-check being on, gave no error during execution.
Jim Xia
Many compilers do have flags to turn on for run-time bounds checking
on array assignment since this is not something uncommon. For
example, both XLF and NAG compilers do this although it means runtime
checking and execution slowdowns. So likely this is a bug in g95.
A comment on the idea of using of subroutine is that it may not be a
natural way how the scientists express the computation solution. That
may be the reason why the function still popular.
Another comment on automatic reallocation for allocatable arrays is
that it wouldn't work for the form of assignment listed in the source
code: c(1:n) = funcThatReturnArrays(...). C(1:n) is an array section
and does not have allocatable attribute. There are compilers
implemented the automatic reallocation on assignment of Fortran 2003,
at least IBM and Cray compilers I know of.
Cheers,
Jim Xia