Defined vs. Intrinsic Assignment

[Andrew Baldwin]

In trying to overload the assignment operator for a derived type in a reallocation on assignment scenario, a friend inadvertently created unexpected behavior in GCC. The test program is the following:


MODULE A_TEST_M
TYPE :: A_TYPE
INTEGER :: INT
!
CONTAINS
!
GENERIC :: ASSIGNMENT (=) => ASGN_A
PROCEDURE, PRIVATE :: ASGN_A
END TYPE

CONTAINS

ELEMENTAL SUBROUTINE ASGN_A (A, B)
CLASS (A_TYPE), INTENT (INOUT) :: A
CLASS (A_TYPE), INTENT (IN) :: B
!
A%INT = 45
END SUBROUTINE
END MODULE A_TEST_M

PROGRAM ASGN_REALLOC_TEST
USE A_TEST_M
TYPE (A_TYPE), ALLOCATABLE :: A(:)
!
ALLOCATE (A(4))
A(1:2)%INT = 7
A(3:4)%INT = 13
!
A = A(1:2)
!
PRINT *, 'SIZE(A)', SIZE(A), 'A:', A
END PROGRAM

When compiled under GCC the results look like this (valgrind output):

==30070== Memcheck, a memory error detector
==30070== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==30070== Using Valgrind-3.11.0.SVN and LibVEX; rerun with -h for copyright info
==30070== Command: ./a.out
==30070==
==30070== Invalid read of size 8
==30070== at 0x402F148: _intel_fast_memcpy (vg_replace_strmem.c:946)
==30070== by 0x400CA1: MAIN__ (asgn_realloc_test.f:29)
==30070== by 0x400DF3: main (asgn_realloc_test.f:22)
==30070== Address 0x50f37c8 is 0 bytes after a block of size 8 alloc'd
==30070== at 0x402BB20: malloc (vg_replace_malloc.c:299)
==30070== by 0x400C13: MAIN__ (asgn_realloc_test.f:29)
==30070== by 0x400DF3: main (asgn_realloc_test.f:22)
==30070==
SIZE(A) 4 A: 45 45 0 0

If you compile with -fcheck=all then you'll get the runtime error:

Fortran runtime error: Array bound mismatch for dimension 1 of array 'a' (2/4)

What I would expect the output to be what you get when using intrinsic assignment:

SIZE(A) 2 A: 7 7

In 7.2.1.4 of my copy of the standard it states:

"A subroutine defines the defined assignment x1 = x2 if"

...

"(5) either
(a) the ranks of x1 and x2 match those of d1 and d2 or
(b) the subroutine is elemental, x1 and x2 are conformable, and there is no other subroutine that defines the assignment."

(5)(a) does not seem to apply since I believe d1 and d2 are rank 0 and x1 and x2 are rank 1. (5)(b) does not seem to apply since x1 and x2 are not conformable, I believe their shapes are [4] and [2] respectively. So, I would not expect the overloaded assignment to be applied (and instead intrinsic assignment is applied).

Is this valid code and/or a valid interpretation of the standard?

[Ian Harvey]

...

> "A subroutine defines the defined assignment x1 = x2 if"
>
> ...
>
> "(5) either
> (a) the ranks of x1 and x2 match those of d1 and d2 or
> (b) the subroutine is elemental, x1 and x2 are conformable, and there is no other subroutine that defines the assignment."
>
> (5)(a) does not seem to apply since I believe d1 and d2 are rank 0 and x1 and x2 are rank 1. (5)(b) does not seem to apply since x1 and x2 are not conformable, I believe their shapes are [4] and [2] respectively. So, I would not expect the overloaded assignment to be applied (and instead intrinsic assignment is applied).
>
> Is this valid code and/or a valid interpretation of the standard?

Maybe, but I don't like where this goes.

I thought the conceptual model was that whether defined assignment was
involved (and if it was involved, the specific binding (or specific
procedure to be called if the generic interface for assignment was not
type bound)) could be resolved at "compile time" based on argument TKR.
Whether two arrays are conformable is a runtime aspect in the general
case.

In the general case, where the shape of the left and right hand side is
unknown, does such a statement violate the requirement that an
assignment statement (a source code concept) meet the requirements of
*either* intrinsic assignment *or* defined assignment?

I hope this is an oversight in the standard.

[Andrew Baldwin]

I would like to overload assignment with an elemental subroutine. And, I would like to be able to utilize reallocation on assignment (or an approximation thereof). As far as I can tell, both are possible on their own but not together. As an outsider to the standard and compiler that seems odd.

Ideally (for me) overloaded assignment with an elemental subroutine would end up used for the assignment within reallocation on assignment. That way, overloading assignment with an elemental subroutine would be sufficient to satisfy both my desires. I have not been able to find support for that ideal in the standard. GCC does not seem to support it, and neither does NAG nor Crayftn from what I've heard.

(Might this ideal scenario allow the "defined or intrinsic" question to be addressed at compile time?)

The lack of the ideal could be worked around with a separate defined assignment, but I cannot seem to convince the compiler to not use the elemental subroutine. For example, with the following program:

MODULE A_TEST_M
TYPE :: A_TYPE
INTEGER :: INT
!
CONTAINS
!
GENERIC :: ASSIGNMENT (=) => ASGN_A
PROCEDURE, PRIVATE :: ASGN_A
END TYPE

INTERFACE ASSIGNMENT (=)
MODULE PROCEDURE REALLOC_ASGN_A
END INTERFACE ASSIGNMENT (=)
CONTAINS

SUBROUTINE REALLOC_ASGN_A (A, B)
TYPE (A_TYPE), ALLOCATABLE, INTENT (INOUT) :: A(:)
TYPE (A_TYPE), INTENT (IN) :: B(:)
TYPE (A_TYPE), ALLOCATABLE :: TMP(:)
!
ALLOCATE (TMP(SIZE(B)))
TMP(:)%INT = B(:)%INT
CALL MOVE_ALLOC (TMP, A)
END SUBROUTINE

ELEMENTAL SUBROUTINE ASGN_A (A, B)
CLASS (A_TYPE), INTENT (INOUT) :: A
CLASS (A_TYPE), INTENT (IN) :: B
!
A%INT = 45
END SUBROUTINE
END MODULE A_TEST_M

!

PROGRAM ASGN_REALLOC_TEST
USE A_TEST_M
TYPE (A_TYPE), ALLOCATABLE :: A(:)
!
ALLOCATE (A(4))
A(1:2)%INT = 7
A(3:4)%INT = 13
!
A = A(1:2)
!
PRINT *, 'SIZE(A)', SIZE(A), 'A:', A
END PROGRAM

If I comment out the bound, defined assignment ASGN_A in the type definition, then everything seems to work fine and REALLOC_ASGN_A is utilized. If I do not, then the result is the same as before.

[Ian Harvey]

Nor ifort.

> (Might this ideal scenario allow the "defined or intrinsic" question to be addressed at compile time?)

Today the standard pretty clearly says you either have intrinsic
assignment, or defined assignment. Defined assignment doesn't involve
automatic reallocation on assignment.

In some hypothetical future, I think you would have to think very
carefully about how this would all work and what it will achieve. The
reallocation of the left hand side to the shape of the right hand side
would have to happen *before* the elemental procedure could be invoked.
That reallocation would imply finalization and loss of the value of
the left hand side in the general case - the left hand side would be
undefined prior to the call - it would only make sense for the
corresponding argument to be INTENT(OUT). That slays a great deal of
applications of defined assignment there and then.

I think that's a compiler bug. Elemental procedures are only supposed
to be considered if there is no non-elemental procedure that matches on
rank, and that consideration covers the merged set of specific bindings
and procedures for the generic identifier associated with assignment.

(If the left hand side of your assignment was not allocatable, I would
expect an error message about an inappropriate actual argument for the
allocatable dummy.)

This does not address your specific query, and may not be a relevant
approach depending on the specifics of what you want to do in the
assignment procedures themselves, but my default approach here would be
to not use defined assignment at all, but author functions that
accomplished the necessary transformation of the original right hand
side expression to something suitable for intrinsic assignment to the
left hand side.

On classic case where defined assignment is required is for some sort of
reference counting approach. But reallocation on assignment kills that
because of the effective INTENT(OUT) of the left hand side discussed
above, so that can't be what you are doing...

[paul.rich...@gmail.com]

Dear Ian,

>
> I think that's a compiler bug. Elemental procedures are only supposed
> to be considered if there is no non-elemental procedure that matches on
> rank, and that consideration covers the merged set of specific bindings
> and procedures for the generic identifier associated with assignment.

I am curious to know what you think that the compiler should give for Andrew's test case - his anticipated reallocation on assignment?

This could be arranged but it will be an SOB to implement!

>
> (If the left hand side of your assignment was not allocatable, I would
> expect an error message about an inappropriate actual argument for the
> allocatable dummy.)
>
> This does not address your specific query, and may not be a relevant
> approach depending on the specifics of what you want to do in the
> assignment procedures themselves, but my default approach here would be
> to not use defined assignment at all, but author functions that
> accomplished the necessary transformation of the original right hand
> side expression to something suitable for intrinsic assignment to the
> left hand side.

I agree with your suggestion :-)

>
> On classic case where defined assignment is required is for some sort of
> reference counting approach. But reallocation on assignment kills that
> because of the effective INTENT(OUT) of the left hand side discussed
> above, so that can't be what you are doing...

Cheers

Paul

[Andrew Baldwin]

Thank you for the suggestion and discussion. Both interesting and helpful.

I do not need to do any of the above (nor, I believe, does my friend); your suggestion and my workaround SUBROUTINE called explicitly both seem like workable solutions to me. For me this is primarily an exercise to improve my understanding of Fortran and ability to interpret the standard by seeing if I could coax assignment into that behavior. And, potentially, an opportunity to help GCC be compliant with the standard.

Another option for the compiler has occurred to me, but perhaps this is not favorable for a reason beyond my ken: have intrinsic assignment applied whenever it cannot be determined whether x1 and x2 are conformable at compile time. That seems compliant with the standard, determinable at compile time, and probably not too onerous to implement.

[Andrew Baldwin]

Sorry, please disregard that "another option". It is simply wrong (in as much as my interpretation is correct) in reverse, by sometimes using intrinsic assignment when defined assignment should be used.

[Ian Harvey]

On 2015-09-05 5:12 AM, paul.rich...@gmail.com wrote:
> On Thursday, 3 September 2015 12:58:01 UTC+2, Ian Harvey wrote:
>> On 2015-09-03 5:10 PM, Andrew Baldwin wrote:

...

Upon encountering the assignment I expect the compiler to have a list of
... lets call them "specific interfaces" - procedures and specific
bindings - that sit behind the assignment generic identifier, whether
those bindings are acquired from Fortran 95 style stand-alone interface
blocks, or from generic bindings in the declared types of the left hand
side or right hand side parts of the assignment statement (the list can
consider generic bindings from any type in scope, but they will always
be eliminated next).

I then expect the compiler to first consider the non-elemental specific
interfaces and see if it can find a match on type, kind and rank of the
actual arguments (and pointer vs allocatable too, in F2008), noting that
the right hand side argument is to be taken as a parenthesised
expression. Because of the rules around generic disambiguation, there
can be at most one such non-elemental specific interface.

In this case I expect the compiler to get a match with the specific
procedure REALLOC_ASGN (which matches on type, kind and rank). The
compiler should then consider the assignment to be defined assignment,
and should then map the left and right hand sides of that assignment
statement to that specific procedure, and then use that procedure to
execute the assignment. Here that procedure [always] reallocates the
actual argument that corresponds to the left hand side of the assignment
to match the right hand side, but note this is not intrinsic assignment,
this is defined assignment directed by a procedure.

Here, actual argument attributes other than type, kind and rank are
consistent with that specific procedure - namely that the left hand side
thing in the assignment is allocatable. If the left hand side thing in
the assignment statement was not allocatable, I would expect the
compiler to issue a diagnostic.

~~~

If there was no non-elemental specific procedure in the list of specific
interfaces sitting behind the generic identifier for assignment (so lets
pretend no REALLOC_ASGN in the code above), then I would expect the
compiler to consider any elemental specific interfaces, and see if it
can find a match on type, kind and
rank-consistent-with-eleemtal-reference (actual argument scalars or
array, but if more than one array, then all arrays must have same rank).
Because of the rules of generic disambiguation, there can be at most
one such elemental specific interface.

This is where I think the standard has an error - in that it appears to
also require conforming arguments, which implies a runtime decision in
the general case. I think this very inconsistent with how generic
resolution works elsewhere in the language. "Conforming" in 7.2.1.4
5(b) should be whatever formal term means
rank-consistent-with-elemental-reference. I think alternatives, such
as requiring a runtime test of whether arguments are conformable to work
out whether intrinsic or defined assignment was in play, or
intrinsic-by-default-unless-it-is-conformable-at-compile-time (note the
language has no explicit concept of compile time) to be a path to madness.

Given my view of what the standard meant to say, and assuming that
REALLOC_ASGN was removed from the example above, then I would expect the
compiler to find a match against the ASGN_A specific binding of A_TYPE.
It would then interpret the assignment statement as defined assignment
- a reference to a procedure with the interface of that specific
binding, the actual procedure referenced depends on the dynamic type of
the passed object, which in this case is simply ASGN_A (the thing with
the matching binding is not polymorphic so its dynamic type matches its
declared type).

At that point the code would then be non-conforming, as the arguments to
the specific procedure are not conformable. In the general case this
could only be detected at runtime, and conforming Fortran processors are
not required to issue a diagnostic for this sort of programming error.
Whether a compiler detected this at compile time, or detected this at
runtime, or just merrily carried on trashing memory somewhere, or ran
off with your teenage daughter, depends on the compiler, supporting
runtime and libido of its developers.

Because the compiler would interpret the assignment as defined
assignment there would be no reallocation on assignment (which is an
intrinsic assignment feature). Given the specific interface was
elemental, there could also be no possibility of the user manually
coding something equivalent to reallocation on assignment, as for the
previous non-elemental matching case - dummy arguments of elemental
procedures cannot be allocatable.

~~~

If there was both no type, kind and rank matching non-elemental specific
interface and no type, kind and elemental-consistent-rank matching
elemental specific interface, then I expect intrinsic assignment to be
used. In this hypothetical case (pretending both the REALLOC_ASGN
specific procedure and the ASGN_A specific binding were deleted from the
example) that would invoke reallocation of the left hand side to match
the shape of the right hand side, as the original shape of the left hand
side does not match the shape of the right hand side.

There you go - it might be a bit long, but that's what I think should
happen, perhaps bar the teenage daughter bit.

[Ian Harvey]

On 2015-09-05 9:46 AM, Ian Harvey wrote:
...

> If there was no non-elemental specific procedure in the list of specific
> interfaces sitting behind the generic identifier for assignment (so lets
> pretend no REALLOC_ASGN in the code above), then I would expect the
> compiler to consider any elemental specific interfaces, and see if it
> can find a match on type, kind and

> rank-consistent-with-elemental-reference (actual argument scalars or

> array, but if more than one array, then all arrays must have same rank).
> Because of the rules of generic disambiguation, there can be at most
> one such elemental specific interface.

...
Note that if the left hand side of the assignment statement was scalar,
and the right hand side of the assignment was an array, I would still
expect a compiler to select an elemental specific interface as being an
appropriate match, and then to regard the statement as being defined
assignment, but to then issue a diagnostic complaining that a reference
to an elemental procedure where any actual argument is an array requires
all INTENT(OUT) or INTENT(INOUT) dummy arguments to be associated with
array actual arguments.

This is consistent with procedure resolution being on the basis of type,
kind and rank (and allocatable/pointer in F2008), the intent
specification of a dummy argument is not considered.

The alternative, where the elemental specific interface was discarded
from further consideration if the left hand side was a scalar and the
right hand side was an array, would still result in a diagnostic,
because the assignment would still not be a valid intrinsic assignment,
so perhaps this clarification is moot, but it assists users if the
nature of the diagnostic is consistent with the conceptual models used
in the language.

(The arrangement of the left hand side of the assignment being an array
of any rank, and the right hand side being a scalar is not problematic -
that would resolve in a straight forward manner to a matching elemental
specific interface if one was available and there was no matching
non-elemental specific interface.)

[Dick Hendrickson]

This has just been discussed on the J3 mailing list. (Bob Corbett (I
think) picked up the original question and passed it on). My
understanding of the discussion so far is that
A) whoops, there's an error introduced by the reallocation on
assignment stuff
B) It has always been the intent of J3 to make the distinction between
defined and intrinsic assignment to be a compile time choice.
C) A (likely? possible?) fix is to add something like "unless the ranks
are different" to one of the rules in the disambiguation text for
intrinsic and defined assignment. (It's possible it was "ranks are the
same", I don't remember)

That's about all I remember of the discussion.

Dick Hendrickson