Scope of content allocated by a pointer inside of subroutines

[Marshall Ward]

If I allocate a pointer inside of a subroutine, and then have an input point to this pointer, and then leave the function, is it valid to reference the content outside of the function? Something like the example below.

program ptr_alloc
implicit none
real, pointer :: x(:)

call xalloc(x)
print *, x
contains
subroutine xalloc(p)
real, pointer, intent(inout) :: p(:)
real, pointer :: q(:)

allocate(q(5))
q(:) = [1, 2, 3, 4, 5]

p => q
end subroutine
end program ptr_alloc

This program works -- I'm guessing because it's just malloc and C pointers under the hood -- but I do not know if one can generally expect such allocated data to persist once you leave the function stack of xalloc.

[Marshall Ward]

BTW I think I made an error by adding `intent(inout)` to the dummy declaration of `p`.

[Steve Lionel]

On 1/7/2020 2:46 PM, Marshall Ward wrote:
> If I allocate a pointer inside of a subroutine, and then have an input point to this pointer, and then leave the function, is it valid to reference the content outside of the function? Something like the example below.
>
> program ptr_alloc
> implicit none
> real, pointer :: x(:)
>
> call xalloc(x)
> print *, x
> contains
> subroutine xalloc(p)
> real, pointer, intent(inout) :: p(:)
> real, pointer :: q(:)
>
> allocate(q(5))
> q(:) = [1, 2, 3, 4, 5]
>
> p => q
> end subroutine
> end program ptr_alloc
>

This is fine. You are associating x (argument associated with p) with
the target of q. Also, the intent(inout) is fine too - intent for
pointers applies to the target.


--
Steve Lionel
ISO/IEC JTC1/SC22/WG5 (Fortran) Convenor
Retired Intel Fortran developer/support
Email: firstname at firstnamelastname dot com
Twitter: @DoctorFortran
LinkedIn: https://www.linkedin.com/in/stevelionel
Blog: http://intel.com/software/DrFortran
WG5: https://wg5-fortran.org

[FortranFan]

On Tuesday, January 7, 2020 at 3:10:31 PM UTC-5, Steve Lionel wrote:

> ..

> This is fine. You are associating x (argument associated with p) with
> the target of q. Also, the intent(inout) is fine too - intent for
> pointers applies to the target.

> ..


No, I don't think so - based on what I understand of the standard, the code in the original post does not conform. Refer to section 19.5 and 19.6 in the current Fortran standard (2018, 18-007r1 document):

--- begin text from the standard document ---
..
20 19.5.2.5 Events that cause the association status of pointers to become undefined
..
29 (6) completion of execution of an instance of a subprogram causes the pointer’s target to become un-
30 defined (item (3) of 19.6.6),
..
34 (9) execution of an instance of a subprogram completes and the pointer is declared or accessed in the
35 subprogram that defines the procedure if the pointer
36 (a) does not have the SAVE attribute,
..


1 19.6.6 Events that cause variables to become undefined
2 1 Variables become undefined by the following events.
..
10 (3) When execution of an instance of a subprogram completes,
11 (a) its unsaved local variables become undefined,
..
--- end text ---


@Marshall Ward,

You're effectively working with an "anonymous" TARGET without a SAVE attribute with your local variable 'q' in your procedure. The resultant behavior is not one you can count on, it's inherently unsafe.

You may want to either look into working with 'classes' aka derived types in Fortran and implementing the necessary 'wiring' to be "safe" with derived type components that have the POINTER attribute. Or working with named targets that have the ALLOCATABLE attribute e.g.,

module m

real, allocatable, save, target :: q(:) !<-- Named target

contains

subroutine xalloc(p)
real, pointer, intent(inout) :: p(:)

! Elided is testing on allocation status of 'q'

allocate(q(5))
q(:) = [1, 2, 3, 4, 5]

p => q

end subroutine

end module

program ptr_alloc

use m, only : xalloc

implicit none

real, pointer :: x(:)

call xalloc(x)
print *, x

end program ptr_alloc

[ga...@u.washington.edu]

On Tuesday, January 7, 2020 at 11:46:03 AM UTC-8, Marshall Ward wrote:
> If I allocate a pointer inside of a subroutine, and then have an
> input point to this pointer, and then leave the function,
> is it valid to reference the content outside of the function?
> Something like the example below.
>
> program ptr_alloc
> implicit none
> real, pointer :: x(:)
>
> call xalloc(x)
> print *, x
> contains
> subroutine xalloc(p)
> real, pointer, intent(inout) :: p(:)
> real, pointer :: q(:)
>
> allocate(q(5))
> q(:) = [1, 2, 3, 4, 5]
>
> p => q
> end subroutine
> end program ptr_alloc

Yes I believe that this is fine.

Note that the pointers target is not q, but a nameless array
allocated by the ALLOCATE statement, which does not go away.

It is, however, close enough to some that aren't fine, that you
should know about them.

Pointers to local ALLOCATABLE variables without the SAVE attribute or
local automatic variables, both of which go away when the subroutine
returns, can't be used after the subroutine returns.

On the other hand, dummy arguments with the ALLOCATABLE attribute
can be allocated in the subroutine and used outside.

[Ron Shepard]

On 1/7/20 3:42 PM, FortranFan wrote:
> On Tuesday, January 7, 2020 at 3:10:31 PM UTC-5, Steve Lionel wrote:
>
>> ..
>> This is fine. You are associating x (argument associated with p) with
>> the target of q. Also, the intent(inout) is fine too - intent for
>> pointers applies to the target.
>> ..
>
>
> No, I don't think so - based on what I understand of the standard, the code in the original post does not conform. Refer to section 19.5 and 19.6 in the current Fortran standard (2018, 18-007r1 document):
>
> --- begin text from the standard document ---
> ..
> 20 19.5.2.5 Events that cause the association status of pointers to become undefined
> ..
> 29 (6) completion of execution of an instance of a subprogram causes the pointer’s target to become un-
> 30 defined (item (3) of 19.6.6),

This restriction would apply to the local pointer q, not to the dummy
argument pointer p.

If xalloc() were called a second time, then the pointer q would be
undefined upon entry, the allocate() statement would associate some new
memory to it, and that new memory would then be associated also with the
dummy argument pointer p, possibly resulting in a memory leak from
whatever p pointed to upon entry.

> ..
> 34 (9) execution of an instance of a subprogram completes and the pointer is declared or accessed in the
> 35 subprogram that defines the procedure if the pointer
> 36 (a) does not have the SAVE attribute,

I'm not sure what this even means. Anyone else know?

I agree with Steve Lionel that the original code looks alright. There
are several seemingly minor variations of this code that would not be
legal, but I think the actual code posted is conforming. For example, if
q were declared as

real, target, allocatable :: q(:)

then the dummy argument p would be undefined upon return. Similarly if
the statement

deallocate(q)

were executed before return, then p would also be undefined upon return.

If the dummy argument were declared as

real, pointer, intent(in) :: p(:)

then the pointer assignment, p=>q, would not be allowed. However, the
declaration

real, pointer, intent(out) :: p(:)

would still be alright. The difference is that you could in principle
use p before the assignment in the original code but not the latter.

Pointers in any language are tricky, including fortran. There are three
possible states to worry about, not just two. A good general rule is to
avoid them if possible. In fortran you can often do this by using
instead allocatable variables or the ASSOCIATE construct.

$.02 -Ron Shepard

[sheikha...@gmail.com]

360assignments is relentless help with Task service, which gives arrangement courses of action sales to various subjects' instruments and frameworks in multi-field learning thought for all points. We provide tasks to help service at a cheap cost. We are a famous professional task help and have been receiving an overwhelmed response globally. https://360assignments.com/assignment-help-usa

[FortranFan]

On Tuesday, January 7, 2020 at 7:09:23 PM UTC-5, ga...@u.washington.edu wrote:

> ..

>
> Yes I believe that this is fine.
>
> Note that the pointers target is not q, but a nameless array
> allocated by the ALLOCATE statement, which does not go away.

> ..


On Tuesday, January 7, 2020 at 8:58:39 PM UTC-5, Ron Shepard wrote:
> ..

> This restriction would apply to the local pointer q, not to the dummy
> argument pointer p.
>

> ..
> I agree with Steve Lionel that the original code looks alright. ..


Show statements from the Fortran standard that support such feedback, my hunch is that will not be possible.

Given the current standard, especially sections 19.5 and 19.6, once the execution of an instance of subprogram 'xalloc' completes, a processor can treat the target of the variable 'p' with the POINTER attribute as undefined. And the subsequent instruction 'print *, x' is non-conforming.

[edmondo.g...@gmail.com]

Section 9.7.1.4

"
2. At the beginning of execution of a function whose result is a pointer, the association status of the result pointer is undefined. Before such a function returns, it shall either associate a target with this pointer or cause the association status of this pointer to become disassociated.
"
Suggesting that you can write a function like

function xppp() result(r)
integer, pointer :: r
allocate(r)
r = 5
end function

or

function xppp() result(r)
integer, pointer :: r

r => some_not_local_variable_with_target_attribute

end function

In the previous example while the pointer q get undefined at the end of the procedure its target no.
The target is not a local variable so is not getting undefined after the end of the procedure.

[Ian Harvey]

On 2020-01-08 07:12, FortranFan wrote:
> On Tuesday, January 7, 2020 at 3:10:31 PM UTC-5, Steve Lionel wrote:
>
>> ..
>> This is fine. You are associating x (argument associated with p) with
>> the target of q. Also, the intent(inout) is fine too - intent for
>> pointers applies to the target.

To be clear, INTENT(INOUT) on a pointer dummy means that the pointer
dummy may be associated with different targets during execution of the
procedure (and the actual argument needs to be suitable for having its
pointer association changed via execution of the procedure). It says
nothing about whether the value of the target (whatever it might be, if
any) might be changed or not.

INTENT(INOUT) might imply to a reader of the code that the pointer
association status of the actual argument should be something useful
when the procedure is invoked (because that association status is
preserved on invocation), but that's a matter of coding style, not
formal language requirements.

>> ..
>
>
> No, I don't think so - based on what I understand of the standard, the code in the original post does not conform. Refer to section 19.5 and 19.6 in the current Fortran standard (2018, 18-007r1 document):
>
> --- begin text from the standard document ---
> ..
> 20 19.5.2.5 Events that cause the association status of pointers to become undefined
> ..
> 29 (6) completion of execution of an instance of a subprogram causes the pointer’s target to become un-
> 30 defined (item (3) of 19.6.6),

As below, 19.6.6 (3) does not apply here.

> ..
> 34 (9) execution of an instance of a subprogram completes and the pointer is declared or accessed in the
> 35 subprogram that defines the procedure if the pointer
> 36 (a) does not have the SAVE attribute,

Only relevant to the dummy pointer, not the actual argument.
Termination of execution of the procedure terminates the argument
association between the dummy and the actual.

(The way the required behaviour is specified by the standard could be
better, but if you think the above is problematic, then your example
below has the same issue..)

> ..
>
>
> 1 19.6.6 Events that cause variables to become undefined
> 2 1 Variables become undefined by the following events.
> ..
> 10 (3) When execution of an instance of a subprogram completes,
> 11 (a) its unsaved local variables become undefined,

The target created by the allocate statement is not a local variable -
the above is not relevant.

> ..
> --- end text ---
>
>
> @Marshall Ward,
>
> You're effectively working with an "anonymous" TARGET without a SAVE attribute with your local variable 'q' in your procedure. The resultant behavior is not one you can count on, it's inherently unsafe.

q is local to the procedure, its target is not.

[FortranFan]

On Wednesday, January 8, 2020 at 7:55:03 PM UTC-5, Ian Harvey wrote:

> ..

> The target created by the allocate statement is not a local variable -
> the above is not relevant.
>

Please show where in the standard it indicates, "The target created by the allocate statement is not a local variable".

> ..

> q is local to the procedure, its target is not.

> ..

Same request here, please show from where in the standard one can determine "target is not" "local to the procedure"

Thanks,

[Marshall Ward]

Thanks all, this was a useful discussion for me.

If I am understanding correctly, it seems that a pointer `q` is distinct from the anonymous variable created when `allocate(q)` is called, which is not the case for an allocatable variable.

I think I understand that all of the Standard citations here describe `q` but not its allocated target. Is there anywhere in the standard which describes how this anonymous allocated content ought to behave?

[FortranFan]

On Thursday, January 9, 2020 at 11:37:30 AM UTC-5, Marshall Ward wrote:

> .. Is there anywhere in the standard which describes how this anonymous allocated content ought to behave?


Before anyone responds to this, it will be very useful to know of OP's effort to look for "how this anonymous allocated content ought to behave" in the standard and what OP found/learnt from it!

[Ian Harvey]

The definition of local variable is in 3.154.2 of F2018.

variable in a scoping unit that is not a dummy argument
or part thereof, is not a global entity or part thereof,
and is not an entity of part of an entity that is accessible
outside that scoping unit.

The object created by allocation of a pointer does not meet that
definition - it is not a "variable in a scoping unit" (noting that
scoping unit is a source concept, not an execution concept), further it
is (or can be, if you want to avoid circular definition) also accessible
outside the scoping unit that is the subprogram xalloc.

Because the object created by allocation of a pointer does not meet the
definition of a local variable, it is not a local variable.

[Ian Harvey]

On 2020-01-11 11:27, FortranFan wrote:

> On Thursday, January 9, 2020 at 4:19:26 PM UTC-5, Ian Harvey wrote:
>
>> ..
>>

>> The definition of local variable is in 3.154.2 of F2018.
>>
>> variable in a scoping unit that is not a dummy argument or part
>> thereof, is not a global entity or part thereof, and is not an
>> entity of part of an entity that is accessible outside that scoping
>> unit.
>>
>> The object created by allocation of a pointer does not meet that
>> definition - it is not a "variable in a scoping unit" (noting that
>> scoping unit is a source concept, not an execution concept),
>> further it is (or can be, if you want to avoid circular definition)
>> also accessible outside the scoping unit that is the subprogram
>> xalloc.
>>
>> Because the object created by allocation of a pointer does not meet
>> the definition of a local variable, it is not a local variable.
>
>

> Thank you for your effort, but I think the standard does not support
> your reasoning.
>
> The standard does state "Allocation of a pointer creates an object
> that implicitly has the TARGET attribute." It really does not
> clarify anything else about this "object".
>
> Then section 3.46 of the standard suggests an object can be "constant
> (7.1.4), variable (9), or subobject of a constant (5.4.3.2.4)". If
> one leaves out "constant" or "subobject of a constant", one can think
> an object is a variable.

It is a variable, as all things with the TARGET attribute are.

> Then one can take 3.46 to mean "the object created by allocation of a
> pointer" is a variable of some sort.
>
> But now section 19.1 para 1 says, "An entity is identified by an
> identifier".
>
> The standard does not indicate "the object created by allocation of a
> pointer" to have an identifier. Should one would think if the
> standard doesn't give the object an identifier one should assume it
> is NOT "identified by an identifier" If so, it must follow this
> variable which is "the object created by allocation of a pointer" is
> not an entity on account of not having an identifier.
>
> If the variable which is "the object created by allocation of a
> pointer" is then not an entity, it cannot then be "a global entity or
> part thereof" and it cannot be "an entity of part of an entity that

> is accessible outside that scoping unit".

The term "entity" (without further qualification) has its common
language use. It is just a fancy way of saying "thing".

Entities within a Fortran program do not always have identifiers. A
simple case is a Fortran main program without a name.

The object created by allocation of a pointer is an entity.

The object created by allocation of a pointer does not appear in the
list of things that are considered global entities, therefore it is not one.

It is trivial to construct examples where the object created by
allocation of a pointer is accessible outside of the scoping unit that
has the ALLOCATE statement, but this starts down the rabbit hole of what
"accessible outside" means ("accessible outside" wording is to knock out
host and use association *into* the scoping unit). Rather than run down
that rabbit hole, see below.

> So then one is left wondering in the context of 3.154.2 whether it is
> a "variable in a scoping unit that is not a dummy argument". May be
> it is, or may be not.

A scoping unit is a piece of source. You can print out the source for a
program and use a highlighter, scissors or similar to delineate a
scoping unit (bear in mind there may be holes due to nesting of scoping
units). If you look at that scoping unit you will not see the object
created by allocation of a pointer, because that is not a source
concept, it is something that arises dynamically from execution of the
source.

Whether something is a local variable or not is a simple question of
source, not of execution.

> This goes back to your recent comment in the other thread
> https://groups.google.com/d/msg/comp.lang.fortran/9_0q0dBRzpk/6bvW3ehiAQAJ.
> What may be clear to you in this context "is .. not .. obvious". For
> anyone then to keep insisting there are no problems with the
> description in the standard when it comes to the target of 'q' with
> the code in the original post is doing a disservice to Fortran.

Don't be silly. "I disagree with your interpretation" is not the same
thing as "the wording of the standard does not require improvement".
Submit an interpretation request if you want. That clause of the
standard is notorious for requiring edits.

But INTENT attribute aside, the behaviour expected in the original post
of this thread was the intent of the authors of Fortran 90 and
subsequent standards, so a fair bit of remedial work to section 4.3 of
F2018 will be required to support your argument that this is not
conforming. Lots of code previously considered conforming would be
broken - expect pitchforks!

[FortranFan]

On Saturday, January 11, 2020 at 5:03:40 PM UTC-5, Ian Harvey wrote:

> ..

> Don't be silly. "I disagree with your interpretation" is not the same
> thing as "the wording of the standard does not require improvement".
> Submit an interpretation request if you want. That clause of the
> standard is notorious for requiring edits.
>

Questions and confusion around the object created in an ALLOCATE statement involving a pointer is one that keeps up popping up here and elsewhere. The information about this "object created by allocation of a pointer" is inadequately described in the standard, to say the least. And pardon me but the comments upthread came across to me as saying "the wording of the standard does not require improvement".

> But INTENT attribute aside, the behaviour expected in the original post
> of this thread was the intent of the authors of Fortran 90 and

> subsequent standards ..

If you say so. My point is with "the object created by allocation of a pointer", the wording of the current standard (as well as that of the previous edition) comes across as seeking to obfuscate any such intent rather than make an attempt to convey it.

I'm not calling at all for any remedial work such as with section 4.3 of the standard. All I'm hoping for is a better description of the relevant parts of the standard so that more and more readers of the standard will find it "obvious" the code in the original post is conformant.

I think some verbosity in the Fortran standard section corresponding to "9.7.1.4 Allocation of pointer targets", even some didacticism (perhaps with one or more NOTES in this section), in the description about the "object created by allocation of a pointer" that is introduced into the standard by those who understand well "the intent of the authors of Fortran 90 and subsequent standards" will be beneficial to Fortranners.

[ga...@u.washington.edu]

On Saturday, January 11, 2020 at 9:32:08 PM UTC-8, FortranFan wrote:

(snip)

> Questions and confusion around the object created in an ALLOCATE
> statement involving a pointer is one that keeps up popping up here
> and elsewhere. The information about this "object created by
> allocation of a pointer" is inadequately described in the standard,
> to say the least. And pardon me but the comments upthread came across
> to me as saying "the wording of the standard does not require improvement".

I don't think I have ever said, or will ever say, that the standard
can't use improvement.

But otherwise, 9.7.1.4 and 9.7.3.3 aren't all that bad in describing
allocation and deallocation with POINTER variables.

Partly, it helps to have used pointers in other languages.

Note, for example, that you can pointer assign the value of pointers
to other pointers. Any of those can be used to access the allocated
value, and any can be used to DEALLOCATE it. It is only allocated
if you actually deallocate it, and if you don't it is likely a
memory leak.

ALOCATABLE variables are somewhat different, and are not especially
useful in understanding pointers. Among others, you can't have
multiple copies of the same allocatable variable. You can MOVE_ALLOC
to a different variable, in which case the original one is not
allocated anymore.

[Ron Shepard]

On 1/12/20 1:04 AM, ga...@u.washington.edu wrote:
> On Saturday, January 11, 2020 at 9:32:08 PM UTC-8, FortranFan wrote:

> [...]

> I don't think I have ever said, or will ever say, that the standard
> can't use improvement.
>
> But otherwise, 9.7.1.4 and 9.7.3.3 aren't all that bad in describing
> allocation and deallocation with POINTER variables.

Is there any context in which the anonymous target allocated through a
pointer is automatically deallocated? I can think of several situations
involving automatic variables, allocatable variables, and even nested
components of allocatable variables that are required by the standard to
be automatically deallocated (or at least to behave "as if",
implementation details aside), but I can't think of a similar situation
involving an anonymous target of a pointer.

$.02 -Ron Shepard

[Ian Harvey]

The standard permits, but does not require, processor implementations to
"automatically" deallocate the object formerly referenced by a pointer,
if the object is no longer accessible through any pointer. Such
automatic deallocation of the object may (or may not) invoke a finalizer
for the object.

This freedom for processors simplifies implementation on platforms that
use garbage collection and the like for memory management.

But I'm not aware of any implementations that actually do this - in all
cases I'm aware of you just end up with memory leaks. I haven't looked
too hard though.

[Themos Tsikas]

Memory leak detected by NAG Fortran Compiler:

>cat -n foo.f90
1 program ptr_alloc
2 implicit none
3 real, pointer :: x(:)
4
5 call xalloc(x)
6 print *, x
7 contains
8 subroutine xalloc(p)
9 real, pointer, intent(inout) :: p(:)
10 real, pointer :: q(:)
11
12 allocate(q(5))
13 q(:) = [1, 2, 3, 4, 5]
14
15 p => q
16 end subroutine
17 end program ptr_alloc
18

>nagfor -g -mtrace=all -C=all -C=undefined foo.f90
NAG Fortran Compiler Release 7.0(Yurakucho) Build 7007
[NAG Fortran Compiler normal termination]

>./a.out 2>&1 |nagfmcheck
11 allocations
***MEMORY LEAK:
LEAK: Allocation 4 (size 20) = Z'14BD4B3B5010' at line 12 of foo.f90
LEAK: Allocation 9 (size 20) = Z'14BD4B3B5040' at line 12 of foo.f90

[ga...@u.washington.edu]

Most often, I don't consider memory that hasn't been released
at the end of the program, but otherwise still has pointers pointing
to it, leaked.

Even considering that, though, I don't see how this program has two
leaked allocations, and both on line 12.

If you have a routine that might later be used inside a larger
routine, and called multiple times, then it should release and
allocated memory. But protected mode OS pretty much have to release
everything at the end of a program.

[ga...@u.washington.edu]

Some time ago, I was wondering about the possibility of a Fortran
compiler generating JVM code as output. As JVM uses garbage
collection, it would also for that case.

Other than JVM, I don't know any where garbage collection is
fundamental to system design.

Though with large enough virtual memory, you don't have to
worry so much about leaked memory.

On Tuesday, January 14, 2020 at 3:12:56 AM UTC-8, Ian Harvey wrote:
(snip)

[robin....@gmail.com]

The XPL compiler has used garbage collection since at least 1969.

[Themos Tsikas]

On Tuesday, 14 January 2020 23:08:19 UTC, ga...@u.washington.edu wrote:
> Even considering that, though, I don't see how this program has two
> leaked allocations, and both on line 12.

This is a side-effect of the -C=undefined option. More runtime data structures are created and maintained to keep track of undefined variables and there's an extra leak associated with them. Without that option we only see one leak at that line number.

Themos Tsikas, NAG Ltd.

[ga...@u.washington.edu]

And the additional data structure is the same size!

Reminds me of some years ago, working with a company on some
big complicated system, and kept having it run out of memory,
so I complained that it had a memory leak. After lots of
testing, they claimed that it didn't.

In the end, it turned out that the system was multithreaded with
three threads: input thread, processing thread, and output thread.

The processing thread would generate some data structure in memory
and pass it to the output thread, which would then write results
out and free the data structure.

There was, however, no system to make sure that one thread didn't
get way ahead and fill up memory with results.

As the output was large (ASCII text file), I piped the output
through gzip. That, in turn, slowed down the output thread, but
not the processing thread, until it filled up memory with results.

(32 bit days, so 3GB limit. The actual output was much less.)

[Marcus Lim]