return of copies vs references
Darren Duncan
return statements, regarding whether copies or references are
returned. It will help me in my p6ification of p5 code.
Say I had a class with 3 private attributes, named [$:foo, @:bar,
%:baz], and I was making an explicit accessor for returning the full
values of each.
Take these 3 example method statements:
return $self.foo;
return $self.bar;
return $self.baz;
For each of the above cases, is a copy of or a reference to the
attribute returned? For each, will the calling code be able to
modify $obj's attributes by modifying the return values, or not?
Going further, what is the exact syntax for each type of attribute to
specify whether a copy or a reference is returned?
In Perl 5, with the latter two, the difference was a "return $bar" vs
"return [@{$bar}]" for reference vs copy. I would like that Perl 6
is also at least as clearly disambiguated.
Note that specifying this in the attribute definition isn't
appropriate, since an attribute could just as easily be an array of
arrays, or hash of hashes, and I am returning an inner array or hash
that I either do or don't want to be modifiable by calling code.
Separate question, just to confirm, I assume that plain '=' always does a copy?
Thank you for any clarification.
-- Darren Duncan
Luke Palmer
> I need some clarification on the semantics of subroutine or method
> return statements, regarding whether copies or references are
> returned. It will help me in my p6ification of p5 code.
>
> Say I had a class with 3 private attributes, named [$:foo, @:bar,
> %:baz], and I was making an explicit accessor for returning the full
> values of each.
>
> Take these 3 example method statements:
>
> return $self.foo;
> return $self.bar;
> return $self.baz;
>
> For each of the above cases, is a copy of or a reference to the
> attribute returned? For each, will the calling code be able to
> modify $obj's attributes by modifying the return values, or not?
Well if you're making accessors, why the heck are you making them
private? But I can't really answer your question, because it depends on
how you write the accessors.
I'll answer your question by rephrasing it to use `$.foo`, `@bar`,
`%.baz`. Assume they are all declared with `is rw`:
a) my $x = $obj.foo;
b) $obj.foo = $x;
d) my $ref = \$obj.foo;
e) $$ref = $x;
a) my @x = $obj.bar; # @x is now a copy, because = copies
b) @obj.bar = (1, 2, 3);
c) @obj.bar[2] = 3;
d) my $ref = $obj.bar;
e) @$ref = (1, 2, 3);
a) my %x = $obj.baz;
b) %obj.baz = (a => 1, b => 2);
c) %obj.baz{b} = 2;
d) my $ref = $obj.baz;
e) %$ref = (a => 1, b => 2);
These are all legal. If you don't declare your attributes with `is rw`,
then the "b"s, "c"s (perhaps), and the "e"s are illegal.
Now back to your question. You could write rw accessors for each of
your private variables that behave like the ones above like so:
sub foo() is rw { $:foo }
sub bar() is rw { @:bar }
sub baz() is rw { %:baz }
If you want to intercept the write and do something with it, you can do
this (see S06 for details):
sub foo() is rw {
new Proxy:
FETCH => sub ($self) { $:foo },
STORE => sub ($self, $val) {
say "Setting to $val";
$:foo = $val;
};
}
Luke
Darren Duncan
> > For each of the above cases, is a copy of or a reference to the
>> attribute returned? For each, will the calling code be able to
>> modify $obj's attributes by modifying the return values, or not?
>
>Well if you're making accessors, why the heck are you making them
>private? But I can't really answer your question, because it depends on
>how you write the accessors.
I am writing accessors to mediate with the attributes, which are all
private, and whose implementation may change over time. What I want,
in the normal case, is that calling code which invokes my methods
will get a copy of attributes which it can modify, that won't affect
the original attribute values.
When I last asked a related question here, I was told that simply
returning an attribute will allow the caller to modify the original
attribute by default. I wanted to make sure this didn't happen. It
is possible that there was a misunderstanding regarding the previous
question, and the default action is in fact a copy.
-- Darren Duncan
Larry Wall
: When I last asked a related question here, I was told that simply
: attribute by default.
That used to be true for arrays and hashes, but I just changed my
mind on it. (Scalars always copy.)
: I wanted to make sure this didn't happen. It
: is possible that there was a misunderstanding regarding the previous
: question, and the default action is in fact a copy.
See my other message--it should make you a little happier. I had
not sufficiently thought through what would happen with autogenerated
read-only accessors in scalar context.
Larry
Larry Wall
: I need some clarification on the semantics of subroutine or method
: returned. It will help me in my p6ification of p5 code.
:
: Say I had a class with 3 private attributes, named [$:foo, @:bar,
: %:baz], and I was making an explicit accessor for returning the full
: values of each.
I am assuming you're talking about read-only accessors, not rw accessors.
: Take these 3 example method statements:
:
: return $self.foo;
: return $self.bar;
: return $self.baz;
Those would have to be:
return $self.:foo;
return $self.:bar;
return $self.:baz;
or
return $:foo;
return @:bar;
return %:baz;
: For each of the above cases, is a copy of or a reference to the
: attribute returned?
Perl 5 always makes a copy of return values, but that just turns
out to not matter for references, since a copy of a reference is as
good as the original reference. Perl 5 also propagates scalar/list
context into subs. For $:foo it doesn't matter--it always behaves
as a scalar value even in list context. In list context, @:bar and
%:baz should probably return copies of their values much like they
do in Perl 5, (or more likely, some kind of lazy COW reference that
can lazily interpolate into the surrounding lazy list context).
Whether $self.:bar and $self.:baz should behave the same is an
interesting question. They *look* scalar, so maybe they should imply
reference return, and you'd have to say
return $self.:bar[];
return $self.:baz{};
to get the equivalent of
return @:bar;
return %:baz;
But bare
return $self.:bar;
return $self.:baz;
would be equivalent to:
return \@:bar;
return \%:baz;
But I could argue it the other way too.
: For each, will the calling code be able to
: modify $obj's attributes by modifying the return values, or not?
The caller can modify the value only if an explicit ref is returned (or
the accessor is marked "rw").
Where we seem to differ from Perl 5 is that in scalar context, a bare
array or hash automatically enreferences itself rather than returning
some kind of size. So in scalar context, it would seem that
return @:bar;
return %:baz;
and
return $self.:bar;
return $self.:baz;
are equivalent to:
return \@:bar;
return \%:baz;
(Again, $:foo is never a problem unless it's already a reference.)
So the issue is whether this interpretation will encourage people to accidentally
return references to things they didn't want to give write access to. On the
other hand, making the private methods context sensitive doesn't actually
seem to fix this particular problem, but just pushes it down one level into
the implicit accessor. Maybe we need to work something up where
references returned from read-only accessors are always COW references.
If we assume that [...] is lazy when it can be, then that would be saying that
scalar context forces
return @:bar;
to mean
return [@:bar];
and you'd have to write an explicit
return \@:bar;
to get around that. But that seems kind of hacky and special-casey.
On the other hand, there are going to be strong cultural forces
discouraging people from writing such accessors in the first place,
so maybe we just go ahead and let people return hard refs in scalar
context on the assumption they know what they're doing. I suspect
that most actual accessors to arrays and hashes will just look like
ordinary getter and setter methods with extra args for subscripts, or
will return an explicit proxy if they want to behave like an lvalue.
And in either of those cases, you don't try to return the entire
array or hash. So maybe we should settle for the clean but slightly
dangerous semantics here.
Except that we've defined default read-only accessors that would,
under the "clean" rules, give people automatic access to arrays and
hashes if called in scalar context. So I think we really only have
three options here for the public accessors:
Don't generate autogenerate accessors at all for arrays and hashes.
Generate array and hash accessors that refuse to work in scalar context.
Generate array and hash accessors that autocopy in scalar context.
Of those three, the last seems the most friendly.
: Going further, what is the exact syntax for each type of attribute to
: specify whether a copy or a reference is returned?
:
: In Perl 5, with the latter two, the difference was a "return $bar" vs
: "return [@{$bar}]" for reference vs copy. I would like that Perl 6
: is also at least as clearly disambiguated.
If we go "dwimmy" rather than "clean", and assume private array and
hash accessors always return refs, then these return refs from public
accessors:
return \$self.:foo; # in any context
return $self.:bar; # in any context
return $self.:baz; # in any context
return \$:foo; # in any context
return \@:bar; # in any context
return \%:baz; # in any context
and these return copies:
return $self.:foo # in any context
return $self.:bar[]; # in list context
return $self.:baz{}; # in list context
return $:foo; # in any context
return @:bar; # in any context
return %:baz; # in any context
That's actually simpler than the tables I made up when I was believin in
the "clean" solution. What it essentially boils down to is that return
from a public accessor always forces list context on its arguments even
if the accessor was called in scalar context (in which case a ref to
a COW list is stored in the scalar).
I'm wondering if we should just up and say that "return" always forces list
context on its arguments. It would simplify some things, and complicate others...
: Note that specifying this in the attribute definition isn't
: appropriate, since an attribute could just as easily be an array of
: arrays, or hash of hashes, and I am returning an inner array or hash
: that I either do or don't want to be modifiable by calling code.
Yes, even with the return context hack, it'd only go one level down.
I think at some point we have to rely on people not to write stupid
accessors, and rely on culture to enforce that. I really, really
want to avoid falling into C++ const hell.
Maybe we can define some kind of deepcow operator for the nested cases.
Or maybe there's some other solution I don't see yet, or that someone has
already told me about and I've forgotten...
: Separate question, just to confirm, I assume that plain '=' always does a
: copy?
Yes. And with := it's mandatory for "is copy" parameters, optional
for "is constant" parameters, and prohibited for "is rw" parameters.
Hopefully if values come back COW from a routine, we can avoid a
duplicate copy on the assignment.
: Thank you for any clarification.
Well, I don't know if I clarified it that much, but you're welcome
to the mud too. :-)
Larry
Larry Wall
: I'll just point out, the rest of this message, with all the autocopy
: complexity (according to /some/ people), uses this assumption. It all
: happily goes away if $self.:bar returns a list if @:bar is declared.
: And I can't, off hand, see any other problems with it. Maybe I'm just
: blind, though.
Yeah, that part's negotiable. But it will seem a little strange if
we push it so far that private accessors default to protecting your
own data from you, which it would if we forced the autocopy on private
accessors in scalar context. I think private accessors will have to be
distinguished from public accessors somehow. With forced ref return on
private accessors we could be more general in how we specify autocopy
on all return statements, but it could be made to work the other way
too, in which case we say that only private accessors pay attention
to context. And maybe that just falls out of the fact that private
accessors are really just subs in disguise. So maybe the distinction
is that real methods always return lists, and only subs (and sublike
private methods) pay attention to the context in which they're called.
That's actually weirdly symmetrical with the notion that only subs can
impose compile-time context on their arguments, while methods always
have to assume list context because you have to generate the argument
list before you can know which method you're going to dispatch to.
So I could buy it either way. Opinions?
Larry
Luke Palmer
> Perl 5 always makes a copy of return values, but that just turns
> out to not matter for references, since a copy of a reference is as
> good as the original reference. Perl 5 also propagates scalar/list
> context into subs. For $:foo it doesn't matter--it always behaves
> as a scalar value even in list context. In list context, @:bar and
> %:baz should probably return copies of their values much like they
> do in Perl 5, (or more likely, some kind of lazy COW reference that
> can lazily interpolate into the surrounding lazy list context).
> Whether $self.:bar and $self.:baz should behave the same is an
> interesting question. They *look* scalar, so maybe they should imply
> reference return, and you'd have to say
>
> return $self.:bar[];
> return $self.:baz{};
I'll just point out, the rest of this message, with all the autocopy
complexity (according to /some/ people), uses this assumption. It all
happily goes away if $self.:bar returns a list if @:bar is declared.
And I can't, off hand, see any other problems with it. Maybe I'm just
blind, though.
Luke
Darren Duncan
For now, I will use the syntax $obj.:bar[] and $obj.:baz{} in the
get_bar|baz() accessor methods to get copies returned; if the rules
change again, I'll update later.
I noticed that the example seemed incomplete, saying only what would
happen with the above in list context, and not saying what would
happen in scalar context.
I would like a way of specifying that the copy always happens
regardless of the caller's context. This doesn't have to be a
language 'default', and I'm fine if its a little more verbose, just
as long as its consistent and unambiguous.
> return $self.:foo # in any context
> return $self.:bar[]; # in list context
> return $self.:baz{}; # in list context
> return $:foo; # in any context
> return @:bar; # in any context
> return %:baz; # in any context
Eg, something that is like lines 5 and 6, but lets me explicitly use 'self'.
-- Darren Duncan
Thomas Sandlaß
> That's actually weirdly symmetrical with the notion that only subs can
> impose compile-time context on their arguments, while methods always
> have to assume list context because you have to generate the argument
> list before you can know which method you're going to dispatch to.
Sorry if it's only me, but I don't understand what this means.
I get, that at the call site of a sub, first of all the lexically
closest sub can be determined at compile time. This means its signature
is available and can be checked and imposed on the arguments---right so far?
Single invocant methods are defined inside classes. Thus the knowledge of
the compiler about their signature hinges on how well the type of the invocant
can be determined. If the data is insufficient this is---depending on compile
mode---either a static type error or dynamic lookup has to be compiled,
with an optional warning.
BTW, do method names have to be pre-declared? Or does the following just
defer the existence check:
sub blubb ( $obj )
{
print $obj.somemethod( "FirstArg", "SecondArg" );
}
What I want to ask is: is ".somemethod()" parsed purely syntactically?
And is there a compiler mode where this is a type error like "type Any
doesn't have .somemethod<Str,Str>"?
With multi subs and methods I guess a lexical definition is needed as for
subs. But that doesn't mean that all dispatch targets are known already.
Thus a dynamic built-up of the arglist and MMD is compiled. Right?
A very nice feature of the compiler here were to perform implementation
side checks when the complete program is loaded? This involves potential
ambiguity and absence failures.
BTW, how far down to pure byte code can Perl6 packages be compiled?
Too much off the mark?
--
TSa (Thomas Sandlaß)
Darren Duncan
If you say 'my %abc' in a method or sub, and later say 'return %abc',
then a reference to %abc will be returned by default.
So its not like I'm treating attributes differently.
-- Darren Duncan
At 3:27 PM -0800 3/17/05, Darren Duncan wrote:
>It occurs to me that I may have requested something before that
>would cause more problems than it solves if implemented. So I have
>a different idea that would hopefully be efficient, powerful, and
>easy to learn. In short, make it work much like Perl 5.
>
>The idea is Perl methods and subs will always return references or
>scalars by default, just as method/sub arguments always take
>references or scalars by default; so we have symmetry between the
>in/out. And they would do this the same way regardless of the
>caller scalar/list context.
>
>If the writer of a method/sub wants to return a copy of a
>non-trivial structure like a hash or array, they explicitly copy it
>and return the copy. This is elegant because when they are
>explicitly copying, they can also fully control to what depth in a
>tree-like structure the items have either their values or their
>references copied.
>
>For example, with the same $:foo, @:bar, %:baz attributes ...
>
>These would return references to the structure root, in all contexts:
>
> return \$self.:foo;
> return $self.:bar;
> return $self.:baz;
> return \$:foo;
> return @:bar;
> return %:baz;
>
>These would return a new structure with level-1 copies of the
>elements, but if any elements are references then the copies point
>to the same things in all contexts:
>
> return $self.:foo;
> return [$self.:bar];
> return {$self.:baz};
> return $:foo;
> return [@:bar];
> return {%:baz};
>
>The exact syntax could vary. The above would scale to, say,
>explicit 2-dimensional copy similarly to Perl 5, like this:
>
> return [$self.:bar.map:{ [$_] }];
> return [@:bar.map:{ [$_] }];
>
>Now, I am operating under the assumption here that in Perl 6, the
>line between having a reference to something or having that thing is
>blurred, and so there is no need to do explicit dereferencing; eg,
>no @{} or %{}.
>
>All that the caller context would then determine is whether what the
>method/sub returned is flattened or not; the caller context would
>not determine whether the method/sub returns a copy or a reference.
>
>Question: Is there a trait for a sub/method that specifies whether
>it forces a scalar or list context?
>
>Any opinions on this?
>
>-- Darren Duncan
Darren Duncan
cause more problems than it solves if implemented. So I have a
different idea that would hopefully be efficient, powerful, and easy
to learn. In short, make it work much like Perl 5.
The idea is Perl methods and subs will always return references or
scalars by default, just as method/sub arguments always take
references or scalars by default; so we have symmetry between the
in/out. And they would do this the same way regardless of the caller
scalar/list context.
If the writer of a method/sub wants to return a copy of a non-trivial
structure like a hash or array, they explicitly copy it and return
the copy. This is elegant because when they are explicitly copying,
they can also fully control to what depth in a tree-like structure
the items have either their values or their references copied.
For example, with the same $:foo, @:bar, %:baz attributes ...
These would return references to the structure root, in all contexts:
return \$self.:foo;
return $self.:bar;
return $self.:baz;
return \$:foo;
return @:bar;
return %:baz;
These would return a new structure with level-1 copies of the
elements, but if any elements are references then the copies point to
the same things in all contexts:
return $self.:foo;
return [$self.:bar];
return {$self.:baz};
return $:foo;
return [@:bar];
return {%:baz};
The exact syntax could vary. The above would scale to, say, explicit
Darren Duncan
be ill-informed.
There's just one thing I need to know that will make everything clear:
Does 'return' always impose a scalar context on its arguments? Has
this been decided for sure, or is it still under debate?
If it does, then I know how to do everything I want based on info in
the Synopsis and Larry's last comments, otherwise I still could find
out what I want to know easily enough.
-- Darren Duncan
Darren Duncan
If that's the case, then we can just drop this discussion, problems solved.
Sorry for wasting your time.
-- Darren Duncan
Larry Wall
: Larry Wall wrote:
: >That's actually weirdly symmetrical with the notion that only subs can
: >impose compile-time context on their arguments, while methods always
: >have to assume list context because you have to generate the argument
: >list before you can know which method you're going to dispatch to.
:
: Sorry if it's only me, but I don't understand what this means.
:
: I get, that at the call site of a sub, first of all the lexically
: closest sub can be determined at compile time. This means its signature
: is available and can be checked and imposed on the arguments---right so far?
Yes.
: Single invocant methods are defined inside classes. Thus the knowledge
: of the compiler about their signature hinges on how well the type
: of the invocant can be determined. If the data is insufficient this
: is---depending on compile mode---either a static type error or dynamic
: lookup has to be compiled, with an optional warning.
In general, most dynamic languages (including Perl 5) assume that it is
impossible to know all the method names even after normal compilation,
since new classes or methods can be added at any time. Perl 5 even lets
you change your @ISA hierarchy on the fly, if you're willing to take
the performance hit.
: BTW, do method names have to be pre-declared? Or does the following just
: defer the existence check:
:
: sub blubb ( $obj )
: {
: print $obj.somemethod( "FirstArg", "SecondArg" );
: }
:
: What I want to ask is: is ".somemethod()" parsed purely syntactically?
In standard Perl 6 this is pure syntax, and doesn't care whether
.somemethod exists yet or not.
: And is there a compiler mode where this is a type error like "type Any
: doesn't have .somemethod<Str,Str>"?
Potentially it could, but it wouldn't by default. You'd have to
explicitly tell the compiler that you aren't adding any more classes
or methods. But then you couldn't use any kind of an autoloader, and
plugable architectures become problematic.
: With multi subs and methods I guess a lexical definition is needed as for
: subs. But that doesn't mean that all dispatch targets are known already.
: Thus a dynamic built-up of the arglist and MMD is compiled. Right?
: A very nice feature of the compiler here were to perform implementation
: side checks when the complete program is loaded? This involves potential
: ambiguity and absence failures.
For various definitions of "complete". Perl potentially runs a lot
of code at compile time and compiles a lot of code at run time.
: BTW, how far down to pure byte code can Perl6 packages be compiled?
That would depend on your definition of "impure byte code", I expect.
: Too much off the mark?
Compile-time type checking is just another thing we're trying to
make possible without actually doing it ourselves. But it's not way
up there on the priority list.
Larry
Piers Cawley
Doesn't that rather depend on the type of the attribute? Personally, if I get
an object back from accessor method then I expect that any modifications of
that object's state will be seen the next time I look at the results of that
accessor method. This is a direct result of the way reference types work, and
the world is a better place because of it. If you want a (deep) copy of the
returned object you should say so:
my $res = $object.attribute.clone;
Darren Duncan
>Doesn't that rather depend on the type of the attribute? Personally, if I get
>an object back from accessor method then I expect that any modifications of
>that object's state will be seen the next time I look at the results of that
>accessor method. This is a direct result of the way reference types work, and
>the world is a better place because of it. If you want a (deep) copy of the
>returned object you should say so:
> my $res = $object.attribute.clone;
I recanted what you're replying to last week. Essentially, I agree
with you that references of non-scalar values should be returned by
default, and that the method must do an explicit copy if that's what
they want returned. Things are much simpler that way, and its how
Perl 5 worked. -- Darren Duncan
Piers Cawley
Bah! Must start keeping up to date with the list again.